olehmell.sub 🇺🇦 retweetledi

gpt-oss uses tokens from Phi-4 tokenizer
You can verify this yourself:
Download the Phi-4-mini-instruct tokenizer:
huggingface.co/microsoft/Phi-…
Compare it with the gpt-oss tokenizer:
huggingface.co/openai/gpt-oss…
They differ in the number of merges and file size, but contain the same tokens
English

















