Mojan Javaheripi

Happy to share 💭 Mixture of Thoughts 💭 A curated, general reasoning dataset that trims down over 1M samples from public datasets to ~350k through an extensive set of ablations 🧑‍🍳 Models trained on this mix match or exceed the performance of DeepSeek's distilled models -- not just on math/code but also on scientific benchmarks like GPQA We also validate that the "additive" methodology from Phi-4-reasoning really works! You can optimise the data mixture independently per reasoning domain and then bring it all together for the final run 🔥 Link to the dataset ⤵️

I am thrilled to share our newest Phi models. This time we went all in on post-training to produce Phi-4-reasoning (SFT only) and Phi-4-reasoning-plus (SFT + a touch of RL) — both 14B models that pack a punch in a small size across reasoning and general purpose benchmarks🧵

Introducing Phi-4-reasoning, adding reasoning models to the Phi family of SLMs. The model is trained with both supervised finetuning (using a carefully curated dataset of reasoning demonstration) and Reinforcement Learning. 📌Competitive results on reasoning benchmarks with much larger top-tier models up to DeepSeek R1 📌 Strong performance on new tests released after data collection (AIME 2025, HMMT) 📌Reasoning transfers/generalizes well to new domains even with only SFT (e.g. k-SAT, Mae Solving, Calendar Planning, etc.) 📌Retains and often significantly improves general-purpose capabilities (e.g. instruction following) In addition to the models, we are also very excited to share a very detailed technical report with insights on model training and evaluation Still have a lot to improve especially with context length, coding and tools. Hope you find the models useful! A big thanks to the amazing team and to all our partners.

We’ve been cooking... a new open weights 14B Phi-4 reasoning model, SFT’d on ~1.4M carefully curated reasoning demonstrations from o3-mini and RL’d for a tiny bit. This model is a little beast.