rishi
2.2K posts
Our open models are designed to support the Genesis Mission by giving the scientists in our national labs the flexibility and sovereignty to work on their own terms. Learn more ⤵️
Reflection@reflection_ai
English
frontier will catch on soon
Pavlo Molchanov@PavloMolchanov
We’re releasing Nemotron-Labs-Diffusion - the first Tri-mode LM family (3B/8B/14B) that switches between 1⃣Autoregressive, 2⃣Diffusion, and 3⃣Self-Speculation decoding by simply changing the attention pattern/mask. One model Three decoding modes. No extra draft models. No architecture changes. Just significantly better efficiency across different concurrency levels. Up to 4× higher real throughput for a single user. 🤗 HF Collection: huggingface.co/collections/nv…, open license 🛜 Project page: research.nvidia.com/publication/20… 📰 Tech report: bit.ly/Nemotron-Labs-… Details below 👇
English
@ishaans22 logo 3 early shot clock to tie the game before double ot? at 7'7?
English
i remember when this model was js homie’s schizo project, he literally took it from 0 to 100.
proud of u homie 😭😭
rishi@rishiiyer01
Leading the training for this model was a privilege. Training diffusion style models will be the future regardless of whether it is discrete/speculative or continuous.
English
rishi retweetledi
@rishiiyer01 cooked here. very excited about scaling ttc with diffusion - sparse active params + diffusion decode means reasoning models can punch above their weight class with competitive latency
x.com/rishiiyer01/st…
rishi@rishiiyer01
Leading the training for this model was a privilege. Training diffusion style models will be the future regardless of whether it is discrete/speculative or continuous.
English
Booom 💥
Ekhem hello @ZyphraAI full tech report?
Zyphra@ZyphraAI
We present ZAYA1-8B-Diffusion-Preview, the first diffusion language model trained on @AMD. Autoregressive LLMs generate one token at a time; diffusion generates a block in parallel, speeding up inference. We show a 4.6-7.7x decoding speedup with minimal quality degradation 🧵
Svenska
rishi retweetledi
very excited about scaling ttc with diffusion. sparse active params + diffusion decode means reasoning models can punch above their weight class with competitive latency
x.com/ZyphraAI/statu…
Zyphra@ZyphraAI
Today we're releasing ZAYA1-8B, a reasoning MoE trained on @AMD and optimized for intelligence density. With <1B active params, it outperforms open-weight models many times its size on math and reasoning, closing in on DeepSeek-V3.2 and GPT-5-High with test-time compute. 🧵
English
rishi retweetledi
Diffusion is the endpoint of inference. Decode becomes as efficient as training. Everything sits on the roofline. Pure FLOPs will be the only bottleneck.
Excited to push the frontier on language diffusion. Massive congrats to the team and an exciting time ahead.
Zyphra@ZyphraAI
We present ZAYA1-8B-Diffusion-Preview, the first diffusion language model trained on @AMD. Autoregressive LLMs generate one token at a time; diffusion generates a block in parallel, speeding up inference. We show a 4.6-7.7x decoding speedup with minimal quality degradation 🧵
English
@rawsh0 @JZWANG_T1 I am additionally excited for RL native in diffusion mode
English
Leading the training for this model was a privilege. Training diffusion style models will be the future regardless of whether it is discrete/speculative or continuous.
Zyphra@ZyphraAI
We present ZAYA1-8B-Diffusion-Preview, the first diffusion language model trained on @AMD. Autoregressive LLMs generate one token at a time; diffusion generates a block in parallel, speeding up inference. We show a 4.6-7.7x decoding speedup with minimal quality degradation 🧵
English
rishi retweetledi
Hosting a research meetup in our North Beach office on Thursday! Come by for food, drinks and talks:
@jyangballin (MSL) will present ProgramBench
@rawsh0 & @rishiiyer01 (Zyphra) will talk about ZAYA-8B
@evan_j_chu and I will speak FrontierSWE and our research bets!
English
@LLMenjoyer @rawsh0 i might nano banana all the pictures I have of you in my phone and put them on main
English
@rawsh0 If you are using ZAYA1-8B in openwebui, make sure you stop doing that and use a bigger model immediately.
English
If you are using ZAYA1-8B in openwebui, make sure you set frequency_penalty 0, presence_penalty 0, repeat_penalty 1 (disabled), and temperature 1, top_p 0.95, top k 0 (disabled).
default settings can cause looping 🔁
English
@HudsonGouge Yes, this is exactly why we decided to release our results (agentic, math, code). Losing on GPQA-d and Mmlu-pro are not the end of the world here. Bigger models should win on these knowledge based evals, though we should still come quite close
English
@rishiiyer01 Now, I don't mean that in a bad way. That's amazing for such a small lab.
English
Lots of potential in this model to exploit in RL. Particularly proud of the context extension engineering I cooked on this one. I’m personally far more confident in scaling some of my crazy arch ideas now as well
Zyphra@ZyphraAI
Today we're releasing ZAYA1-74B-Preview, a major milestone in scaling pretraining on @AMD. ZAYA1-74B-Preview is a 4B active / 74B total MoE. This preview model is a strong pre-RL base checkpoint. The final post-trained reasoning model is coming soon. 🧵
English