Abhi Venigalla

I've been working on a new LLM inference algorithm. It's called Speculative Speculative Decoding (SSD) and it's up to 2x faster than the strongest inference engines in the world. Collab w/ @tri_dao @avnermay. Details in thread.

🚀 Today we’re releasing FlashOptim: better implementations of Adam, SGD, etc, that compute the same updates but save tons of memory. You can use it right now via `pip install flashoptim`. 🚀 arxiv.org/abs/2602.23349 A bunch of cool ideas make this possible: [1/n]

Taalas Specializes to Extremes for Extraordinary Token Speed eetimes.com/taalas-special…

2 weeks ago, we rebuilt our entire product. "Browser automation" fell short of our mission to eliminate all repetitive knowledge work. The new Kaizen is the ultimate digital employee: always on, extremely capable, continually learning. Sign up for access in the tweet below.

InferenceX v2: NVIDIA Blackwell Vs AMD vs Hopper - Formerly InferenceMAX, GB300 NVL72, MI355X, B200, H100, Disaggregated Serving, Wide Expert Parallelism, Large Mixture of Experts, SGLang, vLLM, TRTLLM semianalysis.substack.com/p/inferencex-v…

OpenAI Codex-Spark powered by Cerebras You can now just build things faster—at 1,000 tokens/s.

sim the people, sim the world, join simile. they kick ass. @joon_s_pk @msbernst @percyliang @ElainaYallen @mihikapoor

The world’s most powerful data agent releases today. Sphinx 1.0 is here to power elite data teams.

Excited to announce today that my startup, @positron_ai, has closed a $230M Series B financing round at an over $1B valuation, co-led by great folks at @jumptrading, Arena, Unless Ventures, and strategic backing by @Arm! bloomberg.com/news/articles/…

OpenAI🤝Cerebras openai.com/index/cerebras…

@Arunabh85992578 @firstadopter Your GB200 flop/s are 2x higher than they should be. You're quoting the sparse flop/s number not dense

After ten years, can someone give me an estimate of TPU 2025 revenue for external customers?​​​​​​​​​​​​​​​​ Bueller?

I've got something new for everyone. My first substack article! Not the one I planned to do first, but a fun one! I have made a handy calculator base on the DeepSeek v1 coefficients for finding optimal LR and batch sizes for dense LLMs.

Apologies that I haven't written anything since joining Thinking Machines but I hope this blog post on a topic very near and dear to my heart (reproducible floating point numerics in LLM inference) will make up for it!

🚀 Thrilled to announce our $9.5M funding round led by @buckymoore at @lightspeedvp, alongside an incredible group of investors from the Valley and New York. ✨ With this announcement, we’re also moving Sphinx Copilot -- the state-of-the-art AI agent for data science -- out of closed beta. It’s now available at sphinx.ai (with a generous free tier!). Our early partners have gone from raw data ➝ commercial insights in minutes instead of days. We can’t wait to see what the data community builds with Sphinx. 🌱 This is just the beginning for Sphinx. We’re redefining how AI works with data, from copilots to fully autonomous researchers and analysts. We're excited to keep building best-in-class machine intelligence for a new generation of data-driven innovation. sphinx.ai/blog/sphinx-la…

alexander wang with yann lecun

memory-bound gf, compute-bound bf

presenting: big jeff's trainium hell

Cerebras just beat NVIDIA Blackwell Last week: Blackwell hit 1,000 t/s on Llama 4. Today: Cerebras hit 2,500 t/s on the same model, same benchmarks by @ArtificialAnlys Blackwell smoked Groq, AMD, Google – everyone. Only Cerebras stands – and we smoked Blackwell.

@chase1440 @code_star @mvpatel2000 Good times :)

@code_star @mvpatel2000 @ml_hardware tired: spec sheets wired: MFU mic-drop aka @ml_hardware ripping a live training job on 1024 H100s and screensharing the evals with a skeptical customer

A great way to tell if an org has good ML eng is by backing out their MFU and checking if it's actually good when they brag about their training stack. Super useful to know 1) all the numbers (memorize hardware stats!) and 2) how to drive the math