Mohit Sudhakar

AI memes are getting ridiculously good 😂 This Mr. Bean remake with Sam Altman and Tim Cook was made with Kling 3, and it’s absolute gold.

They’re just creating an artificial supply shortage.

@steipete Bro joined openai and started shilling codex on every tweet.

If you run this workflow, ask Fable to make codex the workhorse. github.com/steipete/agent…

@yadavji_codes I switched away from Claude to GLM 5.2 running on NeuralWatt.

I pay $200/month for Claude Max and hit the limit in under 1 hour. What am I even paying for ?

That actually makes perfect sense

How long after sex is it appropriate to open VS Code

@voicearena_ai @Dheemanthredy @mayaresearch_ai HF link?

Maya-2-Native is the #1 streaming TTS model on the Voice Arena Hindi TTS Leaderboard 🇮🇳 In the overall leaderboard (streaming + non-streaming), a native Indian voice lab has officially broken into the top tier, debuting at rank #2 globally, moving ahead of global heavyweights like ElevenLabs v3 and Cartesia Sonic-3.5. Maya-2-Native is the latest TTS model from @mayaresearch_ai. Built locally to capture the true texture of everyday Indian speech, it handles complex code-switching, local dialects, and natural cadence effortlessly. The model has been highly preferred among native speakers rating on Voice Arena. Results backed by 15.5K+ human-verified, blind, head-to-head listener votes 🧵

The exponential progress has begun

First you have to understand that modern LLM inference already disaggregates weights as models outgrew single chips years ago. You shard either by layer (pipeline parallelism) or by slicing every layer (tensor parallelism), and the two do very different things. As an example, let’s look at Llama 3.3. It has 70B of weights and at FP8 that’s 70 GB of memory which is enough to fit on a single H100. Now that H100 has 3.35 TB/s of HBM, so the fastest it can ever decode for one user is 70/3.35 ≈ 21 ms/token or ~48 tok/s while using under 1% of its FLOPs. Now if we pipeline it across 8 chips: each chip holds ~8.75 GB, which means it only needs 1/8th the bandwidth and 1/8th the FLOPs to sustain the same aggregate throughput. Now crucially the token/sec a user gets is limited by the amount of data that crosses the link. In current LLMs all that is a small amount of activations for LLama 3.3 it’s ~8 KB per token…. Yes, you read that right it’s 8 KILOBYTES we are sending over a <900 GB/s link. That’s only 9 ns of serialization time but the overhead of 224G PAM4 SerDes adds ~100 ns per link traversal with RS-FEC which is 11x longer than the payload itself. And then you have the NVSwitch adding ~300 ns per hop and you need to pay twice. That’s ~600 ns of just hardware latency wrapped around 9 ns of data making a 98% tax before software even shows up. Then NCCL’s collective stack turns 600 ns into 10-20+ us… all to move 8 kilobytes lol. For comparison 8 KB serializes over 10 Gigabit Ethernet NRZ, in just 6.6 us. Pipeline parallelism however doesn’t make a single user faster as the token still needs to visit every layer in the sequence, so per-user speed is still weights / per-chip bandwidth. To get more speed per user token you need to use tensor parallelism and have all the chips work on the same layer simultaneously. TP costs you 2 all reduce OPs per layer, 160 per token on llama 3, that’s still kilobytes of traffic but with NVLink overhead it’s a massive tax and why pipeline parallelism on most models still gives more interactivity per user. However, this gives you a huge latency lever to pull that scales tokens per second with interconnect speed instead of memory BW. The clever amongst you might have also realized that sharding doesn’t just cut memory bandwidth per chip it also cuts FLOPs per chip and is why we have such bad MFU on decode. So once you’ve sized the link for the memory, you need to size the compute for it too. This is called “balancing the pipeline”, and currently no shipping chip does it because they were all designed as standalone monsters. Remember Tokens/sec = ~aggregate memory BW / bytes touched per token. At batch 64 in FP4 you need ~250 FLOPs per byte, and Blackwell ships 1,250. Provisioned 5x more than the narrow pipe of HBM. Nobody saturates shit cause they are all building around HBM. So now it all comes full circle. Parallelism reduces memory bw pressure and thus FLOPs but increases interconnect latency pressure. Despite having HBM and GigaSERDES we aren’t actually doing more work lol. But if you really wanted to balance the pipeline you need to match the memory bandwidth, the flops, and most importantly the interconnect. So what does that look like ? Well if you build around LPDDR’s lower bandwidth, lower your interconnect latency, you actually can beat Nvidia on decode with a fraction of the silicon.

Lots of people are advocating for more American open-source models these days which is amazing but very few people do anything about it! Latest example, Alex Karp came out advocating for American open-source models as a necessity! At the same time, @PalantirTech is a free org on HF with 0 open-source models and 0 public datasets shared. Time to switch from talking to contributing for all!

Mocktails should be $4 and cocktails should be $8 max

Is Sonnet 5 better than Gemini 3.5 Flash? On vibes alone, can't tell.

@quxiaoyin OMG so true

I went a dinner a few weeks ago with a bunch of enterprise execs who told me "we will never use Chinese models." "Even if it's 100x cheaper?" "No, we care about safety and security." 1. They don't understand when they host open-source models with their own GPUs or US data centers, they won't share their data to China. 2. They are giving away all their data to OpenAI and Anthropic rather than owning it privately themselves. 3. They don't understand math. 100x is a big number and lots of profits. It's almost July 2026 now. If your execs still talk like that, fire them now.

@agazdecki Any advice?

Founders who understand sales usually build better products.

@claudeai Great, so we get Fable 5 for One week. On a long weekend. With Claude servers being slow. For 50% usage quota. And it defaults to Opus for coding tasks. On a $200 plan. F this. I'm unsubscribing.

Fable 5 is back.

@gregisenberg Can we use AGI to solve it :)

Pretty funny that we’ll have AGI before we can get Bluetooth devices to pair reliably

@tunguz Possible. But I believe as we have to keep up with smarter AI we will be forced to grow smarter as well.

I saw the greatest minds of my generation get one-shotted by the AI psychosis.

Turns out Elon is right again. The shittiest layer in AI is the model layer. The real money in AI is in compute, energy, and applications. Elon has all 3. Without Chinese open weight models, OpenAI and anthropic would have been happy oligopolies and made trillions. Now, they hired the best talents, burned billions and built the newest model, only to have Chinese free models wiping out all your margins. Every other layer is making money except for you, even though you invented the whole thing. That must feel shitty as hell.

Was bound to happen. The harness was eaten by model advancement.