anand iyer

“If your $500K engineer isn’t burning at least $250K in tokens, something is wrong.”

Am looking at 4 key dimensions, 1. output efficiency = output tokens / total tokens 2. context amplification = cache reads / cache writes 3. Iteration cost = tokens per agent step 4. Tokens per task The key insight: LLM cost scales with context size × iterations, not output length Most token usage isn’t generation. It’s context reuse. Put in other words, The model isn't expensive because it talks a lot. It's expensive because it repeatedly rereads the same context. Here is a simple example, in our dataset: - 17.1B cache reads - 1.36B cache writes Amplification ~ 12.6× Meaning each prompt is reused ~12 times. The result: Output efficiency was only ~0.8% In other words: Less than 1% of tokens were actual model output. The rest were context movement. The biggest inefficiency we found: Large repo contexts + long agent loops. Example: 120k token repo context 50 iterations = 6M tokens But if you reduce context to 40k: 40k × 50 = 2M tokens 67% savings immediately.

Are you fleeing to robotics because Claude Code cooked your job? Yeah... me neither... 😅... As part of building in the open, here are some more learnings from the physical AI ecosystem: Folks under-estimate the number of pain points and problems worth solving for the software-first robotics founder persona!

Excited to launch Gemma 4: the best open models in the world for their respective sizes. Available in 4 sizes that can be fine-tuned for your specific task: 31B dense for great raw performance, 26B MoE for low latency, and effective 2B & 4B for edge device use - happy building!

After 13 years, I can distill all parenting advice to one variable: your kid’s closest friends.