Edward Z. Yang

@fasttosmile In this style of AI programming, you are in fact reading all the code. But you are "reading" the code, in the same way that when you write code you are implicitly reading it. Fast typist, not autonomous software engineer.

@ezyang How does one steer without reading the code? Is the idea to focus more on outputs and tests? Or are you saying one should focus more on the higher level abstractions rather than line by line stuff.

New blog: Read Less, Steer More blog.ezyang.com/2026/03/read-l…

@deepestbrew It's cool for me too, when a tool I use that traditionally had terrible error messages gets better messages, not because I was complaining, but because they're trying to get the agents banging on their API to do the right thing.

@ezyang Cool for them, or for us?

One of the cool thing about the AI revolution is that spending a lot of effort on error messages is cool again, because the agents actually read the error messages!

Similarly, the inability of LLMs to have memories is a blessing, because you can keep tweaking the error message until the LLM one shots the fix

@Jon85Ma My general attitude is that LLMs haven't changed any of the old problems with formal verification. If the project was amenable to formal verification before, LLMs can help you do it faster and cheaper, pushing it into feasibility. If it wasn't, it still isn't.

@ezyang What about writing proofs with LLM, then having some machine learning for the compilation?

A question of intense interest to me is how compilers (and more specifically, compilers for deep learning) should evolve in the era of LLM coding. 🧵

@jamonholmgren I see that the parent post is about day shift / night shift, though, so there probably won't actually be an opportunity to review plans in this case.

@jamonholmgren The point of reviewing the plan is not to /review review/ it (that's what the code review at the end is for), it's to make sure the agent didn't misinterpret your intention or go completely off the rails.

To reiterate a few things so they don't get lost: 1. I never want to review another agent-produced plan again. Waste of my time, overwhelming, not worth it. It's valuable *to the agent*, but not to me. 2. I will burn all the tokens, run all the tests, do all the validations to make sure that when the work product lands on my desk, it's as good as the agents can make it. My time and energy is the most important thing here. 3. The feedback loop is critical: I'll work on the process, docs, and specs as much as I need to, in order to reap the benefits in future sessions. No more manual guidance via interactive sessions (with the exception of exploratory hacking).

@boyuan_chen I am extremely simulator-pilled but also for some reason no one uses them so there must be some sort of Chesterton's fence effect going on here

The Triton vs cuteDSL framing in (1) is sharp. Curious where you think the escape hatch should be for graph-level optimizations though - fusion, memory planning, operator scheduling. Those feel harder to verify cheaply than kernel-level perf, which makes the reward signal problem from (2) even worse at higher IR levels.

@spillai I too am puzzled what to do about distribution of libraries of bespoke kernels. Humans hate having to interface with this sort of thing. Perhaps it is best used as grist for the next LLM kernel gen...

There’s at least another 20% more we can squeeze from this approach before we see diminishing returns. The crazy bit here is that we’re really folding all the known optimizations for a specific model architecture which vllm might not be willing to upstream because it becomes too bespoke for general LLM inference.

@spillai Richard was telling me about the kernel bench rms norm: cool stuff, and also a bit scary!

@ezyang Such an insightful thread - we’ve been adding a handful of LLM-driven optimisations to vllm especially for VLM inference, and so far we’re seeing 30-40% improvement in throughput/latency.

Opus and I chatted about this thread, I think the convo taught me some things about the question claude.ai/share/7e57477a…

Claude's opinion on this thread (Claude doesn't have a personality -- said nobody, ever)