WFH retweetledi
🎙️Introducing Max Agency
Max Agency is a new podcast where we go deep on how the best agents are actually being built: architecture decisions, tradeoffs, evals, and everything in between. Each episode, I sit down with engineering leaders who are doing this work in production.
Our first episode features Izzy Miller (@isidoremiller), AI Engineer at Hex (@_hex_tech). Hex has been shipping data agents since before most teams were even thinking about them, starting with single-cell text-to-SQL and graduating to a full Notebook agent that can work autonomously for 20 minutes on a complex analysis.
Izzy has a lot of perspective on what it actually takes to get agents working well in production, and what breaks along the way.
A few takeaways from our conversation:
- Keep your eval sets small enough to hold in your head: Izzy runs 30-50 handcrafted "traps" with multiple repetitions, rather than hundreds of variants. If you can't explain why your agent fails each one, your eval set is too big
- Day zero performance is almost irrelevant: The more interesting question is how the agent compounds. Izzy is building a 90-day simulation where the warehouse evolves and the agent has to accumulate understanding
- You can catch agent errors without seeing the raw outputs: By running an LLM-as-a-judge over production usage and clustering the results, you can surface places where something likely went wrong, without needing to read individual conversations
Watch the full episode on:
- Youtube: youtube.com/watch?v=Xyh1Eq…
- Apple Podcasts: podcasts.apple.com/us/podcast/how…
- Spotify: open.spotify.com/episode/1BJlg3…
YouTube
English
