Manish Lad

Full breakdown in this week's issue of AI from the Inside 👇 manishlad.substack.com/p/stop-grading…

If a vendor tells you "we score 92% on our eval" — ask what's actually being measured. Similarity to a reference, or coverage of known failure modes. Not the same thing, and only one predicts whether it'll catch what matters in production.

Made a mistake building my first AI eval set. Wrote "ideal" reference outputs first, then scored the agent against them. Took a few examples to realize I wasn't measuring quality. I was measuring "does this sound like me."

📊 PUG Tech Talk | Session Spotlight — Weaving Data Through Time! At our upcoming PUG Tech Talk on Saturday, 18th July 2026, we're excited to have Aditya Kulkarni, Senior Consultant at Rapid Circle, take us through the evolution of data platforms. Session: Weaving Data Through Time: From Data Warehouses to Microsoft Fabric Data platforms have come a long way — from traditional data warehouses to the unified, all-in-one world of Microsoft Fabric. Aditya will walk us through this journey, helping us understand how the landscape has evolved and what it means for how organisations manage and use their data today. A great session for data engineers, architects, and anyone curious about where the data platform space is headed. 📍 Rapid Circle, 5th floor, The Kode Building, Baner-Pashan Link Rd, Pashan, Pune, Maharashtra 411021 🗓️ Saturday, 18 July 2026 ⏰ 10:00 AM to 1:00 PM 🎟️ Free to attend Register now 👇 🔗 meetup.com/puneusergroup/… Also, join our official WhatsApp Community to stay updated on events, meetups, workshops, and more: bit.ly/pugwc Thank you to @RapidCircle_IN for hosting us! 🙌 #PuneUserGroup #PUG #TechTalk #MicrosoftFabric #DataEngineering #DataWarehouse #PuneTech #DeveloperCommunity #RapidCircle

🚀 PUG Tech Talk is back on Saturday, 18th July 2026! Pune User Group in association with Rapid Circle presents our next Tech Talk — and we're kicking things off with Microsoft MVP Mahesh Sabnis @maheshdotnet ! Session: New AI Capabilities in ASP.NET Core 10 ASP.NET Core 10 ships with powerful new AI features — from streaming AI responses to working with independent models. Mahesh will walk us through what's new and how you can start leveraging these capabilities in your own .NET applications. 📍 Rapid Circle, 5th floor, The Kode Building, Baner-Pashan Link Rd, Pashan, Pune, Maharashtra 411021 🗓️ Saturday, 18 July 2026 ⏰ 10:00 AM to 1:00 PM 🎟️ Free to attend Register now 👇 🔗 meetup.com/puneusergroup/… Also, join our official WhatsApp Community to stay updated on events, meetups, workshops, and more: bit.ly/pugwc A big thank you to Rapid Circle for hosting us this time around! 🙌 #PuneUserGroup #PUG #TechTalk #ASPNETCore10 #DotNet #AI #MicrosoftMVP #PuneTech #DeveloperCommunity #RapidCircle

📅 Save the Date! PUG Tech Talk is back on Saturday, 18th July 2026. This time, we're excited to have @RapidCircle_IN host us at their office in Baner-Pashan! We're finalizing an exciting lineup of sessions and speakers — stay tuned, details dropping soon. If you're part of the Pune tech community and love learning, networking, and geeking out over the latest in tech, this is one to block your calendar for. 📍 Rapid Circle, 5th floor, The Kode Building, Baner-Pashan Link Rd, Pashan, Pune, Maharashtra 411021 🗓️ Saturday, 18 July 2026 ⏰ 10:00 AM onwards 🎟️ Free to attend Register here👇 🔗 meetup.com/puneusergroup/… More details coming your way soon. See you there! 🙌 #PuneUserGroup #PUG #TechTalk #PuneTech #SaveTheDate #DeveloperCommunity #RapidCircle

Without evals, your honest answer to "did this get better?" is "it feels like it." With them: "accuracy went from X to Y, here's what regressed." One of those holds up in a roadmap review. Full article 👇 manishlad.substack.com/p/why-ai-produ…

You don't need fancy tooling to start. 20–30 real examples. Write down what "correct" means. Run before every meaningful change. Grow it as new failures show up. That's genuinely enough on day one.

I once shipped a 2-line prompt change. Tested it on 5 examples. Looked great. 2 weeks later: a whole category of inputs was quietly failing. My QA process was "this looks better to me." That's not QA. That's vibes. Here's why AI products need evals 🧵