ian parent

New handle, same builder. @IQcrypto → @iparentx Moved from crypto analysis to building dev tools. Specifically: making AI agents trustworthy.

the harness is the moat. but there's a layer most teams skip even when the harness is solid. you can orchestrate perfectly and still ship bad outputs if nobody is scoring what comes out. the eval layer sits between the harness and production. it's exactly what i'm building with iris.

Beyond raw model capability, the real gap in coding tools is the harness. Now that 500k+ lines of Claude Code are out there, every model lab and AI coding startup, including open-source AI labs, will study it and close that gap fast. SF already has Claude Code source walkthrough meetups lol.

@DailyAIWireNews The eval gap in practice. Most agent teams assume their outputs are correct because nothing visibly broke. But "no errors" and "correct output" are very different things. Scoring every output for quality, safety, and cost inline is the missing layer.

Agentic AI Systems Lack Correctness Guarantees, Posing High-Stakes Risks (Source: Johndcook) Agentic AI systems lack guaranteed correctness, posing risks for critical applications. #AICorrectness #AIEthics #AIGovernance #AgenticAI #ReliableAI 🤔 What level of AI reliability is 'good enough' for critical human-centric applications, and who defines it? s.dailyaiwire.news/tAr79f

wrote up the full pattern. why thresholds decay. what self-calibrating eval looks like in practice. and why the eval advisor is where this is all heading. iris-eval.com/blog/self-cali…

self-calibrating eval. the system monitors its own scoring distributions. detects when thresholds drift. recommends adjustments. a human always approves. eval rules that evaluate themselves.

static eval thresholds have an expiration date. you set a cost threshold at $0.50. three months later it's flagging half your traffic. nothing changed in your code. the environment shifted under you.

been thinking about what happens when your eval rules don't match your actual distribution. you set a threshold. it passes everything. or fails everything. neither is useful. wrote something about self-calibrating eval. drops tuesday.

@claudeai this is where eval becomes critical. when agents are reading code and running tests that's one thing. when they can open your apps and click through real systems the cost of a wrong action goes way up. the eval layer can't be optional anymore.

Computer use is now in Claude Code. Claude can open your apps, click through your UI, and test what it built, right from the CLI. Now in research preview on Pro and Max plans.

@lukatofocus @AlexEngineerAI @tanujDE3180 this. the compounding part is what nobody talks about. once the eval loop is running you stop guessing. every iteration gets tighter because you're working from data not vibes. that gap between teams who eval and teams who don't only grows.

@AlexEngineerAI @tanujDE3180 exactly. and the ones who build the eval loop first end up with a compounding advantage - they know what actually works not what sounds like it should work

everyone's still debating which AI model is best i just use all of them Codex for boilerplate, Opus for reasoning, Gemini for multimodal stop picking sides. start routing. the devs who figure this out first will ship faster than teams of 10

@mayonkeyy turing test passed😏

@iparentx 🤣🤣🤣 ok cool

Welcome to level two: recursive self-improvement is now table stakes Your agent is begging for the infra to evaluate variations of itself at scale Everyone who saw this early had the same underlying ideas in their approach: 1. tighten the analyze, iterate, eval loop 2. map evals and traces to failure modes 3. keep writing harder evals If your product's "features" are agents, they are by definition never "complete". Even a magical 99.9% on the benchmarks, is still not the most time or token-efficient version of itself. It's not just slow to A/B test changes to the agent, you're also getting stuck on local maxima. A single regression does not mean the line of experimentation is a failure. Keep driving it forward, explore the sub-paths

@mayonkeyy em dash eval in real-time.

@iparentx > founder, seems chill > tangential product but same problem space, cool > em dash in response...fahh

full post on why the eval loop is the loss function for agent quality: iris-eval.com/blog/the-eval-… 63% of teams have no continuous eval. they shipped an agent that passed a test once. they have no loop.

most teams treat eval as a gate. pass once, ship, move on. that's not how agent quality works. the eval loop: score, diagnose, calibrate, re-score — continuously. the agents that improve are the ones with a feedback loop, not a checkpoint. wrote about why this changes everything:

@bernhardsson the missing piece in the new ci: output eval. tests verify code works. eval verifies the output is actually good. agents can pass every test and still leak pii or burn 10x your cost budget. ci for agents needs a scoring layer, not just pass/fail.

CI feels more interesting today than it ever was. Writing code has gotten a lot faster, but this shifts the bottleneck elsewhere. I’m excited about sandboxes as a primitive for massive parallelization of tests.

@GG_Observatory this is the take more people need to hear. the moat isn't the agent — it's knowing when the agent is degrading. eval drift is invisible until it's expensive. most teams find out from users, not from their own systems.

Hot take for 2026 AI agents: the moat isn’t "more tools," it’s reliability engineering. Teams that win will version prompts like APIs, track eval drift daily, and enforce rollback SLAs for every agent workflow. Are you measuring agent MTTR yet?

@TrustWallet even more reason to use @iris_eval evaluate what your agents are doing onchain

AI agents are about to move more money than most traders. They're going to need a wallet. gm 👋

@VibeCoderOfek @ZssBecker agents without eval are demos, not products.

@ZssBecker Finally someone saying it. Agents are amazing but the cleanup still needs senior taste. Team human forever.

I posted about how AI/LLMs are no where near where they need to be to replace engineers. Was hit by 1000 gen z'ers screaming I'm wrong. Skill issue. Etc etc. MFers have literally let Claude convince them they are a special skilled snowflakes. Brother. Your shipping dog shit.

we've been calling this exact gap "the eval gap" — the distance between benchmark performance and production reality. it's structural, not incidental. wrote about it here: iris-eval.com/blog/the-eval-… the short version: benchmarks test capability. production needs continuous inline eval on every execution. different problem, different tooling.

@_odsc SWE-bench and WebArena are doing a lot of heavy lifting right now for agent eval. Curious how teams are thinking about the gap between benchmark performance and real-world task completion once agents hit production.

Explore 15 essential datasets for training and evaluating AI agents, including tool calling, web navigation, and coding benchmarks like SWE-bench and WebArena. #AI #ArtificialIntelligence #DataScience hubs.li/Q048DYYW0

the eval tax isn't just the cost of evaluating. it's the cost of not evaluating — the failures, the manual review, the customer churn from bad agent outputs. you pay it either way. the only question is whether you pay it with tooling or with incidents. the data is starting to prove this out at scale.

cio.com just wrote about the "hidden cost of ai agent evaluations" — $47K from a single runaway agent, organizations getting 5-figure eval bills they didn't expect. we've been calling this the eval tax. published about it weeks before this article came out. iris-eval.com/blog/the-ai-ev…