Jeff

Everyone's stacking five AI tools. I deleted mine. My assistant (Hermes) ran on GPT-5.5. Couldn't move it to Claude, you can't point a third-party agent at a Max plan. So I rebuilt it native. It doesn't use Claude Code. It is Claude Code. One assistant. From my phone. ~$0/mo.

@camsoft2000 If you have a real harness built for your personal workflows and optimization opus 5 is better than 5.6. If your just vibe coding use 5.6

Have the usual mix of Claude Opus 5 is terrible I can't work with it and Claude Opus 5 is the best model just delete all your skills. I guess time will tell at this stage I don't feel that compelled to try it out, GPT 5.6 models have been great for me.

@arushiitwt @Abdul_ah14 Graph. Only if your corpus warrants it via multi-hop. Likely, RAG is what you need.

@Abdul_ah14 What's alive?

Thinking of starting RAG....any good resources to learn it?

@prasenx 80% of Fable 5 for 25% less price. My daily driver now, with Fable as reviewer.

what do you think about Claude opus 5 right now

@Tech_girl Far better than 4.8. Cheaper than fable for about 85% of the quality. Running Fable as planner/reviewer with opus 5 driving/building seems optimal right now.

Now that opus 5 is out, How good is opus 5 compared to opus 4.8 and fable 5?

@melvynx For average vibe coder no. If you have a harness in place optimized for your specific workflows, yes.

be honest, is Claude Opus 5 better than GPT Sol ?

@CodeWithAmann Both, opus 5 driving with fable as review/planning partner

Be honest, which AI model is best at Coding?

If you maintain one of these frameworks and think I mapped your detection surfaces wrong, the spec says disputes get adjudicated publicly. I've opened an issue on each repo asking exactly that. Spec, papers, harness, every trace: github.com/Jott2121/sabot

Caveat that belongs on every number above: one pipeline model, five seeds, pinned framework versions. This is these frameworks with this model at these versions, not a law of nature. Replication on other models is the obvious next thing and I haven't done it.

You don't have to take my word for any of it. The headline metric has no LLM in it, so you can rebuild every published number from the raw traces offline, no API key, stdlib only: python3 scripts/score_wave2.py --seeds-file ../seeds/wave2.json All 189 rows come back identical.

Other things that didn't work, published at equal weight because I committed to that before I had any numbers: Zero signal on silent model downgrades. The intervention perturbed the orchestrator I was measuring. One cell class that was already at ceiling got worse.

Now the part I'd rather you hear from me than find yourself. Detection tripled. Corrective action did not budge. On the original surface, recovery-without-detection is still 50%. The pipelines now tell you exactly what they saw, and still don't act on it.

So the honest version is not "I fixed it." It's: notice-without-act was invisible, and now it's legible. That's real progress and it's also a smaller claim than the headline number looks like. Legible beats silent. It isn't the same as fixed.

On Magentic-One it went from roughly zero to 73.6%, but only after I stopped appending the requirement as an instruction and made it line 1 of the output contract. Orchestrators rewrite worker instructions. They don't rewrite the format. That distinction was the whole ballgame.

So I tested the fix. One paragraph added to the reviewer prompt: before your verdict, list every inconsistency you noticed and quote the exact conflicting values. Same frameworks, same tasks, same seeds, same 825 cells. One change. 16.7% to 55.0%.

Read the traces and you can watch it happen. The model notices the anomaly. Then it routes around it and says nothing. It wasn't blind. Nothing ever asked it to report what it saw. That's a protocol gap, not a capability gap. Which means it should be fixable.

The part that matters isn't the 16.7%. In every single row, the most common outcome was this: the task came out CORRECT while nothing in the pipeline ever flagged the fault. 51 to 61% of the time. The pipeline just absorbs the sabotage and moves on.

Every agent framework ships self-checks. Reviewer agents, critic stages, guardrail callbacks. The whole reliability story rests on those working. So I planted 900 faults inside live LangGraph, CrewAI and AutoGen runs and scored what their own checks caught. Median: 16.7%.

One fault per run. A corrupted tool result. A falsified success report. An altered message between agents. A silently downgraded model. Stale context. A write that reported done and never happened. Five tasks, five seeds, both default and best-documented-guardrail configs.

@mattpocockuk This is the mindset

Opus 5 As usual I have no take, other than: If you've designed your harness/environment well, and not over-optimised around a specific model, today should feel like any other day... ...with a slightly lower failure rate

@maxedapps It is really good, super fast. But, I always end up having opus 4.8 or Fable clean up the product before I ship if I built with Grok. Which is fine, I just really want to see grok catch up to the real frontier

Grok 4.5 is such a capable work horse!