Sabitlenmiş Tweet
New Results!
This will likely be the final set for a couple days (depending on what @Anthropic brings Tuesday). We can't spare the usage to run @claudeai Fable 5 on this right now, which is unfortunate given the fall-off of Opus relative to the new releases.
Strong work from @grok @SpaceXAI @elon where 4.5 just continues to exceed expectations.
Same from @OpenAI @sama @thsottiaux with 5.6 where the whole range is impressive. New levels of quality seen from Sol, and we are working on a planning comparator, where I expect Sol to really stand out (based on my own use of the model).
----DH-SWE29 Updates----
This week the audit found five more scoring checks that were grading the shape of the code, things like exact key names, exact strings, and one specific control-flow path, instead of testing whether the fix actually worked. We corrected them and rescored every saved run. No model was re-run.
The corrections moved the board. Several models gained, and the OpenAI configs gained the most, because they were the ones getting marked down on shape rather than behavior. The result is a split decision with no single winner.
Most solved: Grok 4.5, 18 of 29.
Highest quality: GPT-5.6 Sol, 0.797 mean score.
Best value: Grok 4.5, $1.76 per solved task.
The 10 hardest tasks are still unsolved by every model. Full board and method in the replies.
English
