Kit Fraser-Taliente
4 posts
Kit Fraser-Taliente
@KitF_T
meading rinds at @anthropicai
Bergabung Haziran 2016
692 Mengikuti100 Pengikut
@mikeknoop putting it out there like this increases the chances that they comment on it
English
we might be getting scammed by Anthropic:
"we speculate this is a smaller model (maybe Sonnet-ish?) that runs thinking for longer"
Mike Knoop@mikeknoop
The headline is Opus 4.6 scores 69% for ~$3.50/task on ARC v2. This up +30pp from Opus 4.5. We attribute performance to the new "max" mode and 2X reasoning token budget -- notably task cost is held steady. Based on early field reports and other benchmark scores like SWE Bench, we speculate this is a smaller model (maybe Sonnet-ish?) that runs thinking for longer. If true, ARC v2 is measuring the "CoT search" complexity capability of the AI reasoning system, independent of model knowledge. Pretty cool! To get a sense of the complexity limit, here are all the v2 tasks Opus 4.6 failed to solve: arcprize.org/tasks/?dataset…
English
Kit Fraser-Taliente me-retweet
We just shipped Claude Opus 4.6!
I’m also excited to share that for the first time, we used circuit tracing as part of the model's safety audit!
We studied why sometimes, the model misrepresents the results of tool calls.
English
Kit Fraser-Taliente me-retweet
We recently released a paper on Activation Oracles (AOs), a technique for training LLMs to explain their own neural activations in natural language.
We piloted a variant of AOs during the Claude Opus 4.6 alignment audit. We thought they were surprisingly useful! 🧵
English
