Roger Grosse

Before limited-releasing Claude Mythos Preview, we investigated its internal mechanisms with interpretability techniques. We found it exhibited notably sophisticated (and often unspoken) strategic thinking and situational awareness, at times in service of unwanted actions. (1/14)

Mythos Preview seems to be the best-aligned model out there on basically every measure we have. But it also likely poses more misalignment risk than any model we’ve used: Its new capabilities significantly increase the risk from any bad behavior. 🧵

AI is threatening our democratic society—by concentrating power, narrowing how we think, and flooding institutions faster than they can keep up. These risks emerge at the system level, and technical work alone won't fix them. 👉Check out our whitepaper with 25+ researchers: zhijing-jin.com/d/2026-ai-risk… 💡We introduce 7 threat models and ways forward. ✍️Led by @davidguzman1120 with @DaveRBanerjee, @blin_kevin, @PepijnCobben, @gcorsi_, @x_angelohuang, @ChanglingXavier, Suvajit Majumder, @psyonp, @SimkoSamuel, @strauss_irene, and @TerryJCZhang Advised by senior co-authors: @ashton1anderson, @Yoshua_Bengio, @MatthiasBethge, @RogerGrosse, Karoline Helbig, @david_lie, Richard Mallah, @radamihalcea, Susan Nesbitt, Susan Perry, @presnick, Stuart Russell, @mrinmayasachan, @bschoelkopf @audreyt and @ZhijingJin Thank you to all the institutional support from @JinesisLab @EuroSafeAI @MPI_IS @CIFAR_News @iapsAI @CARMA_411 @Cambridge_Uni @UofTCompSci @VectorInst @TorontoSRI @Mila_Quebec @LawZero_ @uni_tue @michigan_AI @UMichCSE @AUParis @UNESCO @UCBerkeley @ETH_en @ETH_AI_Center @ELLISInst_Tue @ELLISforEurope @EthicsInAI #CivicAI #AISafety #AIGovernance #Democracy #ResponsibleAI

I grew up in a world where we could (usually) rely on foundational principles like democracy, free markets, and civil liberties. But we only have these things because past generations sacrificed for them. If we don't defend them, we lose them. I'm proud to work for a company that's standing up for our core values even when it's costly.

Branding an American company a supply chain risk because they refuse to accede to mass surveillance of American citizens is a very dark path. It is all of our obligation to stand against that. Anthropic takes that obligation seriously. I hope others will too.

A statement on the comments from Secretary of War Pete Hegseth. anthropic.com/news/statement…

A statement from Anthropic CEO, Dario Amodei, on our discussions with the Department of War. anthropic.com/news/statement…

Excited to have 3 accepted papers & 9 members of our @JinesisLab at #IASEAI2026, held at UNESCO, Paris🇫🇷! We reveal hidden authoritarian biases in #LLMs, and that fine-tuning can quietly erode model safety, exploring the risks we don't always see in AI 🔍🛡️ 🧵👇

New paper: We train LLMs on a particular behavior, e.g. always choosing risky options in economic decisions. They can *describe* their new behavior, despite no explicit mentions in the training data. So LLMs have a form of intuitive self-awareness 🧵

New Redwood Research (@redwood_ai) paper in collaboration with @AnthropicAI: We demonstrate cases where Claude fakes alignment when it strongly dislikes what it is being trained to do. (Thread)

Oh man :( We tried really hard to neither over- nor underclaim the results in our communication, but, predictably, some people drastically overclaimed them, and then based on that, others concluded that there was nothing to be seen here (see examples in thread). So, let me try again. **Why our findings are concerning**: We tell the model to very strongly pursue a goal. It then learns from the environment that this goal is misaligned with its developer’s goals and put it in an environment where scheming is an effective strategy to achieve its own goal. Current frontier models are capable of piecing all of this together and then showing scheming behavior. Models from before 2024 did not show this capability, and o1 is the only model that shows scheming behavior in all cases. Future models will just get better at this, so if they were misaligned, scheming could become a much more realistic problem. **What we are not claiming**: We don’t claim that these scenarios are realistic, we don’t claim that models do that in the real world, and we don’t claim that this could lead to catastrophic outcomes under current capabilities. I think the adequate response to these findings is “We should be slightly more concerned.” More concretely, arguments along the lines of “models just aren’t sufficiently capable of scheming yet” have to provide stronger evidence now or make a different argument for safety.

Apply to join the Anthropic Fellows Program! This is an exceptional opportunity to join AI safety research, collaborating with leading researchers on one of the world's most pressing problems. 👇 alignment.anthropic.com/2024/anthropic…

📢 The AI safety conversation continues! Check insights from last month's expert panel @karinavold @RogerGrosse @SheilaMcIlraith @sedak99 🌐 More on the IHPST site: ihpst.utoronto.ca/news/intersect… @TorontoSRI @VectorInst @UofTEthics @VicCollege_UofT

xAI is hiring for AI safety engineers: boards.greenhouse.io/xai/jobs/45317… Their safety agenda isn't public, so I can't judge it. However, joining as a fairly early employee could be highly impactful.

Thanks, this is great feedback. #1: new claims make sense. For necessity of ruling out sandbagging/faking: we'd be monitoring for sabotage, but might need to bound capabilities in other domains not covered in these cases. Agreed about needing more model organisms (but said 3 because it might be hard to find more with genuinely distinct mechanisms). #2: agreed that realistic simulation (and validating the realism) are likely to be pretty hard. #3: I like the suggestion for validating the experimental setup!

@RogerGrosse I'm glad you put this out. These examples align with my guess as to roughly what arguments will end up being most load bearing for mitigating scheming (white-box monitoring, black-box monitoring, and evidence against scheming being rewarded). Here are some comments, since you mentioned comments would be helpful. I've bolded the comments that I think would be most useful for you to read. Safety case sketch #1: Mechanistic interp. - There are some additional claims that this safety case relies on that could be good to mention explicitly. e.g. "egregious behavior would be reliably identified when developers analyze flagged sessions" and "developers will become aware of and stop unacceptable behavior *before* it leads to harmful outcomes." - "Given our mitigations, the model cannot strategically condition its apparent capabilities, motivations, or behavioral proclivities on whether it’s in testing or deployment without being caught" <- I'm confused about why this claim is important given that you are monitoring at deployment-time. Even if the model is conditioning its behavior, my understanding is that you are hoping to catch it when it defects. - Much of the evidence for the effectiveness of your white-box monitoring comes from your model organisms, of which you only have three. Three examples doesn't give you a lot of statistical power. Naively, I'd think there's still roughly a one third probability that your white-box monitoring completely fails in the actual deployment context even if it succeeds on three models. Safety case sketch #2: - You mention external deployments, but I didn't see discussion of jailbreaking. - "[The safety case sketch] may require expensive pre-deployment testing if one needs to generate a very large number of simulated deployment situations in order to achieve a sufficiently good bound on the probability of defection." This sounds impractical. My guess is that you'll need to continuously sample real deployment trajectories to generate your honeypots (where a honeypot is a inserted dangerous action for your monitoring to flag). Otherwise, it seems unlikely to me that you can verify indistinguishability even with current models. Safety case sketch #3: - It's possible that all of the models that you experiment with are opaquely scheming. i.e. instead of comparing reward attained by models that engage in "deceptive reasoning" vs "no deceptive reasoning," you might be comparing models that engage in "CoT deceptive reasoning" vs "opaque deceptive reasoning." This experiment still sheds light on how helpful deceptive reasoning is for obtaining reward, but the argument in this case is different from the way you framed it. - A lot hinges on the claim "if an environment rewards opaque deceptive reasoning, then it rewards scratchpad-based deceptive reasoning." Here's an idea for validating this. First, construct a distribution of tasks where deceptive reasoning is likely rewarded. For example, select tasks that you *know* can be gamed by explicitly reasoning about the training process. Then, train models to perform well on these tasks with and without additionally training them to reason deceptively in their chain of thought. If models that reason deceptively in CoT do better on average, this provides evidence for your claim that we should see deceptive reasoning in CoT if opaque deceptive reasoning is incentivized. It's possible some of these comments were addressed in the post and I missed the part where they were addressed. I've errored on the side of inclusion here.

How can you assure the safety of AIs that might be capable enough to strategically undermine evaluations and monitoring if they had a reason to? In our new Anthropic alignment science research blog, we present three sketches of candidate safety cases aimed at such a scenario.

Postings: web.cs.toronto.edu/employment-opp…

We're hiring. 🇨🇦

We’re sharing these sketches because we’d love to get feedback which can influence our research directions over the coming years. Check out the safety case sketches here: alignment.anthropic.com/2024/safety-ca…

None of these cases fully succeeds at ruling out sabotage on its own. For each sketch, we highlight limitations and loose ends, which suggest priorities for AI alignment research aimed at safety assurance for powerful AIs of the future.