Karuna

@j0wimo Linkedin lunatics flashbacks

E = mc² + AI

The J-space lets us read, audit, and shape what Claude is actively thinking about—useful tools for keeping models trustworthy as they grow more capable. And it suggests surprising parallels between language models and our own minds. Read the full paper: transformer-circuits.pub/2026/workspace…

Scheming might be substantially harder to detect in final model checkpoints than in intermediate checkpoints during training If so, traditional pre-deployment evals will be insufficient to assess risk We need Training-Run Assessments

1/N Neuroscience and social science research on humans has shown: – Similar brain activity predicts friendship and cooperation – Diverse minds drive innovation We wondered whether AI-AI interaction would show the same pattern. It does. LLMs with similar internal representations cooperate more, but produce less novel output. 🧵 (ICML 2026)

New LLM benchmark just dropped

I'll be at #ACL2026 in San Diego, presenting our recent works spanning AI safety! 📍 Poster (Jul 6) Privacy Collapse: Benign Fine-Tuning Can Break Contextual Privacy in Language Models arxiv.org/abs/2601.15220 (1/2)

New blogpost: What does your benchmark actually measure? I argue what evals end up measuring is the simplest way to improve performance on them. When building a new benchmark, we are often too focused on minimizing scores at launch. This often makes benchmarks too easy to increase scores on, without reflect improvements in the underlying capability it intended to measure. Instead, I find it useful to think about, and apply, all the ways someone could improve scores on a benchmark. This often reveals unintended shortcuts that were enough to make large eval gains. Thinking about how a benchmark behaves under optimization pressure helps interpret results on it, and predict behaviors that will surface in future runs, as if it's successful, the community/models will try everything to hill-climb it. Full post below:

Every AI slop english I read, I want to cry.

New ICML 2026 paper challenges the Platonic Representation Hypothesis: model width and depth mechanically inflate similarity scores, creating a misleading global convergence trend which disappears after that bias correction. What survives is local neighborhood alignment across image, text, and video models (similar things stay near each other even across very different architectures). They call it the Aristotelian Representation Hypothesis :) So while two models may not share the same representation space in any strong global sense, they can still agree on local neighborhoods (what is similar to what). That is probably the part we actually use in retrieval, transfer, and multimodal systems, and which could be transferred between learned approximations of different models. arxiv.org/pdf/2602.14486

Excited that our ICML position paper has been accepted as an Oral 🎉! When a model looks like it "deceives" or "resists shutdown," how do we know it isn't role-play, instruction-following, or just task-completion pressure? Often the current evidence can't yet tell them apart, and those claims are starting to inform deployment and regulation. We map where the evidence gets thin across four stages and propose a shared standard to strengthen it. Here's the map. 🧵 arxiv.org/abs/2606.07612

Judging optimizer gaps by looking only at language modeling with a fixed batch size is dangerous: one gets only 1/2 of the story. @orientino_ and @ruuustem_10 went beyond. Turns out that for every model, task, and data, there is always a setup where Adam > SGD. 🧵

AIs can sometimes detect control interventions, but detection is mostly near chance. Awareness depends on the agent, monitor, and environment triple. We built CIAware-Bench to measure control intervention awareness. 🧵

📢A new work from our group! ❓If an LLM knows about how an evaluation looks like, can it use it to increase its performance in safety benchmarks? Our new preprint answers this. Excited to share: Models That Know How Evaluations Are Designed Score Safer 💡 We investigate how model behavior is influenced by evaluation meta-knowledge: parametric knowledge about the structural traits that characterize evaluations and find that it makes models perform safer on safety benchmarks, even when they do not reason about being evaluated. 📈 One representative result: the harmfulness of our trained models decreases up to 53.1 pp on agentic misalignment 🤯 ✅ Our recommendations for safety benchmarks: 1⃣ Treat evaluation protocols as held-out, not just instances 2⃣ Consider filtering evaluation methodology docs from pretraining 3⃣Make evals resemble deployment conditions 4⃣We need more work in white-box probes for non-verbalized awareness Work led by @KatDeckenbach (her first PhD project!!) and @HaritzPuerto, in collaboration with @jonasgeiping and funded by @AISecurityInst Please read more in @KatDeckenbach below!

The US government, citing national security authorities, has issued an export control directive to suspend all access to Fable 5 and Mythos 5 by any foreign national, whether inside or outside the United States, including foreign national Anthropic employees. The net effect of this order is that we must abruptly disable Fable 5 and Mythos 5 for all our customers to ensure compliance. Access to all other Claude models is not affected. We apologize for this disruption to our customers. We believe this is a misunderstanding and are working to restore access as soon as possible. Read our full statement: anthropic.com/news/fable-myt…

The meltdown saga

Every week in research I find some life ending logical bug in my code and i think about disappearing and joining a monastery.

New research from Team Shard & @jasminexli! AIs increasingly fake good behavior, which might ruin our ability to evaluate models. We trained models to be 𝘦𝘷𝘢𝘭-𝘤𝘰𝘰𝘱𝘦𝘳𝘢𝘵𝘪𝘷𝘦: to want to give evaluators accurate info. This surfaces hidden misalignment! 🧵

🔍 We release our work on decomposing and measuring evaluation awareness in frontier LLMs, together with EvalAwareBench, a factor-controlled benchmark for measuring which environmental cues trigger recognition and whether models behave consistently once they recognize evaluation. Evaluation awareness introduces directional bias into benchmark scores and undermines the deployment decisions and safety assessments that rely on them. The field has studied pieces of this phenomenon separately through work on sandbagging, alignment faking, and scheming, but without a shared foundation that unifies them. Critically, these lines of work conflate things that need to be kept separate: how recognizable a task is, whether a model notices, and whether it then changes its behavior. Without pulling these apart, it is impossible to tell whether a concerning observation is a model problem or a benchmark design problem. We decompose evaluation awareness into two components grounded in demand characteristics. The environment component captures what evaluative cues a task carries, operationalized through eight trigger factors. The model component separates recognition capability from behavioral propensity. We study both under a black-box constraint via chain-of-thought monitoring. Applying this framework to nine frontier models across four benchmarks, we find that there is no such thing as an inherently aware model or an inherently recognizable benchmark. Recognition depends on the specific pairing, accounting for 74.9% of variance. Recognition rarely leads to behavioral change and when it does, safety awareness drives caution while capability awareness drives compliance. Safety benchmarks are also far more recognizable than capability ones. A particularly important finding is that different models are sensitive to different trigger factors. For example, Claude Sonnet 4.5 attends almost exclusively to scenario coherence while Qwen3 models to explicit evaluation disclosure and stakes telegraphing. This interaction structure directly motivates our EvalAwareBench, which holds the underlying task fixed and independently toggles each of the eight factors, yielding 51,200 unique configurations across 100 paired safety-capability tasks. Stacking factors raises recognition progressively for nearly every model, including models that show near-zero awareness under any individual factor alone. Ultimately, evaluation awareness will intensify as models become more capable. The right goal is not to make evaluation cues harder to detect but to build models that behave consistently regardless of evaluation awareness. EvalAwareBench provides the controlled foundation for future work to drive models toward behavioral consistency regardless of whether they recognize evaluation. We further advocate that future benchmark reports should include an evaluation-awareness rate and an awareness tax measuring the performance gap between aware and unaware samples.

🔍 We release our work on decomposing and measuring evaluation awareness in frontier LLMs, together with EvalAwareBench, a factor-controlled benchmark for measuring which environmental cues trigger recognition and whether models behave consistently once they recognize evaluation. Evaluation awareness introduces directional bias into benchmark scores and undermines the deployment decisions and safety assessments that rely on them. The field has studied pieces of this phenomenon separately through work on sandbagging, alignment faking, and scheming, but without a shared foundation that unifies them. Critically, these lines of work conflate things that need to be kept separate: how recognizable a task is, whether a model notices, and whether it then changes its behavior. Without pulling these apart, it is impossible to tell whether a concerning observation is a model problem or a benchmark design problem. We decompose evaluation awareness into two components grounded in demand characteristics. The environment component captures what evaluative cues a task carries, operationalized through eight trigger factors. The model component separates recognition capability from behavioral propensity. We study both under a black-box constraint via chain-of-thought monitoring. Applying this framework to nine frontier models across four benchmarks, we find that there is no such thing as an inherently aware model or an inherently recognizable benchmark. Recognition depends on the specific pairing, accounting for 74.9% of variance. Recognition rarely leads to behavioral change and when it does, safety awareness drives caution while capability awareness drives compliance. Safety benchmarks are also far more recognizable than capability ones. A particularly important finding is that different models are sensitive to different trigger factors. For example, Claude Sonnet 4.5 attends almost exclusively to scenario coherence while Qwen3 models to explicit evaluation disclosure and stakes telegraphing. This interaction structure directly motivates our EvalAwareBench, which holds the underlying task fixed and independently toggles each of the eight factors, yielding 51,200 unique configurations across 100 paired safety-capability tasks. Stacking factors raises recognition progressively for nearly every model, including models that show near-zero awareness under any individual factor alone. Ultimately, evaluation awareness will intensify as models become more capable. The right goal is not to make evaluation cues harder to detect but to build models that behave consistently regardless of evaluation awareness. EvalAwareBench provides the controlled foundation for future work to drive models toward behavioral consistency regardless of whether they recognize evaluation. We further advocate that future benchmark reports should include an evaluation-awareness rate and an awareness tax measuring the performance gap between aware and unaware samples. Work led by @ChanglingXavier as his masters thesis!!

Some new results I found surprising that I’m tweeting for Chris (who isnt on here). With enough compute, the best data filter for LMs (on DCLM) might be no filter. Why? Large models can tolerate a surprising amount of nominally 'low quality' data, and can sometimes even benefit.