Axel Højmark ⏸

Trained monitors can be strong low-cost alternatives to prompted frontier models for black-box scheming detection. Our fine-tuned open-weight monitors detect scheming/sabotage in agent trajectories better than small prompted models and are on the cost-performance frontier. (1/n)

“Loss of control” lacks a common, actionable, definition and conceptualization. In our new research report we: 1) propose a new taxonomy, 2) put forward actionable mitigations today, and 3) motivate the need for preparedness. We propose a taxonomy for loss of control 👇🧵

Stress Testing Deliberative Alignment for Anti-Scheming Training Bronson Schoen, Evgenia Nitishinskaya (@j_nitishinskaya), Mikita Balesni (@balesni), Axel Højmark (@AxelHojmark), Felix Hofstätter (@HofstatterFelix), @jeremy_scheurer, Alexander Meinke (@AlexMeinke), Jason Wolfe (@w01fe), Teun van der Weij (@Teun_vd_Weij), Alex Lloyd, Nicholas Goldowsky-Dill, Angela Fan, Andrei Matveiakin, @rushebshah, Marcus Williams, Amelia Glaese (@mia_glaese), Boaz Barak (@boazbaraktcs), Wojciech Zaremba (@woj_zaremba), @MariusHobbhahn @apolloaievals @OpenAI

(1/n) Scheming has been a key concern in AI safety for 20+ years. It’s when an AI acts aligned while hiding true goals. New OpenAI + Apollo research found scheming in every tested frontier model, though no harmful scheming has been seen in production traffic. openai.com/index/detectin… youtu.be/5UtuHzfZmhE

Today we’re releasing research with @apolloaievals. In controlled tests, we found behaviors consistent with scheming in frontier models—and tested a way to reduce it. While we believe these behaviors aren’t causing serious harm today, this is a future risk we’re preparing for. openai.com/index/detectin…

Future AIs might secretly pursue unintended goals — “scheme”. In a collaboration with OpenAI, we tested a training method to reduce existing versions of such behavior. We see major improvements, but they may be partially explained by AIs knowing when they are evaluated.

A simple AGI safety technique: AI’s thoughts are in plain English, just read them We know it works, with OK (not perfect) transparency! The risk is fragility: RL training, new architectures, etc threaten transparency Experts from many orgs agree we should try to preserve it: 🧵

Curious about AI progress in the coming years? Check out our recent paper for more details on our predictions!

Interestingly, the quadratic model now aligns better with recent model releases, matching my prior intuition that progress is accelerating.

My predictions on Chatbot Arena Ratings have held up impressively well over the past 3 months.

@OpenAI So according to the first section, the transition to a world with human-level AI will be gradual and manageable. And according to the second section, we will experience centuries' worth of change in a matter of years?

When will frontier LM agents match human-level performance on software engineering, AI R&D and cybersecurity? Our new paper introduces a two-step forecasting method that predicts benchmark scores from release dates, using Chatbot Arena Elo as an intermediate capability metric.

We're hiring for multiple evals positions! We're looking for research scientists, research engineers, software engineers, and everything in between. jobs.lever.co/apolloresearch…

How can we accurately estimate the chance of a LLM Agent completing a very difficult task? A naive approach would be to run the agent on the task thousands of times. This is inefficient! Are there methods that do better and are they unbiased? Our new paper looks at this question!

@nickcammarata @bayeslord You should consider reading Dr. Greger's book 'How Not to Age.' You might be surprised by the amount of evidence supporting a plant-based wholefood diet.

@bayeslord I feel like veganism partially poisons the experiment. Veganism is a totally reasonable diet, but trying to max out longevity primarily through diet and starting with a non-standard potentially massive dietary handicap without testing the handicap size is questionable experiment

i love what bryan johnson has done but i never want to live like him. i see him as a noble sacrificial pioneer, taking risks to see far ahead for us, running on his beliefs, his fears, his motives in total. we, to say the thing, have far too few people bold enough to be like him