Roland S. Zimmermann

119 posts

Roland S. Zimmermann

Roland S. Zimmermann

@zimmerrol

Research Scientist @GoogleDeepMind // Machine Learning (formerly Physics) // Previously @bethgelab @MPI_IS @uni_tue @GoogleAI

San Francisco, CA Katılım Ağustos 2009
486 Takip Edilen419 Takipçiler
Roland S. Zimmermann
Roland S. Zimmermann@zimmerrol·
Control interventions are a common approach to increase the safety of agents - but how well can they detect such interventions? My MATS fellows investigated this and built a new benchmark for investigating this. Check out @JSchaeff3r's thread on the paper!
Joachim Schaeffer@JSchaeff3r

AIs can sometimes detect control interventions, but detection is mostly near chance. Awareness depends on the agent, monitor, and environment triple. We built CIAware-Bench to measure control intervention awareness. 🧵

English
0
1
9
621
Roland S. Zimmermann retweetledi
Alexander Panfilov @ ICML
Alexander Panfilov @ ICML@kotekjedi_ml·
Our MATS 9.0 project with @zimmerrol is out! We studied whether models can detect interventions on their trajectories from control protocols. So far, they're bad at it. However, they can achieve non-trivial accuracy by inventing their own watermarking schemes and embedding signals in messages for future detection. Now since this this paper (and the watermarking strategy) is public, I expect models to get better at it. We hope our benchmark helps track that progress and inform safety teams!
Joachim Schaeffer@JSchaeff3r

AIs can sometimes detect control interventions, but detection is mostly near chance. Awareness depends on the agent, monitor, and environment triple. We built CIAware-Bench to measure control intervention awareness. 🧵

English
1
5
44
3.8K
Roland S. Zimmermann retweetledi
Joschka Braun ✈️ ICML
Joschka Braun ✈️ ICML@BraunJoschka·
RL assumes that LLMs explore well during training. What if they choose not to? In our new ICML paper with @GoogleDeepMind, we train LLMs that strategically resist RL capability elicitation by under-exploring. We study this threat model, called exploration hacking.
Joschka Braun ✈️ ICML tweet media
English
8
45
393
50.3K
Roland S. Zimmermann retweetledi
Max Kaufmann
Max Kaufmann@Max_A_Kaufmann·
Is training against the CoT always bad? RL training can lead to obfuscated CoT making it difficult to 'read an LLMs thoughts'. How can we predict when obfuscation occurs?🤔 Our new @GoogleDeepMind paper introduces a framework to predict this before training starts!
Max Kaufmann tweet media
English
6
24
161
30.7K
Roland S. Zimmermann retweetledi
David Lindner
David Lindner@davlindner·
New DeepMind x UK AISI paper: what would it take to prevent harm from misaligned AI agents via monitoring in real deployments? We wrote a safety case sketch for control monitoring No flashy results but lots of important details for deploying future AI agents safely!
David Lindner tweet media
English
8
28
94
20.6K
Roland S. Zimmermann retweetledi
Edward Grefenstette
Edward Grefenstette@egrefen·
Extrajudicial killings of suspected drug dealers, bombing another country and kidnapping its leader w/o congressional approval. Yes, Maduro is a dictator, but without due process and international law being upheld, if it's Venezuela today who's to say it's not Greenland tomorrow?
English
1
4
14
2.3K
Roland S. Zimmermann retweetledi
Paul Graham
Paul Graham@paulg·
I used to be able to claim that tech billionaires didn't actually do this — that they just wanted to refine their gadgets. But unfortunately in the current administration we've seen all three.
English
93
210
4.4K
333.6K
Roland S. Zimmermann retweetledi
Roland S. Zimmermann retweetledi
David Lindner
David Lindner@davlindner·
MATS is a great opportunity to start your career in AI safety! For MATS 9.0 I'll be running a research stream together with @emmons_scott @jenner_erik and @zimmerrol If you want to do research on AI oversight and control, apply now!
Ryan Kidd@ryan_kidd44

MATS 9.0 applications are open! Launch your career in AI alignment, governance, and security with our 12-week research program. MATS provides field-leading research mentorship, funding, Berkeley & London offices, housing, and talks/workshops with AI experts.

English
0
2
20
1.1K
Roland S. Zimmermann retweetledi
Roland S. Zimmermann retweetledi
David Lindner
David Lindner@davlindner·
Can frontier models hide secret information and reasoning in their outputs? We find early signs of steganographic capabilities in current frontier models, including Claude, GPT, and Gemini. 🧵
David Lindner tweet media
English
10
18
104
17.7K
Roland S. Zimmermann retweetledi
Sundar Pichai
Sundar Pichai@sundarpichai·
Gemini 2.5 Pro + 2.5 Flash are now stable and generally available. Plus, get a preview of Gemini 2.5 Flash-Lite, our fastest + most cost-efficient 2.5 model yet. 🔦 Exciting steps as we expand our 2.5 series of hybrid reasoning models that deliver amazing performance at the Pareto frontier of cost and speed. 🚀
Sundar Pichai tweet media
English
250
438
4.1K
1M
Roland S. Zimmermann retweetledi
Philipp Schmid
Philipp Schmid@_philschmid·
Gemini 2.5 Technical Report 🧵
Philipp Schmid tweet media
Čeština
8
52
393
41.5K
Roland S. Zimmermann
Roland S. Zimmermann@zimmerrol·
@tkipf Nope, not yet. I was hoping to drop them off at the consulate (deadline is Tuesday morning), but seems less and less likely that the documents arrive in time:/
English
1
0
1
222