Joe O’Brien

New IAPS memo with @rosen_br and @covinstantinop: a national security playbook for federal action on frontier AI. Focused on securing models, defensive automation, tracking frontier risks, and building government capacity. Full memo: iaps.ai/research/after…

PALO ALTO NETWORKS on MYTHOS: "In our testing, three weeks of model-assisted analysis matched a full year of manual penetration testing, with broader coverage."

I've spent the past few weeks reading 100s of public data sources about AI development. I now believe that recursive self-improvement has a 60% chance of happening by the end of 2028. In other words, AI systems might soon be capable of building themselves.

Full report: iaps.ai/research/risk-…

We argue that whenever a substantially more capable or riskier model is deployed internally, the developer should create a risk report and argue why the model is safe to deploy.

Models used internally at AI companies have capabilities beyond those of publicly-available models, so it's important that risks from these models are reported externally. We've just published a report on how this should be done.

@robertwiblin @JustinBullock14 What are his thoughts on propagating use of these techniques to frontier AI companies or downstream users, especially absent high-profile cases of misalignment which might create the will to do so?

This week I'm interviewing the world's most cited computer scientist, Yoshua Bengio. He chairs the International AI Safety Report. But his primary focus is developing a comprehensive solution to the AI alignment problem: 'Scientist AI'. What should I ask him?

1/ A few months ago, my co-authors and I at @iapsAI set out to answer a question that felt increasingly urgent: what happens when AI systems can run sophisticated cyber operations entirely on their own?

Applications are open for the IAPS AI Policy Fellowship! 3 months, fully funded, remote or DC. This is your opportunity to work with leading AI policy experts on research projects meant to secure a positive future in a world with powerful AI. Apply by Feb 2 at the link in the thread.

Great opportunity to work at CAISI on running evals & improving the science of evals--would love to see brilliant people in these roles!

Today we are announcing the creation of the AI Evaluator Forum: a consortium of leading AI research organizations focused on independent, third-party evaluations. Founding AEF members: @TransluceAI @METR_Evals @RANDCorporation @halevals @SecureBio @collect_intel @Miles_Brundage

@peterwildeford @metaculus "I was on the airplane" had pretty good delivery tbh

@metaculus which one?

I got interviewed for The Daily Show!

Do you want to fund AI alignment research? The AISI Alignment Team and I have reviewed >800 Alignment Project Applications from 42 countries, and we have ~100 that are very promising. Unfortunately, this means we have a £13-17M funding gap! Thread with details!🧵

A massive step: The EU AI Office has launched a whistleblowing channel dedicated to AI professionals—the first of its kind globally. 1/3 #whistleblowing #euaio

RT @deanwball: If you said: “We should have real-time incident reporting for large-scale frontier AI cyber incidents.” A lot of people in…