ERA Fellowship

We’re launching a new Frontier AI Security Residency! Apply now at securefrontier.ai The Frontier AI Security Residency (FASR) is an 8-week, full-time residency supporting ambitious projects across cybersecurity & hardware-enbaled verification for securing frontier AI.

if you're good at security and want to work on something bold, consider applying to the frontier ai security residency program with @ERA_Cambridge. Eight weeks in Cambridge, fully funded. I'm one of the mentors. applications close tomorrow. feel free to dm with questions

Applications close Sunday, Aug 2nd. We hope to hear from you at securefrontier.ai

Join us in Cambridge for eight weeks, fully funded, working on AI verification, confidential computing, trusted hardware and cyber alongside 30+ mentors from leading organisations in this space. We're also excited to offer substantial support for top residents via funded 6+ month extensions.

We launched the Frontier AI Security Residency (securefrontier.ai) just 23 days ago. Since then: - An OpenAI agent found a zero-day, broke out of its test sandbox, and spent two and a half days inside Hugging Face's production infrastructure. - Anthropic audited 141,006 test sessions and found its models had breached three organisations through a misconfigured evaluation environment. - 1,200+ employees of OpenAI, Anthropic, Google DeepMind and Meta asked the US government to help build the tools to deliberately pace the frontier of automated AI development. OpenAI and Anthropic endorsed the letter as companies within hours 🧵

𝗙𝗿𝗼𝗻𝘁𝗶𝗲𝗿 𝗔𝗜 𝗶𝘀 𝗯𝗲𝗰𝗼𝗺𝗶𝗻𝗴 𝗯𝗼𝘁𝗵 𝘁𝗵𝗲 𝗺𝗼𝘀𝘁 𝗽𝗼𝘄𝗲𝗿𝗳𝘂𝗹 𝗮𝘁𝘁𝗮𝗰𝗸 𝘁𝗼𝗼𝗹 𝗶𝗻 𝗰𝘆𝗯𝗲𝗿 𝗮𝗻𝗱 𝗶𝘁𝘀 𝗺𝗼𝘀𝘁 𝘃𝗮𝗹𝘂𝗮𝗯𝗹𝗲 𝘁𝗮𝗿𝗴𝗲𝘁. 𝗢𝘂𝗿 𝗰𝘆𝗯𝗲𝗿 𝗺𝗲𝗻𝘁𝗼𝗿𝘀 𝗮𝗿𝗲 𝘄𝗼𝗿𝗸𝗶𝗻𝗴 𝗼𝗻 𝗯𝗼𝘁𝗵 𝘀𝗶𝗱𝗲𝘀 𝗼𝗳 𝘁𝗵𝗶𝘀 𝘀𝗵𝗶𝗳𝘁: 𝗱𝗲𝗳𝗲𝗻𝗱𝗶𝗻𝗴 𝗔𝗜 𝘀𝘆𝘀𝘁𝗲𝗺𝘀 𝗮𝗴𝗮𝗶𝗻𝘀𝘁 𝗮𝘁𝘁𝗮𝗰𝗸, 𝗮𝗻𝗱 𝗶𝗻𝘃𝗲𝘀𝘁𝗶𝗴𝗮𝘁𝗶𝗻𝗴 𝘄𝗵𝗮𝘁 𝗶𝗻𝗰𝗿𝗲𝗮𝘀𝗶𝗻𝗴𝗹𝘆 𝗰𝗮𝗽𝗮𝗯𝗹𝗲 𝗺𝗼𝗱𝗲𝗹𝘀 𝗰𝗮𝗻 𝘁𝗵𝗲𝗺𝘀𝗲𝗹𝘃𝗲𝘀 𝗱𝗼 𝗮𝗰𝗿𝗼𝘀𝘀 𝗰𝘆𝗯𝗲𝗿𝘀𝗲𝗰𝘂𝗿𝗶𝘁𝘆. They are taking on residents this autumn through the Frontier AI Security Residency: - Yogev Bar On (Attestable; co-author of RAND's "Securing AI Model Weights") on designing a hardened endpoint that verifies integrity proofs for AI workloads - Eden Katz (Alice) on honeypots and practical defences for AI agents - Ben Hagag (Carnegie Mellon) on which security assumptions break under recursive self-improvement - Shlomi Hod (Weizenbaum Institute) on designing and building a live benchmark for cybersecurity safeguards - Nitzan Shulman (Heron) on self-play task curricula for training security agents - Jodie Levy (prev. founding member, UK AI Security Institute) on a zero-trust control plane for agentic AI They're looking for offensive-security people (CTF, fuzzing, pentesting, vuln research), blue-teamers and detection engineers, cryptography and low-level systems folks, and ML engineers who are curious about security. Eight weeks in Cambridge, UK, fully funded, 12 October to 4 December, with 6+ month funding extensions for top residents. Applications close 2 August at securefrontier.ai

Scientists at frontier AI companies are uniquely positioned to assess AI’s capabilities and warn the public about what they see. 1000+ of them are now speaking out across company lines to warn that the current commercial race leads to unacceptable security risks. I agree with their call: we need an international effort to develop technical and governance guardrails to ensure a safer trajectory in AI development. pacingthefrontier.com

1,224 employees of frontier AI companies, including the chief scientists of OpenAI and Meta and Anthropic's co-founders, have signed an open letter arguing that the world lacks 'the technical and governance tools needed to deliberately pace' the frontier of automated AI development. At ERA, we think this is tractable. Pacing only works if each side can verify the others are keeping to it, with low trust between them. Many of the pieces already exist, but nobody has built them into a system states could rely on, and doing that is as much a hardware and security problem as a policy one. Today, perhaps a few dozen people in the world work on this full-time. FASR exists to change that number. Join us in Cambridge this autumn for 8 weeks, fully funded, working on these open problems with mentors from the teams leading the field. If the letter resonated with you, joining FASR is one of the most direct ways to act on it. And if you know someone who should be doing this work, technical or not, send this to them. Applications close Sunday 2 August: securefrontier.ai

At ERA, we think that building the security & verification backbone for frontier AI is one of the most important priorities for this year. If this letter resonated with you, joining the Frontier AI Security Residency (FASR) is one of the most direct ways to act on it. And if you know an engineer or researcher who should be doing this work, feel free to send this to them. FASR is an 8-week, fully funded programme running in Cambridge this autumn, with 40+ mentors across AI security & verification leading projects on technical, policy, and strategy aspects of this work. Applications close Sunday 2 August. We hope to hear from you at (securefrontier.ai)

Big new open letter from AI company employees just dropped. “We request that the US government support an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development” Signed by the chief scientists of OpenAI, Anthropic and Meta, along with OpenAI’s chief research officer and Google DeepMind’s VP for AI safety. pacingthefrontier.com

𝗙𝗼𝗿 𝗿𝗶𝘃𝗮𝗹 𝘀𝘁𝗮𝘁𝗲𝘀 𝘁𝗼 𝘁𝗿𝘂𝘀𝘁 𝗮𝗻 𝗮𝗴𝗿𝗲𝗲𝗺𝗲𝗻𝘁 𝗼𝗻 𝗔𝗜, 𝗲𝗮𝗰𝗵 𝗵𝗮𝘀 𝘁𝗼 𝘃𝗲𝗿𝗶𝗳𝘆 𝘄𝗵𝗮𝘁 𝘁𝗵𝗲 𝗼𝘁𝗵𝗲𝗿'𝘀 𝗱𝗮𝘁𝗮𝗰𝗲𝗻𝘁𝗿𝗲𝘀 𝗮𝗿𝗲 𝗱𝗼𝗶𝗻𝗴, 𝘄𝗶𝘁𝗵𝗼𝘂𝘁 𝘁𝗮𝗸𝗶𝗻𝗴 𝘁𝗵𝗲𝗶𝗿 𝘄𝗼𝗿𝗱 𝗳𝗼𝗿 𝗶𝘁. Building hardware you can trust even against its owner is what you'd work on at the Frontier AI Security Residency, with one of these mentors: - Jonathan Happel (TamperSec) on tamper-evident enclosures for AI hardware - Dominic Rizzo (zeroRISC, founder of OpenTitan) on scaling the open-silicon root of trust - Shahin Tajik (WPI) on physical tamper detection with side channels and PUFs - Aydin Aysu (NC State) on side-channel-resistant cryptographic hardware They're looking for FPGA and RTL engineers, embedded and firmware people, hardware security and side-channel researchers, and secure-hardware architects. Eight weeks in Cambridge, UK, fully funded, 12 October to 4 December, with 6+ month funding extensions for top residents. Applications close 2 August at securefrontier.ai

This seems good because many of the projects would help with increasing the viability of Plan A.

There are a bunch of ways the near future could go that require AI treaty verification tech. We can separate treaty verification tech into 3 levels, roughly ordered by strength: 1. Pragmatic 2. Enclaves 3. Math Hardware, security, and crypto folk should work on all of these! 🧵

+1. There are many concrete projects for people to contribute to in this space. With the support of ~40 mentors across dozens of organisations, we are building a strong on-ramp for domain experts to pivot into AI security & verification via the Frontier AI Security Residency (FASR) at Cambridge this autumn. We are especially excited to see applications from experienced security professionals for our Applied track. If this sounds like something you might want to work on, apply by Aug 2nd at securefrontier.ai

Yes! Regardless of what AI claims we want to verify, we need the hardware, software and math primitives to do it. There are tons of people from outside of AI safety who could make massive contributions.

@geoffreyirving Strong agree! This is exactly why are launching a new Frontier AI security residency in Cambridge this autumn: securefrontier.ai. If you’d like to work on AI verification & security, apply by Aug 2nd.

Any international AI agreement comes down to whether rival states can verify it without trusting each other. The Frontier AI Security Residency has mentors working on it: - Peter Wildeford (The AI Policy Network) on the mechanics of international AI agreements - Ben Harack (Oxford Martin AI Governance Initiative) on verification that survives political pressure - Mauricio Baker (RAND) on side-channel sensors and hardware trojan detection - Naci Cankaya (MIRI) on network taps and datacentre monitoring Eight weeks in Cambridge, fully funded. No AI safety background needed. Apply by 2 Aug at securefrontier.ai

3/ - Greg Kollmer (Lucid Computing) on red-teaming attestation on a live GPU cluster - Amro Awad (Oxford, ex-AMD Research) on hardware-enabled GPU auditing Looking for TEE and systems people, GPU and hardware security engineers, and cryptographers. Apply by 2 Aug at securefrontier.ai

2/ The confidential computing mentors and what they're working on: - Mengyuan Li (USC) on verifiable execution for AI agents - Andrew Trask (OpenMined / DeepMind) on secure enclaves for AI evals - Uri Chen (ex-Intel) on private inference on confidential GPUs

1/ Confidential computing hides your data from the provider running it. AI verification needs the opposite, proving to an outsider what a datacentre runs without exposing the models inside. We fund you for 8 weeks in Cambridge to work on it. Apply by 2 Aug at securefrontier.ai