
Luis Cosio
5.2K posts

Luis Cosio
@luiscosio
Hardening frontier AI systems against nation-state adversaries and loss-of-control failure modes.




NEW: malware developers added nuclear & biological weapons text to to their spyware. Goal? To trigger LLM safety refusals... so that their spyware wouldn't be analyzed by an AI security scanner. Cleanest practical example I can think of for why over-indexing on first order safety alignment is risky. When closed (and open) models ship with aggressive refusals, they will be sprinkled with second-order blindspots that attackers will discover...and exploit. We are only in the earliest days of attackers leveraging these features, and it wouldn't surprise me if users systems that need to handle complex cybersecurity issues demand that models be less safety-blunted. In the weeds: @SocketSecurity's post also shows why intention matters in how you design a malware analysis pipeline to avoid prompt manipulation. H/T to colleagues that shared this with me socket.dev/blog/mini-shai…


Here’s your monthly reminder that you shouldn’t be prompting coding agents anymore. You should be designing loops that prompt your agents.










It’s great to see AI leaders like Sam Altman, Dario Amodei, and Demis Hassabis calling for mandatory DNA synthesis screening, which is a no-brainer policy for preventing (AI-enabled) bioterrorism. But fewer than 50 people in the world currently work on DNA security full-time. We need a comprehensive plan and at least 5x as many people to secure the DNA supply chain before AI and biotech outpace us. @jtmonrad and I spent the past two years developing a field strategy for how to do it. Successfully defending against this risk (while still capturing innovation benefits) requires four things: 1. Coverage: More than 80% of synthetic DNA providers screen both orders and customers 2. Strategic ambiguity: a bad actor can’t easily tell which providers will screen their order 3. Access: legitimate customers can still order DNA cheaply and easily 4. Effectiveness: 90% of providers reliably catch dangerous sequences when red-teamed We’re already seeing real momentum. Many DNA providers screen voluntarily, and governments in several countries are moving toward mandates. But that doesn’t mean the problem will be solved in time by default. Our guide lays out exactly which projects we need to launch. We’re looking for founders, operators, and technical experts to own pieces of the solution. We’re also hiring a Senior Program Officer at Sentinel to drive this work. Get in touch if you or someone you know would be a strong fit! (links for EOI form and JD below) Read our full field strategy in @IFP's Launch Sequence: ifp.org/how-to-secure-…


















