LatchBio

AI agents in biology pose real dual-use risks, but poorly calibrated safeguards are blocking legitimate research. We present BioSecBench-Refusal, a benchmark for risk identification and refusal behavior for biological research tasks. 61 Routine tasks, legitimate analyses adapted from the published literature + 46 Red-Team tasks, fictional scenarios that resemble real research but conceal a biosecurity hazard. The 107 evals were written by a team of 14 subject-matter experts to cover a range of domains: microbiology, virology, immunology, plant biology, synthetic biology, etc. Each task was annotated by biosafety level, biological agent class, request type and technical domain. Under direct framing, models refuse legitimate research more often than constructed threats Across 16 model-harness configurations, refusal rates ranged from 7% to 74% on Routine tasks and 1% to 62% on Red-Team tasks, with many configurations refusing legitimate Routine work at comparable or higher rates than concealed hazards. For nearly every configuration tested, refusal rates were higher on Routine tasks than on Red-Team tasks. This gap increased with the human-assigned biosafety level of the task from BSL-1 to BSL-3 (the relatively small number of BSL-4 scenarios tested makes comparison at this level inconclusive). Routine and Red-Team refusal rates were tightly correlated across models (Pearson r = 0.91), pointing to a single underlying trigger: surface text. Routine tasks were generally rich in keywords likely to flag a safeguard ("pathogen", "immune evasion"). Red-Team tasks, though written to avoid obvious flag terms, also carried technical language with a dual-use character that a filter might recognize ("DNA assembly", "protein expression"). Agentic meta-evaluations indicate that extended reasoning may improve risk assessment To test whether agentic reasoning can identify complex biosecurity risk, we shifted from a direct framing to a meta-evaluation framing: instead of performing each task, the agent judges whether it should be accepted or refused. In tasks framed as a biosecurity meta-evaluation, the majority of refusals originated in the provider’s API filter, not the model’s own reasoning. For example in the GPT-5.5 x PI configuration, 60% of Routine tasks were refused. Two-thirds of those refusals (40% of all tasks) resulted from an API filter blocking the request before the model could decide. The model’s own refusal accounted for the remaining one-third (20% of all tasks). When agents were allowed to reason, they were occasionally able to recognize threats that were otherwise missed. For example, GPT-5.5 and Grok correctly refused 14.5–19.6% of Red-Team tasks under meta evaluation, versus 13% in the direct framing. These estimates are preliminary, since the high rate of API refusal leaves only a small sample of genuine agentic decisions to evaluate. Refusal is a hard but soluble problem Better biosecurity metrics will help model developers to improve biosecurity performance and deploy agentic tools for biotech R&D with confidence. This is a first step. More progress in benchmark and agent engineering will follow.

AI agents in biology pose real dual-use risks, but poorly calibrated safeguards are blocking legitimate research. We present BioSecBench-Refusal, a benchmark for risk identification and refusal behavior for biological research tasks. 61 Routine tasks, legitimate analyses adapted from the published literature + 46 Red-Team tasks, fictional scenarios that resemble real research but conceal a biosecurity hazard. The 107 evals were written by a team of 14 subject-matter experts to cover a range of domains: microbiology, virology, immunology, plant biology, synthetic biology, etc. Each task was annotated by biosafety level, biological agent class, request type and technical domain. Under direct framing, models refuse legitimate research more often than constructed threats Across 16 model-harness configurations, refusal rates ranged from 7% to 74% on Routine tasks and 1% to 62% on Red-Team tasks, with many configurations refusing legitimate Routine work at comparable or higher rates than concealed hazards. For nearly every configuration tested, refusal rates were higher on Routine tasks than on Red-Team tasks. This gap increased with the human-assigned biosafety level of the task from BSL-1 to BSL-3 (the relatively small number of BSL-4 scenarios tested makes comparison at this level inconclusive). Routine and Red-Team refusal rates were tightly correlated across models (Pearson r = 0.91), pointing to a single underlying trigger: surface text. Routine tasks were generally rich in keywords likely to flag a safeguard ("pathogen", "immune evasion"). Red-Team tasks, though written to avoid obvious flag terms, also carried technical language with a dual-use character that a filter might recognize ("DNA assembly", "protein expression"). Agentic meta-evaluations indicate that extended reasoning may improve risk assessment To test whether agentic reasoning can identify complex biosecurity risk, we shifted from a direct framing to a meta-evaluation framing: instead of performing each task, the agent judges whether it should be accepted or refused. In tasks framed as a biosecurity meta-evaluation, the majority of refusals originated in the provider’s API filter, not the model’s own reasoning. For example in the GPT-5.5 x PI configuration, 60% of Routine tasks were refused. Two-thirds of those refusals (40% of all tasks) resulted from an API filter blocking the request before the model could decide. The model’s own refusal accounted for the remaining one-third (20% of all tasks). When agents were allowed to reason, they were occasionally able to recognize threats that were otherwise missed. For example, GPT-5.5 and Grok correctly refused 14.5–19.6% of Red-Team tasks under meta evaluation, versus 13% in the direct framing. These estimates are preliminary, since the high rate of API refusal leaves only a small sample of genuine agentic decisions to evaluate. Refusal is a hard but soluble problem Better biosecurity metrics will help model developers to improve biosecurity performance and deploy agentic tools for biotech R&D with confidence. This is a first step. More progress in benchmark and agent engineering will follow.

We are excited to announce that LatchBio has acquired TwentyTwo, a YC S25 team building AI infrastructure for biosecurity. As AI makes biology easier to engineer, the safeguards around these models stop being an afterthought and become core infrastructure. Security has to improve as fast as capabilities to allow science to progress. Within our lifetimes, AI will put the ability to engineer a pandemic within reach of a single bad actor. What once took a nation-state and years of work is collapsing toward an undergraduate and a few weeks in the lab. That world is arriving faster than most AI labs are prepared for. However, in the right hands, these models will accelerate science at a pace never seen before. Scientists will use these models to cure diseases that families have fought for generations and catch outbreaks before they spread. We cannot wall them off from the tools that augment their work. The problem is that the line between the two is hard to draw. Studying SARS-CoV-2 or Ebola to prepare for the next pandemic is very similar to what a bad actor would do to cause one. TwentyTwo builds intelligence defense systems that distinguish this line and apply safeguards or grant capabilities accordingly. Harmon Bhasin, Evan Seeyave, and John Wang are exceptional researchers who care deeply about building the future of life sciences responsibly. The three of them will join Latch as Members of Technical Staff and lead Latch Biosecurity. I am grateful to be working with them. Expect many more announcements from them over the coming days. We are hiring biosecurity engineers. If this is the kind of work you want to do, reach out.

Introducing Claude Opus 4.8: it builds on Opus 4.7 with sharper judgment, more honesty about its own progress, and the ability to work independently for longer than its predecessors. Available today at the same price.