Ben Brooks

Nearly every AI firm is invoking frontier risk to justify a persistent gap between open and closed models. But this application of the precautionary principle deserves more scrutiny than it gets. Restricting access to useful technology—models that will, in their developers' own words, transform the economy—shouldn't be our primary response to uncertain risks. Check it out at @aif_media! There are already a bunch of reasons firms might not release their best models openly: cost recovery, competitive pressure, anxious investors. But we should be skeptical of efforts to freeze open-source behind the frontier under the guise of risk.

It's an important part of our mitigations for acute visual risks in open weights bfl.ai/blog/capable-o… iirc, this was a good overview of the practical limitations in various mitigations, incl. data curation arxiv.org/pdf/2412.06966 The gpt-oss paper includes some helpful eval results that show how CBRN pre- and post-training mitigations might interact arxiv.org/pdf/2508.10925

Pre-training data curation as a means to mitigate certain model behaviors seems like an obvious alignment measure (as explored by @thinkymachines & mentioned by @ARGleave on @CogRev_Podcast). Send me your recommendations for folks who may want to talk about this on @scaling_laws & any homework reading I should dive into!!

@KevinTFrazier (18) Own an outcome, don't just navel gaze. That could be an advocacy outcome, technical outcome, or business outcome. Nothing crystallizes AI policy like actually having to deliver for a community you care about.

Receiving a steady stream of emails from "non-technical" law students asking for guidance on how to get involved in AI policy - here's my **concise** list of recommendations. Please add to it...this will become an Appleseed AI post. Eager to hear from folks like @DoniBloomfield, @sayashk, @NeelGuha, @inspiredcat and others who have found a way to thrive at the intersection of AI & the law / policy. (1) we're all technical now. Write that on a sticky note and put it on your mirror. Stop selling yourself short. Fake it until you make it. You can and must dive into the technical weeds to truly add value to this space. Folks who can "speak AI" and "speak law" will be in high demand for the foreseeable future. (2) treat learning about AI like learning another language - immerse yourself. Take a @BlueDotImpact course, read through @Google's free materials on AI, subscribe to @natolambert & @rasbt and read everything they write. When you do not understand something, chase down whatever information you need to learn to grasp it. (3) use the tools. (4) use the tools. (5) use the tools. (6) talk about the tools with other people (professors, students, tech folks in your community) and learn what they are building. Then go build it for yourself. (7) repeat steps 3-5. (8) follow @hlntnr, @janet_e_egan, @anton_d_leicht, @deanwball, @hamandcheese, and anyone they retweet (9) read every blog post from @OpenAI, @AnthropicAI, @GoogleDeepMind (especially the really technical ones) (10) listen to @scaling_laws (what? I'm a shameless academic after all). (11) apply to join @GovAIOrg, @law_ai_, @CSETGeorgetown, @HorizonIPS, or any org in that universe. Get your foot in the door. (12) write about AI. (13) critically analyze AI policy. Then compare and contrast what @CharlieBull0ck & @AdamThierer had to say about it. (14) call me. Better yet, come to Austin and I'll buy you breakfast tacos. (15) join the AI Opportunity Inventory and help analyze AI tools intended to solve public policy problems (link in next tweet). (16) join an AI club or start one. (17) pat yourself on the back because you're asking the right questions! Stay relentlessly curious. Just start doing stuff. Consider this your invitation to join everyone trying to figure this all out.

One characteristic of anti-open hysteria (open source, open weights, open data, open web) is a tendency to ignore the massive diffuse benefits, and concentrate on limited acute harms. Deeply condescending to describe the beneficiaries of openness as "bored 19 year olds".

@random_walker Too much pop AI safety: "can this engine facilitate a bank robbery"

Imagine if car safety testers only did tests on the engine and called it a day, instead of testing the vehicle itself. This is roughly the situation in our understanding of the mental health risks of chatbots. Most of the research is on models but models aren't what people interact with. Chatbots used to be thin wrappers around models, so this used to be an acceptable approximation that enabled automated testing, but not anymore. The chatbot scaffold drastically affects the safety profile, in ways both good and bad: memory and personalization, search and other tools, additional guardrails and filters, drift during extended interactions, and more. Research needs to keep pace with the changing tech and companies need to provide better access to external researchers.

💯. Tactically, we should also spend more time debating the standard of care and acceptable risk thresholds rather than the mechanics of *how* these controls are implemented. With the right standard and right thresholds, even FAA-for-weights might be a nonissue. With the wrong standard and wrong thresholds, even the most industry-friendly soft audit regime could be a disaster for open source, equitable diffusion, and small developers.

I really like that even when we disagree (e.g. I think an SRO for model weights starts to become sensible with today's model capabilities) we can recognise that the other is reasoning from good faith first principles about the world

In their defense, when this debate really kicked off in 2023, Anthropic was basically the only frontier lab that wasn't pushing some cockamamie scheme to "license model developers" or "send in the AI peacekeepers" or whatever. Don't get me wrong, their "FDA / FAA for weights" is still a bad idea. But Anthropic has been commendably consistent about what it believes, and how it would weigh the competing interests.

@__tinygrad__ Falcon from TII popularized this in 2023, but then the open community tore itself apart arguing over open source vs. open weights. Lost sight of the bigger picture: how to make frontier / near-frontier open weights economically sustainable.

This is a great sustainable business model for open weights. It's free if you run it yourself, but if you are running a cloud providing it to others for money, you should have to share profits. (from Kimi K3 License)

@sebkrier @Google @GoogleDeepMind Kabul really testing the context window

Bill text as introduced: obernolte.house.gov/sites/evo-subs…

The new House FRONTIER Act has the same old problems as before. Public authorities have a hard time deciding when the benefits of drugs, vehicles, and financial products outweigh the risk (not to mention the Internet). Each of these has decades / centuries of actuarial history. But under this bill, the federal government would also need to approve the methodologies that auditors will use to determine the acceptable catastrophic risk posed by 1e26 FLOP models, which have ~1 year of actuarial history. Two likely outcomes: 1. These methodologies are conservative. They overweight speculative risks and underweight diffuse benefits. Auditors cry wolf, obtaining emergency restriction orders at the slightest hint of a capability breakthrough or offense-defense lag. It becomes difficult to release a frontier model publicly via API, and impossible to release one open-weights. or 2. The Department of Commerce is mindful of that ^ possibility, and gives auditors wide latitude to determine their own methodologies. Standards are lax, developers can forum shop, and nothing really changes from the status quo. If you believe that AI is an important technology with dual-use properties, either scenario would seem to be undesirable. This is the problem with proposals that go beyond transparency and try to prescribe acceptable risk thresholds. They jump the gun. We simply do not know enough about the risk-utility profile of this technology to form a consensus view of acceptable catastrophic risk. Drawing a new line in the sand could chill the widespread release of legitimate technology (whether through export controls, liability reforms, or these private-audits-with-public-licensing proposals). To be clear, in my view, a regulatory determination of acceptable catastrophic risk is vastly preferable to the tortious approach endorsed by e.g. the original SB1047 / RAISE Act. Regulatory thresholds can be inspected, contested, and adjusted based on broad public input. State jury verdicts cannot so easily. But even so, we are way over the skis here.

Our @bfl_ai researchers co-developed 3 of the 5 most popular open models on @huggingface. Open R&D is in our DNA. But open innovation faces major headwinds. We're glad to rally together with industry to champion open weights for transparency, competition & security in AI.

3. A lot of "accelerationists" care less about eeking out another percentage point of performance, and more about diffusion and adoption. Open weights are only plausibly decelerationist if performance is the sole criterion. Also, if open models are good enough to deter further capex, the market is doing its job. If frontier lab spending is predicated on a multi-year oligopoly over frontier and near-frontier capability, what are we even doing here? What kind of dystopian economic hellscape is that?

Some observations on Kimi: 1. It's a very good model! I don't think its performance can be explained away by distillation or anything like that. In agentic coding sessions, it seems pretty much on par with the best public models of Q1 2026. In my fairly limited use, it also seemed very token hungry. It's not obvious to me that this model is actually that cheap to run. 2. I am personally surprised the Chinese state continues to allow the open sourcing of models this good, given potential risks. To be clear, I *myself* might be fine with models presenting this level of marginal risk being open weight, but I am surprised that China is fine with it. I suspect the reason they are is 75% explained by strategic blindness/lack of AGI-pilledness (the CCP is very Yann Lecun-y in its views of AI). The other 25% or so is their lack of compute for customer inference (making China's open-weight strategy an unintended byproduct of US export controls) and the normal Chinese strategy of aggressive exports. For the companies, as opposed to the government, the decision to open source is partially ideological and partially because they are behind, and they know that very few people would pay for sub-frontier models from China. 3. Open-weight models are inherently decelerationist, and I'm continually surprised to see the so-called "accelerationists" so excited about open-weight models. I suspect the reason they are is that they know open-weight models are effectively ungovernable, and they simply like the overall cloak of ungovernability open-weight models create over the whole of AI. It's not a bad strategy; it reminds me of James Scott's recounting of the hill people in "the art of not being governed." Still, in the end, open-weight models deter further AI capex. 4. One probable outcome of an open-weight-model-dominant world is full AI communism, which is precisely what China proposes: rather than a market product, AI is a "public good" which will ultimately be provided by the state as a kind of "digital public infrastructure." This future strikes me as a dystopian hellscape, but I've never met an open-weight models advocate who doesn't ultimately concede this is where things end. You'd be surprised how many 'accelerationists' lobbied me, while I was in government, to support an eleven or twelve-figure federally funded data center so that startups could train models at a subsidy and then give them away for free. There was no other way for AI to progress, they said. Perhaps this is the logical end state of things. Nonetheless, I find myself surprised to see supposed accelerationists excited about such an outcome. I think many of them just don't know what they're doing. Many accelerationists do not view the creation and serving of frontier models as a legitimate business. 5. I would guess that the Trump Administration will at some point realize that their best strategy here would be to create large amounts of regulatory risk around the use of open-weight Chinese models. You don't need to "ban open source" (one of the dumber motifs of AI policy discussion). You just need to direct every agency to issue soft law that creates FUD. "A Federal Reserve Advisory Bulletin found that there may be backdoors in Chinese AI models." It needn't be that well justified. You just create enough regulatory risk that every regulated enterprise backs off. You probably don't want to create so much regulatory risk that you scare off the hyperscalers from serving Chinese models; this will just drive startups to sketchier providers. There's a happy middle ground here. I'd assume they will do some version of this. 6. It's probably true that open-weight models of this capability make the world a bit more dangerous, but not so much more that you'll really notice. At some point the models will be capable enough that you will notice. "A nonliving, invisible, dangerous, and infinitely self-replicating agent escaped from a Chinese lab," you say? Color me shocked.

Appropriate use of meme, but no one's denying the importance of democratic oversight over thresholds. Just questioning the order of operations, since these proposals always seem to gloss over important details like "how many naughty words, bad napalm instructions, or zero-days are too many" and "do a handful of theatrically closed-source firms really need all this extra help to feudalize the economy?"

Et tu, Demis? "Once the assessment protocol is shown to be effective and robust, formalisation could quickly follow, meaning that Frontier Models would be required to pass it to be deployed in the US market." Ask ten different AI CEOs, godfathers, and safety institutes when a model release is acceptable (or when it's OK to publish frontier weights on Hugging Face), and you'll get ten different answers. That was the problem with these ideas in 2022, and it's still the problem today.

LIVE from Versailles... J Cal puts on the bow tie!🚨 Open Source Wins, AGI Is Here, and Scorsese’s AI Toolkit with CEOs of Cerebras & Black Forest Labs @jason sits down with @andrewdfeldman from $CRBS and @robrombach from @bfl_ai to talk: (0:00) The AI Buildout: Datacenters Bigger Than Cities (Andrew Feldman) (1:50) Reasoning, Inference, and Breaking Moore's Law (16:28) Open Source, AI Sovereignty, and the Road to AGI (40:54) The Innovation Behind Generative Video (Robin Rombach) (47:31) Martin Scorsese, Robots, and the Future of Hollywood IP ------------------------------ Thanks to our partners for making this possible! Most advertisers have never heard of the platform with an $11B annual run rate in ad spend. @AppLovin Ads — 1B+ daily active users, full-screen video ads watched for a median of 35 seconds, and businesses are profitably spending hundreds of thousands of dollars a day on it. Advertiser access is in closed beta. The window is open at applovin.com/ALLIN Positioned at the nexus of technology and the capital markets, @Nasdaq provides premier platforms and services for global capital markets and beyond with unmatched technology, insights and markets expertise. nasdaq.com

The steady provision of models will encourage global dependency on the labs and countries that share them. And vice versa.

@elonmusk the pen that Buzz Aldrin used to fix the Lunar Module engine arm switch is up for auction on July 15. I spoke to @airandspace but they can't spare the 1M. Could you do America a solid and acquire this for the Smithsonian pls?

@beffjezos @steipete @ATT +💯

@steipete @ATT Just get Google Fi

Tried to sign up to @ATT four times now and they reject me and aren’t telling me why. What’s the next best unlimited phone/data plan for the SF area? Or do I know anyone who can work around corp bs?

Our co-founder and CEO, @robrombach, sat down with President Trump, President von der Leyen, President Macron, and other world leaders at the G7 to stress the vital role of open innovation in AI. With openness under pressure around the world, Robin urged governments and industry to make open and responsible development the norm, not the exception. Check out his speech below! bfl.ai/blog/our-co-fo…