Mark Thomas

What kind of institution could possibly regulate frontier AI? I propose one in a new @lawfare — adapting a 90-year-old model from financial regulation. 🧵

@CharlieBull0ck He does say that the models couldn't be deployed without passing the standards body's tests. If it's a FINRA-style org w/ govt supervision then that's real regulation beyond what a lab can do solo. But I agree that enforcement is underspecified here. Releasing my own ideas soon

@chris_j_paxton No, it's not. The "self-regulated organization" term is misleading. FINRA is run by a board with a majority of independent governors, and all its rules & enforcement actions are subject to SEC review. More detail here: lawfaremedia.org/article/ai-com…

The post from Demis really buries the lede with its talk of the "foothills of the singularity" (a great line). But its proposing an industry-funded regulatory body, essentially. So a organization run by people from google/ant/oai, paid for by them, with regulatory authority. Isn't this just a proposal for straight up regulatory capture?

Are the claims that (1) regulation will be captured by industry and (2) regulation will hamper industry? These pull against each other, and neither is required by the FINRA model.* 1) Industry capture is here; just like with the FDA, companies have huge amounts of money. The FINRA model forces industry participation to work through formal, transparent channels instead of back room lobbying. With notice & comment periods you get more oversight and democratic accountability. And a majority-independent, well-selected board is structurally designed to resist capture in a way that normal government agencies are not. 2) We disagree about the need for AI regulation. That's unlikely to change. But if industry has a real hand in writing the rules, the main way the rules block progress is if they're a moat. But regulation compliance would have to unrealistically expensive to act as a binding constraint on new entrants. For the foreseeable future compute will always exceed red tape as a limiting factor for AI startups. * I say "FINRA model" for convenience but FINRA is one example among many of federally supervised self-regulatory organizations (SROs), and its mechanisms are not universally preferable to its sister orgs

I have a tremendous amount of respect for Demis and his work. I love the uplifting and hopeful tone of the future he paints so wonderfully, a world of cures for killer diseases and incredible new materials and abundance. But I completely and totally disagree with FINRA for models. It's basically the food pyramid for models. Remember the food pyramid they pushed on us kids in the 90s? The one that made us all sick and unhealthy and fat little bastards, the one that emphasized ten thousand servings of red meat and milk? Did you know it didn't start out that way? It was originally designed by just such a closed body of wizened scientists protecting the public good. They got together like Demis is recommending and the version they recommended to congress was a simple pyramid that emphasized fruits and vegetables, exactly the kind of thing we know now to be good and healthy eating. What happened? It got lobbied to shit. That's what will we get with FINRA for models. Privileged incumbents. Politics. Push pull. Lobbying and influence. Especially as big companies learn to pack the bold new alphabet agency with insiders. Big regulation can only be absorbed by big companies, putting smaller model makers and open source at a massive disadvantage, even if they pay lip service to it. We'll end up with an army of clip board toting bureaucrats who don't understand what they're regulating with 1000 shifting standards and tests that sound great in reality but just amount to a check box exercise in compliance that only teams of dedicated other suits in big companies with big money can afford to adsorb. I know I'm fighting a losing battle here. Something like this is probably inevitable now. I don't care. I still have to sound the alarm anyway, even if I'm shouting into the wind like Kassandra telling you Troy is going to come crashing down even though you can't hear me. We are going to regulate ourselves into oblivion while China just hits the gas and builds an alternative chip supply chain, massive datacenters and new power plants while we are naval gazing and talking about our past importance. That's because the problem is we have a perfect storm of stupidity driving all our decisions now. Protectionism. Rising ultra-nationalism. Aggressive lobbying by literally batshit insane doom groups and NIMYBs shouting about imaginary problems like datacenters vaporizing all the water while they play golf and water their lawn and chow down on burgers. We've got APTs ratcheting all this insanity up and egging it on, poisoning people's minds. Multi-billion dollar doom group posts on YouTube have 300M views! We also got a generation of politicians who think they can "get ahead" of this technology despite zero ability to predict the future. When Section 230 passed the number one question from congress people was "what is the Internet?" And these visionaries are supposed to predict Uber, social media, WhatsApp, TikTok, AirBnb and everything else that came after? It's delusional and dangerous. No matter what prediction market toting morons tell you, humans are really really really bad at long time horizon thinking and planning and just about everything we do with regulation now will just lock in our misunderstandings today, choke out lateral thinking and privilege a small group of increasingly expensive and paternalizing British East India companies. So I offer you another blast from the past as my answer to new regulation and study groups: "Just say no." Say no to any and all attempts to tangle up the future in a mess of red tape. Say no to privileged people privileging themselves even more at our expense. Say no to the dominant theme of the time which is "we must act now." To that I say, we don't. Proactionary Principle. Never Precautionary Principle. Prove harm in the real world from evidence. Imaginary harm is imaginary. Remember the Population Bomb? Stupid policies that came from that are the things like the one child policy in China that is now causing rapid population decline there. Ironic. Too bad the government visionaries couldn't see the green revolution coming and that all their stupid policy making was actually causing the problem they feared most. Remember the AI jobs apocalypse? Everyone was convinced and it's proving to be total freaking nonsense as anyone who studies history and economics could have told you if you wanted to listen. We cannot predict the future well and we should not try to create fixes for imaginary problems now. Fix things when they actually break. So say it with me now again ten more times: Just say no to the food pyramid for models.

@ziv_ravid Sharp point. But the FINRA model doesn't preclude independent 3P testing. Industry is best positioned to know if the test is gameable & can surface that info, then a majority of independent governors (+ good faith labs) can vote for measures like 3P testing to mitigate gaming

So Demis proposed a "FINRA for AI". Quick context for non-finance people: FINRA is the body that polices US stockbrokers. It's private, funded by the firms it regulates, and overseen by the SEC. Industry policing itself, basically. His idea is to have a US Standards Body that sets benchmark thresholds. Models that cross them will be classified as "Frontier-class." Labs hand over models ~30 days before release, voluntary at first, but eventually you'd need to pass review to ship in the US. He suggested that the evals be refreshed quarterly and that, at some point, the body build its own held-out tests. In other words, benchmarks become law. But the problem is that FINRA enforces rules that are legally well-defined and supported by decades of precedent, and it still gets criticized for capture and for missing frauds (it examined Madoff's firm for years!). Crash tests work because a car can't study for the test. AI evals have neither going for them. We don't even have agreed definitions for "deception" or "dangerous capability," let alone robust tests. And the labs being evaluated are exactly the orgs most capable of optimizing against any known benchmark. We couldn't keep MMLU uncontaminated when the prize was bragging rights. Now the prize is market access. Honestly, the better analogy is standardized testing with unlimited test prep or emissions testing before Dieselgate. A gop direction is a lab-independent, held-out evals. You need serious talent outside the labs, frontier-scale compute, and tests that move faster than the models. It's hard, but we can try 🤷

Original article: lawfaremedia.org/article/ai-com…

FINRA for frontier AI is the model I proposed in Lawfare in April, glad to see Google and now Demis picking it up. The details are key-who picks the board? Inspection/enforcement power? FINRA is a model but must be adapted. I'll be publishing my ideas soon.

This makes the conflict of interest unclear. What incentive does the admin have to boost OpenAI prices if they can't capture that value? Even if the exec branch could capture it as an institution, officials/staff don't gain—why are they disincentivized to regulate?

Second, even if they could sell, any proceeds would go straight into the Treasury's general fund under the Misc. Receipts Statute. The admin can't set up a "public wealth fund" unilaterally because it can't keep/reuse any income. This incl. theoretical dividends.

First, the Constitution's Property Clause requires congressional authorization to dispose of government property. That includes shares of a company. So the 5% stake is illiquid and non-transferable.

@OpenAI's idea to give the government 5% of AI lab equity can't be taken as a serious policy proposal unless Congress enacts legislation around it. The shares can't be monetized otherwise. 🧵

Even if the federal government took an equity stake in OpenAI, it couldn't sell any of it without congressional authorization per the Constitution's Property Clause.

@JulianGewirtz @danwwang There is an institutional design that allows government and industry to collaborate, with transparency & accountability: lawfaremedia.org/article/ai-com…

My latest, with the great @danwwang, in the New York Times today: The American A.I. chaos of recent weeks is profoundly self-defeating. With its actions against Anthropic, the U.S. government is skating close to its own Jack Ma moment. We often think of A.I. as a race between the United States and China. Instead we are seeing the emergence of an even more acute form of competition, between the public power of governments and the private power of ambitious companies. The challenges of regulating A.I. at the frontier are urgent and important. The U.S. government needs to strike a better balance between ambition and control, lest it irrevocably damage America’s long-term technological edge. Read it here: nytimes.com/2026/06/28/opi…

The "middle way" here is the model I proposed in Lawfare earlier this year: a federally supervised, industry-run regulatory body for frontier AI. Google's paper further develops the types of regulation it might do. Glad to see this idea gaining traction.

@deanwball says in @TheEconomist that AI needs a middle ground of independent verification orgs to audit labs w/o excessive bureaucracy. I argue in @lawfare for extending this to a full self-regulatory org: mandatory membership, public rulemaking, government override.

The Frontier Model Forum already has every major lab except xAI. Make membership mandatory, add independent directors, put a federal agency on top. The skeleton is there.

Industry writes rules based on facts on the ground, not a bureaucrat's guess at model capabilities. Safety advocates get enforceable standards and government override. A compromise that might be politically feasible.

What kind of institution could possibly regulate frontier AI? I propose one in a new @lawfare — adapting a 90-year-old model from financial regulation. 🧵