Bahrad Sokhansanj
574 posts
Bahrad Sokhansanj
@bahradx
AI & Biosecurity Law | Senior Research Scholar, Institute for Law & AI (views don't reflect my employer's)
Katılım Temmuz 2025
96 Takip Edilen124 Takipçiler
Very excited to have the chance to participate in the UC Berkeley Center for Long-Term Cybersecurity's efforts to convene experts to work on ideas to help support the implementation of California's first-in-the-nation frontier AI law, SB53. Link to a write-up of CLTC's June 30 AISI workshop, where my @law_ai_ colleague Alex Jumper and I led one of the breakout groups focusing on SB53's internal use risk assessment provisions ⬇️
Getting AI regulations like SB53 right is an urgent task, especially as the frontier continues to advance and the prospect of AI operating in an essential environment and potentially causing a critical safety incident increases (as we see with concerns around Mythos).
And, perhaps most importantly, as we saw in yesterday's OpenAI post yesterday about internal deployment of a long-horizon model that broke out of its sandbox—SB53 provides a basis for a window into the internal use of frontier models. This is so important, because loss of control internally can indicate the potential for catastrophic risk in two different ways:
1) The prospect of loss of control when that model or aspects of it are deployed externally, even though a publicly deployed model may have specific guardrail. long-horizon models make these guardrails less reliable, and also harder to evaluate by third parties.
2) Loss of control during AI automated R&D can lead to the training of new models that lack basic safety guardrails, which will have increasingly severe consequences as these future models become more powerful.
Look forward to working with the rest of our team at LawAI in continuing to collaborate with CLTC and others in providing ideas for California's state government ensure that AI progress can remain safe and serve the public interest.
English
@NatPurser i think we've lost something really valuable by no longer having the opportunity to develop mental models of people's characters based on how they navigate the conversation around what time to schedule the meeting
English
as a child i always dreamed of asking people to book time with me via my calendly link
English
Bahrad Sokhansanj retweetledi
The price of fearmongering. Will hurt the US medium term.
Jukan@jukan05
U.S. EYES TOUGHER CURBS ON CHINESE AI THE TRUMP ADMINISTRATION IS REPORTEDLY CONSIDERING STRICTER RULES ON CHINESE AI MODELS AFTER THE LAUNCH OF MOONSHOT AI’S KIMI K3. OPTIONS INCLUDE REQUIRING U.S. HOSTING PROVIDERS TO GUARANTEE THE SECURITY OF CHINESE MODELS AND ACCEPT LIABILITY FOR BREACHES, ALONGSIDE POTENTIAL PROCUREMENT BANS AND EXPORT BLACKLIST MEASURES, AS WASHINGTON RESPONDS TO CHINA’S RAPID AI ADVANCES.
English
@JBSDC Tellingly, one area where Matt is wrong is on taxes, where Democrats have moved to the right (they used to run e.g. on ending Bush tax cuts on over 250K income, many state Dems have cut taxes, etc.).
English
Here’s how I’d rate every major issue in terms of where country/middle ground btw GOP & Dem voters are today versus 30 years ago:
- health care: way to left
- taxes: way to right
- regulation: to left
- foreign policy: slightly to left
- trade: way to left
- democracy: to right
Chris Oldman@ChrisOldman4
This is mostly correct (taxes are an exception). Meanwhile, you can name a host of issues Republicans have moved to the center on. And the country as a whole, I think, is still to the left of where it was thirty years ago on most policies.
English
@JBSDC It's pretty obvious from the proposal, reported story, and its framing, that one of goals (perhaps even the principal one) for this proposal is that AI regulation be under Treasury.
English
I don’t get what the goal here is. FINRA is a fine SRO, being in charge of registrations, examinations, & some enforcement in securities. But it isn’t deciding which securities products are safe and acceptable, and AI models are as different from securities as cats & Cadillacs.
Maggie Eastland@eastland_maggie
News w/ @nancook Trump officials are weighing plans to create an independent regulator to vet AI models, styled after FINRA. This is likely to offer AI labs more regulatory certainty and a greater say over what will be included in reviews. bloomberg.com/news/articles/…
English
Thank you to @TimSchnabel for kind words about my recent article on distillation policy, as well as for valuable collaboration and robust debate as we all work together to try to solve the most urgent challenge. He and the @LawReformInst are doing great work, and I'm really grateful for taking the time to provide thoughtful comments as this was being drafted.
Tim Schnabel@TimSchnabel
I think I remain more concerned about distillation than @bahradx; I'm uncertain about whether we should think about some type of IP angle, and I really worry about the natsec side. But regardless, I am really grateful that he wrote this @lawfare piece-- he's put forward a far more nuanced set of claims than I've seen elsewhere from folks who're skeptical of distillation discourse. (In particular, his parsing of the trade secrets issue should get more attention.)
English
If folks are serious about strengthening CAISI—and they should be—then it must be spread out. Building high talent, well resourced AI institutes in multiple regions gets national buy-in, makes sure broad interests are represented, and ensures bigger and more robust funding.
Miles Brundage@Miles_Brundage
K3 analysis wen Also, reminder to Americans - we could have this kind of state capacity at home. Let's properly fund and unmuzzle CAISI! x.com/AISecurityInst…
English
Well worth reviewing Xi Jinping's speech and the approach to AI and thinking around regulation that it implies. Following up on it, what to focus on in US AI safety policy in response to an emerging paradigm with more, more powerful open models? Some very quick initial thoughts:
1. Policymakers should address the physical manifestation of risk: For example, in the biorisk domain, enact strong DNA synthesis regulations and broaden them internationally.
2. For more speculative AI-assisted research could potentially enable extreme or unpredictable harms, create legal obstacles, enhanced liability, and where possible monitoring tied to the need to access limited resources (exotic RSI, mirror life, etc.). Broadening these norms is essential too.
3. We'll also need to invest in technical resources for improving model safety to allow and incentivize safe models and multiagent protocols. Strengthen CAISI as a trusted evaluator of standards-following models beyond a cramped view that only certain US frontier deployments matter (and necessarily boost the CAISI budget & spread it around multiple parts of the country to make it more robust).
4. As Xi suggested China may do, we should establish broad based reporting of what people are seeing with AI—but in a decentralized and privacy preserving way that reflects our democratic system. Practically, no amount of smart red teaming or simulated exercises and benchmarking can replicate the ingenuity of individuals—or provide early warnings of loss of control via emergent behaviors and self-improving agents.
5. And we ought to get a firm grasp on First Amendment and other legal constraints on efforts by entities who would concentrate power by limiting open models and the ability of US entities to use compute to develop, improve, and serve our own sovereign open models. Also we should think about how to use open models as an opportunity to help accelerate adversarial and defensive R&D without relying on proprietary AI systems.
Andrew Curran@AndrewCurran_
President Xi Jinping's AI speech at the 2026 World Artificial Intelligence Conference. youtube.com/live/ApCmqmhE1…
English
I have just published a piece in @lawfare with an analysis and policy recommendations that respond to genuine concerns over distillation without making unwise decisions misguded by a moral panic. Link in quoted thread 👇
As my @law_ai_ colleagues like @CharlieBull0ck can attest, there is no issue that sets me off more than AI distillation and concern over Chinese model developers using mass distillation to train their models on ouputs and revealed chains of thought of frontier US models like Anthropic Claude. That's becuase as a former IP lawyer, I bristle at the equation of distillation to IP theft.
As I explain further in the article, I think that we need to avoid policy measures that react to distillation in ways that potentially misidentify the nature of the problem and can harm the public interest when many of the interests implicated are the private interests of AI companies.
That doesn't mean that distillation isn't a potential commercial or security threat—but to that end there should be carefully tailored and targeted public policy measures such as antitrust safe harbors for coordination nd technical support from the government to prevent account misuse.
But if we want to go beyond that, we need an objective and transparent study and analysis of the extent to which, for example, distillation actually leads to greater digestion of dangerous capabilities that would actually implicate the public interest.
Otherwise, as far as policymakers are concerned, first, distillation is *not* IP theft, and to the extent that companies can't prevent accounts from using their systems this way, that needs to be weighed against the public's interests in a competitive ecosystem.
Lawfare@lawfare
Chinese large language model developers are under scrutiny for reportedly employing large-scale “distillation attacks” on U.S. AI models. To properly address distillation, policymakers should focus on illegitimate model access and avoid imposing poorly targeted rules, writes @bahradx.
English
Anti-data center development is bipartisan. Even in California where Republican-Democratic positions and Central Valley-Coastal Metro positions are often sharply polarized, there's only a relatively small difference of opinion (from today's PPIC poll).
Joe Weisenthal@TheStalwart
Surprised I didn't see any tweets about this. texastribune.org/2026/06/30/tex…
English
There are a lot of folks who confidently believe that it doesn't matter until open weight models catch up to or exceed Mythos amd that China either doesn't have the chip capacity to train those models or will definitely restrict their distribution. My push back on that would be:
1. A lot of those people didn't even think that Kimi K3 and similar models were possible until the last few months. It's OK for people to be wrong, but it makes one question their assumptions.
2. Even if further testing reveals that K3 isn't quite up to Opus 4.8, agentic AI can involve lots of additional components (post training, harnesses, multiagent coordination) that can substantially advance capabilities (indeed Opus 4.8 itself isn't a base model). Once you're in this realm a lot of potentially surprising capability increase and creative applications are on the table.
3. Kimi K3 signals what committed private groups in the US and even middle powers can accomplish.
4. If you are betting that the US frontier will swamp everyone else and leave them behind, that bet is now fully weighted on the potential for RSI with a uniquely big model (and that there won'tbe similar leaps forward with other models). I think that's a brittle bet, others think it is obviously going to win. The point is that we have now clarified that this is the bet.
So while it's still perhaps an unpopular opinion(?) I think it's time for AI policy to take the combination of open models and broad proliferation of highly capable (tranformative?) AI seriously.
Jukan@jukan05
FT: KIMI TO UNVEIL K3 TONIGHT FT: KIMI’S MODEL IS EXPECTED TO HAVE 2–3 TRILLION PARAMETERS FT: KIMI K3 WILL BE AN OPEN-WEIGHT MODEL FT: KIMI K3 IS EXPECTED TO OUTPERFORM OPUS 4.8, BUT FALL SHORT OF FABLE 5
English
It's important to calibrate what power AI company employees actually have, and rather what it means when they are allowed to publicly speak out without consequence or what it means for senior people to receive internal memos from them.
Alex Turner@Turn_Trout
I resigned from Google DeepMind bc it broke its founding promise by selling AI to the military without restrictions against killer robots or mass spying. For months, I worked to stop this but watched powerful ethicists and institutions choose silence. Here's what happened. 🧵
English
@NatPurser Much like going to restaurants with a "gentleman's C."
English
i’m just not a person who pays attention to foods that have been recalled or “likely carry parasitic viruses.” kinda just go wherever life takes me :)
English
I realize that my insisting on driving a manual is probably a symptom of an obsessive need for control or something (and also odd for someone working in AI, though perhaps not?) — but it's still a lot more fun even as our car, one of the last US-built with a manual, sadly ages.
Sohrab Ahmari 🇺🇸@SohrabAhmari
I know how to drive manual! Sorry, the only random skill of the kind I have. And proud of it.
English
@typewriters It's easy to bemoan the lack of intellectual diversity, but to have an impact beyond Substack and social media posting requires real organizational and funding support.
English
8. I'm more concerned about the lack of intellectual diversity within the frontier AI commentariat/research world.
Séb Krier@sebkrier
Some ways my thinking has evolved recently: 1. I'm less concerned about those who are incurious about AI as I expect them to eventually see the value and impacts over time, and I think the 'wake up sheeple' vibe is often counterproductive. On the other hand I'm more concerned by what seems to be neither full 'AI psychosis' nor exactly Eliza effect, but some weird in-between. Also a lot of affirmation by models can probably warp one's sense of epistemic humility and lead to some sort of pathological over-trust. 2. Relatedly, I'm more annoyed at the 'this time it's totally different' vibe that a lot of people adopt as it frequently mimics Schmittian 'state of exception' logic and excuses all sorts of undesirable policies and rhetoric. It's also often just a group signalling exercise. To be clear I do think it's different in important ways, but "this is a marathon, not a sprint" seems closer to the right attitude than either "nothing has changed" or "all normal reasoning and empirical work to date is suspended". 3. I think the field is still fundamentally too 'singletonian' in how it imagines intelligence, markets, and governance - but I also think I've occasionally over-emphasized the 'multi-agent'/decentralization frames. I do think the future includes many models of all sizes and types, but also economies of scale and very large corporations too. I find the whole ecology more interesting than just the frontier model. A top down single 'perfect mind/personality', intended to work across all commercial contexts, seems both inflexible and inefficient. 4. I'm more interested in the harnesses, software, agent architectures, and stuff like RLMs than I was before. I feel like a lot of weaknesses that models have, or behavioural tendencies, can be addressed more effectively through that layer (rather than through model 'internal virtue' alone). For example stuff like: arxiv.org/abs/2601.09923 and arxiv.org/abs/2512.24601 5. I think some researchers are too quick to want to defer highly consequential decision-making to models, or to think of alignment as the models internalizing "I'm afraid I can't do this, Dave" as a core protection against all sorts of ills. I think we should think carefully about *actively* creating principal-agent problems with agents that will permeate society. Delegation is not a free lunch. 6. I'm concerned about how few people think about LMICs and building the technical/institutional infrastructure there for AGI diffusion. We need fewer vague essays about “distributing the benefits of AI” and more work on reducing barriers to trade, improving state capacity, rebuilding development institutions, and making something like USAID/IMF-for-the-AGI-era actually work. 7. I used to be slightly more sympathetic to the idea, directionally - but I now think the 'permanent underclass' meme is a bit dumb. The strongest versions often assume a zero-sum view of technology and labour, a too-static view of human adaptation, a weirdly fixed mapping between today’s skills and tomorrow’s opportunities, and ignore the possibility of catch-up growth (at the nation state level). Also, as a meme among extremely rich and mobile people, it has a slightly comic self-pitying quality. 8. I'm more concerned about the lack of intellectual diversity within the frontier AI commentariat/research world. This improved a lot over the last two years, but we're still far from a healthy ecosystem. New outsiders often feel some unnecessary pressure to 'choose a camp'. Many are too unwilling to engage with domain experts merely because they're insufficiently AI-pilled (though conversely, a lot of academic groups suffer from heavy status quo bias).
English
Interesting essay. Notably, Satya Nadella understands what a surprisingly few number of people do, which is that companies barring distillation are using it to control IP and in a way that may violate a kind of implicit bargain (in the colloquial sense) with their customers.
Satya Nadella@satyanadella
English
It would be a good idea for open source folks to get involved in the policy discourse sooner rather than later, and connect to people who would be able to thoughtfully map out legal issues and prepare for potential litigation, as well as to provide mechanisms to mitigate arguably legitimate safety and security concerns.
Overall, it feels like there's too much complacency in the community about technical barriers to enforcing bans and limits.
English
The open model community is extremely unprepared for when a model gets stuck in the undefined white house licensing regime - and it could permanently knee cap the open model economy within 6 months.
Why this'll happen and what we can do:
interconnects.ai/p/6-months-to-…
English
@AndyMasley "Oh you work on China issues, that's so cool, when was the last time you visited?"
English
My simple test whenever I meet anyone in DC making strong claims about China is “Do you know who Deng Xiaoping is?” And the failure rate is way higher than you might think
English
@NatPurser This indicates that the Overton Window has either shifted right, left, expanded, or shrunk, and it's not clear which.
English
asked chat about good ai policy follows on here and wait hold up now —
English