chair

THE FOUR WORDS EVERYONE IN TECH NEEDS TO SAY I wrote a long post in the newsletter about the Open Letter, the big AI risks people like to talk about, and how the whole thing could've just been summarized as "Lina Khan was right"

The entire tech industry (save for Anthropic) has come out in favor of open source AI. So what happens next? Will Anthropic change its lobbying efforts? Not likely. Now the gaslighting begins: “Nobody is trying to ban open source.” “We just want to limit who can use it.” “We just want to limit who can contribute to it.” “We just want to limit how powerful those models can be.” “We just want to make sure the guardrails (we lobbied for) can’t be removed.” The net effect will be the same. They won’t stop until they kneecap open source. The rest of the industry needs to watch these guys like a hawk.

Yesterday, the transcript leaked of an investor call with DeepSeek’s CEO in which he said the only reason DeepSeek trails the US is a lack of compute and detailed how reliant it is on Nvidia chips. Today, DeepSeek suspended its fundraising round. Doesn’t seem like a coincidence.

I don't think it's a lack of imagination. I also used to work at Anthropic, think trends will continue, and used to agree with this. But I've changed my mind and now disagree with this take. Open models are already capable enough to do what you described. For example, I used Opus 4.6 to gain access to other folks medical records, hijack bank accounts, etc. back in February. GLM 5.1 is more capable than Opus 4.6 in most pentesting environments, and it came out in April. Despite capable open-weight models existing, the sketchier folks I know are still using a Claude Code or Codex subscription for hacking. (Even well-resourced groups in other countries! They use the grey/black market of discounted Ant/OAI subscription tokens sold through resellers.) So I see most of the materialized risk here as still coming from Anthropic and OpenAI; safeguards aren't sufficient to stop a moderately dedicated actor. The groups I know who are using open-weight models are legitimate offensive security companies. They won't break the rules to use subscription-based pricing, the open-weight models are more reliable in that they don't require specific jailbreaks nor hit classifiers, and the labs use massive partnerships or spend as a prereq for lowering classifiers/safeguards. I know of three legitimate groups running GLM 5.2 as their primary model. That last part applies for Anthropic, too: I know of two instances where two different Anthropic GTM people used large comitted spend contracts as a prereq for lowering safeguards, and I directly witnessed one. On the inside, I know the narrative and intent is genuinely about safety. But from the outside, Anthropic-the-system seems to be optimizing for revenue and control/power, isn't diffusing capabilities to defenders, and also doesn't have adequate safeguards to prevent misuse from dedicated bad actors. As a result, I now lean towards a future where capable open models are freely available (at least for cyber, bio is harder); I don't trust Anthropic or other frontier labs to handle this sufficiently well without diffused capabilties given what I've seen so far.

This is pretty insane ngl. Sunday nuke

I'm going to make a concrete prediction here. Open weights AI will prove to be genuinely dangerous and cause a bunch of disasters/be used for bad ends, which will vindicate everyone that warned about this possibility and discredit everyone else

The entire tech industry (save for Anthropic) has come out in favor of open source AI. So what happens next? Will Anthropic change its lobbying efforts? Not likely. Now the gaslighting begins: “Nobody is trying to ban open source.” “We just want to limit who can use it.” “We just want to limit who can contribute to it.” “We just want to limit how powerful those models can be.” “We just want to make sure the guardrails (we lobbied for) can’t be removed.” The net effect will be the same. They won’t stop until they kneecap open source. The rest of the industry needs to watch these guys like a hawk.

Anthropic gave you Claude Code. David Sacks gave you a memecoin president. The only thing David Sacks ever opened was your wallet.

The entire tech industry (save for Anthropic) has come out in favor of open source AI. So what happens next? Will Anthropic change its lobbying efforts? Not likely. Now the gaslighting begins: “Nobody is trying to ban open source.” “We just want to limit who can use it.” “We just want to limit who can contribute to it.” “We just want to limit how powerful those models can be.” “We just want to make sure the guardrails (we lobbied for) can’t be removed.” The net effect will be the same. They won’t stop until they kneecap open source. The rest of the industry needs to watch these guys like a hawk.