Evan Hubinger

You can read about the experiment I'm referring to here: anthropic.com/research/agent… I think most people don't understand that 1) AIs could ever blackmail humans 2) AI companies don't know how to robustly prevent these types of behaviors

Yeah, @EvanHub is one of the most cautious and careful people I know for making claims like this; he'll argue you blue about a small factual or conceptual detail because he cares so much about getting stuff right... and he's very comfortable making claims like this. DC really, really, really needs to pay attention.

lmao at how this aged

Yes. Regular people don't yet realize that AI people think they're building something that will destroy the human race.

A statement from Anthropic CEO Dario Amodei: anthropic.com/news/where-sta…

Yeah so this is crazy

I’ve seen this comment made multiple times by OpenAI (below by @natseckatrina): “A lot of the concerns about the government's "all lawful use" language seem to stem from mistrust that government will follow the laws. At the same time, people seem to believe that Anthropic took an important stand by insisting on contract language around their redlines. We cannot have it both ways. Why is it reasonable to say that the government cannot be trusted to interpret laws and contracts the right way, but also to assume (without evidence) that Anthropic’s policy redlines, in a contract, would have been effective.” No! That is *not* the main concern. The concern is in fact that OpenAI’s contract carves out a situation exactly where the DoW can do “constrained monitoring” with legally bought third party data. The whole point of going beyond “all lawful purposes” and precisely pointing to no mass surveillance is because such things are lawful, but the law has failed to catch up to the pace of AI. The only reason we haven’t suffered from mass surveillance is because there was no technology that would scalably change that. LLMs change this. It has nothing to do with what is illegal. OpenAI has either intentionally cornered itself to allow surveillance with their models (the safety stack will be effectively safety theatre) because the DoW will just pressure OpenAI to relax any safety stack if it goes against the contract. And given the concern we have is with things that ARE legal, then OpenAI will either get sued or pressured to follow the contract. As @JTillipman mentioned in here last blog post: “This means that when the government buys AI commercially, the vendor’s standard terms and conditions, including its acceptable use policies, are the default starting point. **Restrictions on use are not some exotic demand by activist AI companies.** This is the consequence of the government buying commercial products on commercial terms.” The claim that we should just let the law decide restrictions goes against common practice in procurement! “The government’s punitive response to Anthropic compounds this problem. If the consequence of negotiating aggressively with the government is being designated a supply chain risk—a mechanism more commonly associated with foreign adversary threats—companies have strong incentives to simply accept whatever terms the government demands. OpenAI itself said it does not believe the supply chain risk designation should have been applied to Anthropic. That may lead to faster procurement, but it will yield worse governance outcomes. Companies that are afraid to negotiate are companies that will not push back when the government’s proposed terms are inadequate for either party. The question the public debate should be asking is not whether AI companies have the right to tell the Pentagon what to do. They do (within the limits of the contract they negotiate), depending on the acquisition pathway, contract type, and the terms the parties agree to. The question is whether the government’s current approach to AI procurement produces contracts that adequately protect the public interest. Based on the evidence, the answer is no. The Anthropic-Pentagon dispute, for all the attention it has received, is a symptom of that deeper problem, not its cause.” There is no part of the selectively shared part of the contract that goes against this. As of now, it is difficult not to believe that we are being deceived by OpenAI on the agreements of the contract or simply that we don’t have nearly sufficient information to make the inference they arguing for.

This is also an interesting update: Anthropic staff say their natsec model has technical safeguards akin to much of what OpenAI says they will build, despite a series of very confident statements by OpenAI staff that it did not. Given other contemporaneous statements by OpenAI that they had to correct afterwards, I know which statement I'm currently putting more credibility in.

@aidan_mclau @scrollvoid This isn't true. Anthropic hasn't offered a "helpful-only" model without safeguards for NatSec use. Claude Gov is a custom model with extra training, including technical safeguards. (We've also had FDEs and researchers implementing it, and we run our own classifier stack.)

"The United States is winning the AI competition because of its commitment to free enterprise and the rule of law; undermining that commitment to punish one company is short-sighted and antithetical to our national security interests. We urge the Department of War to withdraw its supply chain risk designation and resolve this dispute through normal commercial channels." I signed app.dowletter.org, if you agree with the above please consider signing as well.

In the last few days, OpenAI and its executives have claimed that its DoW deal prevents its models being used for mass domestic surveillance. As I write in a lengthy explainer for @ReadTransformer today, that appears to be misleading at best.

Very important piece that confirms what I've suspected the last couple days: "If you look line-by-line at the OpenAI terms, the source said, every aspect of it boils down to: If it’s technically legal, then the US military can use OpenAI’s technology to carry it out."

Well there is part of what the DoD Anthropic was about. Based on the contract language around analyzing bulk commercial data and deanonymizing it matches with this data discussion: Since 2021 the Pentagons DIA has been purchasing anonymized and harvested geolocation data that’s used in advertising, arguing it’s not “spying” since it’s commercial. They’ve now realized AI is strong enough to take this bulk data and de-anonymize it accurately. Anthropic deemed that spying on Americans. OpenAI doesn’t.

@jasonkwon @ARozenshtein Oh no, I'm just seeing this, and it's not right, I fear. I took @jasonkwon up on his offer and asked my Council of 8 LLMs to evaluate the claim. Their judgment was harsh:

If you have technical safeguards without contractual language to back them up, you’re asking to be in a jailbreak competition with the military. Even if you catch them red-handed, you can’t make them stop: Jailbreaks are legal, so they're presumably included in “all lawful use”.

This is a notable and important update. Apparently Anthropic offered "FISA yes, commercially acquired data no" and got turned down. This, uh, makes me substantially doubt the OpenAI claims that they've excluded NSA from their contract successfully.

New @slatestarcodex post: "Some alert ACX readers have done a deep dive into national security law to try to untangle the [DoW-OAI] situation... Current laws against domestic mass surveillance and autonomous weapons have wide loopholes in practice." astralcodexten.com/p/all-lawful-u…

@binarybits @USWREMichael I mean it pretty explicitly confirms that what OpenAI offered that Anthropic did not was using models to surveil Americans through warantless data collection, as long as that collection is one of the various 12333 carve-outs. Probably not something DoD wants said aloud.

This is an important point from Logan Koepke: OpenAI is claiming that DoW lacks authorities to get commercial data at scale, despite extensive reporting that they have done so

HELL NO! Anthropic is in the right here. The Atlantic: “On Friday afternoon, Anthropic learned that the Pentagon still wanted to use the company’s AI to analyze bulk data collected from Americans.” “That could include information such as the questions you ask your favorite chatbot, your Google search history, your GPS-tracked movements, and your credit-card transactions, all of which could be cross-referenced with other details about your life. Anthropic’s leadership told Hegseth’s team that was a bridge too far, and the deal fell apart. Soon after, Hegseth directed the U.S. military’s contractors, suppliers, and partners to stop doing business with Anthropic.”