I spent a few hours listening to Dan Hendyrcks, who runs the non-profit AI Safety group behind SB 1047, aka the California AI Control and Centralization Bill. I find him charming, measured, intelligent and incredibly dangerous. Some of the most dangerous people in life are ones who can convincingly lie about their intentions and who can easily mask those intentions. After listening to him for several hours and reading his talking points info-graphic, which claims to protect open source AI and fine tunes of models, I'm convinced that not only does he know this is an outright fabrication, but that's it's 100% intentional. Make no mistake, Dan's p(doom) is 80%, meaning he believes that there is a 80% chance that AI will kill us all. Reading the language of the bill and with that understanding held clearly in mind, it's not hard to see what the true intentions of the bill are, despite Dan's measured language and slickly produced info-graphics. The intention of the bill is very clear for anyone who has eyes to read the text. It has three clear goals: 1) Ensure that only a small group of companies, rigidly controlled and overseen by a special government agency, have the right to create advanced artificial intelligence. 2) Destroy open source AI. 3) Make sure that model makers have liability hanging over them like the sword of Damocles for the rest of their life, ensuring that governments can hold model makers responsible for any misuse or crime from those models forever. Usually when I listen to AI doomers it takes about one minute to hear the flaws in their logic and the nonsensical logical leaps. It's a bit like looking at code from GPT. Looks great, reads well but under the surface it's riddled with bugs. Many doomers are fantastic at using the language of rationality without any actual rationality happening below the surface. Not so with Dan, who I find delightfully well spoken and clear. He's the kind of fellow I'd like to have a good meal with and a glass of wine. In the Future of Life podcast he rejected notions of "AI Foom" as unlikely and noted that we were in a slow take off scenario. He also subtly digs at the community by saying that belief in rogue AI was based on "cultural history" aka they read too much sci-fi. He also clearly lays out his strategy to go towards grassroots legislation and work with policymakers at that level. I'd almost like him if we weren't trying to crush American innovation and cripple AI development. And yet I find him tremendously dangerous and I'm not afraid to say it. Unlike other folks in the AI safety space he is good at masking his intentions. He's a bit like Yoda when we first meet him in Empire Strikes Back, cleverly disguised as a harmless old man. In all his talking points and in his communications on the bill he is measured and denies his true intentions up and down while working to ensure that AI is rigidly controlled by the Turing Police and centralized at all costs. He cleverly says that the bill protects open source because it "establish a new advisory council to advocate for safe and secure open source AI development." This is cleverly worded to create the illusion that open source is protected. Really it's an advisory board that fails to protect open source AI in any way whatsoever, and he knows it. The bill is absolutely a de-facto ban on open source AI for advanced models because it requires model makers to have “the capability to promptly enact a full shutdown of the covered model,” aka a remote kill switch, including the ability to force “the cessation of operation of a covered model, including all copies and derivative models, on all computers and storage devices within custody, control, or possession of a person, including any computer or storage device remotely provided by agreement." Of course, with a widely distributed model that is not tightly controlled or surveilled this would be impossible because model developers are held liable for the model and fine tunes of the model no matter where that model lives. The talking points also claim that fine tunes of the model are protected. They're not, because of this language: “(2) “Hazardous capability” includes a capability described in paragraph (1) even if the hazardous capability would not manifest but for fine tuning and posttraining modifications performed by third-party experts intending to demonstrate those abilities.” In other words, someone fine tunes a model they consider dangerous, the model maker is liable. This bill is insidious in nearly every way. It was conceived to sound simple and measured while its real intentions are much darker. It must be stopped. It's goal is to dramatically de-incentivize people and companies from ever releasing a powerful model as open source and to ensure that model makers can be crushed into submission at any time. It's designed to do this in the one state that is responsible for the vast majority technical development and innovation over the last few decades. All because someone believes in a the fantasy that AI will kill us all. I fully support everyone's right to believe whatever they want but they don't get to make laws for the rest of us and crush American innovation. If it wasn't so insidious, I'd almost admire it for its deviousness.
