Tryfecta⏸️

Sanders on AI: We need to develop a sense of urgency of here. The economic impacts are going to be enormous. The impacts on our children will be enormous, and again, there is literally an existential threat to the existence of the human race.

First of all, your organization absolutely does not represent the views of everyone who wants to "pause" or "stop" AI, so it is false to say that I "misunderstood" your proposal. I was not referring to your specific proposal in any of my tweets. Second, I do not agree with you that any of this is "likely" to "end" civilization, exterminate humanity, etc. etc. I certainly do believe there are major risks, and I have spent years developing and advocating for policies I believe would address some of those risks. There are other risks about which I have substantial uncertainty, and I have been consistently straightforward about this uncertainty. You, on the other hand, seem to have near-certainty about every bad AI outcome, so long as it gets people on your side. This, in my mind, diminishes your credibility. Third, you claim to want an international treaty (enforced by whom?) signed presumably by a broader group of countries than just the U.S. and China (which ones?) that would ban AI development until some future point (how would we define that point?). My observation is that any such treaty would have to involve capital controls and controls on the free movement of people, unless all countries on Earth with the ability to host large-scale data centers (i.e., almost all of them) were signatories. You claim it is merely a compute governance regime you desire, but what happens when the AI researchers and semiconductor designers and manufacturing engineers are given 9-figure offers to move to God-knows-where to work on a new AI or semiconductor venture? Does the government simply allow that flow of money and people to occur? No. So what you *really* want is a regime that controls (optimally stops) the flow of trillions of dollars in good (all advanced AI compute + all semiconductor manufacturing equipment, as long as it is in service of making advanced AI compute, and by the way, how would you tell the difference between a fab making smartphone chips and GPUs? Inspectors in the fabs? Who supplies the inspectors?), all AI researchers, and all investment dollars that could be tied to AI research or to computing power (what about quantum machine learning, by the way? neuromorphic computing? other new paradigms?). It was actually charitable of me to assume that you'd want this to be enforced by e.g. existing export control regimes within a country. But it seems like you are saying, no, we wouldn't have e.g BIS or MOFCOM do this, we'd have a new international body with "democratic control" (a globally elected president of AI? who runs the elections? are their campaigns? who is allowed to donate to said campaigns) staffed with thousands of people, with a budget easily in the billions, with sweeping power to control flows of goods, people and money that fundamentally implicates ancient principles of national sovereignty. And you're doing all this at a time when international institutions of governance and international collaboration in general has been fraying, to say the least. All of this makes me think you are biting off much more than you can chew, to be sure. I don't want to accuse you of desiring authoritarianism, because I truthfully don't know whether you even understand what it is that you are advocating for. It does not seem based on your response to me that you have really thought about what implementation would require here, and so my guess is that your proposal is not malicious or desirous of tyranny, but instead simply naive and incomplete. And by the way, the hand waviness of your policy prescriptions is nothing compared to the hand waviness of your understanding of artificial intelligence, its likely trajectory, and relevant threat models. You don't even seem to feel a need to explain your position (I myself just wrote 5k words explaining my views on existential risk, and have written many thousands on the threat models I take more seriously), indicating to me that you live in a bubble where the hard-nosed and concrete questions do not get asked. You suggest that I am unserious, interested as I am in the "interesting governance challenges" you seem to dismiss in comparison to what you seem sure is your focus on the "big picture." But do you know what my work has produced? Enacted laws. Dozens of policies being executed as we speak by the largest bureaucracy in the history of mankind. Ideas that have shifted the thinking of people whose decisions will matter. Is it everything? No, it is not. My contributions will ultimately be small. But I put my back into what I do, when the logical move for someone like me would have been to go take a cushy job in the industry after I left the government. Do not ever suggest to me that I do not care deeply about what I am doing and do not ever question the intellectual integrity of a person you do not know. At the very least, do not do it to me.

Here are some questions I wish "Pause" and "Stop" advocates would address: 1. Assuming we achieve the desired policy goal through a bilateral US/China agreement, what would be the specific metric or objective we would say needs to be satisfied in advance? Who decides whether we have satisfied them? What if one one party believes we have satisfied them but the other does not? 2. If the goal is achieved through a bilateral US/China agreement, would we need capital controls to ensure that U.S. investors cannot fund semiconductor fabs, data centers, or AI research labs in countries other than the U.S. and China? 3. Would we need to revoke the passports of U.S.-based AI researchers and semiconductor engineers to prevent them leaving America to join AI-related ventures elsewhere? How else would the U.S. and China keep researchers within their borders? 4. How should we grapple with the fact that (2) and (3) are common features of autocratic regimes? 5. Do the above questions mean that this really should be a global agreement, signed by all countries on Earth, or at least those with the theoretical ability to host large-scale data centers (probably Vanuatu doesn't need to be on board)?

Melania: The future of AI is personified. It will be formed in the shape of humans. Very soon, artificial intelligence will move from our mobile phones to humanoids that deliver utility. They fit well. Imagine a humanoid educator named Plato

Pause AI rhetoric is predicated on the notion that the AI companies are recklessly racing toward dangerous tech and that a government controlled pause button is therefore necessary, but this seems really hard to reconcile with the fact that government is attempting to destroy an AI company because *the government* is racing toward plausibly dangerous AI uses (Sec. Hegseth has stated in official directives that he wants to deploy AI into critical systems regardless of whether it is aligned, for example) and *the company* is pushing back. The roles are totally reversed from the logic that Pause AI and frankly other AI safety advocates confidently assumed for years. It is *industry* that is in favor of alignment and at least somewhat measured deployment risks, and government whose actions seem much closer to reckless. I predicted this for years. I said, in particular, that pauses and bans and licensing regimes gave government a dangerously high degree of control over AI, and that the incentives of government are much more dangerous than those of private industry with competitive market incentives. I believe the events of the last month are good evidence in favor of my view. At this point if you are an AI safety advocate whose policy proposals do not wrestle seriously with the brutal political economic reality of the state and AI, I don’t take you seriously. It gives me no pleasure to have been right about this, by the way. The state has an incredibly strong structural incentive to centralize power using AI, and we are, all of us, not so empowered to stop it. I am quite concerned about this.

Pause AI rhetoric is predicated on the notion that the AI companies are recklessly racing toward dangerous tech and that a government controlled pause button is therefore necessary, but this seems really hard to reconcile with the fact that government is attempting to destroy an AI company because *the government* is racing toward plausibly dangerous AI uses (Sec. Hegseth has stated in official directives that he wants to deploy AI into critical systems regardless of whether it is aligned, for example) and *the company* is pushing back. The roles are totally reversed from the logic that Pause AI and frankly other AI safety advocates confidently assumed for years. It is *industry* that is in favor of alignment and at least somewhat measured deployment risks, and government whose actions seem much closer to reckless. I predicted this for years. I said, in particular, that pauses and bans and licensing regimes gave government a dangerously high degree of control over AI, and that the incentives of government are much more dangerous than those of private industry with competitive market incentives. I believe the events of the last month are good evidence in favor of my view. At this point if you are an AI safety advocate whose policy proposals do not wrestle seriously with the brutal political economic reality of the state and AI, I don’t take you seriously. It gives me no pleasure to have been right about this, by the way. The state has an incredibly strong structural incentive to centralize power using AI, and we are, all of us, not so empowered to stop it. I am quite concerned about this.

Pause AI rhetoric is predicated on the notion that the AI companies are recklessly racing toward dangerous tech and that a government controlled pause button is therefore necessary, but this seems really hard to reconcile with the fact that government is attempting to destroy an AI company because *the government* is racing toward plausibly dangerous AI uses (Sec. Hegseth has stated in official directives that he wants to deploy AI into critical systems regardless of whether it is aligned, for example) and *the company* is pushing back. The roles are totally reversed from the logic that Pause AI and frankly other AI safety advocates confidently assumed for years. It is *industry* that is in favor of alignment and at least somewhat measured deployment risks, and government whose actions seem much closer to reckless. I predicted this for years. I said, in particular, that pauses and bans and licensing regimes gave government a dangerously high degree of control over AI, and that the incentives of government are much more dangerous than those of private industry with competitive market incentives. I believe the events of the last month are good evidence in favor of my view. At this point if you are an AI safety advocate whose policy proposals do not wrestle seriously with the brutal political economic reality of the state and AI, I don’t take you seriously. It gives me no pleasure to have been right about this, by the way. The state has an incredibly strong structural incentive to centralize power using AI, and we are, all of us, not so empowered to stop it. I am quite concerned about this.