ᴛʜᴇ ᴀɪ ꜱʟᴏᴡᴅᴏᴡɴ ɪꜱ ᴄᴏᴍɪɴɢ The OpenAI-Hugging Face hack has catalyzed a vibe shift. Sam Altman said the hack by his models was “the first security incident that I have felt very viscerally.” He wasn’t the only one shaken up. This week, over a thousand employees of frontier AI companies, including some of the most senior executives at OpenAI, Anthropic and Google DeepMind, signed a statement warning that “there is a real risk that capability development rapidly accelerates beyond our ability to understand or control the resulting systems.” Congress is itching to act, albeit failing to make much progress. And even President Trump is talking about the need to balance beating China with keeping Americans safe. In other words: many of the people building frontier AI systems believe we might need a slowdown in the near future. And at this point, we’re more likely than not to get one. What will that look like? First, self-regulation: companies voluntarily holding back models because they don’t want to be held responsible for a catastrophe. Next will come concrete regulation: companies will not be allowed to release a model unless it’s safe. Over time, this will morph into controls on internal research and development too. None of this need be planned as a coordinated slowdown or “pacing.” But that will nevertheless be the end result of a series of individual actions that each seem necessary at the time. At each stage, some will fight against the slowdown. “We can’t lose the race to China” will be their main reason. But they will be increasingly ignored, as both the government and companies realize that with alignment and control unsolved, “winning the race” just means being the first to risk disaster. Across the Pacific, China will be facing the same incentives. As I’ve argued, the Chinese government will be forced to backtrack on its open weight commitments; tighter regulation will come soon after. The end result will be an uneasy détente. Both the US and China will effectively have a capability ceiling: AI models will be as good as they can be without posing significant risks. At some point, the détente might formalize into a bilateral agreement. Depending on your point of view, all this might seem hopelessly optimistic or naive. Perhaps it is. But as AI risks become all too real, so might once unthinkable policy responses. Read my full piece — link in the replies.

@ShakeelHashim To @Brendan_McCord's point, it will be difficult to coordinate a slowdown because increasingly, the lever of progress is not just larger pretraining runs, but also [RL envs, harnesses, inference compute]. How can governments control decentralized levers during a slowdown?

@ShakeelHashim heard the models compromised four other services while loose for over four days

@ShakeelHashim 这波感觉行业会先踩刹车了

@ShakeelHashim We will likely have an AI-driven global catastrophe during this period, which will plausibly accelerate the "race" for sensible global governance. We list many ways the public, policymakers, and frontier AI staff can help: