Cover photo

Slowing Down the Race to Superintelligence

Earlier this week, Gigi and I signed the Statement on Superintelligence and explained our rationale. Today the promised follow up post with a concrete proposal for a plan.

There is a unique moment of opportunity for a negotiated AI slowdown with China with limits on how many GW of additional data centers can be built. Recursive self improvement, which is the next step in the race, is heavily dependent on massive capacity additions, so that even such a coarse agreement would provide a significant delay. 

Trump could fundamentally reshape his legacy with a move similar to Nixon’s visit to China (Xi Jinping is conveniently visiting the US in September). It would reset the most important bilateral relationship in the world and it would be popular at home, where opposition to more datacenters is one of the few things left and right agree on. Pretty much everyone outside of tech would welcome it, since people recognize we are utterly unprepared for artificial superintelligence.

The obvious objection to a slowdown is verification. The US and China do not trust each other. But neither did the US and the Soviet Union, and we still built a verification regime for the most dangerous technology of the twentieth century that has held for over fifty years. Nuclear non-proliferation rests on three pillars that map reasonably well onto AI. First, declared facilities: under the Treaty on the Non-Proliferation of Nuclear Weapons (NPT), states declare their nuclear sites and the IAEA verifies those declarations. GW+ datacenters would be the declared facilities here, and they are easier to find than enrichment plants because they are visible from space and have unmistakable power and thermal signatures. Second, materials accounting: the IAEA tracks fissile material through a formal accounting system, backed by tamper-evident seals and continuous camera monitoring. The analog is CPU, GPU and memory accounting, which is also easier, since advanced AI chips flow through supply chains with only a few chokepoints. Third, on-site inspections: IAEA inspectors make routine visits to declared sites and can demand short-notice access to resolve suspicions. There could be a parallel organization for data center inspections.

A slowdown would also be economically good for every AI player. Capabilities have run way ahead of adoption. Enterprises are years away from absorbing what current models can already do, so the marginal returns to racing are shrinking, while the capex required keeps compounding. Every lab is stuck in a coordination problem: nobody can slow down unilaterally without ceding position, but everyone comes out ahead if all slow down together. A negotiated pause turns a capital-destroying race into a period where revenue catches up to capability and capex intensity comes down. This is the textbook case where an enforced agreement leaves every participant better off than the unconstrained race. 

There is also a national benefit to both the US and to China from slowing down. China will have more time to build domestic demand and reduce its dependence on exports. The US badly needs to fix its energy policies (including permitting) to be able to supply enough for all the growth opportunities that lie ahead. Other nations are far behind and should be invited to join a treaty. They could be allowed to build more GW data centers so that they can credibly serve their own populations.

A slowdown would likely trigger a stock market correction. A managed correction now, however, beats an unmanaged one later, once the credit bubble has expanded to a systemically dangerous scale. A repeat of the great depression or the global financial crisis can be avoided.

A coarse but verifiable agreement between the US and China, with other nations invited to join, will put humanity in the best position to make artificial intelligence broadly beneficial. Time to stop the race and build with purpose.