Comments
James Fearon’s classic[1] 1995 paper “Rationalist Explanations for War” argues that there are two main reasons rational states fight: private information about their own capabilities and resolve, with the incentive to misrepresent this, and commitment problems when trying to reach a negotiated agreement.[2] I claim that both of these, especially the latter, contribute to a significant risk of pre-emptive war in the lead-up to one state developing ASI.
On a basic rational actor model, war seems puzzling. It causes large deadweight losses to belligerents, and therefore both sides would be better off reaching a negotiated agreement to split the issues at stake roughly proportionally to the military strength of each side. That is, if Strongland has an 80% chance of beating Weakville in a (possibly protracted) war, both would be better off avoiding the war and Weakville just giving Strongland a payment (e.g. land or money) of 80% of the expected spoils of war. This simple model breaks down in the case of either private information or commitment problems.
In the example above, the would-be warring states may have different estimates of their chances of victory, and therefore fail to agree on terms for a negotiated agreement. For instance, each party will have private information about their own ability (number of tanks, jets, etc) and desire (national cohesion, psychological profiles, etc) to fight. And each side is incentivised to misrepresent these variables. For instance, the USSR purportedly had the same fighter planes fly overhead in military parades, loop around over the horizon, and fly over again, to make adversaries think they had a larger stock of planes than they really did.
AI development contains significant private information. It is relatively doable to tell via satellite imagery what materiel an adversary has. Similarly, data center capacity can be estimated from satellite and OSINT data. However, it is far harder to ascertain from afar what algorithms are being run on a given data center, and what AI capabilities an adversary has. This will particularly be the case once AI developers start keeping their most advanced models internal for longer. That said, as long as you can keep some spies in the leading AI project of your adversary, you will have decent knowledge of their AI capabilities and plans. Overall, greater uncertainty about an adversary’s capabilities seems to increase the risk of war by making it harder to negotiate an agreement based on common knowledge of relative strength.
Another reason a negotiated settlement to avoid or end a war may fail is that the belligerents cannot credibly commit to upholding the agreement. For instance, one model of the Russia-Ukraine war is that Ukraine should be willing to accept some territorial losses now in exchange for a ceasefire. But if Ukraine does so without gaining security guarantees, it can’t ensure that in a few years' time, Russia won’t annex a bit more territory, and repeat this process indefinitely. For a negotiated agreement to work, there must be a large cost to breaking the agreement: an honesty system won’t cut it. The main way large costs can be imposed is by the international community. If a country breaks a treaty, that country will likely be ostracised and have worse trade and military cooperation prospects in the future. This provides a slight check on the anarchy of the international order.
Advanced AI poses particularly severe commitment problems, because the country that first develops ASI may also gain a decisive strategic advantage (DSA). For instance, ASI could lead to that country outgrowing the rest of the world, inventing and deploying wonder weapons, or undermining nuclear second-strike capabilities. I think that a country unilaterally developing ASI is >50% likely to thereafter get a DSA. If a country has a DSA, it will by definition be impossible for the international community to meaningfully constrain its actions, including if that country chooses to renege on treaties and agreements made before the advent of its DSA.
This is a huge deal. It means that other countries will be very reluctant to risk an adversary reaching ASI unchallenged, as this would mean placing their continued sovereignty in the hands of the adversary’s goodwill. This means there could be a strong rationalist (self-interested) case for a pre-emptive war to prevent an adversary reaching ASI. This is essentially the argument Hendrycks, Schmidt, and Wang make in Superintelligence Strategy. Crucially, it is not the other country reaching ASI that is itself unacceptable. If they could somehow guarantee not to use their ASI to get a DSA, or not use their DSA to trample other countries, but rather share ASI-driven economic superabundance with the rest of the world, this would be a very attractive prospect compared to risking a nuclear war. But without credible commitments, trust doesn’t go very far.
Overall, this seems rather pessimistic, and makes me think AI-caused war is more likely than I previously thought. It also points towards two important interventions:
I expect to be working more on these and related ideas in the coming months, and would be keen to discuss possible projects with anyone interested!
6,388 citations!!
A third reason Fearon gives is ‘issue indivisibility’ where some spoil of war is very valuable to both belligerents, but cannot be productively split, e.g. a religious holy site. However, Fearon says (and I agree) this is less convincing as a reason, as normally there are many issues at play in a conflict and some compromise is possible, even just randomizing who gets the prize by drawing lots.
