Comments
This is a linkpost for https://www.metaculus.com/notebooks/10439/russia-ukraine-conflict-forecasting-nuclear-risk-in-2022/
With the start of the Russian invasion of Ukraine on February 24 the possibility of a nuclear escalation became an important concern of policy-makers and the general public alike. In light of ongoing geopolitical tensions generated and sustained by the war in Ukraine, Metaculus, an online prediction platform, recruited a team of Metaculus Pro Forecasters to forecast on key questions surrounding nuclear risk in 2022. This report provides background for the current situation, summarizes the aggregate forecasts this project generated, and presents the rationales forecasters provided for their judgments.
Key takeaways
- Forecasters estimate the overall risk of a full-scale nuclear war beginning in 2022 to be 0.35% and to be similar to the annual risk of nuclear war during the Cold War.
- The most likely scenario for a nuclear escalation is estimated to be due to an accident or in response to a false alarm.
- A nuclear test conducted by Russia is estimated to be more likely (~7%) than a nuclear detonation that causes fatalities, as it could achieve some of the intended psychological effects while reducing the risk of NATO retaliation.
- Most forecasters believe that although it is not in Russia’s interest to use nuclear weapons offensively, Vladimir Putin may see this as a way to underline his seriousness and deter NATO from engaging further.
- In the case of what is perceived to be a direct attack (nuclear or otherwise), forecasters find it likely that both NATO and Russia would use nuclear weapons in defense or to retaliate.
- Belarus's role in the conflict is subject to great uncertainty, but forecasters believe there is a considerable probability (30%) that Russia will decide to move nuclear warheads to Belarus.
Summary of forecasts
Background on the present situation
Nuclear threat perceptions
On February 27, 2022, Russian President Vladimir Putin placed Russian nuclear and missile forces on the highest combat readiness orders in response to what he called "aggressive statements" made by NATO. So far the United States declined to update its nuclear readiness levels. On February 28, an anonymous US official told Reuters, "I don't believe we've seen anything specific as a result of the direction that [Putin] gave, at least not yet, in terms of appreciable or noticeable muscle movements". On March 1, NATO's secretary general Jens Stoltenberg told the AP, "We will always do what is needed to protect and defend our allies, but we don’t think there is any need now to change the alert levels of NATO’s nuclear forces." On March 3, Russian Foreign Minister Sergei Lavrov said "The thought of nuclear is constantly spinning in the heads of Western politicians but not in the heads of Russians". TIME magazine reported on the same day that the US and Russia have opened a direct communications hotline to prevent an accidental military conflict.
US and NATO responses to the Russian invasion of Ukraine
Ukraine is not a NATO member or otherwise a US treaty ally, meaning that the US has no treaty obligation to defend it using its own forces. US President Joe Biden stated on 24 February that "Our forces are not and will not be engaged in the conflict with Russia in Ukraine." NATO Secretary General Jens Stoltenberg said on February 24 that “We have made it clear that we do not have any plans and intentional [sic] deploying NATO troops to Ukraine.”
However, the US and other NATO member countries have been involved in supplying weapons to Ukraine. On February 26 the US approved $350M in military aid to Ukraine, and on March 10 the US Senate passed a proposal to provide $13.6B in emergency aid to Ukraine. Discussions are also reportedly ongoing in the Biden administration as to legal questions regarding the types of weapons and intelligence the US can provide to Ukraine, given the concern that crossing certain thresholds might make the US a party to the conflict under international law. There are also other concerns about potential paths to escalation between the US and Russia.
Status of nuclear forces
The Federation of Atomic Scientists estimated on February 23, 2022 that the US has 5,428 nuclear weapons, 1,744 of which are deployed, strategic or non-strategic, weapons. In January 2021, Hans M. Kristensen estimated 100 US nuclear weapons were deployed in Italy, Germany, the Netherlands, Turkey, and Belgium as part of NATO's nuclear sharing policy.
Following the collapse of the Soviet Union, Ukraine relinquished possession of the nuclear weapons it inherited from the USSR upon signing the Budapest Memorandum in 1994. The Memorandum establishes security assurances from the US, Russia, and the UK for Ukraine, Belarus, and Kazakhstan. In return, those states relinquished their nuclear weapons between 1993 to 1996. Following the Russian annexation of Crimea in 2014 and the invasion of Ukraine in 2022, Ukrainians have criticized the Budapest Memorandum. Ukrainian President Volodymyr Zelenskyy said on February 22, 2022:
I am initiating consultations within the framework of the Budapest Memorandum. It was entrusted to the Minister of Foreign Affairs to convene them. If they do not take place again or if there are no security guarantees for our country as a result of them, Ukraine will have every right to believe that the Budapest Memorandum does not work, and all package decisions of 1994 will be called into question.
Methods
To gather available evidence about near-term nuclear risk, i.e. the risk from the detonation of nuclear weapons within the year 2022, Metaculus created a confidential forecasting project and recruited a group of 13 experienced forecasters – we will call them Metaculus Pro Forecasters in this report – to forecast on relevant nuclear risk questions.
Recruitment process for Metaculus Pro Forecasters
Metaculus selected individuals who met most if not all of the following criteria:
have scores in the top 2% of all Metaculus forecasters
- have forecasted on a minimum of 75+ questions that have been resolved
- have experience forecasting for a year or more
- have forecasted across multiple subject areas
- have a history of providing commentary to explain their forecasts
The Pro Forecasters were selected from a forecasting community that in aggregate performs significantly better than chance.¹ Most Pro Forecasters have many years of experience and all have forecasted hundreds of questions. All Pro Forecasters accrued their track records before the Russian invasion of Ukraine.
The forecasting questions
The following questions were asked:
- Will there be a full-scale nuclear exchange between the US and Russia before 2023?
- Will Russia detonate a nuclear weapon before 2023?
- Will the US detonate a non-test nuclear weapon offensively before 2023?
- Will Russia detonate any nuclear weapon that causes a fatality anywhere outside of Russia before 2023?
- Will a Russian nuclear weapon be detonated in a non-US NATO country before 2023?
- Will a Russian nuclear weapon be detonated in the US before 2023?
- Will at least one nuclear weapon be detonated in Ukraine before 2023?
- Will Russia place any nuclear weapons in Belarus before 2023?
For the purposes of all questions, a "nuclear weapon" is understood as a bomb that uses a nuclear fission or fusion reaction as its primary energy source, excluding conventional bombs which spread radioactive fallout (so called "dirty bombs").
All questions were opened on March 5, 2022 and will close on December 31, 2022, meaning that these forecasts will be continuously updated and we are only presenting a snapshot of the forecasts as of March 24, 2022.
Forecasting process
Forecasters were asked to make their predictions in a private forecasting project on Metaculus. They were encouraged to provide their reasoning in the form of comments and to update their forecasts based on new information. In addition, forecasters gathered for two live discussion sessions via Zoom and discussed their predictions and factors that influenced their forecasts.
A note on ranges
For all questions other than No. 1, “Will there be a full-scale nuclear exchange between Russia and the US before 2023?”, forecasters gave their predictions on a scale of 1-99%. In many cases, forecasters believed the true probability to be less than 1%; we therefore take forecasts of 1% to mean ≤ 1% throughout.
Results
This section presents the forecasts and summarizes the reasoning provided by forecasters. An overview of the median, 25th percentile and 75th percentile of the forecasts created by the Metaculus Pro Forecasters can be seen in the Table below.
1. Will there be a full-scale nuclear exchange between the US and Russia before 2023?
Context and additional information
For this question, a "full-scale nuclear exchange" must involve at minimum 200 nuclear weapon warheads being launched via missiles, aircraft, ships, or submarines as part of a nuclear exchange between the US and Russia², with the US and Russia each launching at least 100 nuclear warheads at one another and/or at treaty allies of one another.
Median forecast
0.35%
Forecaster reasoning
Forecasters generally perceived the risk of a full-scale exchange to be around 50% of the probability of a nuclear warhead detonating in the United States.
According to one Metaculus Pro Forecaster, two scenarios are most likely to trigger a full-scale nuclear war:
a) A scenario in which a direct threat of war between Russia and the NATO exists (akin to the Cuban missile crisis) that eventually escalates to a nuclear conflict
b) A scenario in which a mistake, or a failure of detection systems could lead to a premature counterattack. Examples of similar situations in the past are e.g. the story of Stanislav Petrov and the Norwegian rocket incident.
The following sequence of events could potentially lead to an escalation of tensions in the sense of a), as described by a second forecaster:
- A clash between NATO and Russian forces anywhere could lead to a scenario with more than 100 NATO casualties.
- Conditional on (1), any NATO country triggers Article 5 to defend against a Russian attack on its territory.
- Conditional on (2), Russia offensively detonates a nuclear weapon in the territory of any NATO country.
- Conditional on (3), full-scale nuclear war between Russia and the US.
Forecasters agreed that the probability of such a scenario should be considerably below 0.25%
Despite forecasters’ uncertainty about Putin’s psychological state of mind, the second scenario, b), in which a mistake or false alarm leads to escalation, is considered more likely. In the last 50 years there have been at least two incidents of this kind that could have led to nuclear escalation (1983 nuclear false alarm and the 1995 Norwegian rocket incident), giving a baseline probability of 4% per year for such incidents. The risk of nuclear escalation in response to any such incident (e.g. due to mistakes, accidents, or the misinterpretation of unreliable data) is considered increased in times of heightened tensions.
2. Will Russia detonate a nuclear weapon before 2023?
Context and additional information
In 1996, the majority of UN countries signed and ratified the Comprehensive Nuclear-Test-Ban Treaty, including Russia, although some states have signed but not ratified the treaty, including the US, China, Israel, and Iran. Because eight Annex II nations have not ratified the treaty, it is not in effect. While test nuclear weapons detonations were once common, according to the Arms Control Association since 1999 the only nation to conduct test detonations is North Korea.
Nuclear weapons are tested for multiple purposes, including to test the capability and reliability of the weapon system, to study physical effects of the detonation, as well as to test the abilities to detect and defend against nuclear detonations. Beyond the technical aspects of the test, test nuclear detonations can be used for political and diplomatic purposes such as intimidation.
Median forecast
11%
Forecaster reasoning
Though still low, the median prediction for this question is noticeably higher than the predicted probability on other questions about the lethal use of nuclear weapons. The likeliest scenario Metaculus Pro Forecasters considered here is a political demonstration of power. Putin might feel that his denouncements of Western aid to Ukraine and of sanctions against Russia are toothless, and perform a nuclear demonstration to show that he's serious about breaking norms and recent trends.
Forecasters who assigned a 7% likelihood to the event believed a nuclear test could be conducted in a very publicized manner in order to influence public opinion and perception of the risks involved in opposing Russia. A test would have the advantage of possibly avoiding the high risk of provoking retaliation and escalation involved in deploying tactical nuclear weapons, while potentially achieving some of the intended psychological effects. On the other hand a nuclear test may be unlikely as it is unclear whether Russia deems one to be necessary, as overt threats from Putin may already be enough to deter NATO from getting further involved in the conflict without isolating Russia further.
Forecasters generally considered nuclear tests to be an irrational move, but expressed uncertainty about Putin’s current mindset. Tail risks may generally be higher when power is concentrated in the hands of a single person, and it appears likely that Putin has surrounded himself with advisors who are not willing or able to question him or his decisions. In a crisis, there is limited time to think things through, so decisions can be made in haste and accurate information is too often in short supply. Given that Russia seems to have underestimated the costs of invading Ukraine their perception of the costs of a move like this could also be considerably off the mark.
3. Will the US detonate a non-test nuclear weapon offensively before 2023?
Context and additional information
A deliberate, inadvertent, or accidental/unauthorized detonation of a state or non-state nuclear weapon can count towards a positive resolution. Neither detonations for testing purposes nor peaceful nuclear explosions will be considered for a positive resolution, even if such detonations cause significant damage. Weapons provided to NATO countries in NATO's nuclear sharing program will count, even if they are detonated without the US's direct approval (but only if they are from the US arsenal, not eg. France's or the UK's).
Median forecast
≤1%
Forecaster reasoning
Forecasters generally didn’t consider it likely for the US to initiate a nuclear attack. However, the US will likely retaliate if they are attacked directly, and they may also feel compelled to respond in kind if Russia used a tactical nuclear weapon in Ukraine to show it was not willing to back off.
4. Will Russia detonate any nuclear weapon that causes a fatality anywhere outside of Russia before 2023?
Context and additional information
This question will resolve positively if Russia detonates any nuclear weapon before January 1, 2023 that causes at least one fatality anywhere in the world outside of Russia. Detonations in the Donetsk People's Republic, the Luhansk People's Republic, and the partially recognized states of South Ossetia and Abkhazia will count toward positive resolution. A deliberate, inadvertent, or accidental/unauthorized detonation of a state or non-state nuclear weapon will count toward a positive resolution.
Median forecast
2%
Forecaster reasoning
Forecasters mainly argued that the current equilibrium was relatively stable. NATO is careful about not provoking Russia, they have clearly stated they do not want to engage in war with Russia, reject the idea of a no-fly zone, and did not raise the alert-level of their nuclear forces.
The use of nuclear weapons by Russia can be divided into offensive and defensive, as well as into tactical and strategic. Tactical nuclear weapons are used in combat to achieve goals related to a single battle. Strategic nuclear weapons, on the other hand, are used against targets far away from the battlefield (i.e. via intercontinental ballistic missiles) to achieve strategic goals.
Forecasters consider the offensive use of tactical nuclear weapons by Russia unlikely, as this carries a high risk of provoking NATO into entering the war directly, which Russia has strong incentives to avoid. Forecasters could conceive of a scenario (but did not find it likely) where Putin would use offensive tactical nuclear weapons to prove his seriousness and deter NATO from further escalation, believing that NATO would not be willing to retaliate.
Were NATO to invade Russia directly, a use of defensive nuclear weapons seems likely, as Russian conventional forces are much weaker than NATO’s. However, in discussions forecasters did not consider it likely that NATO forces would enter Russian territory.
Forecasters express uncertainty about Putin’s state of mind. For example, he may perceive the West as a strong threat to his and to Russia’s existence and may therefore be more ready to make use of nuclear weapons as a last resort than we are aware of. At the moment, however, his internal position in Russia does not seem to be under considerable threat as of now.
5. Will a Russian nuclear weapon be detonated in a non-US NATO country before 2023?
Median forecast
≤1%
Forecaster reasoning
Forecasters did not provide substantial rationales here; presumably these will have been similar to those for other questions.
6. Will a Russian nuclear weapon be detonated in the US before 2023?
Median forecast
≤1%
Forecaster reasoning
As described in more detail in the question on whether there will be a full-scale nuclear war, forecasters believe the most likely cause for nuclear detonation in the US to be false alarm or an accident interpreted as genuine threat. One forecaster noted that the possibility of regime change in Russia may lead to destabilization and increase the risk of nuclear escalation. The current community prediction on a Metaculus question about whether Putin will be president in Russia on February 1 2022 is currently at 80%. Forecasters generally thought the risk of a Russian detonation in the US at present time to be roughly comparable to the average risk during the Cold War (others have estimated this to be around 0.4% per year)
7. Will at least one nuclear weapon be detonated in Ukraine before 2023?
Median forecast
≤1%
Forecaster reasoning
Only two scenarios contribute meaningfully to the possibility of a detonation in Ukraine: either Putin carries out a first use of nuclear weapons in the country, or it is hit during a wider nuclear conflict.
A recent analysis of Russia’s shelling of a Ukrainian nuclear plant suggested that the shelling put the reactor’s integrity at serious risk. This can be taken as a sign that Russia is willing to take nuclear risks in Ukraine.
Forecasters are divided over how likely it is that Putin would escalate the conflict by using a nuclear weapon in Ukraine; some characterize it as a pointless action that would achieve nothing, while others point to its potential to intimidate the opposition and highlight uncertainty over whether a tactical nuclear strike would provoke nuclear retaliation from NATO. Russia is unlikely to have Ukrainian targets pre-programmed into its nuclear system; this may reduce the likelihood of a Russian nuclear strike in Ukraine.
8. Will Russia place any nuclear weapons in Belarus before 2023?
Context and additional information
Belarus has not possessed any nuclear weapons since 1996, as the country cooperatively returned the weapons in its control following the collapse of the Soviet Union. As Belarus has kept close ties to Russia since, on November 30, 2021, Belarusian President Alexander Lukashenko suggested that in response to a hypothetical movement of US nuclear weapons to Eastern European NATO countries, Belarus would be willing to host Russia's Nuclear Weapons. In the same interview, Lukashenko noted that Belarus has carefully preserved the necessary military infrastructure dating back to the Soviet era. On February 27, 2022, a referendum was passed to remove the commitment to be a nuclear-free state in Article 18 of Belarus' Constitution.
Median forecast
35%
Forecaster reasoning
Belarusian President Alexander Lukashenko made several recent statements suggesting that Belarus will only host nuclear weapons if threatened. However, the definition of “threatened” is vague and leaves considerable room for uncertainty. These statements may even have been designed to disguise Belarus’s intention to host nuclear weapons, given that Belarus reversed its nuclear-free policy in a recent constitutional “referendum”. This referendum resulted in consolidation of power for President Lukashenko and may have been part of a broader strategy to exchange the possibility of Belarusian nuclear armament for economic support from Moscow.
For Putin, moving nuclear weapons into Belarus could serve as a distraction from its primary goal of occupying Ukraine. Nuclear posturing in Belarus may also be one of the few options still available to Putin if he wants to make progress on his geopolitical goals and save face at home.
One the other hand, Belarus has been reluctant to enter the war directly, which may be evidence for Belarusian unwillingness to cooperate with Russia in general. Forecasters expressed uncertainty regarding the strength of Moscow’s control over Belarus, and the possibility of losing nuclear weapons to a disloyal Belarus is something Russia will want to avoid. In addition, only specific types of nuclear weapons can be easily deployed in Belarus at short notice, and Russia has relatively few of these.
Future work
Given that the chances of Russia placing nuclear weapons in Belarus and of Russia undertaking a nuclear test were assessed to be substantial, additional forecasting work directed toward understanding nuclear risk under these scenarios would be valuable.
Acknowledgements
We’d like to thank all the Pro Forecasters who participated in this project, Nikos Bosse and Lawrence Phillips for preparing the majority of this report, and the rest of the Metaculus team members who facilitated the execution of the project. In addition, we greatly appreciate Shahar Avin, Michael Aird, Anthony Aguirre, Greg Laughlin, Jeffrey Ladish, Anders Sandberg, and Peter Scoblic for their helpful comments and suggestions.
¹ The community predictions on the 865 questions that have resolved so far on Metaculus have been, on average, substantially better than chance. Also relevant, the Metaculus track record for the three questions which have been resolved in the “Nuclear Threats” category and 20 questions that have been resolved in the “Armed Conflict” category.
² This threshold was selected arbitrarily as there is no agreed-upon definition of what constitutes a full-scale nuclear exchange.
Is any forecaster or organizer willing to bet with me on "Will Russia place any nuclear weapons in Belarus before 2023?" at 3:7 odds (with me on "no")?