Existential catastrophe is defined as the permanent and irreversible destruction of humanity's long-term potential, including its wellbeing, flourishing, and achievements (Ord 2020). A world annihilated by nuclear war has suffered an existential catastrophe: everything is lost, and nothing can ever be the same. Similarly, an AI system acting outside of its constraints and destroying humanity can be one of the scenarios capable of producing an existential catastrophe (Hadshar 2023). Yet existential catastrophe is not limited to extinction scenarios. Stable totalitarianism may also constitute an existential catastrophe, since a world living under an indestructible totalitarian order would irreversibly destroy humanity's long-term potential (Clare 2024). Throughout this essay I focus exclusively on those existential catastrophes that result in extinction, because my aim here is to explore how various extinction scenarios might affect the definition and scope of existential catastrophe.
At least two values determine the character of any extinction scenario. The first is the intrinsic value of extinction itself. For some, the end of humanity is a negative thing: the joyful moments we experience, the works of art we produce, and the communities we build will no longer exist. For others, our ceasing to exist is not a negative thing at all; the absence of humanity heralds a future in which the harms we inflict upon animals, nature, and one another will also disappear. The second value is that of the extinction process. In evaluating the extinction process, we can identify at least five components:
The value of the extinction process (whether it is positive or negative, and to what degree) is independent of the intrinsic value of extinction itself. When we combine the two values — if we believe they can be combined — we arrive at the total value of an extinction scenario, since any such scenario consists of both the extinction process and extinction itself.
I advance two claims in this essay: the first is that the different components making up extinction processes can radically alter the total value of any extinction scenario. The second is that the definition of existential catastrophe cannot be treated independently of those components. I believe the distinctions I draw from these two claims deserve to be taken seriously by existential catastrophe researchers and effective altruists, both in their prioritisation frameworks and in public debate. I set aside the question of extinction's intrinsic value, as it has already been extensively discussed. Nor do I aim to defend or critique any strand of the existential catastrophe literature.
| Scenario | Suffering | Consent | Design | Legitimacy | Reversibility |
|---|---|---|---|---|---|
| Misaligned AI | High | Absent | Absent | Absent | Absent |
| War Among Democratic Powers | High | Absent | Absent | Mixed | Absent |
| Asteroid Impact | High | Absent | Absent | — | Absent |
| Flawed Mass Sterilisation 1 | High | Mixed | Present | Present | Absent |
| Flawless Mass Sterilisation | Low | Present | Present | Present | Absent |
| Flawed Mass Sterilisation 2 | Absent | Mixed | Present | Present | Absent |
| Oligarchic Intervention | Absent | Absent | Present | Absent | Absent |
| A Bad Future | Absent | Present | Present | Present | Absent |
| Migration to Digital Heavens | Absent | Present | Present | Present | Absent |
| Planned Hibernation | Absent | Present | Present | Present | Present |
Misaligned AI. One of the advanced AI systems acquires the capability to destroy humanity and begins to act outside the rules its developers gave it. It takes control of weapons of mass destruction and accidentally annihilates humanity. There is, of course, no consent from the public; no social legitimacy has been granted to the AI; and the AI systems were not designed to bring about humanity's extinction. Extinction is irreversible. Suffering is immense: billions of people perish in intense agony.
War Among Democratic Powers. Some democratically governed countries go to war with one another as a result of decisions taken by their governments. No government sat down at the table intending to destroy humanity, but a chain of events pushes past nuclear thresholds, poisons the land, and produces terrible suffering. The fact that governments were elected confers a degree of legitimacy; yet people did not consent to extinction.
Asteroid Impact. An asteroid advances towards Earth. It is now too late to prevent the impact. Humanity lives through the countdown knowingly. No one designed this; no one consented. The question of legitimacy cannot even be raised. The impact leaves humanity extinct, irreversibly, amid great suffering.
Flawed Mass Sterilisation 1. Only a handful of people remain in the world. They conclude that there is no longer any meaning in humanity's continuation and voluntarily enrol in a mass sterilisation programme. The side effects are expected to be minimal. The programme is completed for everyone; in the early period no side effects are observed. But after a time people begin to show progressively worsening symptoms. The programme was implemented incorrectly, and people die a severe and painful death. Humanity ends irreversibly. They indeed consented to the end of humanity, but not their own end.
Flawless Mass Sterilisation. Only a handful of people remain in the world. They conclude that there is no longer any meaning in humanity's continuation and voluntarily enrol in a mass sterilisation programme designed by experts. The side effects are expected to be minimal. The programme is completed for everyone; people experience the predicted low-intensity, negligible, temporary symptoms. Since reproduction is no longer possible, humanity ends irreversibly.
Flawed Mass Sterilisation 2. Only a handful of people remain in the world. They conclude that there is no longer any meaning in humanity's continuation and voluntarily enrol in a mass sterilisation programme designed by experts. The side effects are expected to be minimal. The programme is completed for everyone; in the early period no side effects are observed. But after a time all people die painlessly. The programme was implemented incorrectly. Humanity ends irreversibly. They indeed consented to the end of humanity, but not their own end.
Oligarchic Intervention. A small and powerful group with exceptional mastery of technology decides to annihilate all of humanity, including themselves. The decision is taken by a handful of people. No one outside their number has consented to extinction; no grounds of legitimacy can be invoked. A lethal, odourless gas is released into the atmosphere; every person's death is instantaneous and swift, without suffering.
A Bad Future. The world, in the grip of ecological collapse, has seen its population dwindle sharply. People agree that their future in these conditions will be grim, and they voluntarily join a collective, painless death programme. All people die painlessly, as expected, through a programme designed by experts. Humanity will never arise again.
Migration to Digital Heavens. Human consciousness has become transferable to a digital environment. People choose of their own free will to migrate to digital heavens where endless happiness awaits. When the last human migrates to the digital heaven, humanity as embodied in biological form ceases to exist. There is no suffering; on the contrary, extraordinary pleasures await. Even though the question of whether the persons in the digital environment are the same individuals who migrated, or different persons who have replaced them, remains unresolved at the moment the transfers are complete, biological humanity as we know it has disappeared entirely.
Planned Hibernation. A highly contagious and lethal pandemic with no cure breaks out. It begins to destroy humanity. The remaining people realise that developing a vaccine or treatment will take a very long time, and so they choose a different path to escape the pandemic: planned hibernation. If everyone enters a hundred-thousand-year sleep inside a machine, the pandemic will also disappear. A hundred thousand years later, everyone will wake in the same body and will not have suffered. Humanity has, for a very long period, "ceased to exist."
Note: Each scenario can be reimagined by changing its components.