Not all extinctions are equal: the bad and even worse ways humanity could end

Existential catastrophe is defined as the permanent and irreversible destruction of humanity's long-term potential, including its wellbeing, flourishing, and achievements (Ord 2020). A world annihilated by nuclear war has suffered an existential catastrophe: everything is lost, and nothing can ever be the same. Similarly, an AI system acting outside of its constraints and destroying humanity can be one of the scenarios capable of producing an existential catastrophe (Hadshar 2023). Yet existential catastrophe is not limited to extinction scenarios. Stable totalitarianism may also constitute an existential catastrophe, since a world living under an indestructible totalitarian order would irreversibly destroy humanity's long-term potential (Clare 2024). Throughout this essay I focus exclusively on those existential catastrophes that result in extinction, because my aim here is to explore how various extinction scenarios might affect the definition and scope of existential catastrophe.

At least two values determine the character of any extinction scenario. The first is the intrinsic value of extinction itself. For some, the end of humanity is a negative thing: the joyful moments we experience, the works of art we produce, and the communities we build will no longer exist. For others, our ceasing to exist is not a negative thing at all; the absence of humanity heralds a future in which the harms we inflict upon animals, nature, and one another will also disappear. The second value is that of the extinction process. In evaluating the extinction process, we can identify at least five components:

1. Suffering. Was there suffering during the extinction process? If so, how much, and how was it distributed?
2. Consent. Did the people living close to the time of extinction, and affected by it, consent to it?
3. Design. Was extinction a consciously designed outcome, or a by-product of other circumstances?
4. Legitimacy. Was the decision to bring about extinction reached through legitimate means?
5. Reversibility. Is the extinction reversible?

The value of the extinction process (whether it is positive or negative, and to what degree) is independent of the intrinsic value of extinction itself. When we combine the two values — if we believe they can be combined — we arrive at the total value of an extinction scenario, since any such scenario consists of both the extinction process and extinction itself.

I advance two claims in this essay: the first is that the different components making up extinction processes can radically alter the total value of any extinction scenario. The second is that the definition of existential catastrophe cannot be treated independently of those components. I believe the distinctions I draw from these two claims deserve to be taken seriously by existential catastrophe researchers and effective altruists, both in their prioritisation frameworks and in public debate. I set aside the question of extinction's intrinsic value, as it has already been extensively discussed. Nor do I aim to defend or critique any strand of the existential catastrophe literature.

Extinction scenarios

Misaligned AI. One of the advanced AI systems acquires the capability to destroy humanity and begins to act outside the rules its developers gave it. It takes control of weapons of mass destruction and accidentally annihilates humanity. There is, of course, no consent from the public; no social legitimacy has been granted to the AI; and the AI systems were not designed to bring about humanity's extinction. Extinction is irreversible. Suffering is immense: billions of people perish in intense agony.

War Among Democratic Powers. Some democratically governed countries go to war with one another as a result of decisions taken by their governments. No government sat down at the table intending to destroy humanity, but a chain of events pushes past nuclear thresholds, poisons the land, and produces terrible suffering. The fact that governments were elected confers a degree of legitimacy; yet people did not consent to extinction.

Asteroid Impact. An asteroid advances towards Earth. It is now too late to prevent the impact. Humanity lives through the countdown knowingly. No one designed this; no one consented. The question of legitimacy cannot even be raised. The impact leaves humanity extinct, irreversibly, amid great suffering.

Flawed Mass Sterilisation 1. Only a handful of people remain in the world. They conclude that there is no longer any meaning in humanity's continuation and voluntarily enrol in a mass sterilisation programme. The side effects are expected to be minimal. The programme is completed for everyone; in the early period no side effects are observed. But after a time people begin to show progressively worsening symptoms. The programme was implemented incorrectly, and people die a severe and painful death. Humanity ends irreversibly. They indeed consented to the end of humanity, but not their own end.

Flawless Mass Sterilisation. Only a handful of people remain in the world. They conclude that there is no longer any meaning in humanity's continuation and voluntarily enrol in a mass sterilisation programme designed by experts. The side effects are expected to be minimal. The programme is completed for everyone; people experience the predicted low-intensity, negligible, temporary symptoms. Since reproduction is no longer possible, humanity ends irreversibly.

Flawed Mass Sterilisation 2. Only a handful of people remain in the world. They conclude that there is no longer any meaning in humanity's continuation and voluntarily enrol in a mass sterilisation programme designed by experts. The side effects are expected to be minimal. The programme is completed for everyone; in the early period no side effects are observed. But after a time all people die painlessly. The programme was implemented incorrectly. Humanity ends irreversibly. They indeed consented to the end of humanity, but not their own end.

Oligarchic Intervention. A small and powerful group with exceptional mastery of technology decides to annihilate all of humanity, including themselves. The decision is taken by a handful of people. No one outside their number has consented to extinction; no grounds of legitimacy can be invoked. A lethal, odourless gas is released into the atmosphere; every person's death is instantaneous and swift, without suffering.

A Bad Future. The world, in the grip of ecological collapse, has seen its population dwindle sharply. People agree that their future in these conditions will be grim, and they voluntarily join a collective, painless death programme. All people die painlessly, as expected, through a programme designed by experts. Humanity will never arise again.

Migration to Digital Heavens. Human consciousness has become transferable to a digital environment. People choose of their own free will to migrate to digital heavens where endless happiness awaits. When the last human migrates to the digital heaven, humanity as embodied in biological form ceases to exist. There is no suffering; on the contrary, extraordinary pleasures await. Even though the question of whether the persons in the digital environment are the same individuals who migrated, or different persons who have replaced them, remains unresolved at the moment the transfers are complete, biological humanity as we know it has disappeared entirely.

Planned Hibernation. A highly contagious and lethal pandemic with no cure breaks out. It begins to destroy humanity. The remaining people realise that developing a vaccine or treatment will take a very long time, and so they choose a different path to escape the pandemic: planned hibernation. If everyone enters a hundred-thousand-year sleep inside a machine, the pandemic will also disappear. A hundred thousand years later, everyone will wake in the same body and will not have suffered. Humanity has, for a very long period, "ceased to exist."

Note: Each scenario can be reimagined by changing its components.

Implications

1. The distinctions between extinction scenarios are of vital importance for prioritisation. The presence and scale of suffering, the nature of consent, the depth of legitimacy, the authorship of design, and reversibility (along with other components one might add) make one scenario better or worse than another. Preventing worse scenarios takes priority over preventing less bad ones, and more resources should be allocated accordingly. Existential catastrophe researchers must therefore engage systematically with these components; the weighting given to each will alter the hierarchy.
2. In the existential catastrophe literature, extinction is generally treated as the permanent and irreversible elimination of the immense positive value that humanity could create through joyful moments, scientific progress, artistic production, and moral advancement — and is therefore framed as a great evil to be prevented. This framework rests on the assumption that the future will be positive, and rightly accepts an existential catastrophe that forecloses such a future as an evil. Yet it does not typically examine how the extinction process unfolds as a separate matter; since the prominent extinction scenarios in the literature (such as nuclear war) already involve great suffering, a deeper treatment of the extinction process has not seemed necessary. But even if an extinction that forecloses a positive future can be regarded as intrinsically bad, the extinction process need not itself contain any evil: it may, for instance, be painless (A Bad Future) or unfold in a consensual and designed manner (Flawless Mass Sterilisation). In other words, when the future is positive, extinction may still be treated as "an evil to be prevented"; but the extinction process itself need not be bad.
3. Conversely, if the future is negative, extinction ceases to be an evil to be prevented; indeed, in a scenario like A Bad Future, attempting to prevent humanity's extinction might even be characterised as ethically wrong.
4. Someone who believes that extinction would be bad both for people alive today and because it would destroy the immense value humanity could create — and who therefore tries to prevent a consensual, designed extinction (Flawless Mass Sterilisation) — exercises a degree of paternalism: preventing an extinction that people have explicitly requested, in defiance of their will, is precisely what paternalism looks like. Accordingly, intervening in an extinction that would arise from people's own choices may be ethically problematic for those who place value on human autonomy.
5. Some people regard humanity's extinction positively on the grounds that it would end the harms humans inflict upon animals and nature. But extinction may also be considered positive for humans themselves, directly, on certain views. Migration to Digital Heavens can be read as one such example: if persons remain the same individuals after migration in the digital world, extinction is positive under the narrow person-affecting view. If persons lose their identity continuity after migration and come to exist as different individuals, extinction may also be regarded positively under some interpretations of the wide person-affecting view, since each of the newly existing persons will be better off. For someone who adopts a particular interpretation of the non-person-affecting view, extinction may be regarded as positive in terms of overall welfare; both Migration to Digital Heavens and A Bad Future fall within this scope. Counterfactually, for instance, in the A Bad Future scenario humanity's extinction means that present people and future generations will not suffer enormously, making extinction preferable to the continuation of existence. Given the complexity we face, it becomes important for existential catastrophe researchers to state their views transparently, since why they do or do not regard any given extinction scenario as important depends largely on the premises underlying those views.
6. How existential catastrophe is defined is likewise not independent of the views one holds. In Planned Hibernation, since humanity will not exist as it does now and some loss of value is involved, those who count Planned Hibernation as an "extinction" (even if not permanent) will classify this scenario as an existential catastrophe; but since the extinction is reversible, others may not count it as extinction at all, placing it outside the category of existential catastrophe. Indeed, one might also ask where consent should be situated within the definition of existential catastrophe: for those who believe existential catastrophes can only obtain in contexts where people have not consented, Planned Hibernation does not constitute one. The way existential catastrophe is defined and classified is therefore not a neutral or technical operation; it is inextricably bound to the values one holds.
7. Legitimacy and consent can be somewhat linked together, yet they are independent of each other. War Among Democratic Powers illustrates this. In one sense, legitimacy can be invoked: people elected their governments. Yet governments may forfeit their legitimacy during the extinction process. Independently of legitimacy, people may not have consented to their governments' war policy.
8. Someone who regards extinction's intrinsic value as bad may — if they believe this value can be combined with the value of the extinction process — come to see a particular extinction scenario as not bad on the whole. Equally, someone who regards extinction as intrinsically good may — if they believe this value can be combined with the value of the extinction process — come to see a particular extinction scenario as not good on the whole. Discussion in the existential catastrophe literature has been built primarily around extinction's intrinsic value; yet the value of the extinction process is decisive for the total value of an extinction scenario. In other words, as argued at the outset, the value of the extinction process can radically alter the total value of an extinction scenario.