Comment Permalink

Hi there :) I very much sense that a conversation with me last weekend at EAGxVirtual is causally connected to this post, so I thought I'd share some quick thoughts!

First, I apologize if our conversation led you to feel more uncertain about your career in a way that negatively affected your well-being. I know how subjectively "annoying" it can be to question your priorities.

Then, I think your post raises three different potential problems with reducing x-risks (the three of which I know we've talked about) worth disentangling:

1. You mention suffering-focused ethics and reasons to believe these advise against x-risk reduction.

2. You also mention the problem of cluelessness, which I think is worth dissociating. I think motivations for cluelessness vis-a-vis the sign of x-risk reduction are very much orthogonal to suffering-focused ethics. I don't think someone who rejects suffering-focused ethics should be less clueless. In fact, one can argue that they should be more agnostic about this while those endorsing suffering-focused ethics might have good reasons to at least weakly believe x-risk reduction hurts their values, for the "more beings -> more suffering" reason you mention. (I'm however quite uncertain about this and sympathetic to the idea that those endorsing suffering-focused ethics should maybe be just as clueless.)

3. Finally, objections to the 'time of perils' hypothesis can also be reasons to doubt the value of x-risk reduction (Thorstad 2023), but for very different reasons. It's purely a question of what is the most "impactable" between x-risks (and maybe other longterm causes) and shorter-term causes, rather than a question of whether x-risk reduction does more good than harm to begin with (like with 1 and 2).

Discussions regarding the questions raised by these three points seem healthy, indeed.

Hey Jim,

Thanks for chiming in, and you're spot on: our chat at EAGxVirtual definitely got the gears turning! No worries at all about the existential crisis, I see it as part of the journey (and I actively requested it) :) I actually think these moments of doubt are important to progress in my mission in EA (similarly laid out by JWS in his post). I usually don't do this, but the post was a good way for me to vent and help me process some of the ideas + get feedback.

You've broken down my jumbled thoughts really well. It is helpful to see the three points la... (read more)

See in context

Reducing x-risk might be actively harmful

by MountainPath

Nov 18 20242 min read 9

22

Cause prioritizationExistential riskPhilosophyCluelessnessMoral uncertaintySuffering-focused ethics

Frontpage

Great. Another crucial consideration I missed. I was convinced that working on reducing the existential risk for humanity should be a global priority.

Upholding our potential and ensuring that we can create a truly just future seems so wonderful.

Well, recently I was introduced to the idea that this might actually not be the case.

The argument is rooted in suffering-focused ethics and the concept of complex cluelessness. If we step back and think critically, what predicts suffering more than the mere existence of sentient beings—humans in particular? Our history is littered with pain and exploitation: factory farming, systemic injustices, and wars, to name just a few examples. Even with our best intentions, humanity has perpetuated vast amounts of suffering.

So here’s the kicker: what if reducing existential risks isn’t inherently good? What if keeping humanity alive and flourishing actually risks spreading suffering further and faster—through advanced technologies, colonization of space, or systems we can’t yet foresee? And what if our very efforts to safeguard the future have unintended consequences that exacerbate suffering in ways we can't predict?

I was also struck by the critique of the “time of perils” assumption. The idea that now is a uniquely critical juncture in history, where we can reduce existential risks significantly and set humanity on a stable trajectory, sounds compelling. But the evidence supporting this claim is shaky at best. Why should we believe that reducing risks now will have lasting, positive effects over millennia—or even that we can reduce these risks at all, given the vast uncertainties?

This isn’t to say existential risk reduction is definitively bad—just that our confidence in it being good might be misplaced. A truly suffering-focused view might lean toward seeing existential risk reduction as neutral at best, and possibly harmful at worst.

It’s humbling, honestly. And frustrating. Because I want to believe that by focusing on existential risks, we’re steering humanity toward a better future. But the more I dig, the more I realize how little we truly understand about the long-term consequences of our actions.

So, what now? I’m not sure.

I am sick of missing crucial considerations. All I want to do is to make a positive impact. But no. Radical uncertainty it is.

I know that this will potentially cost me hundreds of hours to fully think through. It is going to cost a lot of energy if I pursue with this.

Right now I am just considering to pursue earning to give instead and donate a large chunk of my money to different worldviews and cause areas.

Would love to get your thoughts.

22 Reactions

Comments9

Sorted by

New & upvoted

Click to highlight new comments since: Today at 12:22 PM

Jim BuhlerNov 18 20247

MountainPathDec 18 20242

Hey Jim,

You've broken down my jumbled thoughts really well. It is helpful to see the three points laid out like that. They each deserve their own space, and I appreciate you giving them that.

I think you're right that cluelessness is kind of its own beast, regardless of where one stands on suffering-focused ethics.

Anyway, thanks for the thoughtful response and for helping me untangle my thoughts.

JWS 🔸Nov 19 20244

Thanks for sharing your thoughts. I'll respond in turn to what I think are the two main parts of it, since as you said this post seems to be a combination of suffering-focused ethics and complex cluelessness.

On Suffering-focused Ethics: To be honest, I've never seen the intuitive pull of suffering-focused theories, especially since my read of your paragraphs here seems to tend towards a lexical view where the amount of suffering is the only thing that matters for moral consideration.^[1]

Such a moral view doesn't really make sense to me, to be honest, so I'm not particularly concerned by it, though of course everyone has different moral intuitions so YMMV.^[2] Even if you're convinced of SFE though, the question is how best to reduce suffering which hits into the clueless considerations you point out.

On complex cluelessness: On this side, I think you're right about a lot of things, but that's a good thing not a bad one!

I think you're right about the 'time of perils' assumption, but you really should increase your scepticism of any intervention which claims to have "lasting, positive effects over millennia" since we can't get the feedback on the millennia long impact of our interventions.
You are right that radical uncertainty is humbling, and it can be frustrating, but it is also the state that everyone is in, and there's no use beating yourself up for the default state that everyone is in.
You can only decide how to steer humanity toward a better future with the knowledge and tools that you have now. It could be something very small, and doesn't have to involve you spending hundreds of hours trying to solve the problems of cluelessness.

I'd argue that reckoning with the radical uncertainty should point towards moral humility and pluralism, but I would say that since that's the perspective in my wheelhouse! I also hinted at such considerations in my last post about a Gradient-Descent approach to doing good, which might be a more cluessness-friendly attitude to take.

^{^}
You seem to be asking e.g. "will lowering existential risk increase the expected amount of future suffering" instead of "will lowering existential risk increase the amount of total preferences satisfied/non frustrated" for example.
^{^}
To clairfy, this sentence specifically referred to lexical suffering views, not all forms of SFE that are less strong in their formulation

aleneNov 19 20244

Thank you so much for posting this. This is something I worry about a lot but I’m terrible at explaining it. The way you explain it makes much more sense. Thank you. ❤️

Charlie_GuthmannNov 22 20243

It actually goes more giga brain than this - since aliens are in the picture, or even maybe life can re- evolve on our planet to interstellar intelligence. You might be interested to talk to @Arepo , he's a crucial considerer. I'd especially recommend his post "A proposed hierarchy of longtermist concepts".

shameless self plugs that also might lead you to some related readings (i'm narcissistic enough to somehow remember almost all my comments on the subject)

https://forum.effectivealtruism.org/posts/zDJpYMtewowKXkHyG/alien-counterfactuals
https://forum.effectivealtruism.org/posts/zLi3MbMCTtCv9ttyz/formalizing-extinction-risk-reduction-vs-longtermism
https://forum.effectivealtruism.org/posts/zuQeTaqrjveSiSMYo/?commentId=7s2vrDuxonBqoGrou
https://forum.effectivealtruism.org/posts/Pnhjveit55DoqBSAF/?commentId=wTkFestNWNorB5mG4
https://forum.effectivealtruism.org/posts/YnBwoNNqe6knBJH8p/?commentId=HPsgdWEbdEZH3WN6j
https://forum.effectivealtruism.org/posts/WebLP36BYDbMAKoa5/?commentId=cJdqyAAzwrL74x2mG

Either way, don't be down on yourself. I know exactly how you feel. There is way too much stuff to know. The fact that you are writing this and reflecting means you are one of the best humans alive right now, regardless of if x-risk is important or not. Keep up the good work.

MountainPathDec 18 20242

First of all, thank you for engaging with this post! Your kind words and thoughtful pushback mean a lot to me. Over the last couple of weeks, I have been taking a break from many things to help me regain motivation and courage (hence, I am only starting to reply now). Fortunately, I am feeling much better again and ready to tackle the problem I am facing. Thank you once again, and I hope you have a great day!

jo3Nov 18 20241

If you think the expected value is negative regardless of what you can do or move, you should of course become the existential risk^[1].

But, actually estimating whether humanity will be net-negative requires you to know what you value, which is something you're probably fuzzy about. We lack the technology so far to extract terminal goals from people, which you want to have before taking any irrevocable actions.

^{^}
Future-you might resent past-you for publicly doubting the merits of humanity, since I reckon you'd want to be a secret existential risk.

titotalNov 19 20244

I would like to humbly suggest that people not engage in active plots to destroy humanity based on their personal back of the envelope moral calculations.

I think that the other 8 billion of us might want a say, and I'd guess we'd not be particularly happy if we got collectively eviscerated because some random person made a math error.

Charlie_GuthmannNov 22 20246

So we get to use cold hard rationality to tell most people that the stuff they are doing is relatively worthless compared to x-risk reduction, but when that same rationality argues that x-risk reduction is actually incredibly high variance and may very well be harming trillions of the people in the future we get to be humanists ?