Without getting into the specifics, this donation opportunity only made sense above a certain level.
I did donate, and I'm glad I did, but I think it's valuable to note the inconsistencies of human psychology that appear even in communities that pride themselves on rationality.
Thinking back to my first big (well, big for me) donation and an unusual series of thoughts I had. Sharing in case anyone has experienced the same. --
I've practiced frugality to a significant extent, largely because I want to donate/salary sacrifice so much of my income to effective charities. As a result, whenever I'm spending a significant amount of money on anything, alarm bells go off.
I was surprised that alarm bells went off when donating. I had thought through where I donated extensively, and charity was the reason why I wanted to save money in the first place. But I still felt stressed because I was "spending money."
This is such a clear example of missing the forest for the trees. I think we need to be careful about instrumental values/rules and making sure they don't become absolute limitations that decrease overall impact.
I DO believe that preventing extinction is valuable and there are some projects promising enough to prioritize over work in other cause areas.
I DON'T believe that longtermism dominates all other cause areas in expectation, or that the argument you have made in favor of longtermism is convincing.
I agree that rejecting both A and B would not make sense, if you are informed of both. I think the author is wrong to treat A and B as separate decisions, when the agent knows about both in advance.
Knowing that you have the option to take bet B later fundamentally changes the considerations for bet A. As a result, we are not making 2 independent decisions (A: yes or no, and B: yes or no). We are making 4 (A, B, BOTH, NEITHER).
When considering that list, we can see that BOTH is strictly greater than NEITHER in all worlds and rule out NEITHER. We are left with A, B, and BOTH to choose from, all of which might make sense depending on the agent's choices.
At no point did I need to employ NARROW, PLAN, or SEQUENCE. I didn't even consider the probability of H, let alone whether that probability is sharp. I just considered the available options differently.
EDIT: I think this is close in effect to SEQUENCE. As a result, there might be the objection, "What if, of the 4 options, you choose B? Could you change your mind after rejecting A and then reject B as well?" To this I would say that a rational actor does not change their mind without new information. They would only choose B if they believe B > BOTH > NEITHER. Any rational actor who believes B > NEITHER would end up betting B. They would never bet NEITHER.
What might have muddied the waters:
I separately considered how I might deal with these probabilities separately, WITHOUT knowledge that one will follow the other. This is a distinct problem from the original dilemma. However, I think it's the only situation where a rational actor who follows UNSHARP might behave differently.
Without knowledge beforehand, if you hold UNSHARP, the following can happen:
You receive A, evaluate it, conclude it's optional due to UNSHARP probabilities, and reject it. Then, you are offered B, evaluate it, conclude it's optional, and reject it. You look back and think "I wish I would have known beforehand. I would have taken advantage of the arbitrage. Oh well. I guess rational actors with less information make worse decisions."
I think it is rational for an actor to hold unsharp probabilities for some hypotheses.[1] I think it's rational to not engage in sports gambling when no arbitrage exists. My initial example was designed to connect the two.
I haven't made my mind up on whether it's necessary to hold unsharp probabilities in theory but I'm much more confident in practice.
When you see a new opportunity that you know very little about that might be massively valuable, using your minimally informed baseline model to direct action seems irresponsible. Upon further investigation, everything regresses to the mean.
In the sports gambling example I gave, you should reject unless you see arbitrage because ~all available information is priced in. In the case of impact, new opportunities look more exciting than reality due to (e.g.) selection effects and stable equilibria.
This discussion of whether or not we should have unsharp probabilities is beside the point. My argument is about whether we can have unsharp probabilities without sacrificing rationality. I believe we can.
This seems to me to be another instance of the 1% fallacy (or the 0.1% fallacy, or the 10^-18 fallacy).
In your post, you talk about being skeptical of arguments where infinities cancel. I would argue that uncanceled infinities are generally a sign of a model being applied beyond its range of efficacy.
If you start off with a rough model and extrapolate it out to get a big enough number, all you have to do is come up with a set of conditions under which that number could plausibly be achieved, no matter how improbable. Then, say "Zero isn't a probability, and you don't have enough evidence to show that the probability cancels my big number." from the start and suddenly you have a large expected value.
However, when you refine the model (considering higher order effects, more thoroughly treating counterfactuals, turning exponential curves into more accurate s-curves), the big number drops out.
The base rate for the infinite is 0. As a result, I am much more skeptical of models where infinities don't cancel.
Could you please provide any concrete grounding for the probability of counterfactually shifting from extinction to a vast future (not delaying extinction temporarily) that is not based on a very small subjectively "conservative" probability?
I agree that accepting both bets is consistent with a sharp probability at 50%, though I'm just trying to give an example of a case where I would have an unsharp probability range where I would reject both bets in isolation but take them when they arrive together.
I don't employ any of the 3 strategies. My argument is that you don't need a fancy strategy because, in the example, you know that bet B is coming when you're asked about bet A. I think it's reasonable for a rational actor to reject bet A and reject bet B if the two are presented separately but accept them both if they are presented together. My example is intended to demonstrate that. A rational actor doesn't need NARROW, PLAN, or SEQUENCE. They need to consider the future: "Bet B is coming, so there's an arbitrage opportunity regardless of the probability." The article seems to disagree, treating every action in isolation and requiring that we make the right decision without global thinking.
My recommendation for portfolios is not an argument for, but an implication of, unsharp probabilities. A lot of cause prioritization is about the core philosophical positions you hold underpinning it. If you have a sharp probability, you might be comfortable investing all in one cause. If you have an unsharp one, you might not be convinced that investing in any one cause is net positive. However, you might find a combination of causes that seems robustly better than no action.
For example, you might be concerned about climate policy's constraints on growth as well as growth's effect on the climate. If you believe that the second order effects of investing in growth on the climate are smaller than the direct benefits of donating to climate policy (and vice versa), it is strictly better to donate to both in some combination than to do nothing. Someone with a sharp probability might be comfortable donating to just one in a way someone with unsharp probabilities would not.
As a result, portfolios are better (i.e. are more often optimal) in a world where UNSHARP is true.
I see and agree with your point about marginal returns. Depending on how strong that effect is, portfolios are also good in a world with sharp probabilities only.
Related to/expandng upon @huw 's objection, you not only need to show that longtermist organizations make a difference in the probability of extinction, but that it lasts for a long time. Even if we assume that your $10 Billion would reduce the probability of extinction in 2026, we could counterfactually face extinction in 2027, or 2028, or... etc.
In effect, your calculation assumes that your reduction in existential risk lasts forever. This is a classic example of the "1% fallacy," where the tiny probability is really hard to achieve.
I would say any intervention that moves us from the path of extinction to the path of never going extinct ever for the next 10^(big number) years is exceptionally valuable. But that's a different category of interventions than simply "reducing the probability of extinction."
Even if you believe that the value of utility in the future is equal to the value of utility in the present, you should always discount by a minimum of your probability of extinction per unit time.
This has some really counterintuitive implications.
Let's consider the value of preventing extinction entirely for a year, which would be an absurdly large claim for any organization to make by orders of magnitude.
Maybe the probability of extinction per year is really high. That means that reducing it to zero is a massive change in our odds of survival (yay!). It also means that our probability that everyone will die in the next few years anyway is also large (darn!).
Maybe the probability of extinction per year is really low. That means reducing it to zero is unlikely to actually do anything (darn!). But if it does, we'll probably stick around for a long time (yay!).
It turns out these effects exactly cancel out mathematically. Using an annual probability of extinction x, the value of preventing extinction is:
That means that preventing extinction for 1 year is equal to 1 year of utility. Granted, 1 year of utility is a lot, but it is very different from 10^40 or 10^58 lives.
Using the numbers you used in your post,[1] each dollar reduces the probability of extinction by one in a hundred trillion. We should multiply that by the value of one year of utility. I'll use 10 Billion DALYs (representing 1 year of life for ~10B people) for simplicity, which results in a cost-effectiveness of $10,000 per DALY. That's orders of magnitude worse than Global Health, your "worst" EA Cause Area.
Caveats/objections:
Accounting for population growth/improving livelihoods, the value of each year does increase over time, but unless you assume unconstrained exponential growth and a very small probability of extinction every year, that effect is relatively mild compared to the discounting and that won't shift the endline result much.
You could claim that the extinction probabilities are heavily correlated so by preventing extinction in one year you're also preventing it every year afterwards. I think there's a reasonable argument to be made that probabilities are correlated across short time spans, but I find it much less plausible across long time spans. As a result, this could shift the value up, but I don't think it resolves the underlying issue.
You could claim that the probability of extinction is particularly high this year but will drop off later. However, that requires a very specific view of probabilities that is not argued for here in your post. Even if you believe we are at a "hinge of history," you would need to make the claim that the duration of the hinge is very short and the probability of extinction after the hinge is tiny.
I acknowledge that the numbers in your post are intended to be very conservative. I also have my own reasons to object to your choice of conceptual approach to considering what cause is the best cause area. However, I am considering your numbers as listed for the sake of argument.
This seems to be a breakdown with the consideration of actions in complete isolation rather than with having coarse probability estimates.
At least in practice, there's a clear difference between considering bet A in isolation and considering bet A when you know bet B is coming. If you told me about a sports game between the Snofuls and the Fleertis and offered me 2:1 odds on the Snofuls to win, I wouldn't take it. But if you told me you would also give me 2:1 odds on the Fleertis to win, I would take both bets, guaranteeing a profit.
As a rational actor with no useful information, I have a very broad range of potential probabilities for this bet, and it is permissible to do neither bet in isolation. However, when we consider our options simultaneously, that changes the calculus.
To apply this to altruistic action, there might be actions that we are uncertain about in isolation, but we are willing to pursue as a part of a portfolio approach.
I think you've accurately identified a real tension here, and this connects with a fundamental critique of EA as a movement, which is that it is too often focused on measurable outcomes rather than systemic change. I tend to agree that this critique has teeth and applies to the way EA is often practiced.
I do want to highlight that Global Health work is not inherently a temporary fix. Global Health work frequently can (and should) focus on improving existing health systems, not just having a temporary impact. By addressing the root cause, you can make a more permanent difference (and be more cost-effective while you're at it)
So why are more EAs focused on Global Health instead of Global Development relative to your expectations? In my opinion, two major reasons are
Some people are likely overly focused on measurable outcomes over systemic change.
Some types of Global Health work is more systemic than you give it credit for.
Without getting into the specifics, this donation opportunity only made sense above a certain level.
I did donate, and I'm glad I did, but I think it's valuable to note the inconsistencies of human psychology that appear even in communities that pride themselves on rationality.
Thinking back to my first big (well, big for me) donation and an unusual series of thoughts I had. Sharing in case anyone has experienced the same.
--
I've practiced frugality to a significant extent, largely because I want to donate/salary sacrifice so much of my income to effective charities. As a result, whenever I'm spending a significant amount of money on anything, alarm bells go off.
I was surprised that alarm bells went off when donating. I had thought through where I donated extensively, and charity was the reason why I wanted to save money in the first place. But I still felt stressed because I was "spending money."
This is such a clear example of missing the forest for the trees. I think we need to be careful about instrumental values/rules and making sure they don't become absolute limitations that decrease overall impact.
At least on mobile, my prior feed customization options to reduce certain topics has been removed, and I have been unable to reinstate my preferences.
I figured this was important to keep separate:
I DO believe that preventing extinction is valuable and there are some projects promising enough to prioritize over work in other cause areas.
I DON'T believe that longtermism dominates all other cause areas in expectation, or that the argument you have made in favor of longtermism is convincing.
I agree that rejecting both A and B would not make sense, if you are informed of both. I think the author is wrong to treat A and B as separate decisions, when the agent knows about both in advance.
Knowing that you have the option to take bet B later fundamentally changes the considerations for bet A. As a result, we are not making 2 independent decisions (A: yes or no, and B: yes or no). We are making 4 (A, B, BOTH, NEITHER).
When considering that list, we can see that BOTH is strictly greater than NEITHER in all worlds and rule out NEITHER. We are left with A, B, and BOTH to choose from, all of which might make sense depending on the agent's choices.
At no point did I need to employ NARROW, PLAN, or SEQUENCE. I didn't even consider the probability of H, let alone whether that probability is sharp. I just considered the available options differently.
EDIT: I think this is close in effect to SEQUENCE. As a result, there might be the objection, "What if, of the 4 options, you choose B? Could you change your mind after rejecting A and then reject B as well?" To this I would say that a rational actor does not change their mind without new information. They would only choose B if they believe B > BOTH > NEITHER. Any rational actor who believes B > NEITHER would end up betting B. They would never bet NEITHER.
What might have muddied the waters:
I separately considered how I might deal with these probabilities separately, WITHOUT knowledge that one will follow the other. This is a distinct problem from the original dilemma. However, I think it's the only situation where a rational actor who follows UNSHARP might behave differently.
Without knowledge beforehand, if you hold UNSHARP, the following can happen:
You receive A, evaluate it, conclude it's optional due to UNSHARP probabilities, and reject it. Then, you are offered B, evaluate it, conclude it's optional, and reject it. You look back and think "I wish I would have known beforehand. I would have taken advantage of the arbitrage. Oh well. I guess rational actors with less information make worse decisions."
I think it is rational for an actor to hold unsharp probabilities for some hypotheses.[1] I think it's rational to not engage in sports gambling when no arbitrage exists. My initial example was designed to connect the two.
I haven't made my mind up on whether it's necessary to hold unsharp probabilities in theory but I'm much more confident in practice.
When you see a new opportunity that you know very little about that might be massively valuable, using your minimally informed baseline model to direct action seems irresponsible. Upon further investigation, everything regresses to the mean.
In the sports gambling example I gave, you should reject unless you see arbitrage because ~all available information is priced in. In the case of impact, new opportunities look more exciting than reality due to (e.g.) selection effects and stable equilibria.
This discussion of whether or not we should have unsharp probabilities is beside the point. My argument is about whether we can have unsharp probabilities without sacrificing rationality. I believe we can.
This seems to me to be another instance of the 1% fallacy (or the 0.1% fallacy, or the 10^-18 fallacy).
In your post, you talk about being skeptical of arguments where infinities cancel. I would argue that uncanceled infinities are generally a sign of a model being applied beyond its range of efficacy.
If you start off with a rough model and extrapolate it out to get a big enough number, all you have to do is come up with a set of conditions under which that number could plausibly be achieved, no matter how improbable. Then, say "Zero isn't a probability, and you don't have enough evidence to show that the probability cancels my big number." from the start and suddenly you have a large expected value.
However, when you refine the model (considering higher order effects, more thoroughly treating counterfactuals, turning exponential curves into more accurate s-curves), the big number drops out.
The base rate for the infinite is 0. As a result, I am much more skeptical of models where infinities don't cancel.
Could you please provide any concrete grounding for the probability of counterfactually shifting from extinction to a vast future (not delaying extinction temporarily) that is not based on a very small subjectively "conservative" probability?
I agree that accepting both bets is consistent with a sharp probability at 50%, though I'm just trying to give an example of a case where I would have an unsharp probability range where I would reject both bets in isolation but take them when they arrive together.
I don't employ any of the 3 strategies. My argument is that you don't need a fancy strategy because, in the example, you know that bet B is coming when you're asked about bet A. I think it's reasonable for a rational actor to reject bet A and reject bet B if the two are presented separately but accept them both if they are presented together. My example is intended to demonstrate that. A rational actor doesn't need NARROW, PLAN, or SEQUENCE. They need to consider the future: "Bet B is coming, so there's an arbitrage opportunity regardless of the probability." The article seems to disagree, treating every action in isolation and requiring that we make the right decision without global thinking.
My recommendation for portfolios is not an argument for, but an implication of, unsharp probabilities. A lot of cause prioritization is about the core philosophical positions you hold underpinning it. If you have a sharp probability, you might be comfortable investing all in one cause. If you have an unsharp one, you might not be convinced that investing in any one cause is net positive. However, you might find a combination of causes that seems robustly better than no action.
For example, you might be concerned about climate policy's constraints on growth as well as growth's effect on the climate. If you believe that the second order effects of investing in growth on the climate are smaller than the direct benefits of donating to climate policy (and vice versa), it is strictly better to donate to both in some combination than to do nothing. Someone with a sharp probability might be comfortable donating to just one in a way someone with unsharp probabilities would not.
As a result, portfolios are better (i.e. are more often optimal) in a world where UNSHARP is true.
I see and agree with your point about marginal returns. Depending on how strong that effect is, portfolios are also good in a world with sharp probabilities only.
Related to/expandng upon @huw 's objection, you not only need to show that longtermist organizations make a difference in the probability of extinction, but that it lasts for a long time. Even if we assume that your $10 Billion would reduce the probability of extinction in 2026, we could counterfactually face extinction in 2027, or 2028, or... etc.
In effect, your calculation assumes that your reduction in existential risk lasts forever. This is a classic example of the "1% fallacy," where the tiny probability is really hard to achieve.
I would say any intervention that moves us from the path of extinction to the path of never going extinct ever for the next 10^(big number) years is exceptionally valuable. But that's a different category of interventions than simply "reducing the probability of extinction."
Even if you believe that the value of utility in the future is equal to the value of utility in the present, you should always discount by a minimum of your probability of extinction per unit time.
This has some really counterintuitive implications.
Let's consider the value of preventing extinction entirely for a year, which would be an absurdly large claim for any organization to make by orders of magnitude.
Maybe the probability of extinction per year is really high. That means that reducing it to zero is a massive change in our odds of survival (yay!). It also means that our probability that everyone will die in the next few years anyway is also large (darn!).
Maybe the probability of extinction per year is really low. That means reducing it to zero is unlikely to actually do anything (darn!). But if it does, we'll probably stick around for a long time (yay!).
It turns out these effects exactly cancel out mathematically. Using an annual probability of extinction x, the value of preventing extinction is:
That means that preventing extinction for 1 year is equal to 1 year of utility. Granted, 1 year of utility is a lot, but it is very different from 10^40 or 10^58 lives.
Using the numbers you used in your post,[1] each dollar reduces the probability of extinction by one in a hundred trillion. We should multiply that by the value of one year of utility. I'll use 10 Billion DALYs (representing 1 year of life for ~10B people) for simplicity, which results in a cost-effectiveness of $10,000 per DALY. That's orders of magnitude worse than Global Health, your "worst" EA Cause Area.
Caveats/objections:
Accounting for population growth/improving livelihoods, the value of each year does increase over time, but unless you assume unconstrained exponential growth and a very small probability of extinction every year, that effect is relatively mild compared to the discounting and that won't shift the endline result much.
You could claim that the extinction probabilities are heavily correlated so by preventing extinction in one year you're also preventing it every year afterwards. I think there's a reasonable argument to be made that probabilities are correlated across short time spans, but I find it much less plausible across long time spans. As a result, this could shift the value up, but I don't think it resolves the underlying issue.
You could claim that the probability of extinction is particularly high this year but will drop off later. However, that requires a very specific view of probabilities that is not argued for here in your post. Even if you believe we are at a "hinge of history," you would need to make the claim that the duration of the hinge is very short and the probability of extinction after the hinge is tiny.
I acknowledge that the numbers in your post are intended to be very conservative. I also have my own reasons to object to your choice of conceptual approach to considering what cause is the best cause area. However, I am considering your numbers as listed for the sake of argument.
This seems to be a breakdown with the consideration of actions in complete isolation rather than with having coarse probability estimates.
At least in practice, there's a clear difference between considering bet A in isolation and considering bet A when you know bet B is coming. If you told me about a sports game between the Snofuls and the Fleertis and offered me 2:1 odds on the Snofuls to win, I wouldn't take it. But if you told me you would also give me 2:1 odds on the Fleertis to win, I would take both bets, guaranteeing a profit.
As a rational actor with no useful information, I have a very broad range of potential probabilities for this bet, and it is permissible to do neither bet in isolation. However, when we consider our options simultaneously, that changes the calculus.
To apply this to altruistic action, there might be actions that we are uncertain about in isolation, but we are willing to pursue as a part of a portfolio approach.
I think you've accurately identified a real tension here, and this connects with a fundamental critique of EA as a movement, which is that it is too often focused on measurable outcomes rather than systemic change. I tend to agree that this critique has teeth and applies to the way EA is often practiced.
I do want to highlight that Global Health work is not inherently a temporary fix. Global Health work frequently can (and should) focus on improving existing health systems, not just having a temporary impact. By addressing the root cause, you can make a more permanent difference (and be more cost-effective while you're at it)
So why are more EAs focused on Global Health instead of Global Development relative to your expectations? In my opinion, two major reasons are