In Defence of Fanaticism is a Global Priorities Institute Working Paper by Hayden Wilkinson. This post is part of my sequence of GPI Working Paper summaries.

Hilary Greaves and William MacAskill think objections to fanaticism are among the strongest counterarguments to strong longtermism. Such objections also underpin some of the strongest counterarguments to expected value theory. Thus, contemplating fanaticism is critical for comparing neartermist and longtermist causes. One of Greaves and MacAskill’s responses to this counterargument cites Hayden Wilkinson’s In Defence of Fanaticism, suggesting perhaps we should be fanatical on balance.

Here I’ve done my best to summarize Wilkinson’s argument, making it more easily accessible while sacrificing as little argumentative strength as possible.

Introduction

Dyson’s Wager (modified for brevity)

Say you have $2,000 and must choose to donate it to either a charity that will certainly prevent one death from malaria or a charity that will research ‘positronium,’ which has a nonzero probability of bringing astronomically many blissful lives into existence in the far future. Expected value theory suggests you should give the $2,000 to the speculative positronium research. That conclusion is fanatical.

Fanaticism Definition

• Fanaticism: For any tiny (finite) probability ∊ > 0, and for any finite value v, there is some finite V large enough that L_risky is better than L_safe. 
• L_risky: Lottery with value V and probability ∊ > 0; value 0 otherwise
• L_safe: Lottery with value v and probability 1

Like positronium research, L_risky offers a slim probability (∊) of astronomical value (V), while, like the malaria charity, L_safe offers a modest value (v) with certainty.

Previous justifications for fanatical verdicts relied on expected value theory. However, Wilkinson thinks there is good reason to accept fanaticism, even if we reject expected value theory. Hence, Wilkinson argues that denying fanaticism, as defined above, has implausible consequences—without relying on expected value theory. Wilkinson assumes totalism: we prefer outcomes with a higher total value (as opposed to, for example, average value). The Egyptology and Indology Objections (discussed later) rely on totalism.

Intuitions against fanaticism

Fanaticism is so counterintuitive it must be false.

It seems highly implausible that one should give up a guaranteed good payoff for a tiny chance of something better, no matter how tiny the chance.

However, intuitions about probabilities are often misguided and fall prey to various fallacies^[1]. Additionally, intuitive decision-making often ignores probabilities^[2], radically over- or under-estimates probabilities^[3], and treats low probabilities as having probability 0 (Wilkinson includes multiple examples of this in different contexts^[4]). For instance, one study found jurors are just as likely to convict a defendant based on fingerprint evidence if that evidence has a 1 in 100 probability of being a false positive as if the probability were 1 in 1,000 or even 1 in 1 million^[5].

Thus, intuitions about low probabilities lead us to foolishly ignore probabilities roughly 1% or lower. Our intuitive judgments against fanaticism may be similarly foolish, which warrants at least considering the case in favor of it.

A continuum argument

Consider the following two lotteries:

• L₀: value 1 (e.g., one life is saved) with probability 1 (certainty)
• L₁: value 10¹⁰ (e.g., vastly more lives are saved) with probability 0.999999 (near-certainty); value 0 otherwise

Intuitively, L₁ seems better. But now consider L₂, which has a slightly lower probability of success but, if successful, saves many more lives.

• L₂: value${10}^{{10}^{10}}$ with probability 0.999999²; value 0 otherwise

This seems better than L₁. We could continue with L₃, L₄, and so on until some L_n, such that 0.999999ⁿ is less than ∊, for any arbitrarily small ∊.

Intuition suggests that vastly increasing the payoff can compensate for slightly decreasing the probability, meaning each lottery in this sequence is better than the last, making the final lottery better than the first. But, the final lottery’s probability of any positive payoff is less than ∊. So we have Fanaticism.

Transitivity and Minimal Tradeoffs Definitions

This continuum argument rests on two intuitive principles: 

1. Transitivity: If L_a ≥ L_b and L_b ≥ L_c, then L_a ≥ L_c.
   1. That is, if a lottery (La) is at least as good as a second lottery (Lb), and the second lottery (Lb) is at least as good as a third lottery (Lc), then the first lottery (La) is at least as good as the third lottery (Lc).
2. Minimal Tradeoffs: We can make tradeoffs between probability and value. For instance, we can always compensate for a slight decrease in the probability of success with a vastly greater payoff. (This is highly simplified and deformalized, so if you disagree with this principle or want more detail, please see the section on “Minimal Tradeoffs” on page 13).

Given the continuum argument, to reject fanaticism, you must reject one of these two principles.

A dilemma for the unfanatical

To reject fanaticism, you must also reject Scale Independence (as defined below) or allow your lottery comparisons to be absurdly sensitive to tiny changes.

Scale Independence Definition

• Scale Independence: For any lotteries L_a and L_b, if L_a ≥ L_b, then k · L_a ≥ k · L_b for any positive, real k.
  • That is, if L_a is at least as good as L_b, then after multiplying the value of both by the same factor (k), L_a is still at least as good as L_b.

Scale Independence seems highly plausible. After all, if you’re multiplying the values of both lotteries by k, why should their value relative to each other change?

For fanaticism to be false, there must be some probability ∊ > 0 and value v that makes L_risky no better than L_safe, no matter how big V is.

• L_risky: value V with probability ∊; 0 otherwise
• L_safe: value v with probability 1

Thus, to reject fanaticism, you must alter your comparisons of lotteries based on their scale—i.e., discount larger values (V) beyond the extent that their probability (∊) is lower^[6]. This violates Scale Independence.

Absurd sensitivity to tiny changes

You can avoid this violation by asserting that a guaranteed value (v), no matter how small, is always better than a greater value (V) with probability ∊. However, that assertion means there must be a successive pair of probabilities (p_i and p_i+1) between 1 and ∊ such that no value at p_i (no matter how large) is better than any value at p_i+1 (no matter how small). If there were no such pair, we could create a sequence similar to that in the continuum argument.

Since p_i and p_i+1 can be arbitrarily close together, there could be astronomical value at p_i , and miniscule value at p_i+1, and the latter would still be better. Thus, rejecting fanaticism, our evaluations of lotteries become absurdly sensitive to tiny changes in probability, which is both intuitively implausible and impractical for decision-making.

Egyptology and Indology

Wilkinson presents two objections in this section. As presented in his paper, these objections rely on totalism (that we prefer outcomes with a higher total value).

The Egyptology Objection

The Egyptology Objection is that denying fanaticism makes your moral decisions depend on events that aren't altered by your choice, including those in distant galaxies or ancient Egypt.

Background Independence Definition

• Background Independence: For any lotteries, L_a and L_b, and any outcome Ơ, 
  if L_a ≥ L_b, then L_a + Ơ ≥ L_b + Ơ
  • That is, if L_a is at least as good as L_b, then after adding the constant value of an outcome (Ơ) to both lotteries, L_a is still at least as good as L_b.

Some rejections of fanaticism^[7] violate Background Independence and thus fall prey to the Egyptology Objection. 

For instance, say Ơ is an unaffected “background outcome” that occurred in ancient Egypt. If we violate Background Independence, our choice between L_a and L_b might change based on Ơ. Thus, we fall prey to the Egyptology objection if we violate Background Independence.

The Less Severe Egyptology Objection

But even rejections of fanaticism that don’t violate Background Independence give rise to a less severe Egyptology Objection. Wilkinson uses statistical distributions^[8] to show how, if you are uncertain about an event in ancient Egypt (B), rejecting fanaticism can lead to situations where L_risky + B is considered better^[9] than L_safe + B, even though L_risky is not better than L_safe, again making our moral decisions depend on unaffected events, such as those in ancient Egypt.

The Indology Objection

Take the same L_risky + B and L_safe + B  from above, where the former is considered better^[9], yet L_risky is not better than L_safe. (For fanaticism to be false, such lotteries must exist). But let’s change B to represent a very uncertain value of an event in the ancient Indus Valley.

You could research Indology for years and pin down B’s real value as b, with certainty. If so, instead of choosing between L_risky + B and L_safe + B, you’d choose between
L_risky + b and L_safe + b.

If you accept Background Independence, you’d make the same choice between L_risky + b and L_safe + b as you’d make between L_risky and L_safe, so no matter what b you find, you’d make the same decision.

However, even though L_risky is not better than L_safe, you risk making the wrong decision using L_risky + B and L_safe + B, because, if you reject fanaticism, the uncertainty makes the former considered better^[9]. Thus, you should perhaps spend years finding b—even though, if you accept Background Independence, you will make the same decision between 
L_risky + b and L_safe + b as you’d make between L_risky and L_safe—no matter the value of b! This seems more absurd than accepting fanaticism.

Conclusion

To recap, to deny Fanaticism…

1. We must deny either Transitivity or Minimal Tradeoffs and accept the counterintuitive verdicts that follow. 
2. We must either violate Scale Independence or become absurdly sensitive to tiny differences in probability and value.
3. We must accept a less severe version of the Egyptology Objection: in some cases, morally correct judgments depend on our beliefs about far-off, unaffected events, such as those in ancient Egypt.
4. We either must accept the severe version of the Egyptology Objection by denying Background Independence, or face the Indology Objection: we sometimes ought to make decisions that we know we would reject if we learned more, no matter what we might learn.

Hence, rejecting fanaticism, as intuitive as it may initially feel, has highly unintuitive ramifications. The cure is worse than the disease.

We should accept that it is better to produce some tiny probability of infinite moral gain (or arbitrarily high gain), no matter how tiny the probability, than it is to produce some modest finite gain with certainty.

Accepting fanaticism also removes some of the counterarguments to expected value theory and its implications (which, for example, arguably includes strong longtermism).

1. ^{^}

   Wilkinson lists "the Conjunction Fallacy (Tversky & Kahneman 1983), the Gambler’s Fallacy (Chen et al. 2016), the Hot Hand Fallacy (Gilovich et al. 1985), and the Base Rate Fallacy (Kahneman & Tversky 1982)."

2. ^{^}

   (Baron et al. 1993; Gurmankin & Baron 2005).

3. ^{^}

   "We intuitively overestimate some probabilities due to availability bias (Tversky & Kahneman 1974), and underestimate others out of indefensible optimism (Hanoch et al. 2019)."

^{^}

"When presented with a medical operation that posed a 1% chance of permanent harm, many respondents considered it no worse than an operation with no risk at all (Gurmankin & Baron 2005). And in yet another context, subjects were unwilling to pay any money at all to insure against a 1% chance of catastrophic loss (McClelland et al. 1993)."

^{^}

(Garrett et al. 2018).

^{^}

See the bottom of page 16 and page 17 for a much more formal line of reasoning as to why this is true.

^{^}

"One such proposal is expected utility theory with a utility function that is concave and/or bounded (e.g., Arrow 1971). As Beckstead and Thomas (n.d.: 15-16) point out, this results in comparisons of lotteries being strangely dependent on events that are unaltered in every outcome and indeed some irrelevant to the comparison."

^{^}

The math and full line of reasoning is on pages 23 to 27.

^{^}

I mean 'considered better' by Stochastic Dominance, which "says that if two lotteries have exactly the same probabilities of exactly the same (or equally good) outcomes, then they are equally good; and if you improve an outcome in either lottery, keeping the probabilities the same, then you improve that lottery. And that’s hard to deny!" (See page 10 for the formal definition).

Show all footnotes