I'm okay with balancing trade-offs between rigor and reach when the rigor is reasonably high. But it's important to remember that rigor and effectiveness are correlated with each other.
When it comes to longtermism, as in trying to affect the future a thousand the rigor is essentially non-existent, and precisely because of that, I doubt there is any reach either.
Subtitle: Systemic change or high confidence? Pick one.
Folks used to criticize Effective Altruism for being insufficiently open to (presumed) directionally helpful but hard-to-measure efforts to address large-scale problems at their root. While the charge strikes me as misdirected,[1] I certainly agree with the underlying principle—also known as “hits-based giving”—that we should be willing to consider hard-to-quantify longshots (when supported by sufficiently good reasons—reasons that may be much more nebulous and contestable than the results of a randomized controlled trial).
The funny thing is that once Effective Altruists started pursuing this strategy more prominently—investing more in speculative “longtermist” projects like AI safety and fancy conference venues from which they might better influence policymakers—they were met with a chorus of critics lamenting their betrayal of “old-school EA values” (personal asceticism and a strong focus on reliably helping the global poor) in favor of more speculative strategies.
It’s worth noting that these two popular objections are in tension. If you’re unwilling to countenance uncertainty or “speculation” (perhaps because you discount ungrounded expected value estimates as too easily biased by personal interests) then you’re committed to embracing some form of measurability bias: prioritizing rigor over reach. Conversely, if you’re optimistic about the prospects of pursuing contestable “systemic change” to radically improve the world, you can hardly rule out a priori that influencing the trajectory of what is plausibly the most transformative technology ever invented would be a reasonable way to implement that strategy! It’s obviously highly uncertain, but that’s the tradeoff one accepts when pursuing higher potential impact: prioritizing reach over rigor.
Of course, one could always try to argue on the merits that standard EA views on both global poverty and AI safety are empirically mistaken. That’s fine. All I’m saying here is that one had better not endorse higher-level heuristics, applied prior to any engagement with the specific details, that rule out both rigorously-supported global health charities (simply for being insufficiently ambitious) and speculative longtermist longshots (simply for being insufficiently rigorous). After all, if you prioritize rigor, the prima facie case for global health charities is going to be hard to beat. And if you prioritize reach (potential impact), nobody else is even in the same ballpark as the longtermists. Many worry about the potential for bias in longtermists’ judgments, but does anyone seriously expect radical politics to be less subject to bias?
I think it will be difficult to articulate a principled case for thinking that one’s preferred politics is a better bet in expectation for improving the world than both the neartermist and longtermist wings of Effective Altruism. The odds of your individual political actions (voting, protesting, etc.) making a significant difference are comparable to those from contributing to longtermist projects,[2] but the potential benefits are vastly lower, and the odds of misjudgment and proving counterproductive strike me as higher, if anything. (The risks will depend on the precise details of your preferred politics; the more radical your proposed economic overhauls, the more I worry about them proving counterproductive if successfully implemented.)
The upshot: I’d like to see more of the rigor-seeking anti-longtermists taking the side of the neartermist EAs against their speculative socialist critics. And I’d like to see fans of ambitious “systemic change” acknowledge that the longtermist branch of EA is following the right methodology in principle; the disagreement is just about which systemic changes are most promising. It’s worth figuring out where you stand, in principle, on the question of how to navigate rigor-reach tradeoffs: Pick your poison!
EA principles obviously entail support for “systemic changes” if the evidence supports the claim that they have higher expected value than alternative uses of the resources in question. Open Philanthropy (now renamed Coefficient Giving) did fund various criminal justice reform efforts, for example, which was probably a poor use of $200M. EA-funded animal welfare work is arguably “systemic”—e.g. corporate cage-free campaigns and research into cultivated meat that could eventually displace animal agriculture—and seems fairly promising to me.
My sense is that many people pressing the “systemic change” critique were misapplying a procedural gloss to what was really just a first-order political complaint that EAs don’t sufficiently share their enthusiasm for socialism in particular. (A bit like how deontologists misleadingly gloss their substantive anti-aggregative views as procedurally more respectful of “the separateness of persons”. There’s simply no basis for their claiming sole ownership of the procedural virtue—if anything, I’d argue the opposite.)
A recent tweet seemed to assume it was unacceptable for an individual’s longtermist project to have a “one in a million” chance of success. But have you compared the odds of placing a tie-breaking vote in a presidential election? Both have the potential to be extremely good bets, in terms of impartial expected value, despite the low absolute odds. But the potential longtermist payoff is also many, many orders of magnitude greater.
The key issue, I think, is ensuring that the potential benefit isn’t outweighed by the risk of causing immense negative impact. Many have argued that early AI safety concerns had the unfortunate effect of accelerating AI development, for example. But I’d want to see more argument for expecting today’s marginal AI safety efforts to have the same effect: the race is already underway, and would hardly stop just because AI safety efforts do. (I’m open to further argument; just explaining why I’m not moved by the argument from crude induction.)
Good framing!
A framing I had in a draft post was whether, in the ITN framework, to focus mostly on (disclaimer: very rough):
I do think rigor-reach / tractability-scale is the main axis, and that neglectedness is more minor. I think this can lead to unresolvable disagreements on what interventions should be pursued.
On AI safety harms beyond historical induction, there was a recent short post on ways current AI safety efforts could be net-negative.