This is the fourth (and final) post in a series exploring consequentialist cluelessness and its implications for effective altruism:

• The first post describes cluelessness & its relevance to EA; arguing that for many popular EA interventions we don’t have a clue about the intervention’s overall net impact.
• The second post considers a potential reply to concerns about cluelessness.
• The third post examines how tractable cluelessness is – to what extent we can grow more clueful about an intervention through intentional effort?
• This post discusses how we might do good while being clueless to an important extent.

Consider reading the previous posts (1, 2, 3) first.

The last post looked at whether we could grow more clueful by intentional effort. It concluded that, for the foreseeable future, we will probably remain clueless about the long-run impacts of our actions to a meaningful extent, even after taking measures to improve our understanding and foresight.

Given this state of affairs, we should act cautiously when trying to do good. This post outlines a framework for doing good while being clueless, then looks at what this framework implies about current EA cause prioritization.

The following only make sense if you already believe that the far future matters a lot; this argument has been made elegantly elsewhere so we won’t rehash it here.[1]

An analogy: interstellar travel

Consider a spacecraft, journeying out into space. The occupants of the craft are searching for a star system to settle. Promising destination systems are all very far away, and the voyagers don’t have a complete map of how to get to any of them. Indeed, they know very little about the space they will travel through.

To have a good journey, the voyagers will have to successfully steer their ship (both literally & metaphorically). Let's use "steering capacity" as an umbrella term that refers to the capacity needed to have a successful journey.[2] "Steering capacity" can be broken down into the following five attributes:[3]

• The voyagers must have a clear idea of what they are looking for. (Intent)
• The voyagers must be able to reach agreement about where to go. (Coordination)
• The voyagers must be discerning enough to identify promising systems as promising, when they encounter them. Similarly, they must be discerning enough to accurately identify threats & obstacles. (Wisdom)
• Their craft must be powerful enough to reach the destinations they choose. (Capability)
• Because the voyagers travel through unmapped territory, they must be able to see far enough ahead to avoid obstacles they encounter. (Predictive power)

This spacecraft is a useful analogy for thinking about our civilization’s trajectory. Like us, the space voyagers are somewhat clueless – they don’t know quite where they should go (though they can make guesses), and they don’t know how to get there (though they can plot a course and make adjustments along the way).

The five attributes given above – intent, coordination, wisdom, capability, and predictive power – determine how successful the space voyagers will be in arriving at a suitable destination system. These same attributes can also serve as a useful framework for considering which altruistic interventions we should prioritize, given our present situation.  

The basic point

The basic point here is that interventions whose main known effects do not improve our steering capacity (i.e. our intent, wisdom, coordination, capability, and predictive power) are not as important as interventions whose main known effects do improve these attributes.

An implication of this is that interventions whose effectiveness is driven mainly by their proximate impacts are less important than interventions whose effectiveness is driven mainly by increasing our steering capacity.

This is because any action we take is going to have indirect & long-run consequences that bear on our civilization’s trajectory. Many of the long-run consequences of our actions are unknown, so the future is unpredictable. Therefore, we ought to prioritize interventions that improve the wisdom, capability, and coordination of future actors, so that they are better positioned to address future problems that we did not foresee.

What being clueless means for altruistic prioritization

I think the steering capacity framework implies a portfolio approach to doing good – simultaneously pursuing a large number of diverse hypotheses about how to do good, provided that each approach maintains reversibility.[4]

This approach is similar to the Open Philanthropy Project’s hits-based giving framework – invest in many promising initiatives with the expectation that most will fail.

Below, I look at how this framework interacts with focus areas that effective altruists are already working on. Other causes that EA has not looked into closely (e.g. improving education) may also perform well under this framework; assessing causes of this sort is beyond the scope of this essay.

My thinking here is preliminary, and very probably contains errors & oversights.

EA focus areas to prioritize

Broadly speaking, the steering capacity framework suggests prioritizing interventions that:[5]

• Further our understanding of what matters
• Improve governance
• Improve prediction-making & foresight
• Reduce existential risk
• Increase the number of well-intentioned, highly capable people

To prioritize – better understanding what matters

Increasing our understanding of what’s worth caring about is important for clarifying our intentions about what trajectories to aim for. For many moral questions, there is already broad agreement in the EA community (e.g. the view that all currently existing human lives matter is uncontroversial within EA). On other questions, further thinking would be valuable (e.g. how best to compare human lives to the lives of animals).

Myriad thinkers have done valuable work on this question. Particularly worth mentioning is the work of the Foundational Research Institute, the Global Priorities Project, the Qualia Research Institute, as well the Open Philanthropy Project’s work on consciousness & moral patienthood.

To prioritize – improving governance

Improving governance is largely aimed at improving coordination – our ability to mediate diverse preferences, decide on collectively held goals, and work together towards those goals.

Efficient governance institutions are robustly useful in that they keep focus oriented on solving important problems & minimize resource expenditure on zero-sum competitive signaling.

Two routes towards improved governance seem promising: (1) improving the functioning of existing institutions, and (2) experimenting with alternative institutional structures (Robin Hanson’s futarchy proposal and seasteading initiatives are examples here).

To prioritize – improving foresight

Improving foresight & prediction-making ability is important for informing our decisions. The further we can see down the path, the more information we can incorporate into our decision-making, which in turn leads to higher quality outcomes with fewer surprises.

Forecasting ability can definitely be improved from baseline, but there are probably hard limits on how far into the future we can extend our predictions while remaining believable.

Philip Tetlock’s Good Judgment Project is a promising forecasting intervention, as are prediction markets like PredictIt and polling aggregators like 538.

To prioritize – reducing existential risk

Reducing existential risk can be framed as “avoiding large obstacles that lie ahead.” Avoiding extinction and “lock-in” of suboptimal states is necessary for realizing the full potential benefit of the future.

Many initiatives are underway in the x-risk reduction cause area. Larks’ annual review of AI safety work is excellent; Open Phil has good material about projects focused on other x-risks.

To prioritize – increase the number of well-intentioned, highly capable people

Well-intentioned, highly capable people are a scarce resource, and will almost certainly continue to be highly useful going forward. Increasing the number of well-intentioned, highly capable people seems robustly good, as such people are able to diagnosis & coordinate together on future problems as they arise.

Projects like CFAR and SPARC are in this category.

In a different vein, psychedelic experiences hold promise as a treatment for treatment-resistant depression, and may also improve the intentions of highly capable people who have not reflected much about what matters (“the betterment of well people”).

EA focus areas to deprioritize, maybe

The steering capacity framework suggests deprioritizing animal welfare & global health interventions, to the extent that these interventions’ effectiveness is driven by their proximate impacts.

Under this framework, prioritizing animal welfare & global health interventions may be justified, but only on the basis of improving our intent, wisdom, coordination, capability, or predictive power.

To deprioritize, maybe – animal welfare

To the extent that animal welfare interventions expand our civilization’s moral circle, they may hold promise as interventions that improve our intentions & understanding of what matters (the Sentience Institute is doing work along this line).

However, following this framework, the case for animal welfare interventions has to be made on these grounds, not on the basis of cost-effectively reducing animal suffering in the present.

This is because the animals that are helped in such interventions cannot help “steer the ship” – they cannot contribute to making sure that our civilization’s trajectory is headed in a good direction.

To deprioritize, maybe – global health

To the extent that global health interventions improve coordination, or reduce x-risk by increasing socio-political stability, they may hold promise under the steering capacity framework.

However, the case for global health interventions would have to be made on the grounds of increasing coordination, reducing x-risk, or improving another steering capacity attribute. Arguments for global health interventions on the grounds that they cost-effectively help people in the present day (without consideration of how this bears on our future trajectory) are not competitive under this framework.

Conclusion

In sum, I think the fact that we are intractably clueless implies a portfolio approach to doing good – pursuing, in parallel, a large number of diverse hypotheses about how to do good.

Interventions that improve our understanding of what matters, improve governance, improve prediction-making ability, reduce existential risk, and increase the number of well-intentioned, highly capable people are all promising. Global health & animal welfare interventions may hold promise as well, but the case for these cause areas needs to be made on the basis of improving our steering capacity, not on the basis of their proximate impacts.

Thanks to members of the Mather essay discussion group and an anonymous collaborator for thoughtful feedback on drafts of this post. Views expressed above are my own. Cross-posted to LessWrong & my personal blog.

Footnotes

[1]: Nick Beckstead has done the best work I know of on the topic of why the far future matters. This post is a good introduction; for a more in-depth treatment see his PhD thesis, On the Overwhelming Importance of Shaping the Far Future.

[2]: I'm grateful to Ben Hoffman for discussion that fleshed out the "steering capacity" concept; see this comment thread. 

[3]: Note that this list of attributes is not exhaustive & this metaphor isn't perfect. I've found the space travel metaphor useful for thinking about cause prioritization given our uncertainty about the far future, so am deploying it here.

[4]: Maintaining reversibility is important because given our cluelessness, we are unsure of the net impact of any action. When uncertain about overall impact, it’s important to be able to walk back actions that we come to view as net negative.

[5]: I'm not sure of how to prioritize these things amongst themselves. Probably improving our understanding of what matters & our predictive power are highest priority, but that's a very weakly held view.