Value on the Table: Neglected Interventions in Longtermism & AI Safety

Introduction

I've had the opportunity to take a zoom-out look at AI safety & longtermism as a field the past couple weeks. As I've thought and read, I realize that there's a lot of "white space": good ideas that haven't yet been tried or implemented.

I make three major assumptions going into this piece:

1. You care about doing good in the world as effectively as possible, in expectation
2. You care about improving the lives of sentient beings, regardless of species or substrate
3. You believe that transformative artificial intelligence (TAI) could be developed in the near-term (2-10 years) and think that this will dramatically alter the world we live in

This piece has the following sections:

1. Cause Areas. This section surveys the causes of suffering, and considers how this might be impacted by TAI.
2. Supply Chains of Interventions. This section considers what turns a good idea into a positive intervention. This framework then allows us to see where the weaknesses in the pipeline are.
3. The Major Gaps. Cursory thoughts on which parts of the ecosystem are lacking ideas, talent, or funding.

Section 1: Cause Areas

1.1 Causes of Suffering

Most people are already invested in a cause area, but if you're not, here are some observations:

• When thinking about tractability, you can consider:
  • How likely it is we get to a dystopia for a particular cause, how you can prevent that
  • How desirable the utopia is for a particular cause, and how you can help create that future
• Regarding importance:
  • I'd say the number of sentient beings affected by a given cause seems to increase by several orders of magnitude as you go down each row in the table.
  • Consider bad you think the status quo post-TAI outcome is
• And for neglectedness, read on!

1.2 Actions to Take

So now we know what is at stake. Given the Causes of Suffering (and joy!), and the potential impact of TAI, one might wonder -- what should we do next?

Luckily, there are ideas out there. These bullets are primarily drawn from William MacAskill's Effective Altruism in the Age of AGI (reason #1). I have bucketed them in a sensible manner.

Direct Work
Many people think that TAI might lead to a "lock-in" of present values / institutions. If this is the case, doing direct work now could help improve outcomes in the long-run:

• global health & development
• factory farming
• AI welfare / digital minds

Avoiding Bad Futures & Creating Good Futures
We can also preempt bad things from happening, and set up the foundations for good futures. For example, if your threat model is a risk of bio-terrorism from an AI-empowered bad actor, then you could create a stockpile of masks (see ProEquip).

• Gradual disempowerment
• Biorisk
• Space governance
• AI safety (scheming, alignment)
• AI character
• The risk of (AI-enabled) human coups
• AI-driven persuasion and epistemic disruption
• S-risks

Improving Meta Functioning of Humanity
This bucket assumes that we don't have all the answers right now, but we should prepare ourselves (individuals, organizations and large institutions) to make the right decisions when we do know what will matter.

• AI for better reasoning, decision-making and coordination
• Democracy preservation
• Promoting rationalism / EA
• Forecasting / better understanding AI right now

Section 2: Supply Chains of Interventions

Anything that impacts the longterm future starts as an idea and goes along the following supply chain to become a concrete intervention:

• Idea
  • A plausible idea about how to do good
• Proposal
  • A specific Idea is elevated to "we should actually do this"
• Validation
  • External people decide that the Proposal is worth doing something about, i.e. by giving funding or time
• Initiative
  • A Proposal with actual humans & funding attached to it
• Intervention
  • A government policy change, a corporate policy change, increased awareness about a cause

Additionally, you have the following which are not strictly part of the supply chain but do strengthen it:

• Capacity building. Getting more talent and financial resources into any part of the supply chain.
• Exchange of ideas. An Idea can become a Proposal, but then not get validated and go back to the drawing board. On online forums, at conferences, or via. email, individuals and organizations get feedback on their work that informs what they do next.
• Advocacy. Each stage requires someone making an active case for moving to the next. Importantly, ideas only become policy when they go through our wonky political system.

Section 3: The Major Gaps

In an ideal world, each of the causes mentioned in Section 1.2 would have a flourishing ecosystem of organizations developing ideas, proposals, and interventions for a better future. Sadly, this seems to not (yet) be the case!

If you're interested in a cause area, I'd recommend using an LLM to do a deep research on the intersection of a particular cause and supply chain. For example, "find all the organizations doing theoretical / legal research for digital sentience." In many cases, you'll find that there are only a handful of orgs, and that their efforts are spread very think.

Some claims about ecosystems:

• Diversity is important. Just because one organization is working on "policy advocacy" in "AI safety" (fill in your field/cause area) doesn't mean other orgs can't be started.
  • The benefits: different approaches, some of which might work, some might not; if one organization is sunk because of political affiliations/poor management/scandals, other orgs can continue doing the work; orgs can collaborate/share tactics, and spread their efforts across geographical regions (eg. CA vs DC).
  • The cons: you're potentially competing for funding, and it's harder to distinguish yourself.
• We can't let corporations monopolize any cause area. If Anthropic is the only group working on "AI Character", then they have no accountability/incentive to do higher quality work than their competitors.
• If you have shorter timelines, you should favor interventions over research. Hitherto, most AI safety organizations have been in the research / capacity-building side of things. These parts of the supply-chain are important, but only if you're able to translate them into actual interventions. I have short timelines (~2-3 years) and I would expect to see a more fluorishing ecosystem of policy & grassroots advocacy for AI safety, for example.

Without having done extensive research on every cause, I do have some rough takes on where "the ecosystem" stands on various issues. I say this having done some amount of reading on this forum and Forethought's writings, gone to SFS and EAG for two years, and having worked at Constellation for about a year and chatted with many researchers. But again, I encourage you to do your own reading on what exists out there.

For developed cause areas:

• Global poverty. This is an issue that has been studied a long-time in the EA world, so I imagine we're doing a lot of what can be done. It seems to me like poverty is a political problem, so work advocating for more international aid, and marginal interventions like malaria bednets, are both important.
• Farmed & wild animal. I've worked in this field a lot before... and my personal thoughts are that TAI is our best chance at solving this cause area. Animal advocacy is a slog -- median timelines for farmed animal liberation & a welfare state for all insects (assuming no TAI) would be hundreds or thousands of years off. But TAI gives us a chance, if only we can steer it towards positive values towards humans and animals.
• Bio-risk. This is one of the "classic" EA cause areas, but there are probably still more organizations that could be started. For instance, early-pandemic detection, improving ventilation infrastructure throughout the US, having proposals ready for contact tracing, etc.
• AI alignment. Plenty of research organizations here -- but to me, there seems to be a concerning dearth of concrete policy recommendations. SB1047 was some progress to that end.
• AI safety governance. Another bottleneck here is advocacy -- organizations turning reasonable ideas into actual policies. There's also a shortage of politicians who are willing to spend political capital on AIS. One thing that would definitely be helpful would be grassroots, political advocacy to shift the Overton window and build political power for policy-advocates and legislators

For nascent cause areas, I must admit I don't know the frontier of each cause area here. I generally feel that these fields are neglected, and at most have 1-2 EA-aligned causes working on them. Many of these seem to be bottle-necked on ideas / proposals.

• Space governance. More ideas needed!
• Digital sentience. More proposals, like legal frameworks, needed here. Notable work by Eleos AI and Kyle Fish.
• AI coups. A lot of ideas out there. Maybe this is covered by the intelligence community / RAND who are AGI-pilled.
• AI-driven persuasion and epistemic disruption. I'm not sure at all.
• AI for better reasoning, decision-making, and coordination. More ideas needed!
• AI character. This seems to be just Anthropic!
• Forecasting / evaluations. This is one of the more developed fields. These model cards should have more teeth if they're going to actually have political power.

Closing Thoughts

Having done some research and thinking, I believe that the world is dramatically under-prepared for the challenges of transformative artificial intelligence. This will probably affect the long-term future of humanity, and potentially trillions of beings who will live throughout the galaxy. But, I do think that it's worth trying to do something about it.

Writing this piece was helpful for me to orient in my next career steps, and to think from an impartial viewpoint about the work that needs to be done. When you have a good idea of the "white space", you can then start to think about your personal fit, or perhaps what areas need more funding.

I'd like to close with two points:

1. Be calm, cool, and collected. It's easy to feel anxious or overwhelmed as we're nearing the advent of transformative artificial intelligence. If it's not something you can do anything about, then there's no use worrying. If it is something you can do something about -- go do it! There's still no use worrying.
2. Work smart, not hard. I believe in the Pareto Principle, ie. 20% of your decisions drive 80% of your impact. Working 80 hours a week and not having friends might make you more productive in a narrow sense, but does it make you more impactful? I'd argue that regularly taking time off from these problems, zooming out, and thinking with greater strategic altitude is what will really drive impact.

Good luck!

More reading:
William MacAskill, Effective Altruism in the Age of AGI
Forethought, How to Make the Future Better
Holden Karnofsky, Rowing, Steering, Anchoring, Equity, Mutiny
80,000 Hours, Problem Profiles
Forethought, Preparing for the Intelligence Explosion
David Pearce, A Welfare State for Elephants?
Niki Dupuis, How I'm Thinking About the Next 3 Years