This post is part of the Cross-Cause Fund (CCF) series by Rethink Priorities — a sequence of articles dedicated to effective giving across philanthropic areas.

TL;DR: 

There is a seemingly plausible argument that donors should save almost everything until after the transition to AGI, when returns on capital and the cost-effectiveness of philanthropy may both be much higher. This argument relies on particular assumptions about the future that we’re not confident will hold across cause areas. The “save now” strategy is not robust to some very live possibilities for how the future will unfold. In particular, the role of diminishing returns and the risk that saved money is lost or diverted ("drift") has been underappreciated, and it's far from clear that post-AGI opportunities will be cheaper across cause areas. Even moderate risk aversion can also push one toward giving a larger share sooner, and we think that no one should be completely certain in any worldview, including ones committed to risk neutrality or longtermism. Donors ought to be cautious about saving everything now, given uncertainty about the future.

At Rethink Priorities, we invite you to sign up for our webinar, where we’ll go much deeper into this topic. Also, stay tuned to the EA forum for more developed arguments and modeling in the coming weeks.

An argument for giving after AGI

How much of your money should you donate now, and how much should you save and donate later? And how should this depend on your views about how the near-term future will unfold?

One line of reasoning offers a simple answer to this question: If you are considering giving to anything other than AI safety, you should be saving (almost) everything. The argument, voiced by Will MacAskill on Forethought's ForeCast podcast,^[1] goes something like this:

• We are at a particularly turbulent time for AI, but a key transition will occur in the next few years, after which we will settle into a post-AGI world (call this time t).

• (Non-AI safety) philanthropic projects will be more cost-effective after t than before t.^[2] AGI will unlock technological advances that make philanthropic projects much more cost-effective, so a dollar given after t will purchase more value than a dollar given now.

• There will be very high rates of return on capital saved before t. Transformative AI will cause massive increases in productivity and economic growth, so capital invested before t will have extremely high returns.
• If you save now, you will have much more money to give, and you will be able to spend that money more effectively after t. Therefore, you should save your money until after t.

This argument offers sound advice given some assumptions about how the future will unfold. However, we think there are reasons to be highly uncertain about these assumptions. In brief:

• In some cause areas, it will probably become far more expensive to do good. Entrenched interests will be even harder to battle when they are empowered by superintelligence. Bad values could be baked into AGI, making them very hard to reverse.
• In other cause areas, abundance created by a superintelligence boom could improve material conditions, such that a large proportion of highly cost-effective opportunities to do good exist only prior to such a boom.
• Philanthropic funds may be lost or diverted before they can be spent. We don’t need to look far for examples: The recent crash and liquidation of Strategic Awareness Fund assets make salient the possibility that a market correction could eliminate wealth overnight. When FTX collapsed, the Future Fund’s donations were canceled or clawed back. Furthermore, some high-profile donors have lost their motivation to give.
• A massive influx of AI wealth will likely result in strongly diminishing returns.

Huge returns on capital may correlate with these factors pushing in the opposite direction. For example, capital appreciation is likely to cause higher probabilities of drift (creating greater pressure for political responses to wealth inequality), decreases in cost-effectiveness (as entrenched interests also become wealthier), and more strongly diminishing returns (as AI wealth saturates giving opportunities).

Preliminary results from our optimal giving models^[3] show that the “save now” strategy is not robust to some very live possibilities for how the future will unfold. Our modeling shows that if we assume even conservative diminishing marginal returns, then there are plausible probabilities of loss and changes in cost-effectiveness that can recommend giving significant percentages before AGI. These values will probably not hold across the board, but they may be plausible in certain cause areas that interested donors should be responsive to.

Additionally, even if saving everything now has the highest expected value, by doing so, donors could be taking significant risks and forgoing significant opportunities to do good. For donors with even moderate risk aversion and uncertainty about how the future will play out, it can make sense to hedge one's bets by giving a larger fraction earlier.

Which factors matter

Our investigations looked at four main factors:

• Large AI investment returns: the market reprices AI equities after demonstration of transformative capabilities
• Capital productivity for impact: how much philanthropic good each dollar can buy
• Diminishing returns: how effectiveness changes as donations are scaled up
• Drift probability: yearly probability that the money vanishes or is spent elsewhere

We explored scenarios for how AI could affect each of these factors, individually or in combination, either continuously or as a shock at some future time t. We modeled the optimal rate of spending under each scenario if we want to maximize the total amount of expected good. We also constructed models that include risk aversion and those that assume a spend-down timeline. We will present fuller versions of these models in future work, but for now, we can extract some general claims about the direction that these factors will push us toward, with regard to giving now vs. giving later.

On the capital side, the two main considerations are interest rates and drift probabilities. How much will our money grow if invested wisely, and how much will that money be worth (adjusted for inflation) after an intelligence explosion? High interest rates push us toward giving later, when the pot of money is larger. While proponents of giving later emphasize the tremendous potential growth rates of invested capital, the possibility of drift is given less attention. As the period of saving extends, the risk that the money will disappear before it can be spent or will be reallocated toward other ends increases.

On the opportunity side, the two main considerations are capital productivity for impact, i.e., the baseline cost-effectiveness of our best projects, and diminishing returns. How much good can our projects do with a dollar, and how much will that decrease as those projects take on more funding?

Why we should be uncertain: Capital-side considerations

The recent history of AI investment shows very high returns (broadly over 100% over the last five years). If an intelligence explosion yields economic productivity gains on the level of the Industrial Revolution, we may see 10x or even 100x returns on investment. This alone would seem to recommend saving (unless the cost-effectiveness of philanthropic opportunities declines to a commensurate degree).

We are doubtful that something like 100x returns is the mean outcome of transformative AI. We’re uncertain whether an intelligence explosion will occur and what the economic effects would be. There are plausible scenarios in which political forces halt AI development, or where AGI causes an economic depression. Even if AGI yields massive productivity gains, these may already be priced in, in which case we’ll see most of the return on AI investments in the very near future.^[4] But even if we assume that the returns will be extreme, significantly diminishing returns and high probabilities of drift can still tip the scales in favor of giving early.

It’s an obvious point but still worth keeping in mind that even a 100x return on investment doesn’t automatically yield 100x spending power. Even at conservative levels, diminishing returns quickly take a bite out of capital returns and philanthropic impact. For illustration, under the diminishing returns Coefficient Giving assumes, each doubling of funding makes the next dollar about a quarter less valuable. At that rate, a 100x return on investment buys only about 16x the impact, and simple models demonstrate a shift toward giving earlier. Incorporating diminishing returns also creates the conditions for additional changes to cost-effectiveness and probabilities of loss to have a significant effect.

Transformative AI may significantly change the rate at which returns diminish (or perhaps increase). A massive influx of wealth at t would move us to a spot on the curve where returns diminish very rapidly, as physical and logistical obstacles limit the rate at which aid can be dispersed and absorbed. Granted, there are bottlenecks that will be loosened by AGI. However, it is hard to imagine that a sudden influx of even 10x the money of the existing philanthropic landscape could be spent at today’s levels of effectiveness. Even with significant innovations, there can be physical, logistical, and human bottlenecks in implementation. For example, consider a charity that vaccinates children. AGI may make it easier for it to scale up on intellectual and bureaucratic tasks, but it won’t go the last mile to get shots in arms (at least not anytime soon).

There is enough uncertainty about when others will give and when organizations will get saturated that it may make sense to disperse giving over time. Further, given the significant effect of diminishing returns in influencing expected outcomes, spending now to build capacity, and thereby flattening the curve in the future, can be enormously cost-effective.   

Another risk is that we are banking on philanthropic money that won’t exist by the time we get around to spending it. Every year that money sits in savings or investments, there is some chance that it will be lost or diverted from the philanthropic purposes for which it was intended. Investments can vanish for external reasons. For example, a market crash could wipe out investments. Political pressure to rein in AI risk or safeguard military secrets could lead to AI companies being nationalized, with uncertain buyouts for their stakeholders. Short of nationalization, we should expect very high tax rates on AI wealth, especially if it is deemed necessary to offset AI-induced job losses. The turbulent period around the AGI transition may involve repeated attacks on financial systems, wiping out wealth.

There are also internal threats to money earmarked for philanthropy. For example, potential donors may get a taste for extravagant purchases, spending on sports teams (if times are good) or private islands (if times are scary). The chaotic post-AGI information environment may hasten such value shifts. If AGI alignment goes poorly, the superintelligent philanthropic advisers of the future might mislead us.

It may be wise for donors to act now, based on their current preferences, to guard against potential (and recently all too common) shifts in values after living with wealth and power.^[5] There are also things donors can do to safeguard against internal drift that are consistent with saving. They can make binding commitments, or yield control to others, with instructions to spend patiently. But there’s only so much individual donors can do to protect their savings from external exigencies while ensuring the money is liquid enough to be spent when the time is right.

It’s easy to get lulled into doing nothing by thinking that saving is always the safe thing. Saving has a cost: the good you didn’t do with your money. Sometimes this risk is worth taking, but it’s important to recognize that it is a risk.

Why we should be uncertain: Opportunity-side considerations

In our modeling, one of the strongest determinants of optimal timing is capital productivity for impact. What are the opportunities available for doing good with your money, and how much good does each dollar purchase? Increases in impact productivity alone can make it smart to save now and give later, especially if there is a jump in productivity some time in the near future.

MacAskill identifies several reasons why we should expect there to be much more cost-effective philanthropic opportunities after an intelligence explosion. Cognitive labor will become much cheaper, so anything currently bottlenecked by expertise will be unlocked with AGI. Organizations will be able to scale very quickly. This includes technological innovation. It also includes strategic cause prioritization itself; we may make much better philanthropic decisions with the help of AGI.

We urge caution here too. As we’ve argued elsewhere, innovation and economic growth don’t always lead to commensurate growth in impact. For many projects, especially megaprojects, success requires progress on many fronts (technological, cultural, political, legal, economic, etc.), and AGI will not facilitate each of these equally. Costs for labor and resources may increase dramatically, with philanthropic actors getting outbid by for-profit actors.

There are also some kinds of information that AGI may not significantly speed up. The interventions EA most celebrates in global health and development earned that status through decades of RCTs and pilot studies. In nascent fields, we should expect building comparable evidence to take time as well. AGI may speed up study design and analysis, but it can’t shorten the years needed to see whether an intervention actually reduces mortality or raises incomes. A donor who waits to start running pilots may have far more money but little idea where to spend it. Funding now can make later spending better targeted.

AGI may make ground conditions worse while also making the treatments more effective, leaving philanthropic organizations in a Red Queen scenario (running as fast as you can just to stay in the same place). Of course, the harms of AGI might also undo any progress achieved by spending now. But there are cases where existing inequalities might be easier and cheaper to fix before AGI than after. If AGI causes explosive growth, then those with capital will benefit much more than those without. The inequality between those on the AI rocket and those left behind will widen.^[6] 

A chief obstacle to many philanthropic projects is the presence of powerful, well-resourced interests opposing them, and AGI will make these opponents even more powerful. AGI will give animal welfare activists better tools to persuade people and better technology to make farming more humane. But it will also give factory farm owners better tools. Likewise, it will give better tools to opponents of democracy, criminal justice reform, and clean energy. Fighting these fights will probably not get any cheaper than they are now. If we think that success now will get locked in, it makes a lot of sense to spend now. If, on the other hand, success can be rolled back in the future, it may make sense to save resources to fight these expensive battles in the future.

More optimistically, abundance created by a superintelligence boom could improve material conditions and lifespans. If so, money is more necessary now than after AGI, when many of the low-hanging fruit will already be taken care of. Investing in a manner that is correlated with the overall welfare conditions reduces the impact of the marginal dollar, since spending when conditions are poor has a greater impact. As MacAskill notes, saving a life today could be incredibly valuable, as that person will be around to experience the benefits of superintelligence.

It’s difficult to assess cost-effectiveness in a post-AGI world. We don’t know what will be locked in at the point of transition. Spending right now may increase the chances that good governance is in place when the transition occurs. AGI might be very hard to budge, so now may be the only time to influence its values. AI safety work is important here, but so are many other kinds of research and advocacy. If the post-AGI world strongly depends on what we do now, opportunities before t may be far more cost-effective than after.

In summary, some kinds of progress will probably be much cheaper to purchase after AGI. Others will be more expensive. Donors should reflect on which kinds of progress they care about, and consider whether it will be easier or harder to achieve after AGI. And we think that for most cause areas, we should be quite uncertain. 

What to do with uncertainty?

We have argued that there is a wide range of plausible values for key variables: marginal returns curvature, probability of loss, cost-effectiveness after AGI, etc. If you are solely focused on maximizing expected good, then you can average over your uncertainty in these variables and may still end up saving nearly everything. However, things are much different if you are risk-averse.

One way to model risk aversion is to put lower weight on the more optimistic projections of the impact your money can buy.^[7] This acts like a second layer of diminishing returns on top of the first. Recall that under the diminishing returns Coefficient Giving assumes, each doubling of funding makes the next dollar about a quarter less valuable. Add moderate risk aversion, and each doubling makes the next dollar about a third less valuable. An expected 100x return on investment then buys only about 6x the risk-adjusted impact, rather than 16x. When we incorporate uncertainty over returns with these values, risk aversion can have doubling effects on the share being spent before AGI. 

Most donors who are considering giving to animal welfare and GHD causes are probably at least a bit risk-averse. We think that no one should be completely certain in any worldview, including ones committed to risk neutrality or longtermism.

Significant uncertainty about AGI futures might lead us to think that the safest option is to save our money and reassess the landscape in a few years. We don’t know which philanthropic opportunities will be available, and we can always reassess when we get there (under the guidance of our superintelligent philanthropic advisers). However, saving is risky too, and donors should balance both risks when designing their giving.

Future research

We have done more research to model optimal giving strategies in light of uncertainty, which we will be presenting in an upcoming webinar and publishing soon afterward. We are also planning more research to resolve uncertainty in how AGI will affect each of the factors we’ve focused on here, especially cost-effectiveness across cause areas. This will inform the recommendations made by our Cross-Cause Fund.

Acknowledgements:

Thank you to Francis Priestland, Matthew Zipple, Arvo Muñoz Morán, David Moss, and Marcus A. Davis for reviewing this post, and Shane Coburn for copyediting.

1. ^{^}

    This is a simplified version of a strong argument for saving now, and we don’t claim that MacAskill defends this particular version. In that podcast, he does not recommend saving all of your money and recommends some good spending opportunities before AGI.

2. ^{^}

    This argument isn’t intended to apply to projects like AI safety work. Those projects are argued to be cost-effective now, but not cost-effective after an aligned AGI takes over AI governance. Therefore, the argument goes, if you spend before t, you should spend on AI safety. We’ll leave this appended argument to the side for now, since it relies on some specific claims about the cost-effectiveness of AI work that we can’t do service to here.

3. ^{^}

    For another recent model, see this recent post.

^{^}

 If the putative advantages of AGI are already baked into the sky-high valuations of AI companies, then we shouldn’t assume that post-IPO AI stock will be worth 100x what the market currently thinks they are worth now. Further, if there’s one AI winner, then the value of competing companies could plummet. Conversely, hoped-for growth might come from other smart investments down the line, though again, there’s a lot of uncertainty.

^{^}

 You might also factor your AGI timelines into your commitments. Currently you might believe that AGI is due in 8 years, and so be willing to save big for that world. But in 8 years, if you still find yourself without AGI, then you’d probably want to spend more.

^{^}

 Inequality is primarily a problem for positional goods, where how well one is doing is not determined by one’s absolute possession of a good but how much one has relative to others.

^{^}

Here we use a Buchak-style REU with r(p) = pᵃ, applied to the total impact of your spending plan, at a moderate level of risk aversion. There are other ways to incorporate risk aversion. For example, you might set a threshold like “I want to minimize my probability of doing nothing” or “I want to ensure I do x amount of good with at least probability p,” in which case risk will largely be a function of drift probabilities. There are also risk functions that might tell in favor of giving later, such as ones that put more weight on the worst possible outcomes of AGI.

Show all footnotes