I wrote a paper about how much we should spend on AI safety. This post is a summary, in my own words.
We don't need to appeal to longtermism or even altruism to justify massive increases in AI safety spending. Chad Jones’ new paper shows that a purely selfish person should be willing to sacrifice 1-15% of their income to fund AI safety research. I extend this paper to an altruistic framework but maintain the “non-EA” assumptions. The existence of foreign aid budgets suggests countries are willing to sacrifice a small proportion of GDP to spend on life-saving interventions. Consistency with even this very small amount of altruism roughly doubles the optimal spending on AI safety relative to Jones (2025). I hope that this framework can be used to convince non-effective altruists that AI safety is orders of magnitude underfunded, without any need to first convince them to change their moral philosophy. For the EA readers, who have already decided to sacrifice consumption for altruistic causes, this doesn’t change much. In fact, the model collapses into “allocate resources to whatever saves lives most cost effectively” for the highly altruistic agent.
A common hurdle when advocating for AI safety is that the arguments seemingly rely on ethical considerations for the lives of people far away in space or time. While I personally think that we should put a very significant weight on the lives of these people in our decision making, I often find myself stuck in a situation where the person I’m talking to simply will not be convinced of any change in their moral beliefs. I also suspect that governments, in the face of their own domestic trouble, will be hesitant to make decisions about AI safety spending on the basis of considerations for people outside of their country, or people who are not born yet.
Another approach then, is to argue that the decision to spend more on AI Safety is consistent with one’s existing actions. That’s the argument I’m making here, and in fact I’ll show that most countries’ current actions suggest they ought to be willing to sacrifice at least a few percent of GDP to try to mitigate the risks of AI doom.
I’ll introduce two papers: one by Stanford economist Chad Jones (2025), which sets out the framework for thinking about this problem (and which deserves all the credit for the idea), and then an extension I wrote over the last few weeks, which extends the model slightly and hopefully clears up some confusion I was seeing in the commentary on the paper (e.g. see the comments under this post). Both papers suggest that we are vastly underinvesting in AI safety even under the most conservative, short-termist, and “selfish” assumptions. Thus, this is the “non-EA” economic case for increasing spending on AI safety.
In How Much Should We Spend to Reduce A.I.’s Existential Risk?, Jones models a “selfish” representative agent who cares only about their own consumption.[1] Of course, if they die in an existential AI catastrophe, they cannot consume, and so they have a preference for that not to happen. Ignoring future generations, or any altruism at all, the model suggests the purely selfish person should sacrifice between 1% and 15% of their income (i.e. 1-15% of GDP) for AI safety spending.
One of the key derivations of Jones (2025) is a “back of the envelope”[2] calculation of how much the selfish person should sacrifice to spend on preserving their own future. He calculates a lower bound of 1.8% of GDP. This is based on three calibrated assumptions:
While these assumptions might seem strange, or incorrect, the paper goes on to calibrate the model under many different sensitivities. Under these simulations, the optimal spend is actually much higher, averaging around 8% of GDP.
Currently, the US spends roughly $1-2 billion annually on AI safety. Jones’ “selfish” model suggests that number should be closer to $100 billion (approx. 0.3% of GDP) at the absolute minimum, and likely far higher.
The framework above assumes we care only about ourselves. Obviously, this isn’t accurate. As I point out in my extension paper, countries routinely sacrifice consumption to save the lives of strangers, most notably through foreign aid.[3] Foreign aid budgets have been under pressure in recent years, but many programmes have survived and my paper focuses on those cases. For reasons I discuss in the paper, I actually think foreign aid spending massively understates how much people care about others, but at the very least it gives us a concrete example of countries sacrificing GDP in order to save lives.[4]
I extend the selfish Jones framework by introducing a “trilemma” where agents split income between:
We can “calibrate” how much we care about saving lives by looking at foreign aid budgets. This type of spending reveals that we are willing to sacrifice some amount of our consumption to save the lives of complete strangers. The next step, then, is to note that AI safety is a public good. When you reduce existential risk, you save everyone—roughly 8 billion people, with some probability. And presumably we care about those 8 billion people the same amount as we care about the people we can save through foreign aid interventions. Therefore, we can work out how much we ought to spend on AI Safety based on how much we already spend on foreign aid.
I find that adding this “modest altruism”, calibrated to foreign aid budgets, more than doubles the optimal spending on AI safety, when compared to the selfish baseline. If the selfish baseline is 1.8% of GDP, accounting for our revealed preference for helping others raises this to 4% of GDP.
Interestingly, my argument might go against a common framing of humanitarian aid and long-term risk reduction as competing priorities. My argument is that if we care enough to sacrifice GDP on foreign aid today, basic consistency requires us to spend significantly more in order to prevent the extinction of the entire human race.
This paper can seem a bit confusing to effective altruists deciding between spending on AI safety and charity, so I wanted to clear some things up. So far, I’ve only considered the trade-off between AI safety and consumption, and I’ve only considered a very moderately altruistic agent.
But you might not be interested in that scenario because you’re an effective altruist. Perhaps, you have already decided that you want to sacrifice a significant portion of your income, but you want to know how to allocate those donations between AI safety and life-saving global health interventions.
It’s simple, EAs should spend on whatever saves lives most cost effectively. Nothing new here.
I bring this up because I saw some comments on a previous write up of this paper:
It’s important to note here that the $10m value of saving a statistical life cannot be compared to the $5,000 it costs to save a life by donating to GiveWell. Those two numbers are not in the same units. Also recall that these papers are not designed for EAs operating at the margin of deciding between multiple cost effective life saving interventions with a large charitable budget. The papers are targeted at governments deciding whether to sacrifice GDP in the short-run, in order to fund AI safety.
Section 5 of my paper shows formally that all the standard rules (i.e. you should donate to whichever intervention saves lives most cost effectively) apply to people who are highly altruistic.
For EAs engaging with government officials or policy advisors, these papers offer a different framing to the arguments for increasing AI safety spending. AI safety is a good in high demand not just by effective altruists, but by anyone wishing to not be extinct in 10 years. Additionally, AI safety can be viewed as another foreign aid intervention since it has the chance of saving the lives of individuals in foreign countries cost effectively. When combined, these two arguments suggest AI safety is currently underfunded by at least a few orders of magnitude.
My hope is that Jones’ paper and potentially my extension can appeal to government because:
Please do reach out if you have any comments or suggestions :)
In common speech, consumption has some negative connotations. In economics, consumption just means things you like to spend your resources on. In the model here, time and money would be your resources. Thus, consumption might mean buying a fancy watch, but it could equally mean going for a walk in the hills or going out for dinner with your friends.
It requires pages of mathematical derivation!
Foreign aid budgets have been under pressure in recent years. However, many programmes have survived and my paper focuses on those cases. For reasons I discuss in the paper, I actually think foreign aid spending massively understates how much people care about others, but it was at least a useful lower bound.
Individuals also engage in private charitable donations, but most of that is not for the purpose of saving lives, and the donations which do aim to save lives are pretty skewed towards a small number of people (many of them EAs) who this argument is not really for. It’s also true that foreign aid is not primarily about saving lives so I focus on the subset of foreign aid that is. For more details, see section 4 of the paper.
