Comments
Thanks for sharing this. While I think there are strong reasons to invest heavily in AI safety, I'm concerned this particular cost-benefit framing may not be as compelling as it initially appears.
The paper uses a $10 million value of statistical life (VSL) to justify spending $100,000 per person to avoid a 1% mortality risk. However, if we're being consistent with cost-effectiveness reasoning, we should note that GiveWell-recommended charities save lives in the developing world for approximately $5,000 each—roughly 2,000 times cheaper per life saved.
By this logic, the same funding directed toward global health interventions would save orders of magnitude more lives with near-certainty, compared to reducing AI x-risk with uncertain probability.
This doesn't mean AI safety is a bad investment—there are strong arguments based on:
- The value of preserving future generations (which Jones notes would increase spending estimates)
- Diminishing returns or bottlenecks in scaling proven global health interventions
- The categorical importance of preventing existential catastrophe
- Portfolio diversification across different types of risk
(Note: comment generated in collaboration with AI)
Here’s the PDF.
I haven’t read it, but I feel like there’s something missing from the summary here, which is like “how much AI risk reduction you get per dollar”. That has to be modeled somehow, right? What did the author assume for that?
If we step outside the economic model into reality, I think reducing AI x-risk is hard, and as evidence we can look around the field and notice that many people trying to reduce AI x-risk are pointing their fingers at many other people trying to reduce AI x-risk, with the former saying that the latter have been making AI x-risk worse rather than better via their poorly-thought-through interventions.
If some institution or government decided to spend $100B per year on AI x-risk (haha), I would be very concerned that this tsunami of money would wind up net negative, leaving us in a worse situation than if the institution / government had spent $0 instead. But of course it would depend a lot on the decisionmakers and processes etc.