I have the impression that the most effective interventions, especially in global health/poverty, are usually temporary, in the sense that you need to keep reinvesting regularly, usually because the intervention provides a consumable good; for example malaria chemoprevention: it needs to be provided yearly. In contrast, solutions that seem more permanent in the long-term (e.g. a hypothetical malaria vaccination, or building infrastructure), are typically much less cost-effective on the margin because of their high cost.
How do we balance pure marginal effectiveness vs eventually moving towards more permanent solutions? Could it be that by overly optimising for marginal cost-effectiveness, we might be missing a better ‘global maximum’ in the utility landscape, but we just need to descend from the current ‘local maximum’ to be able to get there eventually?
I think you're conflating intervention durability with outcome durability? A child who survives cerebral malaria due to seasonal malaria chemoprevention gets to live the rest of their life; SMC programs are rerun because (mostly) new beneficiary cohorts are at highest risk, not because last year's cohort's survival expires somehow. Similarly with nets and child vaccinations and vitamin A deficiency prevention (i.e. the GW top charities), as well as salt iodisation and TaRL in education and many other top interventions recommended by the likes of TLYCS and FP and so on.
I'd also push back a bit on the "permanent solutions" phrasing. Infrastructure isn't that permanent and requires ongoing expenditures and has a shelf half-life (I used to work in ops in fluid resource-constrained environments so I feel this keenly), diseases can develop resistance to vaccines so you need boosters, etc. Ex-AIM CEO Joey Savoie has a great blog talking more about how Someone Always Pays: Why Nothing Is Really "Sustainable".
Phrasing nitpicking aside, some big funders are in fact funding more "permanent / sustainable" solutions. Open Phil Coefficient Giving's new $120M Abundance and Growth Fund aims to "accelerate economic growth and boost scientific and technological progress while lowering the cost of living", and Founders Pledge (which is almost OP-scale in giving) just launched a new Catalytic Impact Fund that targets "ecosystem leverage points" where small investments can build "sustainable, long-term solutions to global poverty and suffering".Â
Jason's comment above on timetable speedup is essentially how e.g. GiveWell models their grants for malaria vaccines. The model says their grant would need to speed up co-administration for all highest need children in all of subsaharan Africa by at least 9 months to clear their 10x bar, so you can interpret their grant as a bet that funding that clinical trial would in fact achieve at least 9 months speedup. Notice how it's an uncertain bet; I think most donors (weighted by dollars moved) care quite a fair bit about certainty of direct benefits, so they'd probably donate to e.g. the Top Charities Fund instead of the more experimental EV-maxxing All Grants Fund.
Given EA's small share of the total global health/poverty funding landscape, the most likely effect of its investment on an expensive-but-permanent project is to speed the timetable up. So, for instance, perhaps we would get a hypothetical vaccine a year or two earlier if there had been EA investment. So, in comparing the effects of a yearly intervention vs. an expensive-but-permanent one, we are still looking at near-term effects that are relatively similar in nature and thus can be compared.Â
I don't suggest that is true for all "permanent" interventions, though, so it isn't a complete answer. It also might not scale well to a field in which EA funding is a large piece of the total funding pie.
I disagree with your point that saving the child's life is something you need to continuously reinvest in[1]. But I do think that you're pointing at something adjacent more along the lines of:
Giving out bed nets doesn't fundamentally solve global poverty
Solving global poverty is better than continuously giving out bed nets
Therefore, EAs should focus less on bed nets and more on solutions for global poverty.
I kind of agree with this. Imo the only real long-term solution is economic growth. But that said, two points:
Saving a child's has positive flow through effects that help with growth (for example, it returns many productive years back into the economy)
There are EA people who are also thinking about growth (here, here, here, or here) or science for global health interventions which also help (here).Â
Funnily enough, your Malaria vaccine is an example of something that Open Philanthropy Coefficient Giving has funded. They funded a bunch of malaria vaccine research, notably the stage 3 trials of one of the two vaccines (I forgot which one)
If AI agents use LLMs as their brain, how much of the harm that they end up doing is up to the chance of which token gets sampled from the output distribution?
As a dumb example, say some agent using an LLM as its backbone is reasoning "[...], so humans deserve to", and the LLM's output distribution for the next token assigns 98% probability to "die" and 1% to "live". The reasoning that follows will be influenced by which of these gets sampled.
Crossposted to Less Wrong
‘Like most of life’s dangers, the dangers of artificial intelligence are not easily identified from the armchair.’
J Dmitri Gallow
Epistemic status
I’m not an expert in this field, and herein I make some sweeping generalisations educated guesses about the overall...
Let’s imagine it’s November 2031. We’re both still here, AI is much more capable than it is today, but governments are governing, civilisation is still (mostly) civilised, and there has been no catastrophic loss of control. It just kind of didn’t happen.Â
What happens to the AI safety field then?Â
I expect people who currently believe catastrophe could happen within two or three years will start saying five or ten years. Then 15. Then ‘who cares if we’re a few decades of...
Thanks to Bruce Friedrich, Seth Ariel Green, Tobias Leenaert, Jan Willem Bolderdijk, Martine van Haperen, Herman Rohrbach, Chris Popa and Elizabeth Alvarez for comments. Claude helped structure the text. Ideas and mistakes are my own. I currently work with ProVeg International. The prize I propose would be defined and judged by an independent committee, and its money would fund scaling the winning solution(s), not ProVeg or the winner.
TL/DR
* EA could d...
I have the impression that the most effective interventions, especially in global health/poverty, are usually temporary, in the sense that you need to keep reinvesting regularly, usually because the intervention provides a consumable good; for example malaria chemoprevention: it needs to be provided yearly. In contrast, solutions that seem more permanent in the long-term (e.g. a hypothetical malaria vaccination, or building infrastructure), are typically much less cost-effective on the margin because of their high cost.
How do we balance pure marginal effectiveness vs eventually moving towards more permanent solutions? Could it be that by overly optimising for marginal cost-effectiveness, we might be missing a better ‘global maximum’ in the utility landscape, but we just need to descend from the current ‘local maximum’ to be able to get there eventually?
I think you're conflating intervention durability with outcome durability? A child who survives cerebral malaria due to seasonal malaria chemoprevention gets to live the rest of their life; SMC programs are rerun because (mostly) new beneficiary cohorts are at highest risk, not because last year's cohort's survival expires somehow. Similarly with nets and child vaccinations and vitamin A deficiency prevention (i.e. the GW top charities), as well as salt iodisation and TaRL in education and many other top interventions recommended by the likes of TLYCS and FP and so on.
I'd also push back a bit on the "permanent solutions" phrasing. Infrastructure isn't that permanent and requires ongoing expenditures and has a shelf half-life (I used to work in ops in fluid resource-constrained environments so I feel this keenly), diseases can develop resistance to vaccines so you need boosters, etc. Ex-AIM CEO Joey Savoie has a great blog talking more about how Someone Always Pays: Why Nothing Is Really "Sustainable".
Phrasing nitpicking aside, some big funders are in fact funding more "permanent / sustainable" solutions.
Open PhilCoefficient Giving's new $120M Abundance and Growth Fund aims to "accelerate economic growth and boost scientific and technological progress while lowering the cost of living", and Founders Pledge (which is almost OP-scale in giving) just launched a new Catalytic Impact Fund that targets "ecosystem leverage points" where small investments can build "sustainable, long-term solutions to global poverty and suffering".ÂJason's comment above on timetable speedup is essentially how e.g. GiveWell models their grants for malaria vaccines. The model says their grant would need to speed up co-administration for all highest need children in all of subsaharan Africa by at least 9 months to clear their 10x bar, so you can interpret their grant as a bet that funding that clinical trial would in fact achieve at least 9 months speedup. Notice how it's an uncertain bet; I think most donors (weighted by dollars moved) care quite a fair bit about certainty of direct benefits, so they'd probably donate to e.g. the Top Charities Fund instead of the more experimental EV-maxxing All Grants Fund.
Given EA's small share of the total global health/poverty funding landscape, the most likely effect of its investment on an expensive-but-permanent project is to speed the timetable up. So, for instance, perhaps we would get a hypothetical vaccine a year or two earlier if there had been EA investment. So, in comparing the effects of a yearly intervention vs. an expensive-but-permanent one, we are still looking at near-term effects that are relatively similar in nature and thus can be compared.Â
I don't suggest that is true for all "permanent" interventions, though, so it isn't a complete answer. It also might not scale well to a field in which EA funding is a large piece of the total funding pie.
I disagree with your point that saving the child's life is something you need to continuously reinvest in[1]. But I do think that you're pointing at something adjacent more along the lines of:
I kind of agree with this. Imo the only real long-term solution is economic growth. But that said, two points:
Funnily enough, your Malaria vaccine is an example of something that
Open PhilanthropyCoefficient Giving has funded. They funded a bunch of malaria vaccine research, notably the stage 3 trials of one of the two vaccines (I forgot which one)Once the child is >5y/o their chance to die drops a lot. Once you save their live once, you really don't have to reinvest to save their life again.Â
(I couldn't find the graph for sub-Saharan Africa, but it looks kind of the same)