EA Forum Bot Site
EA Forum

Take the 2025 EA Forum Survey to help inform our strategy and prioritiesTake the survey
Hide table of contents
Comment Permalink
alamo 29141
0
0

I might've used too strong of a language with my original post, such as the talk about being a sucker. For me it's useful to think about donations as a product I'm buying, but I probably took it too far. And I don't think I've properly emphasized my main message, which was (as I've added later) - the explore-exploit tradeoff for causes is really hard if you don't know how far exploration could take you. Honestly, I'm most interested in your take on that. I initially only used GiveWell and CEARCH to demonstrate that argument and show I how got to it.    

The drug analogy is interesting, although I prefer the start-up analogy. Drug development is more binary - some drugs can just flat-out fail in humans, while start-ups are more of a spectrum (the ROI might be smaller than thought etc.). I don't see a reason to think of CEARCH recommended programs or for most other exploratory stuff as binary. Of course lobbying could flat-out fail, but it's unlikely we'll have to update our beliefs that this charity would NEVER work, as might happen in drug development. And obviously with start-ups, there's also a lot of difference between the initial market research and the later stages (as you said).  

GiveWell has a lot of flaws for cause exploration. They really focus on charity research, not cause research. It's by design really biased towards existing causes and charities. The charities must be interested and cooperate with GiveWell. They look for the track record, i.e. charities operating in high-risk, low tractibillity areas such as policy have a harder time. In most cases it makes sense, sometimes it can miss great opportunities. 

Yes, they've funded some policy focused charities, but they might've funded much more if they were more EV maximizing instead of risk-averse. Seeing the huge leverage such options provide, it's entirely possible. 

Also, they aren't always efficient - look at GiveDirectly. Their bar for top charities was 10x GiveDirectly for years, yet they kept GiveDirectly as a top charity until last year??? This is not some small, hard to notice inefficiency. It literally is their consistent criteria for their flagship charities. Can you imagine a for-profit company telling their investors "well, we believe these other channels have a ROI of at least 10x, but please also consider investing in this channel with x ROI", for multiple years? I can't. Let alone putting that less efficient channel as one of the best investments…

That's exactly what I mean when I say altruism, even EA, can have gross inefficiency in allocations. It's not special to GiveWell, I'm just exemplifying. 

If GiveWell can make such gross mistakes, then probably others can. Another example was their relative lack of research on family planning, which I've written about. They're doing A LOT of great things too. But I must say I am a bit skeptical of their decision making sometimes. 

Keep in mind, CEARCH would have to be EXTREMELY optimistic in order for us to say that it hasn't found a couple of causes 10x GiveWell. We are talking about 40x optimistic. That might be the case, but IMO it’s a strong enough assertion to require proof. Do you have examples of something close to 40x optimism in cost-effectiveness? 

I agree that a lot of the difference in EAs donations can come from differing perspectives, probably most. But I think even some utilitarian, EV maximizing, 0-future discount, animal equalists EAs donate to different causes (or any other set of shared beliefs). It's definitely not impossible. 

As for other examples of 10x GiveWell cost-effectiveness in global health: 

  • CE has estimated another charity yields $5.62 per DALY.
  • An Israeli non-profit, which produced an estimate of 4.3$ QALYs per dollar, in cooperation with EA Israel. A volunteer said to me he believed they were about 3x too optimistic, but that’s still around 10x GiveWell. 

Also here is an example of 4x disagreement between GiveWell and Founders Pledge, and an even bigger disagreement with RP, on a mass media campaign for family planning. Even the best in the business can disagree. 

Sorry for this being a bit of a rave
 

Reply
See in context
by alamo 2914
2 min read1 answer 0

4

Cause prioritizationEffective givingForecastingPhilosophyCause XOpinionThreads
Frontpage
Show all topics

TL;DR: The explore-exploit tradeoff for causes is impossible if you don't know how far exploration could take you - how good the best causes may be. 

Recently, I found out that Centre for Exploratory Altruism Research (CEARCH) estimates, with a high confidence level, that advocacy for top sodium reduction policies is around 100x as cost-effective as top GiveWell charities, in terms of DALYs. This made me feel like a sucker, for having donated to GiveWell. 

You see, when I'm donating, I think of myself as buying a product - good for others. The hundreds of dollars I donated to GiveWell could've probably been replaced with a couple of dollars to this more effective cause. That means that I wasted hundreds of dollars, that could've done much more good. 

But let's use milk as an analogy. GiveWell is the equivalent of buying a liter of milk for 200$. If that happened with a product I wanted to buy for myself, I would probably feel scammed. A literal lifetime of donations to GiveWell might be replaced with 6 months of donating to this cause.  I'm not saying I got scammed, but thinking about it from the perspective of buying good intuitively helps. Nowadays I don't donate to GiveWell anyways, but it sucks.

This - being a sucker who pays too much for doing good - is really bad. It's exactly what we try to avoid in EA. It can decrease our impact by orders of magnitude. 

And that's not even the end. CEARCH also estimated (although with a low level of certainty) that nuclear arsenal limitation could be 5000x cost-effective as top GiveWell charities. If they're wrong by an order of magnitude, that's still 5x times better then even the hypertension work. Now donating to GiveWell is like buying a liter of milk for 1000$. And who's to say that there's nothing 50x more effective than the nuclear cause? 

At some point, you might consider to stop buying milk for the moment, and looking around for the cheapest prices. And you'll probably not get the cheapest, but you might be able to figure a reasonable estimate of them, and buy at a rate close to it.

So this situation made me think - can we put a reasonable limit to the maximum cost-effectiveness of our money and time? Within a 50% CI? 80 CI? 99% CI... And can we have a reasonable estimate for the time it will be reached?

For someone extremely risk-averse it's probably easy, as there are only so much RCTs in our world. And it's seems likely that GiveWell is within a close range. But for anyone that's risk-neutral, I can't think of a good way. So I've come to ask you wonderful people - what do you think?

The end result I'm thinking of is something like: 

"The maximum cost-effectiveness we can expect is 500x-50,000x that of GiveWell (95% CI). The year we expect to find a cause within an order of magnitude of that cost-effectiveness is 2035-2050 (50% CI)".

But of course any input will be interesting. 

Things that could potentially limit cost-effectiveness, off the top of my head:

  • A weak form of the 'stable market hypothesis' for doing good.
  • Hedonic adaptation - people adapt to better circumstances. The amount of good we can induce in anyone's lives is thus limited. 
  • Caps on the amount of humans & other beings that are likely to ever live.

 

Note: I'm emotionally content with my past donations to GiveWell, don't worry. Also, this is not a diss on GiveWell, they're doing a great job for their goals. 

4

0
0

Reactions

0
0
New Answer
New Comment


1 Answers sorted by
Top

Jason

7
5
1

CEARCH's work is, as the name implies, exploratory. It searches for new potential programs and cause areas. It recognizes that early-stage CEAs often overstate the effectiveness of a program. Even its deep CEA on hypertension relies on a lot of guesswork in my view.

I personally think its work is often seriously overoptimistic, especially when it comes to predicting the success and ease of lobbying efforts against considerable opposition. But that's more okay given its exploratory function. If there were a shovel-ready sodium-reduction charity ready to receive and deploy significant funds, I would view that as akin to buying equity in a brand-new startup.

GiveWell does something entirely different with its Top Charities -- you're getting a specific implementation of a program by a carefully vetted charity, with a pretty tight cost-effectiveness analysis based on either the charity's actual performance or efficacy of similar real-world interventions in a randomized controlled trial. 

The extent to which people should prefer tried-and-true interventions versus novel-but-potentially-higher-impact ones is a recurrent question. But most of us retail global-health donors who have done our research, as well as the bulk of the money from big donors in EA global health, goes to the tried-and-true. That is a pretty good clue that we are not all being "scammed" here!

Finally, to the extent that work with insanely good cost-effectiveness exists, that work often has limited room for more funding. Here, for instance, both of your examples involve lobbying.

Joel Tan🔸13
2
0

Hi Jason,

I agree with the broad thrust of your comment that there's a tradeoff between guaranteed performance vs potentially higher impact stuff.

That said, I would push back on two points:

(1) With respect to our work being "seriously overoptimistic, especially when it comes to predicting the success and ease of lobbying efforts against considerable opposition." I think some of our earlier/shallower  CEAs definitely do suffer from that, but our deep CEAs are (a) highly comprehensive, factoring in a multiple of relevant temporal/epistemic discounts; and (b) are far more sceptical of advocacy efforts (e.g. we think there is a 6% chance of success for sodium reduction advocacy in a single country over 3 years. i.e. if you did 48 country-years of lobbying you'll get one win). Amongst the various considerations informing this estimate is, of course, the reality of industry pushback.

(b) I disagree to a limited but fairly significant extent, on the notion that such work has limited room for more funding. Scaling works differently in advocacy than in direct delivery. You pay more to buy increased chances of success (e.g. the difference between funding a small NGO vs hiring a professional lobbying outfit vs hiring ex-government officials to lobby for you). Of course, diminishing marginal returns apply (as opposed to economies of scale in direct delivery - though in the long run, there's always DMR as we work our way down the list of priority countries).

Reply
alamo 29143
0
0

Thanks for the reply!

If I understand your main arguments correctly, you're basically saying that high cost-effectiveness options are rare, uncertain, and have a relatively small funding gap that is likely to be closed anyways. Also new charities are likely to fail, and can be less effective. And smart EAs won't waste their money. 

Uncertainty and rarity: Assume that CEARCH is on average 5x too optimistic about their high confidence level report, 20x too optimistic about their low confidence stuff (that's A LOT). Still, out of 17 causes they researched,... (read more)

Reply
2
Jason
I think we agree on somewhat more than it seems at first glance. I don't think the current GiveWell top charities are the pinnacle of cost-effectiveness, support further cause exploration and incubating the most promising ideas into charities, and think it's quite possible for EA funders to miss important stuff.  The crux is that I don't think it's warranted to directly compare cost-effectiveness analyses conducted on a few weeks of desktop research, expert interviews and commissioning of surveys and quantitative modelling to evaluations of specific charities at scale and in action, and I think your original post did that with allusions to scamming, GiveWell charities as $1000 liters of milk, and being a sucker.  Although CEARCH is too young to us retrospectively compare its analyses to the cost-effectiveness of launched charities, I think something like drug development is a good analogy. Lots of stuff looks great on paper, in situ, or even in animal models, only to fall apart entirely in the multi-phase human clinical trial process on the way to full approval. Comparing how a drug does in limited animal models to how another drug does in Phase III trials is comparing apples to oranges. Moreover, "risk the model/drug will fall apart in later phases" is distinct from "risk after Phase III trials that the model/drug will not work in a specific country/patient." To be very clear, this is not a criticism of CEARCH -- as I see it, its job is to screen candidate interventions, not to bring them up to the level of mature, shovel-ready interventions. The next step would be either incubation or a deep dive on a specific target charity already doing this work. I would expect to see a ton of false positives, just as I would expect that from the earliest phases of drug development. It's worth it to find the next ten-figure drug / blockbuster EA intervention. I think this should make you question your assumptions to some extent. GiveWell has evaluated tons of interventions
alamo 29141
0
0

I might've used too strong of a language with my original post, such as the talk about being a sucker. For me it's useful to think about donations as a product I'm buying, but I probably took it too far. And I don't think I've properly emphasized my main message, which was (as I've added later) - the explore-exploit tradeoff for causes is really hard if you don't know how far exploration could take you. Honestly, I'm most interested in your take on that. I initially only used GiveWell and CEARCH to demonstrate that argument and show I how got to it.    

The drug analogy is interesting, although I prefer the start-up analogy. Drug development is more binary - some drugs can just flat-out fail in humans, while start-ups are more of a spectrum (the ROI might be smaller than thought etc.). I don't see a reason to think of CEARCH recommended programs or for most other exploratory stuff as binary. Of course lobbying could flat-out fail, but it's unlikely we'll have to update our beliefs that this charity would NEVER work, as might happen in drug development. And obviously with start-ups, there's also a lot of difference between the initial market research and the later stages (as you said).  

GiveWell has a lot of flaws for cause exploration. They really focus on charity research, not cause research. It's by design really biased towards existing causes and charities. The charities must be interested and cooperate with GiveWell. They look for the track record, i.e. charities operating in high-risk, low tractibillity areas such as policy have a harder time. In most cases it makes sense, sometimes it can miss great opportunities. 

Yes, they've funded some policy focused charities, but they might've funded much more if they were more EV maximizing instead of risk-averse. Seeing the huge leverage such options provide, it's entirely possible. 

Also, they aren't always efficient - look at GiveDirectly. Their bar for top charities was 10x GiveDirectly for years, yet they kept GiveDirectly as a top charity until last year??? This is not some small, hard to notice inefficiency. It literally is their consistent criteria for their flagship charities. Can you imagine a for-profit company telling their investors "well, we believe these other channels have a ROI of at least 10x, but please also consider investing in this channel with x ROI", for multiple years? I can't. Let alone putting that less efficient channel as one of the best investments…

That's exactly what I mean when I say altruism, even EA, can have gross inefficiency in allocations. It's not special to GiveWell, I'm just exemplifying. 

If GiveWell can make such gross mistakes, then probably others can. Another example was their relative lack of research on family planning, which I've written about. They're doing A LOT of great things too. But I must say I am a bit skeptical of their decision making sometimes. 

Keep in mind, CEARCH would have to be EXTREMELY optimistic in order for us to say that it hasn't found a couple of causes 10x GiveWell. We are talking about 40x optimistic. That might be the case, but IMO it’s a strong enough assertion to require proof. Do you have examples of something close to 40x optimism in cost-effectiveness? 

I agree that a lot of the difference in EAs donations can come from differing perspectives, probably most. But I think even some utilitarian, EV maximizing, 0-future discount, animal equalists EAs donate to different causes (or any other set of shared beliefs). It's definitely not impossible. 

As for other examples of 10x GiveWell cost-effectiveness in global health: 

  • CE has estimated another charity yields $5.62 per DALY.
  • An Israeli non-profit, which produced an estimate of 4.3$ QALYs per dollar, in cooperation with EA Israel. A volunteer said to me he believed they were about 3x too optimistic, but that’s still around 10x GiveWell. 

Also here is an example of 4x disagreement between GiveWell and Founders Pledge, and an even bigger disagreement with RP, on a mass media campaign for family planning. Even the best in the business can disagree. 

Sorry for this being a bit of a rave
 

Reply
More from alamo 2914
56
Copenhagen Consensus Center's newest research on global poverty - we should be talking about this
alamo 2914
· · 2m read
13
13
View more
Curated and popular this week
 ·  · 1m read
 · 
This morning I was looking into Switzerland's new animal welfare labelling law. I was going through the list of abuses that are now required to be documented on labels, and one of them made me do a double-take: "Frogs: Leg removal without anaesthesia."  This confused me. Why are we talking about anaesthesia? Shouldn't the frogs be dead before having their legs removed? It turns out the answer is no; standard industry practice is to cut their legs off while they are fully conscious. They remain alive and responsive for up to 15 minutes afterward. As far as I can tell, there are zero welfare regulations in any major producing country. The scientific evidence for frog sentience is robust - they have nociceptors, opioid receptors, demonstrate pain avoidance learning, and show cognitive abilities including spatial mapping and rule-based learning.  It's hard to find data on the scale of this issue, but estimates put the order of magnitude at billions of frogs annually. I could not find any organisations working directly on frog welfare interventions.  Here are the organizations I found that come closest: * Animal Welfare Institute has documented the issue and published reports, but their focus appears more on the ecological impact and population decline rather than welfare reforms * PETA has conducted investigations and released footage, but their approach is typically to advocate for complete elimination of the practice rather than welfare improvements * Pro Wildlife, Defenders of Wildlife focus on conservation and sustainability rather than welfare standards This issue seems tractable. There is scientific research on humane euthanasia methods for amphibians, but this research is primarily for laboratory settings rather than commercial operations. The EU imports the majority of traded frog legs through just a few countries such as Indonesia and Vietnam, creating clear policy leverage points. A major retailer (Carrefour) just stopped selling frog legs after welfar
 ·  · 10m read
 · 
This is a cross post written by Andy Masley, not me. I found it really interesting and wanted to see what EAs/rationalists thought of his arguments.  This post was inspired by similar posts by Tyler Cowen and Fergus McCullough. My argument is that while most drinkers are unlikely to be harmed by alcohol, alcohol is drastically harming so many people that we should denormalize alcohol and avoid funding the alcohol industry, and the best way to do that is to stop drinking. This post is not meant to be an objective cost-benefit analysis of alcohol. I may be missing hard-to-measure benefits of alcohol for individuals and societies. My goal here is to highlight specific blindspots a lot of people have to the negative impacts of alcohol, which personally convinced me to stop drinking, but I do not want to imply that this is a fully objective analysis. It seems very hard to create a true cost-benefit analysis, so we each have to make decisions about alcohol given limited information. I’ve never had problems with alcohol. It’s been a fun part of my life and my friends’ lives. I never expected to stop drinking or to write this post. Before I read more about it, I thought of alcohol like junk food: something fun that does not harm most people, but that a few people are moderately harmed by. I thought of alcoholism, like overeating junk food, as a problem of personal responsibility: it’s the addict’s job (along with their friends, family, and doctors) to fix it, rather than the job of everyday consumers. Now I think of alcohol more like tobacco: many people use it without harming themselves, but so many people are being drastically harmed by it (especially and disproportionately the most vulnerable people in society) that everyone has a responsibility to denormalize it. You are not likely to be harmed by alcohol. The average drinker probably suffers few if any negative effects. My argument is about how our collective decision to drink affects other people. This post is not
 ·  · 5m read
 · 
Today, Forethought and I are releasing an essay series called Better Futures, here.[1] It’s been something like eight years in the making, so I’m pretty happy it’s finally out! It asks: when looking to the future, should we focus on surviving, or on flourishing? In practice at least, future-oriented altruists tend to focus on ensuring we survive (or are not permanently disempowered by some valueless AIs). But maybe we should focus on future flourishing, instead.  Why?  Well, even if we survive, we probably just get a future that’s a small fraction as good as it could have been. We could, instead, try to help guide society to be on track to a truly wonderful future.    That is, I think there’s more at stake when it comes to flourishing than when it comes to survival. So maybe that should be our main focus. The whole essay series is out today. But I’ll post summaries of each essay over the course of the next couple of weeks. And the first episode of Forethought’s video podcast is on the topic, and out now, too. The first essay is Introducing Better Futures: along with the supplement, it gives the basic case for focusing on trying to make the future wonderful, rather than just ensuring we get any ok future at all. It’s based on a simple two-factor model: that the value of the future is the product of our chance of “Surviving” and of the value of the future, if we do Survive, i.e. our “Flourishing”.  (“not-Surviving”, here, means anything that locks us into a near-0 value future in the near-term: extinction from a bio-catastrophe counts but if valueless superintelligence disempowers us without causing human extinction, that counts, too. I think this is how “existential catastrophe” is often used in practice.) The key thought is: maybe we’re closer to the “ceiling” on Survival than we are to the “ceiling” of Flourishing.  Most people (though not everyone) thinks we’re much more likely than not to Survive this century.  Metaculus puts *extinction* risk at about 4
Recent opportunities in Cause prioritization
99
Can for-profit companies create significant, direct impact?
T.K., Ambitious Impact, Jacintha Baas
· · 9m read
6
6
66
1Day Sooner Open Phil Funding Request
1Day Sooner
· · 1m read
3
3
63
AIM Grantmaking Program for Funders: Upcoming Cohorts
Judith Rensing, Joey🔸, Ambitious Impact
· · 2m read
0
0
View more