Principal — Good Structures
I previously co-founded and served as Executive Director at Wild Animal Initiative, and was the COO of Rethink Priorities from 2020 to 2024.
Thank you for reading some of the article. I hope that you find some time to read the rest.
To be clear, I read the whole thing - I meant that I think the fact that a pretty important issue jumped out to me within a few minutes of starting reading struck me as a reason that getting feedback from ACE seems really important.
Did you ask ACE to review this before publishing?
I really think you should! I also really think you should ask for feedback from other people who have done charity evaluations, and the charities you evaluate. You should definitely still publish them, but they'll be better critiques for having engaged with the best case for the thing you're critiquing!
What do you mean by conceptualizing funds? In this hypothetical, they simply spend $200k less on the lawsuit. LIC did not spend their entire budget, and charities oftentimes do not. Under ACE’s methodology, LIC’s cost-effectiveness would improve if they spent $200k less and achieved the exact same total outcomes as a charity. The calculations we’ve done are 100% objective, and if you can find an error that we made, please let us know. You can find those calculations here:
Yep, this seems right, but it's also the case that if they did something else with that funding, the effectiveness of that action would be rated much more highly, which also seems correct. I think the issues you point to are interesting, but they strike me as intentional decisions, which ACE may have internal views on, and for which I think getting their feedback might be really important. You are correct about a mathematical fact, but both you and ACE seem to have different goals (calculating historic cost-effectiveness vs marginal impact of future dollars), and there are assumptions underlying your analysis that if changed, might change the output.
What assumptions are you referring to?
I meant ACE's assumptions - I thought your post raised some really good questions. They are issues that if I saw, I'd email to ACE and ask why they made the choices they made, then choose whether or not to publicly publish them based on their response. Maybe these choices are reasonable, and maybe they aren't - you raised some really good points I think. But it just seems hard to evaluate in a vacuum.
LIC has a historical track record, and it is a bad one. People should have the opportunity to start something new. However, they shouldn’t be rated a top charity after receiving over a million dollars in funding and failing to achieve any positive legal outcomes.
Again, I don't really see good evidence for this - what is the typical track record for legal campaigns? How much do they cost? How long do they take to work? These all would be important questions to answer before claiming cost-effectiveness or lack thereof. In this case, I could easily be persuaded to agree with you, but not for any of the reasons in your analysis — the fact that they spent some money some lawsuits and it didn't work isn't the only evidence I'd want to think about whether or not donations to them will be useful.
Did you ask ACE to review this before publishing? It seems like the kind of thing that would be worth getting feedback on before publishing. I didn't look at this for more than a couple minutes, but I saw immediately that there might be some conceptual disagreements between you and ACE - for example, I noticed that in your first example, you assume in your example (I believe), that if LIC didn't spend 200k on the lawsuit against Costco, they wouldn't spend it on anything else. It's unclear to me that this is the counterfactual, or how ACE is conceptualizing those funds. There might be reasoning behind their decisionmaking that would be useful to your critiques they could share.
I also felt like this felt pretty politically motivated. Not sure if that is your intention, but paragraphs like this:
ACE's recommendations determine which animal charities receive millions of dollars in donations.[1] Thus far, we have reviewed 5 of ACE's "Top 11 Animal Charities to Donate to in 2024" and only one of them (Shrimp Welfare Project) appears to be an effective charity for helping animals. ACE's poor evaluation process leads to ineffective charities receiving recommendations, and many animals are suffering as a result.
Without any evidence feels pretty intense. ACE is kind of low hanging fruit to pick on in the EA space, so this read to me like more of that, without necessarily the evidence base to back it. Reading your report, I felt kind of like "oh, there are interesting assumptions here, would be interested to learn more", and not "ACE is doing an extremely bad job."
E.g. I think the questions that would be good to ask in a critique of ACE might be:
I also think getting feedback on statements like this would be really helpful:
The correct formula for calculating cost-effectiveness is simply impact divided by cost. Rather than using this simple formula, ACE has elected to create a methodology that does not properly account for impact or cost.
I think ACE has wanted to do this at points in their history — my impression is just that it is incredibly difficult, so they've approached it from other angles instead. I also don't think it's clear to me that ACE's goal is to report cost-effectiveness. I think clarifying this with them, and getting a sense of why they don't do what you see as the simple approach would be useful for making this critique stronger. And, I don't think people should make giving decisions based only on historic cost-effectiveness - just because an opportunity was impactful doesn't mean the organization needs more funds to do that work, that it will scale, work in the future, etc.
I don't disagree that ACE might be directing funds to ineffective charities! I don't really think non-OpenPhil EA donors should give to farmed animal welfare, for example. But, I don't think it is obvious to me that ACE going away means money going to more effective charities - I expect it would mostly be worse - people giving to animal charities with basically no vetting.
That being said, critique of critical organizations is great in my opinion, so appreciate you putting this out there!
Note that we now have raised $3,015 in pledges for November, and the marginal $25 donation will influence the allocation of around $234. In October, the average donation influenced $221.50, so if you're excited to directly influence the allocation of funding between EA causes, this is still a great way to give right now! You can sign up here.
Thanks! Strongly agree with making it more democratic via some mechanism, and if it survives beyond the first 6 months, I plan on moving it to having some kind of elected oversight group or similar (mainly will figure out how to do that with input from the members). Interesting note on sortition - this seems plausibly like a good use for it. Thanks!
The first month of Equal Hands is complete!
Here are the results: 20 donors pledged to give $4,430 according to the collective preference of the pool.
This resulted in the following donations:
October 2024 | AW | GHD | GCR | EA Community | Climate Change | Total |
Pledge Breakdown | 39.25% | 26.20% | 27.40% | 3.75% | 3.40% | 100% |
Implied Donations | $ 1,738.78 | $ 1,160.66 | $ 1,213.82 | $ 166.13 | $ 150.62 | $ 4,430.00 |
Pseudo Counterfactual | $ 2,711.00 | $ 1,172.25 | $ 354.75 | $ 81.50 | $ 110.50 | $ 4,430.00 |
Implied Change | -$972.23 | -$11.59 | +$859.07 | +$84.63 | +$40.12 |
All but 2 donors met their pledge, and $4,355 was given following the system. Backstopping funders covered the $75 gap left by the two donors.
Interestingly, the net effect (compared to the pseudo counterfactual of the money being distributed by each donor according purely to their preferences) of Equal Hands in October was roughly to move ~$900 from animal welfare to GCR areas. From the data, it looks like the primary cause of this was that animal welfare-motivated donors were most likely to give the largest amounts, but GCR donors were more likely to sign up (especially at the minimum, $25).
We're running 5 more months of this trial, and you can sign up here.
I think it would be pretty hard for me to make that trade off in a workplace context (I think I'm still a deep sucker for impact and in any real version of this is X would be whatever the organization is indifferent towards and I'd donate it). If you forced me to in some hypothetical I'd guess X is quite low for many junior roles (<$10k), but higher for more mid/senior roles (>$50k?). But I think something like the following are true:
Thanks! My wording in the above message was imprecise, but I mean something like farmed vertebrates. SWP is probably among the two most important things to fund, in my opinion.
Basically I think the size of good opportunities in farmed animal advocacy is smaller than OpenPhil's grantmaking budget and there are few scalable interventions, though I don't think I want to go into most the reasons publicly. Given that they've stopped funding many of what I believe are more cost-effective projects, and that EA donors are basically the only people willing to fund those, EA donors should be mostly inclined to fund things OpenPhil can't fund instead.
So some combination of 1+2 (for farmed vertebrates) + other factors