Comments
Congratulations, I'm very excited about the work that CE is doing.
BTW, I'd be interested in reading a post of post-mortems on CE-incubated projects that didn't pan out.
Congratulations, I'm very excited about the work that CE is doing.
BTW, I'd be interested in reading a post of post-mortems on CE-incubated projects that didn't pan out.
Already in our content calendar :)
Thank you for writing this and for all the work you (and others) have put in over the years.
My question is to what extent you think CE’s impact measurement is tautological. If you determine something to be a high impact opportunity and then go and do it, aren’t you by definition doing things you estimate to be high impact (as long as you don’t screw up the execution or realise you made an error). To full adjust for selection effect, you would have to ignore all research conducted before the decision was made and rely solely on new data, which is probably quite hard to come by.
The 40% seems very high. For-profit start-ups have a much higher failure rate. If that’s true, that’s incredible, but I’d expect to see more like 5% of charities and 50% of funds.
I think tautological measurement is a real concern for basically every meta charity, although I'm not sure I agree with your solution. I think the better solution is external evaluation, someone like GiveWell or Founders Pledge who does not have any reason to value CE charities. Typically, these organizations do their own independent research and compare it across their current portfolio of projects. If CE can, for example, fairly consistently incubate charities that GW/FP/etc. rank as best in the world, I think that is at least not organizationally tautological (it is assuming that these charity evaluators are in fact identifying the best areas/charities, and replicating any flaws they have though).
In terms of success rate, I agree 40% is high but I would expect many NGO incubators to be considerably higher than in the for-profit space, for a few reasons (a couple listed below):
General competition: There are just not that many charities aiming for pure impact (in an EA way), unlike the for-profit market. The general efficiency of the charity market is pretty low, and thus there are lots of fairly easy wins.
Scale sensitivity: Generally, successful for-profits are seen as really large-scale ventures (e.g., unicorns) and the market is consistently hunting for that. Debatably, the only charity currently seen as highly impactful that can get to that sort of scale is GiveDirectly. Thus the bar for success in the charity sector is significantly lower in terms of a money spent scale. For example, if we founded a charity running with a 1m a year budget, that was x2 as effective as top GW ones, we could count that as a success. But an organization of the same size would be considered a rounding error by YC. If we take size expectations into account, it might be like 1/25 charities that we incubated that have any significant chance of getting to unicorn-level size.
Thanks, Joey. Really appreciate you taking the time to engage on these questions.
To be clear, I’m not seriously suggesting ignoring all research from before the decision. I’m just saying that mathematically, an independent test needs its backrest data to exclude all calibration data.
It strikes me that there are broadly 3 buckets of risk / potential failure:
The high success rate almost makes me think CE should be incubating even more ambitious, riskier projects, with the expectation of a lower success rate but higher overall EV. Very uncertain about this intuition though, would be interested to hear what CE thinks.
The CE incubatees are an absolutely amazing bunch and exactly the types of people I would want on these world-bettering projects. Charity Entrepreneurship is no doubt one of the EA projects I am most excited about due to pure impact, research prowess and future potential.
When I compare the CE program to YC (see also OWID@YC), it feels even better due to the great co-founder matching process and the success rate along with the excellence within the focus areas (people don't come with their own esoteric tech startups).
For other commenters who talk about the use of reach numbers instead of e.g. QALY or WELLBYs, having seen some of their spreadsheets and programs, I am in deep awe and respect of some of the superheroes that have saved hundreds of lives (with many of these being counterfactual given the effectiveness and new vectors of impact) though I'll let them summarize their numbers.
I am very excited about where the projects will be in one and even five years and commend everyone involved.
Although I think this is incredible, there is a nagging feeling in the back of my mind that it may be difficult to sustain the kind of success rate that CE is currently achieving. Do you anticipate that, as you run more incubation programs, you will see more incubated charities failing or shutting down?
Do you have a sense of whether you will eventually exhaust cost-effective charity ideas as you continue to run more incubation programs?
Thank you for your comment, Yadav.
Regarding success rates, we expect the net impact per charity to generally improve because both our program and outreach have gotten stronger. This improvement could still happen even with the overall success rate of our charities decreasing, with more charities shutting down but more charities having large wins, especially in policy. Over time, we expect some of our charities to shut down, with our general estimate being that one in five will.
Regarding cost-effective ideas, we recognize this as a legitimate concern and something we think about quite a bit. There might be some diminishing returns on the impact of ideas we can find, but researching multiple cause areas has slowed this down a lot. We expect some level of a natural refresh rate as more studies and new evidence comes out on what are the most effective things to do. For example, we expect one new great idea per cause area to become newly available per year. Our current estimates suggest we could start about 10 charities across four cause areas a year for the foreseeable future.
- 40% of our charities reach or exceed the cost-effectiveness of the strongest charities in their fields (e.g., GiveWell/ACE recommended).
- 40% are in a steady state. This means they are having impact, but not at the GiveWell-recommendation level yet, or their cost-effectiveness is currently less clear-cut (all new charities start in this category for their first year).
- 20% have already shut down or might in the future.
Thanks for writing this up. I am excited about the work you are doing, but to be blunt, these success rates strike me as implausibly good. Here are a few reasons why I am skeptical:
I have read your response here, and agree that it'd be good to have an external organization do a comprehensive evaluation.
TL;DR: Charity Entrepreneurship have helped to kick-start 23 impact-focused nonprofits in four years. We believe that starting more effective charities is the most impactful thing we can do. Our charities have surpassed expectations, and in this post we provide an update on their progress and achievements to date.
At Charity Entrepreneurship (CE) we launch high-impact nonprofits by connecting entrepreneurs with the effective ideas, training and funding needed to launch and succeed. We provide:
(Applications are now open to our 2023/2024 programs, apply by March 12, 2023).
We estimate that on average:
To date, our CE Seed Network has provided our charities with $1.88 million in launch grants. Based on the updates provided by our charities in Jan 2023, we estimate that:
1. They have meaningfully reached over 15 million people, and have the potential to soon reach up to 2.5 billion animals annually with their programs. For example:
2. They have fundraised over $22.5 million USD from grantmakers like GiveWell, Open Philanthropy, Mulago, Schmidt Futures, Animal Charity Evaluators, Grand Challenges Canada, and EA Animal Welfare Fund, amongst others.
3. If implemented at scale, they can reach impressive cost-effectiveness. For example:
FWI is an ACE standout charity that we helped launch in September 2019. They work with farmers, corporations, and governmental agencies to improve the welfare of fish. At the end of 2022, the organization hit their first million fish helped, and they transitioned 89 farms to higher welfare standards. With a team of 20, FWI is pioneering effective animal advocacy for fish on multiple levels. From conducting some of the first research on improving the welfare of Indian major carp, through creating the Alliance for Responsible Aquaculture, with 58 fish farms currently enrolled, to signing India’s first corporate commitment for fish. FWI’s successes inspired CE to launch a similar organization in the Philippines through our current February-March 2023 program.
LEEP drives effective policies to eliminate lead poisoning across the globe. Incubated in 2020, they secured their first policy change in Malawi after eight months of operation. The Malawi program alone will decrease lead poisoning in ~215,000 children and avert 43,000 DALY-equivalents. Since then, LEEP has received government commitments of regulation implementation in five countries. Overall, LEEP is now running lead paint elimination programs in ten countries, which (as mentioned above) they estimate will have a similar impact on average.
SWP is the first ever organization that aims to significantly improve the lives of billions of farmed shrimp. Since launching in September 2021, the organization has signed 11 MoUS (Memoranda of Understanding) with relevant stakeholders that may lead to improving the lives of >2.5 billion shrimp per annum. With 625 shrimp potentially helped per dollar spent (their internal estimation), SWP could become the most cost-effective animal advocacy organization in the world.
Incubated in 2021, FEM prevents maternal deaths and other health burdens due to unintended pregnancies through an evidence-based solution. FEM produces informative radio broadcasts that shift knowledge, attitudes, and behavior around family planning in Nigeria. Since launch, the organization has managed to reach an estimated audience of 5 million listeners in Kano State, Nigeria, and has built the partnerships and operational capacity to reach 10 million more listeners across three other Nigerian states: Anambra, Kogi, and Ondo. By 2030, FEM expects to reach a total of 50 million people across the country. FEM’s rapid and significant success has inspired CE to devote our next research cycle to mass media interventions.
Many of the forum users are probably not aware that we incubated Happier Lives Institute (HLI) in its early stage through our 2019 Incubation Program. Since then, the organization published break-through papers on subjective well-being, including the effects of cash transfers that led to discovering an error in GiveWell’s analysis of deworming, which reduces the cost-effectiveness estimate by 10%-30%.
In 2020, we launched another mental-health-focused organization called Canopie that evolved to be a scalable and cost-effective digital program to prevent and treat perinatal mood disorders. The organization partnered with the Mayor of Washington, DC to provide Canopie to all expecting and new mums, and is now working to secure relationships with healthcare organizations in the US and the UK. This would allow Canopie to become accessible to millions of new mums per year. Once scaled, Canopie would cost $1.06 per mother reached, and $42.43 per mother protected from depression.
In our 2022 program, we additionally incubated two mental-health charities: Vida Plena, an organization focused on implementing cost-effective programs in Latin America (the team estimates they can be at least 8x more cost-effective than GiveDirectly in terms of improving subjective well-being, if run at scale) and Kaya Guides, an organization that aims to reduce depression among youth in India through a guided self-help program. The team estimates they could be 45x more cost-effective at increasing subjective well-being than direct cash transfers in LMICs. Once at scale, the marginal cost of helping an additional person could be as low as $3.93.
We have created an Our Charities page on our website, where you can find up-to-date information about all of our incubated interventions, their progress, achievements, cost-effectiveness and funding gaps. We highly encourage you to visit it to learn more.
We believe that we have a proven methodology for reliably creating effective charities. We’ve refined our approach, strengthened our team, and are now moving to scale Charity Entrepreneurship. We will run the Incubation Program twice per year, and start between eight and ten new organizations annually. We have learned from our mistakes and challenges (we will write a post about them too), as well as from our incubatees’ successes. There remains considerable uncertainty and a huge amount of scope for growth, but we are increasingly confident that our model represents a clear and counterfactually significant benchmark to beat. Few, if any, careers have evidence showing that they do as much good or reduce as much suffering.
We hope that even more people in the impact-focused community will decide to choose nonprofit entrepreneurship as their full-time job. As shown above, when the right ideas are launched, it can lead to changing millions, or even billions, of lives for the better.
Applications are now open, with the deadline on March 12 (apply here) and you can launch the interventions below through our July-August 2023 Incubation Program:
If interested, you can read more about them here.
If you have any questions or hesitations about applying, contact us at: [email protected]
Nice one. The success rate is quite phenomenal - especially how committed the founders are to bringing their concepts to fruition. Your biggest strength might be in selecting people even more than selecting causes.
My one slight issue with the data presentation is the use of "people reached" or "animals reached" as a headline metric. To some extent I understand using it outside of EA circles as we know that the biggest numbers sound the most impressive, but I don't think it s a impact measurement with integrity. Basically any org that does mass media will reach millions very fast which is great, but it doesn't necessarily translate to impact.
Endless NGOs spend millions on fairly useless media messages here - give me 10,000 dollars tomorrow in Uganda and I can reach 1 million people with whatever message you like - that's not an impact measurement, what matters is the result of that message - which looks to be great with CE orgs. What sets your orgs apart is that their approach is backed by evidence and is likely to lead to real positive impact - not the fact that they can reach millions over the radio, anyone can do that!
Not the biggest deal, but I think within EA we can do better with our headline metrics.
Small question also, why no mention of Fortify Health, who I think are the CE org which has got the most funding to date and have done an amazing job?
Thank you for your comment, Nick. I used the word "reach" because it is difficult to objectively measure the impact of some of our charities, given their age. We wanted to give people a sense of their current progress despite this. Reach can also give a sense of the potential scale of the intervention. We also tried to include some estimated cost effectiveness to show the sense of level of impact per dollar. However, overall we do agree with the general sentiment that, sadly, people tend to look at more vanity level metrics like reach, funding, etc. Of course, many of our organizations will need to run an RCT (which some are already planning) before we can get a real sense of their impact, and we believe our organizations are committed to do this.
We indeed agree that Fortify Health has done a really great job, but we had to limit the number of charities we could talk to for this post, and some charities were more cautious about being associated with the EA movement. Feel free to check our website for updates.