A handful of us from EA Salt Lake City are dipping our toes into program evaluation, looking at the impact of an organization run by one of our members. This post highlights the progress we’ve made so far, as well as a number of questions that we’re hoping EA folks with a bit more experience can help us answer. 

TLDR: We’ve found that digging wells in Niger easily clears Open Philanthropy's “Bar”, and we are looking for feedback to see if our analysis makes sense.

The Organization: Wells4Wellness

Wells4Wellness has been operating since 2009, digging (and importantly, maintaining) wells across Niger. Niger has one of the highest rates of death from unsafe water sources in the world at around 80 deaths per 100,000 people per year. Most of those deaths are children under 5 dying from diarrhea. A majority of the country is also sitting on top of the Irhazer-Iullemeden Basin, a large aquifer that Wells4Wellness has found surprisingly easy to tap in locations where clean water is needed most.

The founder (Pat Herath) and director of operations (Willie Herath) recently gave a presentation to the EA SLC group, where they gave us an overview of their organization and explained that they are dedicated to maintaining all their wells, which separates them from other well-drilling efforts that often don’t cover maintenance costs. And in the presentation, they shared a very compelling number - that Well4Wellness delivers clean water for $9 per person for life.

A group of us wanted to dig into this number a bit more. What is the impact of clean water for life? How does it compare to other similar interventions like chlorination and to cost-effective interventions recommended by GiveWell?

Our Current DALY Estimates

We borrowed heavily from GiveWell’s research into chlorination, using the (likely conservative) assumption that clean water from wells would be as effective as chlorination at averting deaths from diarrheal diseases. We used available statistics on mortality in Niger, the mortality rate reduction from clean water interventions, and the cost of creating a new well to estimate the cost-effectiveness of Wells4Wellness in dollars per DALY.

Mortality reductions:

• GiveWell found the mortality rate reduction in children under 5 years old to be between 6-12% depending on the specifics of the population.
• GiveWell found Over 5 mortality rates reductions between 1-4%.
• Given that Niger’s diarrhea burden is higher than the 3 countries GW looked at, we took 12% as our Under 5 reduction and 4% as our Over 5 reduction.

DALYs saved:

• According to UNICEF, there are 23 under-5 deaths in Niger per 1,000 each year. A 12% reduction would mean 2.76 lives are saved per 1,000.
• Assuming an average of 57 years lived per life saved (just taking Niger’s life expectancy of 62 and subtracting 5), that means an intervention on 1,000 children would save 157 DALYs.
• There are 5 over-5 deaths per 1,000 each year in the three countries GW looked at, and we’ll use that number since we couldn’t find a Niger-specific one. A 4% reduction would mean 0.2 lives are saved per 1,000.
• Assuming an average of 47 years lived per life saved (just taking Niger’s life expectancy of 62 and subtracting the median age of 15), that means an intervention on 1,000 over 5’s would save 9 DALYs.
• 20% of the population is under 5.
• An intervention on 1,000 people would save 39 DALYs per year of operation (under 5 DALYs: 157 * 0.2, over 5 DALYs: 9 * 0.8).

We also used a second method to calculate the DALYs saved:

• First, we found an in-depth study which determined that in the year 2016 Niger had 442.1 DALYs lost from diarrheal diseases per 1000 children under 5.
• A different study said that providing a clean drinking water supply reduced diarrhea risk by 52%.
• Considering that 20% of the Nigerian population is under five, we get 442.1 * 52% * 20% = 46 DALYs saved per 1000 people who are given a clean source of drinking water for a year.
• This estimate ignores the benefit to adults, but it includes both mortality and long-term health effects.
• It is also reasonably close to our other estimate. 

Going forward, we will use the smaller estimate of 39 DALYs saved per 1000 people per year to be conservative.

Costs and ROI

We found it difficult to estimate the costs per person per year of having access to well water.

Wells4Wellness provided us with some key numbers:

• Wells provide water for around 1,200 - 1,600 people
• Basic wells cost $5k - $6k to dig
• Chalet wells cost $8k - $22k to dig

We don’t know the percentage of wells that are basic vs chalet, but if we assume ~ 50-50, we get an average of $10,000 for building a well. We also did a quick Google search and found a different organization that quotes their average cost to add a new well at $9,000, which is close enough to give us confidence that our number is reasonable. 

Another key part of a well’s lifetime cost comes from maintenance. Minor repairs are covered by a “well purse” that is budgeted into the original well construction cost. Based on some data from Wells4Wellness on their financials and the major repairs they’ve performed each year, we estimate that major repairs will typically cost around $2,000. And we assume wells need some major repairs every 10ish years on average. 

To keep things simple, we make the (conservative) assumption that the well will only last for ~ 50 years, and the number of people using it will be constant every year. So over 50 years, there will be $8,000 of repairs and $10,000 of initial costs, which works out to around $360 a year in costs. With 1,200 people using the well, that’s $0.30 per person per year! That works out to about double the $9 cost per life from their presentation, but is still very impressive!

Cost per DALY:

• If 1,000 people are served in a year, 39 DALYs will be saved at a cost of $300, or $8 per DALY
• GiveWell also found that the economic effects of chlorination (see rows 147-152 HERE), through reduced healthcare costs and “development effects” (meaning children grow up healthier and earn more throughout their lives as a result), are worth roughly as much as the reductions in mortality. Incorporating those benefits changes our units from DALYs to something less straightforward, but we end up with a cost per DALY-equivalent of only $4!

$4 per DALY is more than 10x under Open Philanthropy’s bar! Too good to be true?

Comparison to Chlorination

We wanted to check how our results compare to water chlorination, which has previously been identified as a compelling cause area. This meta-analysis from 2023 estimates $25-65/DALY averted for chlorination, depending on the intervention. They found the most cost-effective intervention was integrating chlorine distribution and education into existing maternal and child health services, for an estimated $27/DALY averted. Chlorination interventions that involved more infrastructure - point-of-access chlorine dispensers or in-line chlorination - cost an estimated $62-65/DALY averted. 

Compared to our estimates for Wells4Wellness, chlorination is ~5-10x more expensive per DALY averted! 

Chlorination has received much more interest from EA groups than digging wells, and those groups have done more research than we have, so we are pretty surprised and confused by this result!

There are a few other points to note: 

• In the point-of-access chlorination interventions, uptake was around 50%, and in maternal and child health services uptake was only about 30%. The DALY estimates above take adoption rates into effect, but even still an intervention with a nearly 100% uptake (wells) is a lot more compelling.
• Inline chlorination wouldn’t have the same problems with adoption rates, but it does require existing infrastructure to deliver the chlorinated water to people’s homes. None of the villages prioritized by Wells4Wellness have existing wells, let alone plumbing, so this doesn’t seem like a fair comparison for providing clean water in those areas.
• Even if we assume the effects of chlorination and well digging are the exact same, chlorination costs around $1.70 per person per year, which is ~5x more than Wells4Wellness. Even if scaling up chlorine gets the price down to $0.50 per person per year as given here, our estimate for digging wells is still lower. 

Time Horizon and Discounting

Our calculation used a 50-year window and divided the total cost by the DALYs saved over that time. For long time horizons, GiveWell has some discussion around discounting benefits that occur in the future, but there is no clear consensus on what the discount rate should be. 

We think that even a 50 year lifetime for the wells is a conservative estimate, and that in reality they will last much longer. But there may be reasons to discount the benefits occurring that far in the future. For example, other sources of clean water could be built during that time to make the wells less important. Other reasons to discount future benefits are discussed in this article, though we did not find a clear recommendation on what discount rate to use. 

Our Questions

1. Do our DALY estimates make sense? Are there any assumptions that we missed?
2. Is it really plausible that it only costs $0.30 per person per year to supply clean water via wells? Chlorination is considered very cost effective and costs around $2 per person per year.
3. Is our naive method of dividing total cost by total DALYs saved appropriate, or are there any established guidelines for discounting the DALYs saved in the future?
4. A source of fresh water seems like a major quality of life improvement that isn’t captured in DALYs or economic effects; how can we quantify this?