Background
Let's say you're choosing between donating to a charity that helps people living in a developed country and donating to a charity that helps people living in a developing country. Where will your donation do more good? According to GiveWell,
your dollar will go further overseas.
As far as I'm aware, there are four comparisons that are used to support GiveWell's claim.
The
first comparison is between a) how much utility an American living at the U.S. poverty line would gain from a cash transfer and b) how much utility a Kenyan living at the global extreme poverty line would gain from the same transfer. Since the relationship between utility and income
roughly follows a logarithmic curve, it can be modeled using the function u = ln(x) where u is utility and x is income. The derivative of this function, du/dx = 1/x, shows how much a person's utility changes as their income changes. For the American at the U.S. poverty line, who has an
income of around $12,000 USD, an extra dollar results in a gain of 1/12,000 units of utility. For the Kenyan at the global extreme poverty line, who has an
income of around $500 USD (purchasing power parity adjusted), an extra dollar results in a gain of 1/500 units of utility. Thus, the Kenyan benefits 24 times as much as the American [1/500 / 1/12,000 = 24]. The biggest problem with this comparison is that it is
highly sensitive to the function selected, and the function used above does not come from experiments involving cash transfers. The correct function could be quite different, meaning the true ratio may be much lower or higher.
The
second comparison is between a) how much a developed country is willing to pay to save the life of one of its citizens and b) how much it costs a highly effective charity to save a life in a developing country. The United Kingdom National Health Service is willing to pay approximately $1,000,000 to save the life of one of its citizens (click
here and go to footnote 10). By contrast, Malaria Consortium can save a life in a developing country for
around $2,500. Assuming that the United Kingdom fully funds every intervention that can save the life of one of its citizens for less than $1,000,000, the most effective intervention in the United Kingdom that is still in need of funding will cost at least as much as $1,000,000 for each life saved. This would mean that it costs at least 400 times as much to save a life in a developed country than a developing country [$1,000,000 / $2,500 = 400]. The primary issue with this comparison is that it is highly unlikely that developed countries have fully funded all interventions that can save the life of one of their citizens for less than their willingness to pay to save the life of one of their citizens.
The
third comparison is between a) how much it costs to provide a blind person in a developed country with a guide dog and b) how much it costs to prevent blindness in a person living in a developing country through trachoma surgery. Since
it costs $50,000 to train a guide dog but only $125 to prevent one case of blindness through trachoma surgery* (and assuming that curing blindness is at least as good as providing a guide dog), donating to trachoma surgery is at least 400 times as effective as donating to a guide dog charity [$50,000 / $125 = 400]. This comparison is flawed since it is highly unlikely that guide dog provision is one of the most cost-effective developed country interventions, whereas it is at least plausible that trachoma surgery is one of the most cost-effective developing country interventions. Additionally, this comparison may create the perception that effective altruism is ableist (which may be reason enough to avoid the comparison even if you believe the perception is unfair).
The
fourth comparison is between a) how much it costs for a GiveWell top charity working in a developing country to save a life and b) how much it costs for various U.S. charities previously recommended by GiveWell to serve one person or family. Since the cost for Malaria Consortium, a GiveWell top charity, to save a life (
$2,500) is less than the cost for the U.S. charities to serve a person or family (
generally around $10,000) and since saving a life has a bigger impact than the impact of the U.S. charities serving one person or family, giving overseas is likely to be significantly more effective than giving in the United States. The biggest limitation to this comparison is that it does not say how much more impactful saving a life is than serving one U.S. person or family, making it difficult to estimate how much more cost effective the overseas charities are.
Thus, none of the existing comparisons meet the following two criteria: 1) the comparison measures the benefits of the two interventions using the same unit 2) the estimates of the impact for both interventions are based on experimental evidence. In this post, I will attempt two comparisons that meet both criteria. Specifically, I will compare Malaria Consortium's seasonal malaria chemoprevention program with a) anti-smoking mass media campaigns in developed countries and b) the Nurse Family Partnership program. I am choosing these two programs because I believe they are the two most cost-effective interventions among scalable developed country interventions that can absorb more funding and that benefit people in the same way that Malaria Consortium benefits people (by preventing premature death). (I came across anti-smoking mass media campaigns via
this comment, and Nurse Family Partnership was a GiveWell top charity
for three years.)
Methodology
In this post, I will use three units of measurement: premature deaths prevented, life years gained, and discounted life years gained.
The first metric, premature deaths prevented, is simply the number of people who did not prematurely die but would have prematurely died absent the intervention. One drawback of this metric is that it gives equal weight to preventing the premature death of a teenager with many years of life ahead and preventing the premature death of someone who is only likely to live a few additional years.
The second metric, life years gained (LYG), corrects for this by estimating, for the people whose premature death was prevented by the intervention, the number of extra years they lived as a result of the intervention. One flaw of this metric is that it considers a life year gained today as equally valuable to a life year gained in the future despite the fact that a life year gained today could bring about greater benefits in the future because the benefits of living additional years (and being economically productive during those years) could
compound over time.
The third metric, discounted life years gained (DLYG), corrects for this flaw by treating life years gained in the future as being a certain percent less valuable for each year further in the future they occur. The percent is called the discount rate. In this post, I will use a discount rate of 3% because that is the rate typically used in the literature on anti-smoking mass media campaigns. (Note that the
median discount rate among GiveWell staff is 4.5%.) [Some other
reasons for discounting, such as the option of investing money now so that more can be donated later or the expectation that the cost per life year gained will be greater in the future, do not apply when deciding which of two interventions to fund in the present.]
Additionally, there are some problems common to all three metrics.
1. They treat developing country lives/life years as being as valuable as developed country lives/life years. This could be flawed if you think the quality of life is higher in developed countries. Even if you think quality of life is the same, you might still think lives/life years in developed countries are more important because developed country citizens tend to produce more economically and global economic growth
benefits the global poor.
2. They treat lives/life years gained by a 1 year old as being as valuable as lives/life years gained by an 11 year old (whereas you
might think that lives/life years gained by infants are less valuable). (Indeed, nearly every GiveWell analyst
places greater value on averting the death of someone over the age of 5 than on averting the death of someone under the age of 5.)
3. They only consider whether an intervention is extending how long a person lives and not whether it is improving the quality of the years that person lives. One metric that does consider quality of life is the quality adjusted life year (QALY), which considers how much an intervention improves years that would have been lived anyways as well as how many years of life it adds (and the quality of those added years). I will not be using QALYs because calculating quality of life benefits is too complicated, but I do note some quality of life benefits below.
4. The life years gained metric treats extending a life from 10 years to 15 years as being as valuable as extending a life from 60 years to 65 years, whereas you might think that life years at some ages should be given more weight. (The premature deaths prevented metric also shares this flaw.)
5. The life years gained metric treats 40 years gained for 5 people as equally valuable to 5 years gained for 40 people, whereas you might think that the former outcome is better. (The premature deaths prevented metric is even worse for this scenario because it treats the former outcome as being only an eighth as good.)
Seasonal Malaria Chemoprevention (Malaria Consortium)
The first intervention I will consider is seasonal malaria chemoprevention. GiveWell estimates that the seasonal malaria chemoprevention program run by Malaria Consortium prevents one premature death of a 3-59 month old child for every $2,359 it spends (before accounting for leverage and funging effects) (click
here and go to cell B81).
A 1-4 year old African child can expect to live for another 61.2 years (click here and on the x axis find "2013" then "Both sexes" and on the y axis find "expectation of life at age x" then "1-4 years"). Since each premature death prevented by Malaria Consortium equates to 61.2 life years gained, one life year is gained for every $38.55 it spends [$2,359 / 61.2].
Alternatively, using discounted life years, 27.87 discounted life years are gained for every premature death Malaria Consortium prevents [(1 - (1 + 0.03)^-61.2) / 0.03]. This implies that one discounted life year is gained for every $84.64 it spends [$2,359 / 27.87].
Finally, there is
research that suggests that preventing malaria mortality could increase fertility, which may be relevant depending on your views on
population ethics.
The second intervention I will consider is anti-smoking mass media campaigns (MMCs). According to
a 2014 systematic review, there are eleven studies that have produced estimates for anti-smoking MMCs. (You can find the search strategy for the review
here.) Of the eleven studies, three were estimates of the cost-effectiveness of hypothetical anti-smoking MMCs (
Fishman et al. 2005,
Raikou and McGuire 2008,
Higashi et al. 2011) and one was an estimate based on an anti-smoking MMC directed at an ethnic community in London with a high smoking rate and low information about the health effects of smoking, meaning that it likely has limited external validity (
Stevens et al. 2002). After excluding those four studies, there are seven studies remaining. The table below displays the estimates from those seven studies and an eighth one that was published after the systematic review was published.
*This is the cost to the organization running the campaign, not the societal cost.
**Discounting has a larger effect when benefits are realized only after many years.
All of the eight studies were based on anti-smoking MMCs in developed countries. The most favorable estimate shows anti-smoking MMCs targeted at adults being more cost-effective than seasonal malaria chemoprevention ($32 per DLYG vs. $84.64 per DLYG) and several estimates show them being no more than 16 times less cost-effective than seasonal malaria chemoprevention (<$1,300 per DLYG vs. $84.64 per DLYG).
However, there are several reasons why someone might favor seasonal malaria chemoprevention over anti-smoking MMCs targeted at adults (even if they believe the most favorable estimate for the cost-effectiveness of anti-smoking MMCs).
1. The evidence in favor of seasonal malaria chemoprevention (
seven randomized controlled trials) is stronger than the evidence in favor of mass media anti-smoking MMCs (where none of the studies have a true control group). Additionally, the evidence for anti-smoking MMCs being more cost-effective is weak (only a single study).
2. You might think that anti-smoking MMCs are only highly cost effective when the spending crosses some threshold and that the EA community is unlikely to be able to pool together enough money to fund a campaign that crosses that threshold.
3. You might think that adding a small number of years to a large number of people's lives (what anti-smoking MMCs do) is less valuable than adding a larger number of years to a smaller number of people's lives (what seasonal malaria chemoprevention does). On the other hand, you could think that anti-smoking MMCs are more important because they benefit adults rather than young children.
4.* You might think that it is less morally important to help people who could quit smoking on their own than to help people who cannot do anything to prevent their death from malaria. You might also think that since people can quit smoking on their own and since the dangers of smoking are widely known, it would be paternalistic to run a campaign encouraging people to do so.
5.* You might think that it is less morally important to help people who made the decision to start smoking than to help people who did not cause themselves to be at risk of malaria. You might also think that there are greater benefits to helping someone who did not cause themselves to be in a situation because it also makes other less fearful of what would happen if they end up in that condition. (In the case of seasonal malaria chemoprevention, that benefit would be to the parents, not the young children.)
6. Even if you disagree with the above two points, you might think that others will agree with them. This could make you want to donate to seasonal malaria chemoprevention because it would be more likely to result in others donating when they hear about your decision.
7. As far as I'm aware, there is no charity that focuses on running mass media anti-smoking MMCs (though if this is the only objection, EAs could start one).
*You might accept these two arguments as to cessation campaigns targeted at adults but reject them as to prevention campaigns targeted at youths. Since youth prevention campaigns are significantly less cost effective than seasonal malaria chemoprevention, you would still end up choosing seasonal malaria chemoprevention over anti-smoking MMCs.
[Note: It is difficult to compare the cost effectiveness of developed country anti-smoking MMCs and developing country anti-smoking MMCs because the systematic review cited above did not uncover any studies based on a developing country anti-smoking MMC. The one developing country study that it found was for a hypothetical anti-smoking MMC. That study,
Higashi et al. 2011, estimated that an anti-smoking MMC in Vietnam would result in one DLYG (discount rate = 3%) for every 78,300 VND (
about 4 USD). Additionally, the
Giving What We Can report that shows tobacco control in developing countries being highly cost effective is based on the cost-effectiveness of tobacco taxes, not the cost-effectiveness of anti-smoking MMCs, and the estimated cost-effectiveness of tobacco taxes is
based on the cost to the government, not the cost to the organization lobbying for a tobacco tax.]
Nurse Home Visiting (Nurse Family Partnership)
From 1990 to 2011, a group of researchers conducted a randomized controlled trial in Memphis, Tennessee to study the effect of home visits from nurses on low-income first time mothers and their children. (This program later became known as Nurse Family Partnership (NFP).) Each mother was randomly assigned to one of four categories; those in categories one and two did not receive home visiting, while those in categories three and four did receive home visiting. Although maternal mortality data was available for all four groups, child mortality data was only available for groups two and four. (Click
here for the study reporting the results. If you want to closely following along for the next few paragraphs, open two new browser tabs with the study in each. On one of the browser tabs, click on the "Figures/Tables" tab and scroll down to Table 2. For the other, stay in the tab with the text of the study.)
The study found that preventable-cause child mortality rate for the first children of mothers who were assigned to group 2 (the control group for child mortality) was 1.84% [9 / 489] and that preventable-cause child mortality rate for the first children of mothers who were assigned to group 4 (the treatment group for child mortality) was 0.00% [0 / 217]. (The difference was statistically significant (see "Child Mortality" under the "Results" section of the study).) This implies that one premature death of a first child was prevented for every 54.348 families served [1 / (0.0184 - 0.0000)].
I will assume that Nurse Family Partnership also reduced child mortality for subsequent children and that the effect on subsequent children was identical to the effect on the first child. (There is
some evidence that NFP has a lasting effect on parenting.) I will also assume that mothers who received nurse home visits (i.e. mothers in the treatment group) had an average of 2.08 children (since they had an average of
1.08 subsequent births at six years of follow up). (Note that this is lower than 2.9 children, which is the
average number of children of mothers without a high school diploma who are currently in their mid-40s.) This implies that one premature child death was prevented for every 26.129 families served [1 / (2.08 * (0.0184 - 0.0000))].
The study also found that external-cause maternal mortality for mothers assigned to groups 1 or 2 (the control group for maternal mortality) was 1.62% [(0 + 11) / (166 + 514)] and that external-cause maternal mortality for mothers assigned to groups 3 or 4 (the treatment group for maternal mortality) was 0.22% [(0 + 1) / (230 + 228)]. (This difference was statistically significant (see "Maternal Mortality" under the "Results" section of the study).) This implies that one premature maternal death was prevented for every 71.429 families served [1 / (0.0162 - 0.0022)].
[Mothers in group 3 received home visiting during pregnancy and two postpartum visits; mothers in group 4 received home visiting during pregnancy and for two years after the child was born (see "Treatment Conditions" under the "Methods" section of the study). The current NFP program
provides for visits during pregnancy and for two years after the child is born. Thus, group 4 would probably be a better treatment group than groups 3 and 4 combined. However, the difference between the external-cause mortality for mothers in groups 1 and 2 and mothers in group 4 was not statistically significant, perhaps due to group 4 being too small by itself (see "Maternal Mortality" under the "Results" section of the study).]
Combining the two numbers, I estimate that, during the study, one premature death was prevented for every 19.131 families served [1 / (1 / 26.129 + 1 / 71.429)].
To calculate the impact of NFP today, I need to take into account the decrease in infant mortality (death before age 1) and child mortality (death between age 1 and age 20) between when the study took place and today as well as the decrease in fertility between when the study took place and today. From 1990 to 2013, infant mortality decreased by about a third and child mortality decreased by half. (Click
here and then download "Infant and neonatal mortality rates: United States, 1915-2013" and "Childhood Mortality Rates, by Age at Death: United States, 1900-2013.") Since infant mortality accounted for 40% of infant child/deaths in the study (with child mortality accounting for the remaining 60%) (go to the "Figures/Tables" tab of the study and scroll down to Figure 2B), I will assume that the effect of NFP on infant/child mortality today is 0.567 of what it was during the study [2/3 * 0.4 + 1/2 * 0.6]. Additionally, since the average number of second and subsequent children born per first child born has
decreased in the U.S. by about 10% between 1990 and 2015, I will assume that the number of second and subsequent children born to mothers served by NFP has gone down by 10%. This implies that NFP today prevents one premature infant/child death for every 48.606 families served [1 / ((1 + 0.9 * 1.08) * 0.567 * (0.0184 - 0.0000))], which implies that it prevents one premature death (mother or infant/child) for every 28.924 families served [1 / (1 / 48.606 + 1 / 71.429)]. (Mortality among females aged 15-44
has changed little since 1990.)
[In the above paragraph, the term "child mortality" refers to death between age 1 and age 20. Throughout the rest of the post, the term "child mortality" refers to death before age 20, including death before age 1.]
According to an
old GiveWell estimate (go to "What do you get for your dollar?"), it costs Nurse Family Partnership $10,800 to serve one family. If this estimate remains accurate today, then, on average, each $312,379 [28.924 * $10,800] spent today by Nurse Family Partnership on home visiting programs similar to the one studied in the randomized controlled trial prevents one premature death.
I turn now to calculating the cost per life year gained. Based on Figure 2B (go to the "Figures/Tables" tab of the study and scroll to Figure 2B), I estimate that of the children whose lives were saved by NFP, 40% would have otherwise died around age 0, 30% would have otherwise died around age 5, and 30% would have otherwise died around age 19. Since 0-20 year old children in the U.S. can
expect to live until they are 79, each premature child death prevented results, on average, in 71.8 life years gained [0.4 * 79 + 0.3 * (79 - 5) + 0.3 * (79 - 19)] or around 24.28 discounted life years gained [[(1 - (1 + 0.03)^-79) / 0.03] - 0.3 * [(1 - (1 + 0.03)^-5) / 0.03] - 0.3 * [(1 - (1 + 0.03)^-19) / 0.03] = 30.11 - 1.37 - 4.29 = 24.44]. Since one premature child death is prevented for every 48.606 families served, one child life year is gained for every 0.677 families served [48.606 / 71.8]. If discounted life years are used instead, around one discounted child life year is gained for every 1.989 families served [48.606 / 24.44].
Based on Figure 1B (go to the "Figures/Tables" tab of the study and scroll to Figure 1B), I estimate that of mothers whose lives were saved by NFP, 25% would have otherwise died around 4 years after pregnancy, 60% would have otherwise died around 13 years after pregnancy, and 15% would have otherwise died around 19 years after pregnancy. On average, mothers who survived the entire 20.9 year follow period were 39.4 years old at the end of the follow-up period, which means that mothers who survived had an average age of 18.5 at the beginning of the follow-up period (scroll to "Mortality Outcomes" under the "Methods" section of the study). Since 20-40 year old females in the U.S. can
expect to live until they are 82, each premature child death prevented equates to 51.85 life years gained [0.25 * (82 - (18.5 + 4)) + 0.6 * (82 - (18.5 + 13)) + 0.15 * (82 - (18.5 + 19))] or around 22.74 discounted life years gained [[(1 - (1 + 0.03)^-82) / 0.03] - 0.25 * [(1 - (1 + 0.03)^-4) / 0.03] - 0.6 * [(1 - (1 + 0.03)^-13) / 0.03] - 0.15 * [(1 - (1 + 0.03)^-19) / 0.03] = 30.38 - 0.93 - 6.38 - 2.15 = 20.92]. Since one premature maternal death is prevented for every 71.429 families served, one maternal life year is gained for every 1.378 families served [71.429 / 51.85]. If discounted life years are used instead, around one discounted maternal life year is gained for every 3.414 families served [71.429 / 20.92].
Combining the results from the above two paragraphs, I estimate that one life year is gained for every 0.454 families served by NFP [1 / (1 / 0.677 + 1 / 1.378)]. This implies that one life year is gained for every $4,903 NFP spends on home visiting [0.454 * $10,800]. Alternatively, using discounted life years, I estimate that one discounted life year is gained for every 1.257 families served by NFP [1 / (1 / 1.989 + 1 / 3.414)]. This implies that one discounted life year is gained for every $13,576 NFP spends on home visiting [1.257 * $10,800].
(Note that NFP has
benefits other than saving lives. See
here for the research.)
(It's also worth noting that mothers who participated in NFP had
0.2 fewer children than mothers who did not (2.08 vs. 2.28). Under certain views of
population ethics, this could mean that NFP does more harm than good.)
The table below compares the cost effectiveness of Malaria Consortium and NFP.
|
|
Malaria Consortium
|
NFP
|
Ratio of NFP to Malaria Consortium
|
|
Cost per premature death prevented
|
$2,359
|
$312,379
|
132.42
|
|
Cost per life year gained
|
$38.55
|
$4,903
|
127.19
|
|
Cost per discounted life year gained
|
$84.64
|
$13,576
|
160.40
|
[Note: According to one estimate, bail funds prevent one day of pretrial detention for every $10 donated to them. This implies that a $3,650 donation would prevent a year of pretrial detention [$10 * 365]. If you consider preventing one year of pretrial detention to be equivalent to or better than extending a person's life by one year, then the cost per life year gained of bail funds is better than the cost per life year gained of NFP. If you discount future benefits, then bail funds are likely several times more cost effective because their detention prevention benefits occur more quickly than the mortality reduction benefits of NFP.]
If the ratios are taken at face value, then Malaria Consortium is at least 100 times as cost effective as Nurse Family Partnership. (This lends some support to William MacAskill's 100x Multiplier.)
Of course, there are many reasons why my estimates for NFP (and hence the ratios above) should not be taken at face value.
1. My estimate for NFP is based on a single randomized controlled trial, making it less likely that the NFP estimate is accurate. (GiveWell's estimate for Malaria Consortium is based on
seven randomized controlled trials.)
2. My estimate for NFP does not consider on the ground realities such as attrition. (By contrast, GiveWell carefully adjusts for attrition in its
Malaria Consortium estimate.)
3. You might think that one of my assumptions is wrong, such as my assumption that NFP affects second and subsequent children the same way it affects first born children.
4. GiveWell's
estimate (go to "What do you get for your dollar?") of the cost of serving one family through NFP was based on limited information and may no longer be accurate.
5. You may be uncertain about the extent to which donations to NFP would result in additional families being served. While there are
states without NFP (where NFP could presumably expand if given more funding), the federal government and state governments could conceivably reduce their funding of NFP if there is more private funding.
6. You might disagree with using the same discount rate for developed countries and developing countries.
7. NFP has large quality of life benefits that may be more significant than its mortality reduction benefits. By contrast, Malaria Consortium's mortality reduction benefits account for the vast majority of its benefits (click
here and go to row 78). Thus, a comparison that looks only at benefits from mortality reduction favors Malaria Consortium over NFP.
8. Malaria Consortium only reduces the mortality of young children, while NFP reduces the mortality of children of all ages and of mothers. You might think that reducing the mortality of young children is much less important, which could lead you to think that the cost per unit impact of NFP is significantly less than 130 times the cost per unit impact of Malaria Consortium.
(The first five points are reasons why the ratio may be biased in favor of Nurse Family Partnership, while the last two points are reasons why the ratio may be biased in favor of Malaria Consortium.)
Conclusion
In this post, I have compared two of the most cost-effective scalable developed country interventions that can still absorb additional funding (anti-smoking mass media campaigns and nurse home visiting) to one of the most cost-effective scalable developing country interventions that can still absorb additional funding (seasonal malaria chemoprevention). There is a single study that shows anti-smoking mass media campaigns being more cost-effective than seasonal malaria chemoprevention and several studies showing them being no more than 16 times less cost-effective than seasonal malaria chemoprevention, but none of these studies have true control groups. Using data from a single randomized controlled trial on nurse home visiting, I estimate that, under certain assumptions, nurse home visiting is 125-160 times less cost-effective than seasonal malaria chemoprevention (depending on which metric is used). Based on these two comparisons, it appears that your dollar probably goes 10-150 times further overseas.
Disclaimer: I am not a current or former GiveWell employee.
This report briefly discusses MMCs as well as tax increases. It mentions MMCs are likely to be much more effective than those in the UK, due to the comparatively far lower awareness of the harms of smoking in developing countries, and far higher incidences in smoking. I wonder if we could learn more about the potential efficacy of such campaigns by comparing them to campaigns to try to lower road traffic injury? My impression is that in the latter case there has been a bit more study done specifically in developing world contexts.
Yeah, my hunch is that in developed countries, it's higher-leverage to help people quit than spread awareness of smoking harms. For example, there's a supplement called NAC that might help. (I assume that no large trials have been done because no pharmaceutical company can patent it.) Making e-cigarettes easier to get could also be a good idea.