Ah, ok. That's a fair point. There is a substitution effect. My main intuition here, though, is that extremely effective NGOs like BRAC basically provide a parallel public health / social safety net for people, yet often-times what would be great is if the government itself was able to provide these services. There is a substitution effect no matter what organization you donate to. For example, I would bet money that public health departments where Against Malaria is more active counterfactually spend less on malaria prevention than otherwise.
e.g. here's an excerpt from Stefan Dercon's recent book, Gambling on Development:
Success in delivering effective health services stands out, and although the government expanded services, the most dynamism at scale was offered by NGOs. The role of BRAC (originally the Bangladesh Rural Advancement Committee) was pivotal. In 1990, it developed a model of community health workers, some paid but many volunteers, who offered advice but were equipped with basic health and sanitary products they were allowed to sell. By 2005, BRAC workers were outnumbering government community health workers. With other NGOs following suit, more than three-quarters of health workers are now supplied by NGOs. BRAC alone reached up to 110 million people with health information and basic services, such as detecting the vast majority of malaria and tuberculosis cases in the country.
One idea I've been toying with is for individual donors to donate to subnational government agencies for capacity building in low-income countries (e.g. donate to the public health department of a city). It has been exceedingly easy to donate to causes like the recent conflict in Ukraine, or donate to the US government, but there doesn't seem to be many opportunities for individual donors to give to subnational government agencies. I'm not sure how to go about implementing this idea, but I think it could be highly effective if done correctly.
You're right. His critique is mostly about the decision cutoff rule, and assumes that Givewell has accurately measured the point estimate, given the data. On the other hand, the url you provided shows that taking into account uncertainty can cause the point estimate to shift.
Thank you again for taking the time to share your thoughts. I hadn't seen that link before, and you make a fair point that using distributions often doesn't change the end conclusions. I think it would be interesting to explore how Jensen's inequality comes into play with this, and the effects of differing sample sizes.
Hi David, I really enjoyed this post. Your comment on the potentially infinite standard errors on ratio distributions has been something I have been mulling over for the last few months.
Because the numbers going into the ratio are themselves averages from samples of Indonesians, each comes with its own margin of error....as far as the math goes, there’s a nontrivial chance that Inpres led to zero additional years of schooling, then there’s a nontrivial chance that the ratio of wage increase to schooling increase is infinite.
Outside of the context of weakly identified instrumental variable regressions, I'm wondering how much Givewell takes this into account in its cost effectiveness analysis, and how much EA in general should be considering this. In one sense, what we care about is not E(benefit)/E(cost), but rather E(benefit/cost). i.e. what we care about is if we reproduced the program elsewhere, that it would have a similar cost-effectiveness. If what's inside the expectation of the latter is the ratio of two independent normals, then we get a Cauchy, which has infinite fat tails and undefined standard errors. Am I right to say that cost-effectiveness analysis only has meaning if either
the distributions on the numerator or denominator are non-independent-normal, perhaps skewed (e.g. ratio of two independent Gammas gives us an F, which does have a well-defined standard error), or
the denominator, the distribution of "cost", ends up converging to a constant (e.g. GAVI, where we have a well-defined/nonrandom cost that's mandated by the government).
Does that imply that simply picking projects that have highest E(benefit)/E(lives saved) may not be the best solution? Have you read any good papers on this topic? (or perhaps this isn't an issue at all?) The only thing I could find is abstract, which has seen 0 citations in the last 2 decades since it was published.
(Also, this is trivial, but I thought the addition of the photos added a really nice touch as compared to the typical academic journal article.)
Ah, ok. That's a fair point. There is a substitution effect. My main intuition here, though, is that extremely effective NGOs like BRAC basically provide a parallel public health / social safety net for people, yet often-times what would be great is if the government itself was able to provide these services. There is a substitution effect no matter what organization you donate to. For example, I would bet money that public health departments where Against Malaria is more active counterfactually spend less on malaria prevention than otherwise.
e.g. here's an excerpt from Stefan Dercon's recent book, Gambling on Development:
One idea I've been toying with is for individual donors to donate to subnational government agencies for capacity building in low-income countries (e.g. donate to the public health department of a city). It has been exceedingly easy to donate to causes like the recent conflict in Ukraine, or donate to the US government, but there doesn't seem to be many opportunities for individual donors to give to subnational government agencies. I'm not sure how to go about implementing this idea, but I think it could be highly effective if done correctly.
You're right. His critique is mostly about the decision cutoff rule, and assumes that Givewell has accurately measured the point estimate, given the data. On the other hand, the url you provided shows that taking into account uncertainty can cause the point estimate to shift.
Ah, another article. It seems
uncertainty analysis is getting more traction: https://www.metacausal.com/givewells-uncertainty-problem/
Thank you again for taking the time to share your thoughts. I hadn't seen that link before, and you make a fair point that using distributions often doesn't change the end conclusions. I think it would be interesting to explore how Jensen's inequality comes into play with this, and the effects of differing sample sizes.
Hi David, I really enjoyed this post. Your comment on the potentially infinite standard errors on ratio distributions has been something I have been mulling over for the last few months.
Outside of the context of weakly identified instrumental variable regressions, I'm wondering how much Givewell takes this into account in its cost effectiveness analysis, and how much EA in general should be considering this. In one sense, what we care about is not E(benefit)/E(cost), but rather E(benefit/cost). i.e. what we care about is if we reproduced the program elsewhere, that it would have a similar cost-effectiveness. If what's inside the expectation of the latter is the ratio of two independent normals, then we get a Cauchy, which has infinite fat tails and undefined standard errors. Am I right to say that cost-effectiveness analysis only has meaning if either
Does that imply that simply picking projects that have highest E(benefit)/E(lives saved) may not be the best solution? Have you read any good papers on this topic? (or perhaps this isn't an issue at all?) The only thing I could find is abstract, which has seen 0 citations in the last 2 decades since it was published.
(Also, this is trivial, but I thought the addition of the photos added a really nice touch as compared to the typical academic journal article.)