Joel Burke and I recently co-founded Good Policies through the Charity Entrepreneurship Incubation Program. We are currently piloting campaigns to increase taxes on tobacco products in Armenia and Mongolia and are fundraising so that we can keep investigating tobacco taxation and other policy interventions.
We thought that laying out the rationale for someone to donate to this organisation compared to other fantastic giving opportunities might provide a useful space for the EA community to give us feedback on our ideas. We also want to build our case for donors who are keen to improve their impact.
How good are the current top GiveWell Charities?
GiveWell does a fantastic job of evaluating charities in the working to save or improve lives in low-income countries and their top charities undeniably produce a large impact per dollar donated compared to the average charity. Their current estimate of the cost-effectiveness of their most cost-effective program is $480 to produce an outcome as good as averting the death of an individual under 5 years old (Evidence Action: Deworm the World). There is some scepticism around the effects of deworming programs (as the majority of the benefit comes from long term effects like better access to education or raised economic outcomes). Their second most cost-effective program is the Against Malaria Foundation which produces a similar outcome for $1690 with a significantly stronger evidence base. To do the same amount of good via unconditional cash transfers would, they say, costs $29,068 (GiveDirectly).
Brief Overview of the Value of Advocacy Orientated Interventions
We think that there could be advocacy orientated opportunities with significantly higher expected cost-effectiveness than that of GiveWell’s top charities. Some of the highest value opportunities might be in influencing governments of low/middle-income countries to implement evidence-based policies that would significantly improve the lives of their citizens. There are a few reasons that we think that global health advocacy organisations might have promise.
- There seems to be a lot of research on policies that improve lives. There are already excellent groups that rigorously test health policies and have identified policies that can do a large amount of good.
- There are many governments that haven’t implemented impactful policies that aren’t particularly controversial. There are multiple opportunities to build support for a small set of high value policy asks, and speeding up the implementation of said policies could save a huge number of lives.
- There are activities that charities are simply not able to do without the involvement of the government, for example reforming fiscal policy. Governments also have access to unique opportunities to scale interventions across entire populations in time frames that aren’t realistic for charities.
We think that there is an opportunity for a targeted organisation to speed up the implementation of policies that would improve a huge number of lives for a relatively small amount of money.
Evaluation of our Current Activities
Our calculation of the expected value of our activities is not nearly as advanced as GiveWell’s cost-effectiveness analysis. We have however done our best to estimate the cost-effectiveness of our interventions in a way that will let us compare to GiveWell’s current activities. We will start by examining the effect of our current intervention. When tobacco taxes are increased there are two impact streams. The first and primary source of impact is that tobacco consumption decreases. The second is that revenue is generated which the government can then spend on its programs.
We have built a cost-effectiveness model to estimate the impact of our program which can be found here. In order to estimate the decrease in smoking consumption based on the current literature, we have implemented a model called TETSim which estimates the effect on health outcomes and fiscal revenue for various proposed excise tax changes. You can read more about the model here and a case study of its use here. To convert lives saved to DALYs, we estimate the number of additional healthy years of life added for various demographics of quitters.
We are keen to note that we think that this CEA is far less reliable than GiveWell’s analysis of their current top charities. However, we want to share it to make a stronger case for what we think is a competitive giving opportunity and to get feedback on mistakes in our analysis that we might have made.
Limitations
Our model has a variety of limitations that I think would be helpful to quickly discuss here. Firstly, it does not take into account the effects of secondhand smoke, which account for a significant proportion of tobacco-related deaths. It also doesn’t account for different estimates particularly well, ideally, we would like our estimates to have some kind of probabilities attached to give a better sense of the distribution of possible outcomes. Currently, we give our best guess, and a pessimistic and optimistic guess that are towards the tail ends of the distribution. We may rebuild our model in guesstimate or similar software in the future to rectify this issue. It is also challenging to estimate the attribution of any success to us and the likelihood of getting a policy win. To rectify this we have tried to be fairly conservative in our estimates, considering we have chosen countries that are particularly neglected. We also assume that people are equally likely to quit independent of age, we think if we did account for this it would marginally improve the cost-effectiveness of this intervention as there is some evidence that points towards younger people being more sensitive to price changes than older generations.
Results
We compared the short term (one year) effects of our program on a $/DALY basis to the Against Malaria Foundation. Our best guess is that our program has cost-effectiveness of 40-49 $/DALY. Based on our rough estimates our best guess is that our programs in Armenia and Mongolia are 3.82-4.72 times more cost-effective than AMF at a 5% chance of successfully increasing excise taxes on tobacco products by 20% with full attribution to our charity. You can make a copy of our CEA and modify parameters as you wish, but we have tried to justify and provide references to parameters as necessary. Of course, there is a large degree of uncertainty in our findings, mostly due to us not having a particularly good grasp of the hit rate of teams that work on this type of intervention and how clearly we’ll be able to attribute impact to our organisation. We have previously talked about why we think that we may be able to attribute a significant proportion of a future policy win to our team, but we have little empirical evidence to back this up. We have tried to be conservative in our estimates to account for this, in particular, we currently don’t model any of the secondhand smoke effects and we only model the policy being sped up by one year due to our actions. We also do not currently model other realistic policy asks like step increases in taxes year on year.
Funding
We are currently fundraising to support our work in Mongolia and Armenia in 2020 and we are still looking to fill our gap for funding. We are looking to raise $200k so that we can both continue to work on Good Policies full-time; if we raise between $100k and $200k, it is likely that we will continue to work in one country but one cofounder will step back and look at other ways to assist the project; if we raise less than $100k, it’s unlikely that we’ll be able to continue putting serious work into the charity.
If you’re interested in donating, or you think that you could help us find other donors, we would love to hear from you! We’d be glad to share more information about our plans, and what different levels of funding could enable us to work on. You can reach us directly at [email protected].
The model looks great! I think it's well-formulated and the data are well-researched, so it seems informative.
Substantive things:
-You might want to add pessimistic guesses for the cost of your advocacy. Intuitively, $100k for 5% attribution seems high when I consider travel costs, salaries, lobbying costs, etc. Generally when we assess policy change, we've considered the benefits to be the benefits of an org's most successful campaigns, and the costs to be the org's total costs because it's inherently hard to predict in advance which campaigns will be successful (you can't just use the costs for that campaign).
-Consider adding a note in the sheet explaining what you mean by "Percentage of the consumption decrease due to decreased smoking prevalence" (cell A9) and "Percentage of lives saved due to quitting" (A14). Not sure exactly what those are referring to at the moment.
Some aesthetic things:
-in cells B17:D20 your units are "millions of lives saved", but the outputs are on the order of thousands of lives saved, so it's a bit hard to read. You might want to change that to just "lives saved"
-you generally might want to snip the number of significant figures in the cells because it makes it harder to read. e.g. for the cells showing proportion of smokers per age group, snip it off at 2 sig figs
(I'll come back with some more comments later - just wanted to give quick initial impressions!)
Thank you so much for giving really actionable feedback. I'll add those notes but just address them here too
Percentage consumption decrease ... is referring to the fact that some people will reduce their intensity and some people will quit (which we call a reduction in smoking). In our model quitters will experience improved health outcomes and people that reduce won't (this is partly us trying to be conservative as we struggled to find good studies looking at the link between reduced intensity and health outcomes).
Percentage of lives saved ... is because not all quitters will have improved health outcomes. Some will only quit after the damage has been done and there will be marginal differences to their exposure to risk. In the model this just discounts the effect of quitting.
I'll have a think about how to go about coming up with more pessimistic guesses for costs/attribution etc. In any case, the feedback is useful. Our model admittedly just goes for a low number guided a by the opinion of some experts in the tobacco control space that we spoke to. Maybe in future we should look at surveying experts a wider variety of experts or using some kind of prediction market as this is a pretty key part of the model.
I agree that a 5% chance of a 20% tax increase over one year with £100k seems optimistic