What is the cost-effectiveness of researching vaccines?

This essay was jointly written by Peter Hurford and Marcus A. Davis.

Note that because of technical length restrictions on the EA Forum, this essay is broken up into three parts: Part 1, Part 2, and Part 3. To see all three parts in one part, you can view the article on our research site.

Previous articles also include an analysis of how beneficial vaccines have been, an analysis of how much it costs to roll-out a vaccine, how much it costs to research and develop a vaccine, and how long it takes to research a new vaccine. However, this series is structured so that starting with this article should be all you need to do.

Summary

We looked at academic literature for vaccine cost-effectiveness as a whole and we also performed individual case studies on seven contemporary and historical vaccines to try to estimate the total cost-effectiveness of researching and developing a vaccine from scratch. Looking back historically, we find a range of $0.50 to $1600 per DALY, depending on the vaccine. Using this historical information, we derive an estimate for the total cost-effectiveness of developing and rolling out a “typical” / ”average” vaccine as being $18 - $7000 / DALY. The smallpox vaccine, malaria vaccine, and rotavirus vaccine may all be more cost-effective investments in total than marginal investments in distributing bednets (see Appendix C), especially when pursued to the point of completely eradicating the disease. However, there are many important assumptions made by these models, and changing them could strengthen or undermine these conclusions.

Section 1 and Section 2 detail the raw cost-effectiveness figures, section 3 provides a detailed analysis of all the key assumptions and model uncertainty, and section 4 provides our takeaways. Appendix A provides an analysis of comparing our models to other models in the literature, Appendix B provides an estimate of the cost-effectiveness of the Ebola vaccine, Appendix C compares vaccination cost-effectiveness to distributing bednets, and Appendix D provides a rough assessment and comparison of some estimates of the cost-effectiveness of GAVI.

Table of Contents

1. What are the costs and benefits?
2. What is the cost-effectiveness?
   1. Scenario 1: Vaccinate 60% of the relevant populations in SSA
   2. Scenario 2: Eradicate the disease completely
   3. Individual Vaccines
      1. Smallpox Vaccine
      2. Measles Vaccine
      3. Rotavirus Vaccine
      4. HPV Vaccine
      5. HIV Vaccine
      6. Malaria Vaccine
3. Why do these estimates vary? What assumptions are we making?
   1. What population are you targeting?
   2. How are you calculating the DALY burden for that population?
   3. How do you want to handle benefits that are not captured by DALYs?
   4. How effective are you assuming the vaccine will be?
   5. How expensive do you assume the roll-out will be?
   6. What assumptions are you making about the R&D cost?
   7. How will population and DALY burden change over time?
   8. How do you consider “leveraged funds” / “unlocking funding” / “crowding out funding”?
   9. How do you handle discount rates, if at all?
   10. For how many years should you consider benefits?
   11. How do you adjust for counterfactuals?
4. Our Takeaways
5. Appendices
   1. Appendix A: What Do Other Articles Say?
   2. Appendix B: Forecasting Cost-Effectiveness for Ebola
   3. Appendix C: Comparing to Against Malaria Foundation
   4. Appendix D: Cost-Effectiveness of GAVI
6. Endnotes

1. What are the costs and benefits?

While developing a vaccine is a huge accomplishment, it is not really of any significant humanitarian benefit until the vaccine is scaled up and rolled out to a significant population. Thus, spending a lot of money to develop a vaccine merely “unlocks” the opportunity to roll out the vaccine and the hope is that this roll-out is cost-effective enough to make up for the cost of the R&D once amortized across the entire vaccinated population.

We, therefore, take the following as our equation for the total cost-effectiveness of vaccine R&D:

Total cost-effectiveness of vaccine R&D ($/DALY) = ((Total R&D costs of making vaccine) + (Total roll-out costs of vaccine)) / ((DALY burden of disease) * (% reduction in disease attributable to the vaccine))

Note here that we’re looking at the total cost-effectiveness across the entire investment in R&D, which requires us to also look at the total investment across vaccine roll-out too. We’re not looking at the marginal investment, or the cost-effectiveness you would get if you added more funding to vaccines today, which could be a very different number (see section 3.8 for details).

To try to answer this question, we previously modeled how long it would take to develop a new vaccine, how much it costs to research and develop a vaccine, how much it costs to roll-out that vaccine and then how beneficial we can expect vaccines to be.

These data were calculated from available evidence about vaccines generally and also for specific case studies: smallpox (which was chosen as it was one of the first vaccines and the only human disease successfully eradicated by vaccination); measles (which was chosen as it was one of the first vaccines); HIV, malaria, and Ebola vaccines as they are modern and under current development, and rotavirus and HPV as they recently finished vaccine licensing. For each of these vaccines, we calculated how much they would cost for both R&D and roll-out, and then what benefits we would expect.

We aggregated all this data in a spreadsheet that makes all the calculations between the multiple sections much more clear, with a good amount of detail. Based on that data, we come to the following conclusions. (For more detail, see the spreadsheet and the previous articles.)

[a] - Estimates for DALYs prevented per person vaccinated over a 20 year period.
[b] - These figures, except for smallpox and measles, are derived from their 2016 global DALY burdens from Global Burden of Disease, Results Tool (2016e)
[c] - This figure is 70% of the 2016 global burden from cervical cancer of 7,390,002.82.
[d] - All ebola estimates here are a 5 year average from 2012-2016 and assumes a vaccine that is 50% effective.

2. What is the cost-effectiveness?

Based on the data in the above tables and the considerations we’ve made, we can then apply our formula: Total cost-effectiveness of vaccine R&D ($/DALY) = ((Total R&D costs of making vaccine) + (Total roll-out costs of vaccine)) / ((DALY burden of disease) * (% reduction in disease attributable to the vaccine))

As stated before, these calculations were originally derived from research in prior articles (e.g., for R&D costs, roll-out costs, and total benefits) for a particular basket of vaccines. We then aggregated all this data in a spreadsheet , used the spreadsheet to create individual Guesstimate models for each vaccine, and created 90% confidence intervals from each model. We made calculations for each of our vaccines, except for Ebola, where we felt like there was insufficient information to make a confident calculation. (We still attempt to estimate for Ebola in Appendix B.)

The models varied targeted populations across two scenarios – one where 60% of the relevant population in Sub-saharan Africa (SSA) is vaccinated and another where the disease is completely eradicated. We also varied assumptions about the DALY burden, vaccine effectiveness, roll-out costs, and R&D costs. We then created these 90% confidence intervals.

2.1) Scenario 1: Vaccinate 60% of the Relevant Populations in Subsaharan Africa

For this scenario, we assume that there is a one-time fixed cost investment for researching and developing the vaccine and building infrastructure for rolling out the vaccine that is amortized over a time period to consider benefits for (in our model, we cap it at 20 years), then an annual cost to keep rolling out the vaccine to the relevant population. Then, every year, we save a certain amount of DALYs from preventing that disease via the vaccine. Together, we can use this to calculate DALYs averted over the benefits time period compared to the cost spent during the time period.

2.2) Scenario 2: Eradicate the Disease Completely

For this scenario, we consider one-time costs for researching and developing the vaccine and then rolling out the vaccine enough to achieve eradication. In the case of smallpox, as it really has been eradicated, real estimates of spending and disease burden were used. In all other cases, extrapolations were made from the estimated costs of vaccination per person and the intended target population of the vaccination[3]. The population targeted was different depending on the disease, with HPV and HIV roughly targeting reproductive age populations and malaria and measles targeting children under 5. The efficacy of the vaccines was also assumed to be high enough to achieve eradication at the time eradication is attempted[4].

After eradication is achieved, we assume that ongoing costs are essentially $0[5]. We then consider this large cost compared to a period of accrued benefits from the eradication (in our model, we cap it at 20 years; see discussion below on this cap).

2.3) Analysis of Individual Vaccines

2.3.1.) SMALLPOX VACCINE

When making direct estimates using inputs from Fenner, et al. (1988), by our estimation before the eradication campaign in the early 1960s, smallpox prevented a DALY from death for ~$20-64 in developing countries and ~$1550-3800 in developed countries. Weighting this by population in each region, this implies a total cost per DALY of $527 for both roll-out costs alone and roll-out costs plus R&D costs. Over the full population since eradication, not accounting for post-eradication prevention spending, the eradication campaign cost $26-70 per death prevented, $3.30-11 per case prevented, and $0.44-3.80 per DALY prevented at the rates of death and disease of 1967.

These estimates are absolute as they are not compared to the cost of maintaining the 1967 vaccination and monitoring costs. As smallpox is eradicated, these figures will fall over time, unless and until there is a new smallpox outbreak or significant enough threat of an outbreak to cause an increase in spending.

Assuming the pre-vaccination world would be like the pre-eradication world of smallpox endemic countries in the 1960s, given a case rate of roughly 1% and a death rate between 5 and 20% per case of smallpox, our Guesstimate suggests a cost per DALY prevented of $4-62 with or without the R&D costs included.

2.3.2.) MEASLES VACCINE

Assuming a very rough conversion that a life saved from a measles death is ~30 DALYs[6], this would imply very rough figures of $9-320/DALY in general, given a cost range per person vaccinated of $1-38, weighted so cheaper prices are more likely. In areas with pre-existing strong vaccine infrastructure such that measles vaccines cost about $2 a dose, the price would be roughly $9 per DALY. Given the wide range of possible cost per person, it should be noted that holding everything else constant, the cost-effectiveness including R&D cost scales roughly linearly with the price of vaccination per person.

Perhaps we could make sense of these figures by further untangling marginal vaccine costs from the investments needed to make vaccine pathways. If we assume a one-time investment of $100M, similar to the more directly estimated fixed costs of the malaria vaccine, is sufficient to get all lower income countries to a point where marginal vaccination costs $2 per child in total vaccine and logistics costs, and we assume that we’re rolling out the measles vaccine to 96.3M people, the population of 60% of children under 5 in SSA, and we take our prior estimate that the measles vaccine cost $38.3M in R&D, the total yearly costs are ~$151M for the vaccinations at ~$10/DALY, only slightly higher than with no investment spending at all. Given our earlier calculations, this would avert about 177K measles deaths or 5.3M DALYs.

A more worse-case analysis assuming $23 per child, with the same $38.3M in R&D, and rolling out to the same population, the total cost would be $587M at $110/DALY.

2.3.3.) ROTAVIRUS VACCINE

Given vaccine efficacy of 39-85% (RotaCouncil, 2016, p11) and a reduction of diarrhea of 30-54% (Madhim et al., 2010, Msimang, et al., 2013), if we assume we can achieve a 60% vaccination rate of the under-five population in SSA, vaccinating 96.3M children (see Population Pyramid), we would avert ~7.5-13M DALYs per year using the 2005 DALY rate or 4.6-8.3M DALYs using the 2016 rate. Assuming an initial $50-100M investment in building the pipeline to roll out the vaccine, with a per person cost between $3-7 that would be $6-63/DALY looking just at roll-out costs and $10-68/DALY when adding in R&D costs.

There’s great uncertainty about these estimates given fixed roll-out costs are a guess and the efficacy of the vaccine is fairly wide.

2.3.4.) HPV VACCINE

If the HPV vaccine was rolled out to all children aged 5-15 in SSA with a 60% vaccination rate and given ~780,000 DALYs in women aged 15-49 (Global Burden of Disease, Results Tool, 2016b)[7] then the HPV vaccine could prevent ~328,000 DALYs per year in SSA. With a fixed roll-out cost guessed to be an additional $50M, at $3-13.50 per person this would imply $240-1300 per DALY based on just roll-out costs and $370-1600 per DALY when including R&D costs.

2.3.5.) HIV VACCINE

Given estimates of a vaccine that is 50% effective (or more) and distributed to 60% of the eligible population in SSA, it would avert 30% of all HIV, saving 9.7M DALYs. Relative to other vaccines, many of the parameters for our HIV estimates vary widely, particularly our estimated cost per person of $30-160. Ultimately, our model suggests an estimated cost-effectiveness of $180-570 per DALY. Our individual scenarios indicate the final cost-effectiveness depends significantly on which population that vaccine is ultimately distributed to and the cost of per person of vaccination 8 .

2.3.6.) MALARIA VACCINE

Previously we noted that the malaria vaccine involved spending ~$605M in fixed costs to unlock the ability to roll out the vaccine for $22/child. The Global Burden of Disease estimated a yearly malaria burden of 56.2M DALYs, with 44.3M global DALYs for those under five, and 42.1M of those in SSA (Global Burden of Disease, Results Tool, 2016c). Modeling in Guesstimate, accounting for uncertainty by using a range of vaccine efficacy between 39-75% (the upper end of which is closer to vaccine efficacy for established vaccines), an expected R&D cost between $600-1,000M and a cost per person between $18-25, we get an estimate of $12-23/DALY for roll-out costs alone and $13-33/DALY including R&D costs.

Note that because of technical length restrictions on the EA Forum, this essay is broken up into three parts. Please continue on to Part 2, which contains a discussion of model uncertainty. All the appendices and footnotes are in Part 3. To see all three parts in one part, you can view the article on our research site.

Thanks to Max Dalton, Joey Savoie, Tee Barnett, Palak Madan, and Christina Rosivack for reviewing this piece.