Doing a Basic Life-Focused Cost-Benefit Analysis

This post is designed to teach people with basic medical and operational knowledge how to do a simple cost-benefit analysis with life-years as the unit of analysis. I am distilling cost-benefit analysis down to the simplest procedure that will allow a non-expert to produce something useful. This framework has some 'magic numbers' and heuristics I do not explain, but I want a clean presentation. To do a proper analysis, consult this guideline. This post may be updated in the future due to feedback.

This framework is appropriate for the analysis of a relatively simple change in operations. This could include interventions, procedures, or controls. For this post, I'm going to call the thing you are doing the policy. Even if the policy is meant to be applied in a large area, the analysis should always focus on a single model site. This makes it concrete, and makes all of the costs and benefits easier to visualize and list.

Estimating costs

A proper cost-benefit analysis includes an analysis of both the time to implement the program and the monetary costs of things that must be purchased. But far too often, people only look at the money cost, and ignore the time. So to avoid that mistake, this framework focuses almost entirely on the time cost. This is a better approach, because for most policies, the time cost is bigger than the capital and supply costs. If you have an estimate of the annualized money cost, divide it by the GDP per capita of the country the money is coming from, and add that number to the annualized FTE cost you calculate. But if not, just ignore the money and focus on the time.

The first step to estimating costs is to consult the full operational plan for implementing the new policy. Look at everyone who needs to know about the change, and everyone who will have to do anything different. Tally things on a person-by-person basis. Estimate how much time an individual in a particular role will spend on the policy, ideally by consulting several of them, and then multiply by the number of people in that role. Never round any time cost down to zero. Five additional seconds per procedure can be significant, when multiplied by the annual number of procedures.

Be sure to include

1) line staff. How much time will they spend learning about the policy, reviewing it, asking questions about it, and then actually doing it? Think concretely, and if possible, look back at the training and verification time involved in similar policies. 

2) customers or patients. How much time, if any, are they required to spend as a result of the policy?

3) managers and executives. How much time will they spend learning about the policy, running training sessions for the staff, monitoring the staff to ensure that the policy is being implemented properly, and doing disciplinary procedures for staff who fail to comply?

4) any outside people who are affected, like suppliers or partners or governing bodies.

Estimate and add up the initial time estimates and the yearly time estimates separately. Then divide the initial number by five. However, if employee turnover time is less than 5 years, their training costs should instead be divided by the turnover time. Then add this to the yearly time, for the total annualized time cost. Divide the hours by 2000 for the annualized FTE cost of the policy.

Estimating benefits

To estimate benefits, you need a study or evidence showing the health benefits of what you're doing. If you cannot find a published study, do an expert elicitation: ask neutral experts for their best-case estimate, their worst-case estimate, and their expected estimate. Then make a triangular distribution from that and aggregate the estimates.

Often the benefit estimate will be in terms of an odds ratio or a relative risk ratio. Turn this into a number of individual cases by multiplying by the number of people in the site each year, and if applicable, the existing prevalence.

List the expected annual number of lives saved (which could be a small percentage chance of saving a life) and all illnesses prevented per year of operation of the new policy. You are going to convert them all into life-years gained, and add them up. To do so, look up the DALY value of all of the illnesses or conditions the policy is expected to prevent here.

1) Every life saved is worth 20 life-years (future years are discounted).

2) Preventing a permanent illness or injury is worth 20 life-years times its DALY adjustment.

3) Preventing a temporary illness or injury is worth its DALY adjustment, times the expected length of the condition.

Social benefits can be converted into DALY values by estimating how the policy will affect the mental health of the affected population, and using the DALY scores for anxiety or depression. This will be speculative, so be sure to add appropriate caveats to the conclusion if social benefits are a large factor.

If the policy produces any monetary benefits, divide the benefit number by the GDP per capita of the country receiving the the benefits.

Decision

Compare the annualized FTE cost to the annualized life-years gained. Ideally you have a 90% confidence interval for one or both, and this uncertainty should be presented and considered (Here's a Google Sheets template for doing Monte Carlo analysis). But if you only have estimates for the averages, use this heuristic:

If the FTE cost is less than the life-year benefit, the policy passes the cost-benefit test. It is good, and should be a priority, assuming it does not have a high money cost you were unable to estimate.

If the FTE cost is greater than the life-year benefit, it's not recommended as a priority. There are probably better things you can be doing.

If the FTE cost is more than five times the life-year benefit, the policy fails the cost-benefit test. It is probably a waste of time, and you should only do it if you are legally forced to.