Zvi's objections seem pretty reasonable:

I’ll say in advance this is based on a quick read, so it’s plausible some of these issues are my misunderstanding of what was going on, if only because it seems so stupid. If I did misunderstand one or more points here, I apologize, but the whole thing seems pretty terrible.

1. The average dollar spent is going to be vastly more effective than the marginal dollar spent going forward, since the low hanging fruit will already be gone.

2. Cancer treatments are neither cheap nor pleasant. The calculation here counts lives saved without counting the time and money spent on treatment. Yes, you won’t be paying that bill, but the bill is real and represents the loss of real resources that would have otherwise likely gone to other ‘life saving’ spending.

3. Crowding out effects are real. At least a large portion of these findings would have happened anyway, and the Feds are taking credit for the full effect of the treatments studied.

4. That’s right, this essentially counts any life saved through cancer treatment, ever.

5. Study results are being translated into estimated gains at the population level, at face value. Which we all know won’t hold up in practice, it never does – they won’t work as well in the field, and also won’t be implemented where you’d want.

6. Also I don’t see anything in the analysis of impact on how often the treatments actually got administered, at all? As in, maybe I’m missing something, but I can’t find the part where they check how many people actually got cancer treatments in order to estimate how many lives got saved.

7. Instead, they seem to be using the formula: On the basis of a previously published method, for each trial-proven new treatment for a given type of cancer, life-years gained (LYG) at the population (Pop) level was calculated as the product of model-estimated additional life accrued to the average patient (Pt) and multiplied by the number of patients in the cancer population (NCaPop) who would benefit from the new treatment (ie, LYGPop = LYGPt × NCaPop).9.

8. And then: To derive the number of patients in the cancer population to whom the new treatment would apply (NCaPop), we matched the major cancer type, stage, tumor characteristic, prior cancer, surgery, sex, and age (ie, ≥ 18 years) eligibility criteria from the clinical trial to corresponding cancer population data using the Surveillance, Epidemiology, and End Results (SEER) program.

9. That’s not how many people did benefit. That’s how many people they say in theory would benefit if we gave everything to everyone.

It's a very interesting study and a compelling idea. I think the big issue is that we need to look at the marginal impact of extra dollars on cancer research, this is looking at the average impact of money spent on cancer drug trials. Expected effectiveness of more money should be lower, as the most promising drugs are more likely to already have funding.