Comment Permalink

FWIW I'm also suspicious of the 0.001 DALYs per person number.

AFAICT, the way you get it is by combining two methods: method 1 is to ask people a chain of questions like "as a fraction of death, how bad is life imprisonment", "as a fraction of life imprisonment, how bad is not being able to eat tasty stuff", "as a fraction of not being able to eat tasty stuff, how bad is not being able to have sugary drinks", multiply their answers to get how bad losing sugary drinks is as a fraction of dying, and then multiply by the fraction 64/74 (for remaining life years? this was opaque to me), to get a DALY loss of 0.016 +/- 0.009 [1]. You then do method 2: ask people how much of their annual income they'd give up to get a 1-year exemption from a ban on drinking sugary drinks, take the binary logarithm of 1 + that fraction, and multiply by 2 to get DALY loss. This gives you a loss of 0.0012 +/- 0.0009 [1]. You then average each respondent's result from each method to get a per-respondent DALY loss estimate, before aggregating that accross respondents. Because the standard deviation of responses from method 1 is 10 times higher than that of method 2 [2], you weight method 2 10x higher in the per-respondent average, meaning that the overall loss is basically just that of method 2.

But I don't think you're right to conclude that method 2 is more accurate than method 1: it's just that method 2 gives ~10x smaller results for whatever reason, so it makes sense that its error is also ~10x smaller. If you look at the spread in responses as a fraction of the mean response, methods 1 and 2 are pretty close (if anything, it looks like method 1 is a bit more precise). If you instead weighted the methods equally, you would get 3x the per-person DALY loss [3], and if I'm right in the parent comment, that would net out to a 15% reduction in the value of the program.

(also more fundamentally, the fact that the methods give 10x different values suggests that they plausibly are just measuring different things, and we should be unsure which (if either) is actually measuring the disvalue of the loss of freedom to drink sugary drinks)

[1] My error here is the standard error of the mean of each result: basically, how much we'd expect our calculated mean to vary if we resampled. It's equal to the empirical standard deviation divided by the square root of the number of samples (which is 4).

[2] You also list one benefit of method 1 and one benefit of method 2, which I'm assuming cancel out in your considerations.

[3] Sanity check: the mean of the first method is 10x bigger than the second method, previously we were ~ignoring the first method, now we're taking the geometric mean, and the geometric mean of 1 and 10 is 3 (because 3^2 is about 10), so this looks right.

See in context

Deep Report on Diabetes

by Joel Tan🔸

Oct 6 20234 min read 41

28

Cause prioritizationGlobal health & developmentPolicyLess-discussed causesCentre for Exploratory Altruism ResearchCost-effectiveness analysisResearch summary

Frontpage

Show all topics

Note: This post (and the underlying report) was updated after additional research and analysis.

Summary:

We find that advocacy for sugar-sweetened beverages taxes to control diabetes mellitus type 2 to be highly-cost effective (~72,131 DALYs per USD 100,000, which is at least 10x as cost-effective as giving to a GiveWell top charity).
Beyond raw cost-effectiveness estimates (which are highly uncertain), the cause looks highly promising, given the high quality of evidence underlying the theory of change.
Reinforcing this, the expert consensus supports the fact that this is an effective solution for a growing problem.
Further support comes from the fact the harder-to-quantify downsides to the intervention (e.g. lower freedom of choice) are marginal.

Our detailed cost-effectiveness can be found here, as can the full report be read here.

Introduction: This report on diabetes mellitus type 2 is the culmination of three iterative rounds of research: (i) an initial shallow research round involving 1 week of desktop research; (ii) a subsequent intermediate research round involving 2 weeks of desktop research and expert interviews; and (iii) a final deep research round involving 3 weeks of desktop research, expert interviews, and the commissioning of surveys and quantitative modelling.
Importance: Diabetes is a chronic disease characterized by elevated levels of blood sugar, which over time leads to serious damage to the body, with type 2 diabetes in particular occurring when the body becomes resistant to insulin or becomes unable to produce enough insulin. Globally, diabetes mellitus type 2 is certainly a problem, and causes a direct health burden of 90 million disability-adjusted life years (DALYs) in 2024, as well as an indirect health burden of 2 million DALYs from an increased risk of depression amongst diabetes sufferers. There is also an accompanying net economic burden equivalent to foregoing the doubling of income for 191 million people; note that people typically value such income doublings at around 1/5th of a year of healthy life. And this problem of diabetes is only expected to grow between 2024 and 2100, as a result of factors like economic development, ageing, and population growth.
Neglectedness: Government policy is far from adequate, with only 20% of the potential reduction in diabetes burden from implementing sugar-sweetened beverages taxes already being captured by existing government policy; this is not expected to change much going forward – based on the historical track record, any individual country has only a 1% chance per annum of introducing such policies. At the same time, while there are NGOs working on diabetes and sugar-sweetened beverages taxes (e.g. a big Bloomberg-funded organization does work in Brazil, Colombia, Jamaica, Barbados and South Africa), expansion of such work is conditional on funding, and fundamentally, efforts in the area are not equal to the massive and growing disease burden.
Tractability: There are many potential solutions to the problem of diabetes mellitus type 2 (e.g. sugar-sweetened beverages taxes, or mandatory reformulation, or pharmaceutical interventions); however, we find that the most cost-effective solution is likely to be advocacy for sugar-sweetened beverages (SSB) taxes. The theory of change behind this intervention is as such:
- Step 1: Lobby a government to implement a sugar-sweetened beverages tax.
- Step 2: A sugar-sweetened beverages tax reduces consumption of sugar-sweetened beverages in a single country.
- Step 3: Reduced consumption of sugar-sweetened beverages in a single country reduces the global disease burden of diabetes mellitus type 2.
Using the track record of past SSB tax and sodium control advocacy efforts and of general lobbying attempts (i.e. an "outside view"), and combining this with reasoning through the particulars of the case (i.e. an "inside view"), our best guess is that policy advocacy for top sodium reduction policies has a 8% chance of success per campaign. Meanwhile, based on various meta-analyses, and after robust discounts and checks (e.g. for a conservative theoretical prior of a null hypothesis; for endogeneity; for study populations being unrepresentative; or for publication bias), we expect that a WHO-recommended 20% tax on sugar-sweetened beverages will reduce consumption of sugar-sweetened beverages by 12%; and in turn, a 100% reduction in sugar-sweetened beverages consumption in a single country will reduce the global disease burden of diabetes mellitus type 2 by 0.02%.
There are additional externalities and complications to an SSB tax. On the positive side, (a) an SSB tax also reduces ischaemic heart disease (n.b. an increase in impact relative to the diabetes-only baseline by 200%). On the negative side, (b) we expect on average a 7 year gap between when an advocacy intervention begins and when the health impact actually kicks (-5% impact); (c) taxing SSBs risks causing consumers to switch to alcohol (-30% impact); and (d) taxing SSBs leads to reduced freedom of choice (-0.5% impact). Most significantly, however, is the potential gain in cost-effectiveness (+600%) from implementing the intervention in the most promising countries rather than the average one – that is, those countries suffering from some combination of a higher national disease burden, greater neglect by their governments and NGOs, and state fragility.
Implementation Issues: From our interviews with NGOs working in the space, we find that funding is scarce. On the issue of whether a talent gap exists, however, opinion was more mixed.
Outstanding Uncertainties: There are a number of outstanding uncertainties, of which the three most important involve: (a) our use of point estimations (n.b. relying on them is reasonable given that we are ultimately interested in mean estimates, but caution is also warranted, as significant variance is possible); (b) the very simplified methodology we use to project the future disease burden of diabetes mellitus type 2; and (c) the highly uncertain estimates of the probability of advocacy success.
Conclusion: Overall, our view is that advocacy for sugar-sweetened beverages tax to control diabetes mellitus type 2 is an extremely cost-effective cause , and we recommend that charity incubators, grantmakers, policy advocacy organizations, and governments themselves, consider directing more resources towards this area.

Postscript: We've updated our analysis of the issue of whether and how much freedom of choice is diminished by a soda tax. We would like to thank the various commentators who gave feedback, though we also understand that they may still have reasonable disagreements over our methodology and results, especially given different philosophical and epistemic priors on the matter.

28 Reactions

Mentioned in

66Cost-effectiveness accounting for soil nematodes, mites, and springtails

28Intermediate Report on Diabetes Mellitus Type 2

More posts like this

Comments41

Sorted by

New & upvoted

Click to highlight new comments since: Today at 11:09 PM

DanielFilan2y25

Who are the 16 people you surveyed to determine the disvalue of reducing people's freedom to buy sugary drinks?

[EDITED TO ADD: this comment burys the lede a bit, in a great-grandchild I do some napkin math that, if correct, indicates that the survey results mean that the reduced value of freedom entirely negates the health benefits]

DanielFilan2y17

OK I'm really confused - you calculate ~0.001 DALYs (which i guess is ~9 disability-adjusted life-hours) lost per person to eliminating people's freedom to consume sugary drinks, adjust that down because you're not eliminating it but just restricting it, make a second adjustment which I don't understand but which I'll assume is OK, multiply by the population of the average country to get a total number of DALYs, then:

Fourthly and finally, we divide by overall diabetes mellitus type 2 disease burden, thus creating an estimate -0.001% to be used as a downward adjustment on impact.

But you estimate that taxing sugary drinks won't eliminate the DMT2 disease burden, but instead reduce it by 0.02%. So shouldn't this factor instead be 0.001% / 0.02% = 5%?

DanielFilan2y20

Let me try to do a rough calculation myself: if you world-wide banned sugary drinks, each person would lose 0.001 DALYs total over the rest of their lives [EDIT: this is wrong, it's per annum, see this comment for a corrected version of the following calculation].

What's the disease burden caused by DMT2? Your report says roughly 90 million DALYs, I'm going to assume that's per year (you probably say this somewhere or it's probably an obvious convention, but I couldn't find it easily and don't know the conventions in this field). The global average age is ~30 and average life expectancy is ~70, so let's multiply that by 40 remaining years to say 3,600 million DALYs of total DMT2 burden for the present population over the rest of their lives. Banning sugary drinks would reduce that burden by 5%, for a gain of 180 million DALYs. There are ~8 billion people on earth, so that's 0.02 DALYs per person gained by banning sugary drinks.

So loss of freedom of 0.001 DALYs reduces the benefit of 0.02 DALYs by 0.001 / 0.02 = 5%, agreeing with my guesstimate above (assuming that the ratio is the same for a tax vs an all-out ban, which seems right to first order).

DanielFilan2y20

FWIW I'm also suspicious of the 0.001 DALYs per person number.

[2] You also list one benefit of method 1 and one benefit of method 2, which I'm assuming cancel out in your considerations.

DanielFilan2y12

Wait: your survey numbers are for DALYs lost per year. So if the disease burden is 90 million DALYs per year, banning sugary drinks gives a benefit of 0.0006 DALYs per person-year, compared to 0.001 DALYs lost per person-year, meaning that the loss of freedom reduces the benefit by 167% (or 500% if you believe this comment). So now I'm really curious how your adjustments are bringing that down so much.

MHR🔸2y4

EDIT: reading the full report, the 0.02% reduction in diabetes burden is from eliminating sugary drink consumption in a single country. I've updated my comments below to correct for that

Overall, we expect a 100% reduction in sugar-sweetened beverages consumption in a single country to reduce the global disease burden of diabetes mellitus type 2 by 0.02%.

I'm also confused here, but I get different numbers than you do.

My BOTEC:

For 100% elimination of sugary beverages in all countries, the benefits seem like they'd be 0.0002*193*90,000,000 = 3,474,00 DALYs averted /year
For 100% elimination of sugary drinks, the costs in reduced freedom seem like they'd be 8,100,000,000 * 0.001 = 8,100,000 DALYs/year

So this looks net negative in expectation

DanielFilan2y7

Sounds like you roughly agree with me - 8.1 / 3.5 = 230%, which is close to 167%. Difference is I use the 5% reduction number for proportion of burden due to sugary drinks, getting 90 mil / 20 = 4.5 mil, 8.1 / 4.5 = 180%, and the rest is error built into these calcs.

MHR🔸2y2

Gotcha, makes sense!

MHR🔸2y2

Another way to look at this is that according to the report's numbers, a 100% reduction in sugary beverages would prevent 0.02%*193 = 1 in 26 cases of diabetes. But it would also make life 1/1000th worse for everyone.

Joel Tan🔸2y11

Hi Daniel!

To answer the various issues you raised (in order of when they were posted):

(1) We surveyed EAs, via the forum mainly. Obviously not a particularly representative sample, but bigger surveys are more expensive (even if you do them online via e.g. Lucid) and my sense coming in was that this was unlikely to affect the CEA materially (a lot of people don't have libertarian intuitions, for better or for worse), and so we didn’t commit too much time or money towards this issue. I will say that I suspect EAs – being highly educated and more likely to be concerned with abstract considerations like freedom – the estimate may higher than the population average; this would be consistent with larger panel data in political science on attitudes with respect to abstract concerns like democracy, civil rights, rule of law etc.

As for relative accuracy of the two methods – assuming there’s a true mean both are trying to get at, my view is that for the method with far higher variance, the sample mean we calculate is almost certainly much further off from the true mean, and I would be uncomfortable using it for critical calculations, if that makes sense.

(2) For the adjustments, the first, as you note, is just for the fact that sugary drinks are merely being taxed, rather than being banned. The second is just for diminishing marginal returns to freedom on income – the basic intuition here is that your freedom/ability to (say) read books increases as your income increases and you can buy more books; but the increase from 0->10 is far different from 1000->1010, since we would intuitively say there is a massive increase in your freedom to read books in the first case (you can read at all!), and a fairly marginal one in the latter. On top of this, there’s just the fact that income is no longer the binding constraint at high levels of income (we lack time to read, not money with which to buy books with).

(3) The calculation of an overall downward adjustment (the issue you raised with “But you estimate that taxing sugary drinks won't eliminate the DMT2 disease burden, but instead reduce it by 0.02%. So shouldn't this factor instead be 0.001% / 0.02% = 5%?”). I think you’re right! This is almost certainly a mistake we made – I’ll have to go recrunch the numbers myself (also noticed that there are a number of issues – probability of success, and also substitution with diet coke – which should be factored in). Thanks for flagging this out!

(4) On the per annum issue: In our analysis of tractability, we’ve generally stuck to trying to calculate “proportion of problem solved in a single year”, and bring in how said benefits persist in a different section. Similarly, for the adjustments, we stick to the per annum numbers.

(5) On the issue of whether the intervention is potentially net negative given BOTECs if we compare headline numbers of disease burden of diabetes * effect of banning SSBs against global population * 0.001. Beyond potential a prior scepticism as to whether such a significant number of people not dying or suffering ill health from diabetes really is less valuable than loss of freedom to drink sugar drinks, there are a number of adjustments here that are probably important (the massive cardiovascular benefits, economic impacts, plus deliberate geographic prioritization increasing cost-effectiveness).

DanielFilan2y16

Beyond potential a prior scepticism as to whether such a significant number of people not dying or suffering ill health from diabetes really is less valuable than loss of freedom to drink sugar drinks

I guess I want to add something here about why one would have the opposite prior: by and large, a decent model of people is that when they make decisions, they roughly weigh costs and benefits to themselves (or follow a policy that they adopted when weighing costs and benefits). Diabetes is mostly a cost to oneself. At least in the US, people are broadly aware that drinking sugary drinks makes you a bit less healthy, via an increased chance of obesity, diabetes, heart disease etc. But people drink them anyway because they're tasty - that is, in their judgement, the value of being able to drink them is higher than the health risk.

It seems like you have a strong prior that people are wrong about this, and that they significantly underweight the health impacts of soft drinks, such that limiting their intake by 20% is worth it to reduce their risk of getting diabetes by 1%. This isn't impossible - it could be that people don't know that sugary drinks are unhealthy (of course, people could also overestimate how unhealthy they are), or that socialized healthcare means that there are massive externalities to diabetes cases - but I didn't see any arguments to that effect in your executive summary.

Joel Tan🔸2y4

Consolidating over the number of comments:

(1) Per the formula (1-ABS(((1000/(0.83*1.2))-(1000/0.83))/(1000/0.83)))^0.1, it would appear that we have 98.2% remaining freedom, and a 1.8% reduction?

(2) I think you're right on the averaging issue - in a number of other areas (e.g. calculating probabilities from a bunch of reference classes), we've tended to use the geomean, but that's for extrapolating from estimates to the true value, as opposed to true individual means to the true population means. The other related issue, however, is whether the sample is representative, and whether we think that highly educated people are more likely to report caring about abstract concerns. Will have to think about this, but thanks for the feedback!

(3) As to your larger point, I'm not sure if it's reasonable to interpret individual behaviour when it comes to trading off short term against long term gains as maximizing, given time inconsistent preferences (relative to valuing your welfare equally at all times), people just not thinking too much about daily choices, lack of awareness of the precise degree of risk (even if the risk were 10x or 0.1x, it's not like you would likely shift a meaningful shift in behaviour), and of course motivated reasoning.

DanielFilan2y5

(1) Sorry, I was copy-pasting the formula in the spreadsheet, but missed the extra 0.1 factor you added at the end. First of all, I still think the factor of 50 you're shaving off due to diminishing marginal returns seems quite extreme, given the lack of articulable justification for why it should be so big. I guess the extra factor of 10 is because diet cola exists? I'm not sure why you're adding that - as I mentioned, the questions you asked people already mentioned that people could substitute to diet sodas. Quoting from the instructions to participants of cell E1 of your survey spreadsheet, "Note that this does not include artificially-sweetened beverages (i.e. Coke Zero, Diet Pepsi, Sprite Zero etc) – you would still be able to drink those.".

(2) Makes sense. Regarding composition, we could look at polling of soda taxes to get a sense for this. One thing to note is that politicians typically don't implement these taxes (hence you looking into lobbying for them), suggesting that they're not very popular. Of the two polls I could find, it seemed like either there were no significant differences between groups, or that Republicans were more opposed to soda taxes than Democrats. Given that the vast majority of American EAs are Democrats, this suggests that the poll could be underrating the disutility of reduced sugar consumption.

(3) Note that many of these factors go both ways - people could be motivated to appear self-abnegating and healthy, think sugary drinks are less healthy than they are (anecdotally, I've asked two people how much diabetes would be reduced if sugary drinks didn't exist, and they both overestimated relative to your cited number), or not think about how sugary drinks are actually OK. It's indeed plausible that people value their future selves less than they ought to according to standard utilitarianism, but I don't think it's a priori clear that they do so by a massive factor.

DanielFilan2y11

(1) Fair enough re sample (altho it obviously limits how much of a conclusion you can draw). Re: the different variances, I basically dispute that the cash value method has a meaningfully lower variance than the hypothetical sequence method, because the relevant error is relevant to the mean. That said, this factor of 3 is the smallest issue I have with your calculation.

(2) Could you show the calculations (perhaps in a simplified format, like I did)? The tax vs ban seems like it should affect the value of freedom similarly to how it affects the disease burden reduction, and it's very unclear to me what you're actually doing to adjust for diminishing marginal returns to freedom - and I'm skeptical that the heavy discount relative to the BOTEC model is justified.

(3) Fair enough! Easy to make this sort of slip-up in such a big survey [EDIT: by "such a big survey" I meant "such a big report" - surveys were just on the mind]

(4) Yeah, the per annum issue was just a problem with my BOTEC, I have no particular reason to think you got it wrong (other than the extreme divergence from the BOTEC).

(5) For what it's worth I don't share your skepticism that taxing sugary drinks could be net negative. At any rate: I'd guess the factors you first thought of are probably the most important (deferring to your process of creating the initial report), and I suppose that there are probably further effects that go either way - especially since, as you noted, cost effectiveness analyses tend to look less promising as more effort is put into them.

Joel Tan🔸2y4

Hi Daniel - thanks for all the previous feedback. I did a tentative update to the CEA (https://docs.google.com/spreadsheets/d/1kdnvaeP5iAUAF_Fgcf_ZgZDci2cYV02M51N7HMTMOBQ/edit#gid=1248986269), taking into account the points you raised plus some other considerations. I also separated out the analysis into its own tab, hopefully for better clarity.

(1) One issue we've had issues with is large variance in estimates - and we typically try to use the geometric (rather than arithmetic mean) to better capture the differences in magnitudes. Our previous average of different people's estimates used a pure arithmetric mean (because of the presence of zeroes - people didn't value drinking sugary drinks at all, I guess).

Partly because of our discussion on weighing, I tried a different approach of getting a geomean of the non-zeroes, and then creating a weighted average with the zeroes. The results are pretty sensitive - 0.0002 with this mixed method vs 0.001 with a pure arithmetric mean. I do think the former is probably the better method, insofar as arithmetric means are too sensitive to the high-end number, and have updated accordingly - but I'm not sure if there's a good answer one way or another!

(2) For diminishing marginal returns - I basically think of it in terms of a graph (x axis is number of sugary drinks you can buy with your current income, y-axis is "freedom of choice"), and I take the relationship to be y=x^0.1. The proportional reduction in the number of sugar drinks you can buy is estimated like this: (a) calculating how much you can buy without the tax (an arbitrary USD 1000, divided by prevailing prices sourced from Walmart), (b) calculating how much you can buy after the tax (USD 1000, divided by prices subject to 20% tax), and then (c) taking ABS((b-a)/a) to get the proportional reduction (0.17, for a 20% price rise). (d) 1-c then gives how much you can still buy, proportionally (0.83). (d) Then, using the y=x^0.1 formula, we take (1-d)^0.1 to find that 98% of "freedom of choice" still remains, and correspondingly, there was a 2% reduction.

Obviously, a whole bunch of assumptions are made, most significantly about the curve and DMR - any thoughts you have would be welcome!

MHR🔸2y4

Regarding the choice of means, I personally think that the arithmetic mean may make more sense in this case. Suppose you had 3 respondents who gave responses of 0.1, 0.001, and 0.001 as their DALYs from full banning of sugary drinks. If we assume that the individuals are accurately responding regarding their views and (as you are though the rest of this analysis) that utility gains and losses can be linearly aggregated across different individuals, and then the total harm that occurs to the three individuals in the case of a drink ban is 0.102 DALYs. If you had used the geometric mean instead, you would estimate that the total harm from a drink ban was just 0.014 DALYs.

It seems that an arithmetic mean would make more sense if one thinks that differences between the survey respondents' views reflected true underlying differences in preferences, whereas a geometric mean would make more sense if one thinks that differences between the survey respondents' views reflected noise in estimates of similar underlying preferences.

Joel Tan🔸2y4

As mentioned to Daniel in the other thread: I think both of you are right on the averaging issue - in a number of other areas (e.g. calculating probabilities from a bunch of reference classes), we've tended to use the geomean, but that's for extrapolating from estimates to the true value, as opposed to true individual means to the true population means. The other related issue, however, is whether the sample is representative, and whether we think that highly educated people are more likely to report caring about abstract concerns. Again, will have to think about this, but grateful for the feedback!

DanielFilan2y4

Then, using the y=x^0.1 formula, we take (1-d)^0.1 to find that 98% of "freedom of choice" still remains, and correspondingly, there was a 2% reduction.

The number in the sheet is a 0.2% reduction, not a 2% reduction. [EDIT: my bad, it's a 2% reduction, there's just another factor of 10 reduction that I mistakenly lumped into that]

DanielFilan2y4

I still disagree with your belief that the accuracy of the iterated questions format was lower than the accuracy of the fraction of income format - both questions had standard deviations that were approximately the same multiple of their means.
I think your original strategy of aggregating across the population using the arithmetic mean made sense, and don't understand what the justification is supposed to be for replacing it with a geometric mean [1]. Concretely, imagine a decision that affects two friends lives, making one 50% worse, and the other 0.005% worse. Presumably you wouldn't take the geometric mean and say "this basically makes both your lives 0.5% worse, which is not very much". Instead you might conclude that your friends are different in some way. Similarly, it seems like probably some people like sugary drinks and others don't, causing significant variation in how much they care about sugary drinks being banned.
As DMR said, that curve seems kind of weird to me - it seems like an unjustified assumption is being used to cut a BOTEC by a factor of 50, which strikes me as suspicious. The real curve is presumably not linear (because otherwise people would buy more sugary drinks on the margin), but intuitively I feel like a factor of 5 adjustment makes way more sense than a factor of 50.

By my analysis of your sheet, if you use a factor of 5 rather than 50 for the decreasing marginal utility, and use the arithmetic mean rather than the geometric mean to aggregate across participants, you get the disutility of freedom as 500% higher than the gains. If you also weight both estimation methods equally, it goes up to 1,100% - which is bigger enough than my BOTEC that I worry you might be making some errors in the opposite direction?

[1] Consider that this analysis is done in the genre of a utilitarian calculation, which usually uses the arithmetic mean of welfare rather than the geometric mean, as is used implicitly in the disease reduction component.

MHR🔸2y2

Very interesting, thanks for the detailed explanation! I'll be curious to see what Daniel says, but my intuition is that y=x^0.1 is a generous assumption for the shape of the freedom of choice utility curve. I do agree that I'd expect utility losses from freedom of choice to be nonlinear, but the function you're using would imply that 80% of freedom of choice is preserved when sugary drink consumption has been cut by 90%. Moreover, my intuition is that the welfare losses due to reduced sugary drink consumption are also partially due to not getting tasty drinks anymore (thinking for myself, I wouldn't really mind not having sugary drinks, but I would definitely be sad in the similar case of not having sugary foods). That component of the utility losses seems like it would be closer to linear.

DanielFilan2y6

also noticed that there are a number of issues – probability of success, and also substitution with diet coke – which should be factored in

Just reread this - surely these don't need to be factored in? Probability of success affects the numerator and the denominator equally, and your poll respondents probably already knew about diet coke.

DanielFilan2y4

Also in cell 1F of the results spreadsheet, "salty food" should be "sugary drinks".

Vasco Grilo🔸1mo2

Hi Joel,

Has this research led to any concrete grants, or prompted GiveWell to estimate the cost-effectiveness of advocating for taxing sugar-sweetened beverages?

Joel Tan🔸1mo4

Hi Vasco, I made a consolidated comment here: https://forum.effectivealtruism.org/posts/k7NjuGEKdRSrrJHmZ/deep-report-on-hypertension?commentId=weQHRcDt53BkCuLct

Larks2y14

Further support comes from the fact the harder-to-quantify downsides to the intervention (e.g. lower freedom of choice) are comparatively insignificant.

I'd like to start off by thanking you for actually trying to estimate this, rather than just sweeping it under the rug. I really appreciate this, and think it reflects very poorly on other EA researchers that they don't do this. This is a major step towards better capturing the true moral tradeoffs.

Unfortunately, that is the last good thing I have to say about this, because your actual methodology seems quite terrible.

You essentially asked 16 of your friends if it was a bad idea, the sort of sample size that should raise major red flags. There was no attempt to sample people disproportionately affected by the ban.
You ignore the fact that some of the responses make zero sense - like the person who, if understood correctly, thinks that a sugary drink ban would reduce their total lifetime utility by 2% but is not willing to spend a single dollar to avoid this colossal hit.
Despite this, if you take a simple average of their responses, the policy looks like a bad idea.
Instead of reporting this fact, you used a bizarre function, consisting of an average of the arithmetic mean of weighted arithmetic means for some respondents, the geometric mean of weighted arithmetic means for some respondents, and the geometric mean of weighted geometric mean of other respondents, to show the 'average' was low.
The weights appear designed to reduce the estimate, because they are (I think) based on the size of the standard errors, which naturally scale with estimate size.
The number of degrees of freedom in how you defined this formula is very large compared to your number of data points. I would be very surprised if you pre-registered this formula.
What's more, aside from looking p-hacked, the formula is totally absurd. Here is a simple example:
- In the final column, cell I18, you are using the arithmetic mean for 4 of the respondents as a hack to get around the fact that it would be embarrassing to report the geomean as it is zero.
- But this makes your function non-monotonic! If we took the four respondents who gave literally zero as their answer (I5, I13, I16 and I17) and replaced '0' with 0.0000000001, the utility cost of the program has surely increased (as some people dislike it more, and no-one dislikes it less). But because they are now non-zero, we apply the geomean (at least I assume we did - the logic is hardcoded, never a good sign) which means the bottom line number actually falls, going down by a factor of 96%.
- So basically the existence of a single person who disvalues something by a sufficiently small but positive value can outweigh any number of other people saying something is a major cost.

Yelnats T.J.2y12

Also in support of the sugar tax is that we've seen health taxes used as a cost-effective way improve health/save lives in tobacco, alcohol, and salt. For tobacco, taxation is the most cost-effective of all the tobacco control measures. I'm not surprised to see the evidence point to a sugar tax.

Joel Tan🔸2y2

Agreed! Though in terms of both evidence base and tractability, I would say tobacco > soda > salt - we've done a lot more tobacco taxes than SSB taxes, which in turn is far more common than sodium taxes (or taxes on high salt food). That is consequential for (a) whether we have empirical data to inform our cost-effectiveness analyses and general prioritization, and (b) whether we can persuade policymakers, who care a lot about whether other countries have done something (and successfully).

Vasco Grilo🔸2y6

Thanks for sharing, Joel!

I think it is worth noting your cost-effectiveness estimate increased with further research:

In the intermediate report, it was 20 times that of GiveWell's top charities. "CEARCH finds that the marginal expected value of policy advocacy for these top sugar control solutions to eliminate DMT2 to be 11,036 DALYs per USD 100,000, which is around 20x as cost-effective as giving to a GiveWell top charity (CEA)".
In this report, it is 7.13 (= 78647/11036) times the one of the intermediate report, i.e. 143 (= 7.13*20) times as effective as GiveWell's top charities.

I think this is interesting because I agree with the caveat you have in the intermediate report that:

The headline cost-effectiveness will almost certainly fall if this cause area is subjected to deeper research: (a) this is empirically the case, from past experience; and (b) theoretically, we suffer from optimizer's curse (where causes appear better than the mean partly because they are genuinely more cost-effective, but also partly because of random error favouring them, and when deeper research fixes the latter, the estimated cost-effectiveness falls). As it happens, CEARCH intends to perform deeper research in this area, given that the headline cost-effectiveness meets our threshold of 10x that of a GiveWell top charity.

Any thoughts on why this did not happen?

Joel Tan🔸2y9

Sharp eye, Vasco! Two things probably drove this result.

(a) Most significantly, we've taken into account the fact that different countries have different levels of cost-effectiveness (driven by differing disease burdens, neglectedness, tractability etc); and so an EA funder or fund-adviser targeting the top end of countries (as we are doing ourselves in our charity evaluations) can probably get far higher impact (we estimate the top end is 7x the average, which is plausible to us insofar as even top GiveWell charities like DtW see similarly drastic cost-effectiveness differences across their country projects).

That said, making these geographic adjustments is very time consuming (since you basically have to build a whole geographic prioritization model, which in turn requires a lot of data sourcing and talking to NGOs etc) - so it's not usually feasible to do in shallow/intermediate research rounds.

(b) Additionally, in the intermediate CEA, we based our costing estimates on a couple of big global NCD charities. However, subsequent work suggests that this would not be realistic - in the hypertension space, we've been following up on our cause priorization research by doing actual evaluations of sodium policy charities, to circulate to our grantmaking and E2G partners. The organizations we've been looking at have tended to be smaller local LMIC charities - partly because we are unlikely to move enough money to really help those big global orgs, but also because those local charities are probably more cost-effective (they tend to have pre-existing government connections, but also crucially, much lower salaries). Expecting that the kind of charities we (and other impact-oriented EA donors) will look at in the diabetes space, will similarly be these smaller local LMIC charities, we've changed our costing reference class accordingly.

We also looked at an additional bunch of negative factors (like SSB taxes cause substitution towards alcohol, or how much disvalue people place on reduced freedom of choice from taxes) and added more epistemic discounts to the theory of change (particularly around whether taxes impact consumption), but these did not turn out to be as significant as the above two factors.

Ian Turner2y3

A 100% reduction in sugar-sweetened beverages consumption will reduce the global disease burden of diabetes mellitus type 2 by 0.02%.

Oh. 😞

MHR🔸2y5

I think that line in the summary is misleading. The full report says (emphasis mine):

Overall, we expect a 100% reduction in sugar-sweetened beverages consumption in a single country to reduce the global disease burden of diabetes mellitus type 2 by 0.02%.

So eliminating 100% of sugary beverage consumption in all 193 countries (# of countries used in the CEA) would prevent 3.86% of diabetes cases.

Joel Tan🔸2y4

You're right - we should probably should update that to be clearer!

Joel Tan🔸2y2

It's unfortunate, but the kinds of policies that would eliminate diabetes will probably be extraordinarily expensive (a whole bunch of fiscal, regulation, educational and environmental measures to prevent poor diet and exercise habits, plus mass production and distribution of insulin/ozempic/renal drugs etc) - and the closer we get to solving the problem, the more costs increase, since the easier cases of diabetes to prevent/treat would all be dealt with first.

Vasco Grilo🔸1y2

Hi Joel,

Have you reached out to GiveWell about this? I imagine it would be interested in investigating interventions in this area given your estimate that taxing SSBs is 155 times as cost-effective as their top charities.

Joel Tan🔸1y6

Hi Vasco,

The GiveWell team handling the nutrition portfolio reached out to me to discuss salt policy, and SSB taxes are on their longlist, iirc. Of note is the fact that Vital Strategies (which have gotten GiveWell grants for their alcohol policy work, also does SSB tax advocacy).

Ian Turner2y3

One thing I don’t see here is a discussion of the taxes themselves, either in terms of their effects on those who pay them or on the governments that collect them.

I suppose we have something like:

− some CE for the amount of taxes paid
+ some CE for the government’s use of the money (either lower taxes elsewhere, increased government programs, etc.)
− some CE for the effect on government corruption (either that the tax revenues themselves are stolen, or that the administration of tax is used as an opportunity for bribery)
− some CE for the cost (to the government) of administering the program

What do you think?

Joel Tan🔸2y3

To clarify, on top of charity costs, we do look at the cost to the government of implementing the tax (the WHO has a whole table of useful data on the cost of various health interventions); that said, we discount this cost to some extent because it's counterfactually less valuable than EA expenditure.

Beyond the health impact of the taxes, we only looked at the disvalue people place on reduced freedom of choice. We chose not to model the use of the taxes and took them to be just a lump sump transfer - not just quite accurate given regressivity concerns (since the poor are disproportionately likely to suffer the tax) but this is balanced out (since the poor are disproportionately likely to benefit in terms of improved health and subsequent greater earnings).

Corruption is an interesting issue, but I'm unsure if the evidence is particularly strong, or even which direction it goes - if you have higher revenues you can pay your officials more, reducing the risk of corruption. It's probably a choice, at the end of the day, whether or not you want to model this as an extraneous adjustment factor - my sense is that we wouldn't have much confidence in the adjustment, regardless, and so it wasn't worth trying given lack of time.

Yelnats T.J.2y2

I can't speak for a sugar tax, but if it's anything like a tobacco tax, the numbers end up being pretty good.

Tobacco taxation according to WHO is not only the most effective but also the most cost effective for enforcement.

Yes, corruption can reduce the positive value of the government revenue, but tobacco taxes are such a windfall that you need Mobutu-level embezzlement for revenue to not make it's way back to government programs.

A recent success story was Colombia that tripled their tobacco tax over the course of 2 years. Consumption was cut by a third and the revenue doubled the budget of their national health service.

Tobacco taxes (and maybe a sugar tax?) are technically regressive taxes, however the poor disproportionately reap the financial and health benefits because they are the most price sensitive and the ones most targeted by Big Tobacco.

Vasco Grilo🔸2y2

Hello again Joel,

Further support comes from the fact the harder-to-quantify downsides to the intervention (e.g. lower freedom of choice) are comparatively insignificant.

Personally, I am worried about the meat eater problem. GiveWell's top charities decrease mortality in low income countries, where there is low consumption per capita of animals with bad lives, but I assume most disease burden linked to type 2 diabetes respects high income countries, where such consumption is higher. Do you know how much of the reduction in the disease burden would come from less morbidity (instead of less mortality)?

Joel Tan🔸2y4

Per the GBD, it's around 53:47 years lived with disability to years of life lost (so slight majority of the disease burden is morbidity).

For the fundamental question of whether it's net positive to save lives given the animal suffering it causes - my own view from very rough CEAs + philosophical priors is that it's probably net positive, but the uncertainty here is somewhere between high to impossible.

Vasco Grilo🔸2y2

Per the GBD, it's around 53:47 years lived with disability to years of life lost (so slight majority of the disease burden is morbidity).

Thanks for sharing. If roughly half of the benefits come from decreased mortality, I think the uncertainty of the effects on animals is high enough for the overall effect to be unclear (my view is informed by this BOTEC), i.e. I do not know whether the taxes would increase/decrease welfare.

For the fundamental question of whether it's net positive to save lives given the animal suffering it causes - my own view from very rough CEAs + philosophical priors is that it's probably net positive, but the uncertainty here is somewhere between high to impossible.

If you do not mind sharing, did you use Rethink's median welfare ranges in your BOTECs? Since uncertainty is so high, would you also say that, upon further investigation, or with different assumptions from yours which are still reasonable, it is quite possible that one would conclude decreasing the disease burden of type 2 diabetes decreases welfare (accounting for humans and farmed animals)? If the intervention is not resiliently positive in terms of increasing welfare due to effects on farmed animals, would it make sense to flag these as a potential downside?

As a meta point, do you think there are better ways (than what I am doing here) of commenting about the effects on animals of global health interventions?