Comments
With the AI windfall, mental health may be taken increasingly seriously by mainstream EA. There’s increasingly room to explore the cause area and answer some of the biggest uncertainties; at the same time, costs of the most effective interventions are continuing to fall. StrongMinds is now treating for ~$35, about a third of GiveWell’s best guess from a few years ago, and is expected to fall further.
This presents a unique opportunity to demonstrate that mental health interventions should be considered as part of broader global health portfolios. We can do so by making methodological improvements to our cost-effectiveness analyses.
The Happier Lives Institute have pioneered (and are constantly improving) methods for CEAs of mental health in WELLBY terms. However, their mission doesn’t cover other forms of outcomes, such as DALYs and income doublings. Furthermore, they haven’t yet publicly explored models of psychotherapy that leave lasting impacts on individuals (which may be of interest to HLI). Therefore, with an eye to increasing the estimated impact of mental health interventions in ways that are legible to mainstream EAs, I propose three directions for EA-flavoured mental health research.
About me: I’m Huw, I work at Kaya Guides, an EA mental health charity[1]. This is based on my own experience of building our CEA, and reviewing other people’s CEAs of us. As such, I only consider CEAs for psychotherapy—other interventions are much more neglected and they would also be promising, but I have no expertise in these areas. I don’t have formal psychotherapy training or research experience; I only have a degree in statistics. As such, I am likely missing important facts and many of my estimates may not be high quality.
Due to HLI’s excellent focus on the WELLBY, combined with CG/GiveWell’s neglect of mental health, there hasn’t been a lot of work in writing DALY conversions. Prior art is:
I’d like to focus on the Founders Pledge approach, because the Global Burden of Disease Study (GBD) is the generally-accepted source for DALYs within EA. A DALY in EA is calculated by multiplying out the GBD’s disability weight for a condition, by the length of time under which that condition is averted.
As such, if we could determine a single, ‘canonical’ way of estimating the disability weights produced by an intervention, we could get to DALYs very quickly. The challenge with disability weights today is that they are determined under the GBD by asking global survey respondents which of two described health conditions is ‘healthier’. For depression, respondents are given three descriptions:
The GBD then does some fancy statistical fitting to determine a weight (0–1) for each condition. Mild is 0.145, moderate is 0.396, and severe is 0.658. We also know from Salomon et al. 2015 that the weights are fit in logit-space, so we can reasonably interpolate between them.
However, we have no ability to assign PHQ-9 scores to this scale confidently—PHQ-9 scores weren’t used anywhere in the construction of the GBD. Existing approaches have assumed some kind of linear interpretation, but we must be careful about assuming that a GBD ‘mild’ is the same thing as a PHQ-9 ‘mild’ (to say nothing of the PHQ-9s ‘moderately severe’).
One method that hews closely to the GBD would be to assess SF-12 scores. This is because the GBD’s statistical estimates work by comparing the survey preferences to a dataset of SF-12 scores for the same conditions. In Burnstein et al. 2015, the authors map SF-12 scores to GBD weights, and then assess different conditions in terms of SF-12 scores (as well as using third-party datasets for the same).
If we had a useful dataset for mapping PHQ-9s to SF-12s, then we could convert those through to GBD weights by either recreating Burnstein’s mapping function, or by asking Burnstein to produce their function directly.
I was able to identify two usable studies for this purpose:
However, I think this direction is less likely to be useful because (a) the two datasets we have aren’t representative of LMIC populations and (b) it involves an extra conversion step, for which we’re likely to see some loss. The PHQ-2 is particularly suspect, because although it’s useful as a diagnostic tool, people who are merely sad will score highly on it but not be generally depressed—this would potentially underestimate the disability weight of saturating the scale, relative to a depressed population.
Another direction to explore is how to directly map PHQ-9s to disability weights. One way of doing this is for a researcher to subjectively decide how the three GBD bands correspond to PHQ-9 scores, using their intuition. This is how existing approaches have functioned. However, the choices of bands and mappings here impact the resulting DALY estimates quite meaningfully. It’s my opinion that a small amount of work to standardise this method could pay off fairly substantially.
Another somewhat more objective way would be to use a survey. Respondents could adopt a ‘persona’ as one of the three (+neutral) states, and then fill in a PHQ-9 survey as that person. Respondents could be human, or potentially AI. We could then fit a reasonable mapping between the scores.
The final option is to find populations of depressed people, have them fill in a PHQ-9, and then ask a single vignette:
One last question. Here are three people. Thinking about the past 2 weeks, which person is most like you?
I’ve invented Person A’s description as the inverse of the others, while the others are taken from the GBD (I believe a ‘none of these’ option would be too confusing). We could then fit a curve on these.
Because the EA mental health space is fairly well-connected, a coordinated effort here between organisations could produce fairly large datasets for this question across the global south—we could partner with charities in India, Ecuador, South Africa, Kenya, Zambia, Zimbabwe, Uganda, the Phillipines, and so on. Kaya Guides will be asking this question ourselves and I expect to put together a dataset soon (reach out if you’d like to partner on this!).
Cuijpers et al. 2026 is an important meta-analysis, which was unable to rule out the hypothesis that depression’s effects may last indefinitely. If this is true, then existing EA CEAs may be dramatically underestimating the impact of psychotherapy, potentially by a factor of 2–15x. However, there isn’t enough existing evidence that can be used to estimate this.
If this were true, CEAs could also start to access the value of suicide prevention, since we could argue that psychotherapy permanently averts suicides, rather than merely postponing them. (In my own BOTECs, suicide prevention wouldn’t provide a meaningful part of a CEA today, because of how rare suicide is, combined with the assumption that suicide prevention only lasts until the next episode of depression).
In my opinion, we should model depression as an episodic disorder. If this is true, then a course of psychotherapy may shorten, or lessen the severity of, an existing episode of depression. Under the GBD’s own analysis (Appendix 1), the mean episode of depression lasts around 0.65 years. This means that by the time of 12-month follow-up, most study participants would have completed the episode they started the study with, in both arms. As such, 12-month follow-ups are only catching (1) participants in the midst of very long episodes or (2) participants who have begun a subsequent episode.
Such a study is then almost certainly underpowered to detect effects (if it were powered for post-treatment), as only ~10% of the study population would be in an episode at 12 months (according to a BOTEC I did to simulate a bunch of episodes). The same can be said for later follow-ups. It’s very rare that large enough studies are conducted to detect effects under this measurement paradigm—but this could be a direction for future research.
Another direction could be to develop or standardise methods for determining long-term effects. Under an episodic model, we would need to capture a longitudinal sense of the participant’s trajectory. We could do this in two ways:
I suspect this is a good research direction for a funded academic study, although perhaps some easy wins could be taken with basic research.
Some studies of the economic impacts of depression put the value at a percentage of GDP. Given depression’s prevalence, this implies economic losses of ~20% of income—nothing to sneeze at. Meta-analyses of the effects of psychotherapy on labour-market outcomes find around ~0.15 standard deviations on most outcomes.
More work needs to be done to establish the precise effect sizes, and then convert these to income doublings, to be taken seriously by CG. Similarly, more work needs to be done in establishing appropriate discounts and methodologies for including these effect sizes in CEAs. Many outcomes simply don’t show up as statistically significant.
Similarly, another concern worth researching is that causal impacts of psychotherapy don’t seem to recover a proportionate amount of the burden of depression on GDP. It would be worth understanding why this is, and doing more work to quantify the effects.
Much of the work in this field has involved Johannes Haushofer, who Kaya Guides is collaborating with on our RCT. In the next few weeks, we’ll start our RCT with a very large sample size (around 3,500 participants), which by our count will be one of the largest ever studies of any psychosocial intervention in South Asia. The hope is that at these sample sizes, we’d be able to find effects on measures that other studies have struggled with (such as income at 12 months).
Overall, however, an impact of 0.15sd seems unlikely to move the needle. Improving PHQ-9/DALY conversions improves the credibility and legibility of mental health CEAs for relatively cheap, and studying the long-term effects of mental health interventions could yield substantially larger effects than previously estimated. So economic impacts seem like the least promising direction.
As well as work to improve mental health CEAs, organisations in the space need to communicate better about what has changed since GiveWell’s CEA of StrongMinds. Kaya Guides are guilty of this ourselves, and I’m hoping to share more on this soon.
If you’re interested in pursuing any of these research questions further, please reach out to me!
Sorry, I owe everyone an update on this ↩︎
I am glad mental health is also being seriously considered within the EA community. It was one of the causes I missed seeing in the forum when I entered, and this post was spot on.
As a psychologist myself who has been treating several years with clients regarding self-esteem issues, I really hope your charity and this cause as a whole has a deep impact!