Seems interesting! I have some newbie questions:
Assuming that these results hold, do you know how good it's outcomes are relative to similar interventions (say MBSR or group therapy)?
They also seem to be worried about the meaning of having no change in biomarkers. How important is that?
I'm also pretty uninformed on the biomarkers aspect. Beyond what they say in the paper:
One reason why we do not find significant effects on biomarkers at conventional levels may be power issues combined with relatively noisy measures. Another, related reason may be the composition of our sample: high levels of pro-inflammatory cytokines have been found for major depression; respondents in our sample, however, report, on average, only mild depressive symptomatology, pre-treatment. In fact, we find that only eight out of 133 respondents (about 6%) report strong depressive symptomatology, as indicated by PHQ-9 scores of fifteen or higher. Moreover, even amongst these, only about a third show associated elevated inflammation (Wium-Andersen and Nielsen, 2013). For cortisol, individual differences and timing of measurement matter; it has been found to be a rather short-term measure for stress (Miller et al., 2007).
I found Table 5 here gives a bit more context on the correlation between these biomarkers and different outcomes.
As far as comparisons, they say:
Impacts on subjective wellbeing, mental health, and pro-sociality are large: the course increases life satisfaction on a zero-to-ten scale by about one point, more than being partnered as opposed to being single (+0.6) or being employed as opposed to being unemployed (+0.7) (Clark et al., 2018). It is more than double the effect of ENHANCE, a 12-week course focusing primarily on positive habits, skills, and attitudes, which is probably the most comparable intervention (Kushlev et al., 2017). 28 However, the authors are able to track outcomes over a longer period of time, up to six months post-treatment. Finally, the effect on life satisfaction is somewhat larger than effects found in trials by the UK Big Lottery Fund, which funded a wide range of wellbeing programmes (fourteen portfolios, each consisting of three to 34 actual trials) from 2008 to 2015 at a volume of £200 million. Trials typically included community-based activities such as horticultural activities, cooking lessons, or sports events. As a conservative estimate, they increased life satisfaction on a zero-to-ten scale by, on average, 0.5 points for six months post-treatment (New Economics Foundation-Centre for Local Economic Strategies, 2013). Different from our intervention, however, these trials all targeted specific groups with mental health needs, including overweight adults, families with young children, or people with substance use disorders.
I am just thinking if this course does not perpetuate the problem of those from advanced economies taking care of their close ones while leaving those in developing economies further behind, unattended to.
Then, students of this course would feel great about making differences that are net-negative, considering the opportunity cost of not aiding more distant people much more cost-effectively (viz the amount it takes (few hundred USD) to change one's life through GD).
I acknowledge that helping the world is briefly mentioned at the end, but overall, this course much more advocates for local involvement. Thus, unless this course is run in 'globally diverse' groups, it may not improve the world structures toward greater inclusion and generosity very effectively. Please correct me if I am wrong.
Although correlates of mental wellbeing have been extensively studied, relatively little is known about how to effectively raise mental wellbeing in local communities by means of intervention. We conduct a randomised controlled trial of the "Exploring What Matters" course, a scalable social-psychological intervention aimed at raising general adult population mental wellbeing and pro-sociality. The manualised course is run by non-expert volunteers in their local communities and to date has been conducted in more than 26 countries around the world. We find that it has strong, positive causal effects on participants' self-reported subjective wellbeing (life satisfaction increases by about 63% of a standard deviation) and pro-sociality (social trust increases by about 53% of a standard deviation) while reducing measures of mental ill health (PHQ-9 and GAD-7 decrease by about 50% and 42% of a standard deviation, respectively). Impacts seem to be sustained two months post-treatment. We complement self-reported outcomes with biomarkers collected through saliva samples, including cortisol and a range of cytokines involved in inflammatory response. These move consistently into the hypothesised direction but are noisy and do not reach statistical significance at conventional levels.
Note that it's not peer-reviewed, but I didn't see any red flags when looking through the paper. If the estimates in the paper turn out to be reliable, it has external validity, persistence, etc., this seems like a surprisingly impactful and scalable intervention.
TL;DR: Big, if true.
Course material can be found here.
I don't have much time to spend on this, but here are a few thoughts based on a quick skim of the paper.
The study was done by some of the world's leading experts in wellbeing and the study design seems okay-ish ('waitlist randomisation'). The main concern with internal validity, which the authors acknowledge, is that changes in the biomarkers, while mostly heading in the right direction, were far from statistically significant. This could indicate that the effects reported on other measures were due to some factor other than actual SWB improvement, e.g. social desirability bias. But biomarkers are not a great metric, and measures were taken to address these concerns, so I find it plausible that the effects in the study population were (nearly) as large as reported.
However:
- The participants were self-selected, largely from people who were already involved with Action for Happiness ("The charity aims to help people take action to create more happiness, with a focus on pro-social behaviour to bring happiness to others around them"), and largely in the UK. They also had to register online. It's unclear how useful it would be for other populations.
- It's quite an intensive program, involving weekly 2–2.5 hour group meetings with a trained facilitator two volunteer facilitators. ("Each of these sessions builds on a thematic question, for example, what matters in life, how to find meaning at work, or how to build happier communities.") This may limit its scalability and accessibility to certain groups.
- Follow-up was only for 2 months, the duration of the course itself. (This limitation seems to be due to the study design: the control group was people taking the course 8 weeks later.)
- The effect sizes for depression and anxiety were smaller than for CBT, so it may still not be the best option for mental health treatment (though the CBT studies were done in populations with a diagnosed mental disorder, so direct comparison is hard; and subgroup analyses showed that people with lower baseline wellbeing benefited most from the program).
- For clarity, the average effect size for life satisfaction was about 1 point on a 10-point scale. This is good compared to most wellbeing interventions, but that might say more about how ineffective most other interventions are than about how good this one is.
So at the risk of sounding too negative: it's hardly surprising that people who are motivated enough to sign up for and attend a course designed to make them happier do in fact feel a bit happier while taking the course. It seems important to find out how long these effects endure, and whether the course is suitable for a broader range of people.
This is how they describe their facilitators:
Not sure if that's what you had understood and meant with 'trained facilitator' (just wanted to make it clear that it doesn't mean licensed behavioral therapist or something).
Thanks - "trained facilitator" might be a bit misleading. Still, it looks like there were two volunteer course leaders for each course, selected in part for their unspecified "skills", who were given "on-going guidance and support" to facilitate the sessions, and who have to arrange a venue etc themselves, then go through a follow-up process when it's over. So it's not a trivial amount of overhead for an average of 13 participants.
Thanks for your thoughts!
Yes, regarding persistence they also note:
Thanks - I missed that on my skim. But the "extended" follow-up is only for another two months. It does seem to indicate that effects persist for at least that period, without any trend towards baseline, which is promising (though without a control group the counterfactual is impossible to establish with confidence). I wonder why they didn't continue to collect data beyond this period.