Great that you did this, really appreciate it.
I'm no expert on the biology, but my intuition would in any case have been that the effect size would be tiny/negligible for 6 weeks of supplementation, and that for non-trivial effects, you would need sustained supplementation over a longer time period.
Is there any reason to doubt my intuition on this?
I believe prior work showed large effects from short periods of supplementation. (Edit: Note that this work seems to debunk prior work, but this should explain the study design)
Bravo! This really sets a bar for the quality of inquiry we should strive for in this community.
Thanks for doing this, Fabienne! Relatedly, readers may be interested in The effects of creatine supplementation on cognitive function in adults: a systematic review and meta-analysis, which was published this year, and includes your study.
Background: This study aimed to evaluate the effects of creatine monohydrate supplementation on cognitive function in adults and explore its potential role in preventing and delaying cognitive impairment-related diseases.
Methods: Following the PRISMA 2020 guidelines, a systematic review with meta-analysis was conducted. Randomized controlled trials (RCTs) published between 1993 and 2024 were retrieved from PubMed, Scopus, and Web of Science databases. The study protocol was registered with PROSPERO (registration number: CRD42024533557). The impact of creatine supplementation on overall cognitive function, memory, executive function, attention, and information processing speed was assessed using standardized mean differences (SMD) and Hedge’s g with 95% confidence intervals (CI).
Results: Sixteen RCTs involving 492 participants aged 20.8–76.4 years, including healthy individuals and patients with specific diseases, were selected. Creatine monohydrate was the form used in all included studies. Creatine supplementation showed significant positive effects on memory (SMD = 0.31, 95% CI: 0.18–0.44, Hedges’s g = 0.3003, 95% CI: 0.1778–0.4228) and attention time (SMD = −0.31, 95% CI: −0.58 to −0.03, Hedges’s g = −0.3004, 95% CI: −0.5719 to −0.0289), as well as significantly improving processing speed time (SMD = −0.51, 95% CI: −1.01 to −0.01, Hedges’s g = −0.4916, 95% CI: −0.7852 to −0.1980). However, no significant improvements were found on overall cognitive function or executive function. Subgroup analyses revealed that creatine supplementation was more beneficial in individuals with diseases, those aged 18–60 years, and females. No significant differences were found between short- (<4 weeks) and long-term (≥4 weeks) interventions for improving cognitive function. Low-to-moderate risk of bias was found, and no significant publication bias was detected. The GRADE assessment indicates that the certainty of evidence for memory function is moderate, suggesting a reasonable level of confidence in the positive effects of creatine on memory. However, the evidence for processing speed, overall cognitive function, executive function, and attention is of low certainty, indicating that further research is needed to confirm these potential benefits.
Conclusion: Current evidence suggests that creatine monohydrate supplementation may confer beneficial effects on cognitive function in adults, particularly in the domains of memory, attention time, and information processing speed. Larger robust clinical trials are warranted to further validate these findings. Furthermore, future research should investigate the influence of different populations and intervention durations on the effects of creatine monohydrate supplementation, as well as elucidate the precise mechanisms underlying its potential cognitive-enhancing properties.
I have just published a break-even analysis of creatine supplementation. I estimate the improvement in cognitive function caused by creatine supplementation is worth it for a net income above 10.6 k$/year.
Hi @Fabienne,
Have you considered running a similar study for long-chain omega-3 supplementation @Paul_Christiano may be interested too. Witte et al. (2013) ran a randomised, double-bind, placebo-controlled trial involving 65 healthy people which found it “enhanced executive functions by 26%, whereas performance remained constant after placebo (paired t-test, t(31) = 3, P = 0.005; Fig. 3A)”. Based on Fig. 3A, they are saying the mean of the following composite score for executive function among participants in the treatment group increased by 0.26. “[z phonemic fluency + z semantic fluency - z TMT [trail making test] (part B - part A)/partA - z STROOP [Stroop Color-Word test] (part 3 - (part 1 + part 2))/2]/4”, where z refers to the z-score of a person on a test. “Regarding the composite score for memory, both groups showed a similar retest effect at follow-up, with no significant effect of group (ANOVARM, group X time: P = 0.6; time: F1,63 = 19.8, P < 0.001)”. “For sensorimotor speed, a global retest effect was noticed that did not differ between groups (composite score, ANOVARM, time: F1,63 = 14, P < 0.001)”.
Thanks for comparing vegetarian and omnivore diets. Interested to see vegan diet results too since vegan diets restrict even more sources of dietary creatine than vegetarian diets
Strongly upvoted. I'd love to see more work like this: well-powered, preregistered, and directly answering an important question.
Would be interesting to see a SEM analysis of this looking at the impact on a latent IQ measure behind these test results that each try to get at it separately. This model should have much better power to find an effect and not suffer from multiple hypothesis testing issues looking at the different tests separately. Model would be of the form
Latent variable definition
IQ =~ RAPM+BDS+Other tests
Regression
IQ ~ Creatine+Other explanatory variables(Age,vegetarian etc.)
Our paper “The effects of creatine supplementation on cognitive performance - a randomised controlled study” is out now!
→ Paper: https://doi.org/10.1186/s12916-023-03146-5
→ Twitter thread: https://twitter.com/FabienneSand/status/1726196252747165718?t=qPUghyDGMUb0-FZK7CEXhw&s=19
Jan Brauner and I are very thankful to Paul Christiano for suggesting doing this study and for funding it.
My understanding of the results: for the preregistered tasks you measured effects of 1 IQ point (for RAPM) and 2.5 IQ points (for BDS), with a standard error of ~2 IQ points. This gives weak evidence in favor of a small effect, and strong evidence against a large effect.
You weren't able to measure a difference between vegetarians and omnivores. For the exploratory cognitive tasks you found no effect. (I don't know if you'd expect those tests to be sensitive enough to notice such a small effect.)
At this point it seems a bit unlikely to me that there is a clinically significant effect, maybe I'd bet at 4:1 against the effect being >0.05 SD. That said I still think it would be worthwhile for someone to do a larger study that could detect a 0.1 SD effect, since that would be clinically significant and is very weakly suggested by this data (and would make supplementation worthwhile given how cheap it is).
(See also gwern's meta-analysis.)
Unless I'm misunderstanding, does this mean you'd bet that the effects are even smaller than what this study found on its preregistered tasks? If so, do you mind sharing why?
Yes, I'd bet the effects are even smaller than what this study found. This study gives a small amount of evidence of an effect > 0.05 SD. But without a clear mechanism I think an effect of < 0.05 SD is significantly more likely. One of the main reasons we were expecting an effect here was a prior literature that is now looking pretty bad.
That said, this was definitely some evidence for a positive effect, and the prior literature is still some evidence for a positive effect even if it's not looking good. And the upside is pretty large here since creatine supplementation is cheap. So I think this is good enough grounds for me to be willing to fund a larger study.