Nick, just wanted to share that I personally strongly held the view that RCTs usually produce smaller effects than observational trials until reading a 2024 Cochrane study examining this specific question - in short, I think it supports a prior view that well-conducted observational studies (at least in healthcare) are not much more likely to overestimate effects than RCTs. To your credit you explicitly include "larger cohorts" among your example of "better studies", so perhaps you already know this! But it was news to me. (Of course I also agree that small and/or low-quality studies, regardless of methodology, should be taken with caution).
UPDATE Dec. 1, 2025: We recently completed a literature review and meta-analysis of randomized controlled trials evaluating the impact of complex neonatal health interventions on neonatal mortality. We found that increased rates of early initiation of breastfeeding (EIBF) and skin-to-skin contact (SSC) correlated with reduction in neonatal mortality. Substantial improvements in EIBF and SSC were seen even though interventions did not appear to emphasize these practices, increasing our confidence that they can be readily improved. A summary of our findings, as well as a link to the full report, can be found on the HealthLearn blog.
Nick, just wanted to share that I personally strongly held the view that RCTs usually produce smaller effects than observational trials until reading a 2024 Cochrane study examining this specific question - in short, I think it supports a prior view that well-conducted observational studies (at least in healthcare) are not much more likely to overestimate effects than RCTs. To your credit you explicitly include "larger cohorts" among your example of "better studies", so perhaps you already know this! But it was news to me. (Of course I also agree that small and/or low-quality studies, regardless of methodology, should be taken with caution).
UPDATE Dec. 1, 2025: We recently completed a literature review and meta-analysis of randomized controlled trials evaluating the impact of complex neonatal health interventions on neonatal mortality. We found that increased rates of early initiation of breastfeeding (EIBF) and skin-to-skin contact (SSC) correlated with reduction in neonatal mortality. Substantial improvements in EIBF and SSC were seen even though interventions did not appear to emphasize these practices, increasing our confidence that they can be readily improved. A summary of our findings, as well as a link to the full report, can be found on the HealthLearn blog.
This is great- really curious to hear (in due time) about your experience and learnings!