We would like to clarify several methodological points raised above regarding the analyses and interpretation of findings reported in Ben et al. (2026).
We would first like to emphasize that we do not claim to provide conclusive evidence regarding the impact of IMPALA on HDU mortality; this limitation is explicitly acknowledged in the manuscript. The study was underpowered for mortality in the HDU (Ben et al., 2026, p. 3, 9-10), and the confidence interval is wide and includes zero (estimated effect −9.8 percentage points [95% CI: −26.5 to 5.0]).
However, the fact that the estimated mortality reduction of 9.8 percentage points exceeds the observed pre-intervention mortality rate of 8.1% is not in itself contradictory. The estimate refers to the post-intervention population, whose patient mix differed substantially from that in the pre-intervention period. If children in the post-intervention period were more severely ill, their counterfactual mortality risk in the absence of IMPALA could have been considerably higher than the observed pre-intervention mortality rate.
Importantly, the study finds a large and precisely estimated reduction (of almost 50%) in critical illness events, an outcome for which it was adequately powered. Since critical illness events are associated with heightened risk of mortality through rapid deterioration, this finding is consistent with a potential reduction in mortality relative to the counterfactual. Nevertheless, given the small sample size and limited information on patient composition, we cannot draw firm conclusions about the exact effect size.
The first version of the manuscript did not accurately describe the analyses. The TMLE models included the entire post-period HDU cohort, not just the children who were monitored with IMPALA (Online Supplement 11). We corrected this description in the final version. Not all children in HDU were put on a monitor during their stay, as admissions during peak times were sometimes higher than the number of available monitors (p.10). The estimates remained unchanged because the correction concerned only the description, not the underlying analyses.
We accounted for changes in patient-mix as much as possible with the available data. We underscore that, contrary to the suggestion above, we did include indicators of clinical severity in the estimation models for HDU: this is proxied by the Blantyre Coma Scale and abnormal vital signs at admission (Online Supplement 11). The models for both hospitals also included available covariates that differed significantly between the pre- and post-intervention cohorts, including diagnoses (HIV, malaria) and an indicator of multimorbidity, to proxy for potential cause of mortality. The models also included additional indicators chosen in discussion with the research team and clinicians familiar with the study setting (Online Supplement 11).
To further address changes in patient-mix, we used a Super Learner ensemble to estimate the propensity score (p. 4), combining multiple candidate algorithms (here, a logistic regression model with a simple baseline model) according to their cross-validated predictive performance, rather than relying on a single pre-specified model. We also conducted sensitivity analyses (p.5), including Coarsened Exact Matching as an alternative matching approach and E-values to assess the potential influence of unmeasured confounding. These analyses yielded results broadly consistent with the main findings, although they cannot rule out residual confounding.
That being said, we fully agree that the evidence from a pre-post analysis is, at best, suggestive. Such analyses remain vulnerable to bias due to unobserved confounders that cannot be adequately accounted for with the available data. A randomized evaluation would provide stronger causal evidence, but we have not yet been able to secure sufficient funding for such a study. In the meantime, we believe it is valuable to report and critically interpret our analyses using the best available data and methods, while being transparent about the limitations of the study design.
Finally, to clarify the respective roles of the co-authors, Goal3 staff contributed to data collection and manuscript preparation, while the academic researchers were responsible for the statistical analyses and reporting of the study findings.
Best regards, Wendy Janssens On behalf of all co-authors
Hi Martijn,
We would like to clarify several methodological points raised above regarding the analyses and interpretation of findings reported in Ben et al. (2026).
We would first like to emphasize that we do not claim to provide conclusive evidence regarding the impact of IMPALA on HDU mortality; this limitation is explicitly acknowledged in the manuscript. The study was underpowered for mortality in the HDU (Ben et al., 2026, p. 3, 9-10), and the confidence interval is wide and includes zero (estimated effect −9.8 percentage points [95% CI: −26.5 to 5.0]).
However, the fact that the estimated mortality reduction of 9.8 percentage points exceeds the observed pre-intervention mortality rate of 8.1% is not in itself contradictory. The estimate refers to the post-intervention population, whose patient mix differed substantially from that in the pre-intervention period. If children in the post-intervention period were more severely ill, their counterfactual mortality risk in the absence of IMPALA could have been considerably higher than the observed pre-intervention mortality rate.
Importantly, the study finds a large and precisely estimated reduction (of almost 50%) in critical illness events, an outcome for which it was adequately powered. Since critical illness events are associated with heightened risk of mortality through rapid deterioration, this finding is consistent with a potential reduction in mortality relative to the counterfactual. Nevertheless, given the small sample size and limited information on patient composition, we cannot draw firm conclusions about the exact effect size.
The first version of the manuscript did not accurately describe the analyses. The TMLE models included the entire post-period HDU cohort, not just the children who were monitored with IMPALA (Online Supplement 11). We corrected this description in the final version. Not all children in HDU were put on a monitor during their stay, as admissions during peak times were sometimes higher than the number of available monitors (p.10). The estimates remained unchanged because the correction concerned only the description, not the underlying analyses.
We accounted for changes in patient-mix as much as possible with the available data. We underscore that, contrary to the suggestion above, we did include indicators of clinical severity in the estimation models for HDU: this is proxied by the Blantyre Coma Scale and abnormal vital signs at admission (Online Supplement 11). The models for both hospitals also included available covariates that differed significantly between the pre- and post-intervention cohorts, including diagnoses (HIV, malaria) and an indicator of multimorbidity, to proxy for potential cause of mortality. The models also included additional indicators chosen in discussion with the research team and clinicians familiar with the study setting (Online Supplement 11).
To further address changes in patient-mix, we used a Super Learner ensemble to estimate the propensity score (p. 4), combining multiple candidate algorithms (here, a logistic regression model with a simple baseline model) according to their cross-validated predictive performance, rather than relying on a single pre-specified model. We also conducted sensitivity analyses (p.5), including Coarsened Exact Matching as an alternative matching approach and E-values to assess the potential influence of unmeasured confounding. These analyses yielded results broadly consistent with the main findings, although they cannot rule out residual confounding.
That being said, we fully agree that the evidence from a pre-post analysis is, at best, suggestive. Such analyses remain vulnerable to bias due to unobserved confounders that cannot be adequately accounted for with the available data. A randomized evaluation would provide stronger causal evidence, but we have not yet been able to secure sufficient funding for such a study. In the meantime, we believe it is valuable to report and critically interpret our analyses using the best available data and methods, while being transparent about the limitations of the study design.
Finally, to clarify the respective roles of the co-authors, Goal3 staff contributed to data collection and manuscript preparation, while the academic researchers were responsible for the statistical analyses and reporting of the study findings.
Best regards,
Wendy Janssens
On behalf of all co-authors