I'll pick up on specifically the Liberia thread, because I think it provides an interesting context to explore!
My (again, very rough) understanding is that much of the Liberian CHW program was funded/run externally to Liberia (lots of international donor dollars from bilaterals, implementation details were handled by partners like Last Mile Health or others depending on the state). I think a lot of this has gone off the rails post-USAID, essentially revealing this wasn't really a domesticated program (which doesn't mean it is inherently bad, just worth being clear-eyed about if true).
Assuming that understanding is correct, I think this is somewhat instructive: Liberia may be a good example of where proCHWs have worked, but it is not a great example of where 'lobby the government with a 5 year catalytic grant and then the government will pay for it forever' has worked. As we think about what specific interventions we might want to get funded in this space, what we're analyzing with CEAs, etc., I think it will be important to accurately model what we really want. Do we want:
i) to do a pure policy change play ii) to directly fund an NGO to do proCHWs iii) some mixture of both and so on - there's a ton of options here!
My guess is in the right context any of these could be cost-effective (or at least they are equally probable to be so), but that ensuring we have evidence for each part of the theory of change we attempt to enact will be vital.
Thanks for posting this Madeleine, it is great to see people from outside the traditional EA global health space engaging here!
This isn't an area I'm super familiar with, but I'll try to throw in some questions/thoughts to perhaps draw out the argument a bit, because I think this is valuable to think about!
Let's assume for the sake of discussion here that the creation of a CHW program in an area where core CHW-delivered care (like vaccination, malaria bed nets, SMC, vitamin A, deworming, etc.) is completely unavailable is cost-effective at a typical EA bar.
I think it is interesting that the recommended thing to get funded notably slightly different/more indirect, which is to fund policy change to get governments to pay for the creation of more professional CHW programs. I think I'd be really interested in hearing more about the evidence base behind this recommendation (e.g., the systematic review you linked pertains more to academic/NGO interventions designed to improve CHW performance, rather than efforts to improve government rollout of CHWs). Questions I'd be really interested in hearing some more about:
What evidence do we have of externally funded health systems strengthening campaigns successfully changing government implementation of CHWs, in a way that has both (a) created professional, paid CHW programs and (b) which has then actually changed health outcomes in an identified way afterwards?
I imagine this has worked somewhere - I'm only vaguely familiar with CHIC, but from that vague knowledge I know CHWs have been gaining steam. That said, it certainly hasn't worked everywhere. My impression for instance is that lots of funders have spent quite a lot of money trying to get a Nigerian CHW program off the ground only to end up with some policy documents that look nice on paper but CHWs not actually getting paid or doing anything meaningful most of the time.
What has defined the difference between contexts where externally funded programs to change government behavior have worked and where they have not? How can we predict in advance which areas are worth spending money on and which are not?
I think it is then interesting to revisit our assumption at the top here. The counterfactual we're talking about here is probably not zero treatment to CHW treatment. It probably looks more like a reasonably competent government rolls out CHWs in an area that has some existing primary healthcare services - in this situation, how many more people get treatment? At what marginal additional cost? Is that marginal benefit worth that cost?
Super possibly! But just flagging that it isn't as straightforward as the base case assumption we make might be.
I think intuitively we expect this to be really cost-effective in places that are underserved/hard to reach - but those places probably don't have super competent government in the first place, and so is our notional health systems strengthening campaign going to help? Maybe! Maybe not! Would love to hear more thoughts.
Anyway in closing - you guys should put together an EA-style CEA of this! I think that'd be the best way to make this case.
Always happy to answer questions! Thanks for your support + belief in what we do! Means a lot (which sounds very "charity language" but really is true for us and for the other people running charities hanging out around here).
Hi! I'm Justin - I run Taimaka. We're an EA org, but pretty quiet on the forum - keep meaning to get around to writing up something about our work, but hasn't happened yet, so this is a good excuse to say hello!
This is a good question, and our cost-per-life-saved figure is also obviously a bold claim, so I'll share a bit about our thinking here. One disclaimer I'll make for clarity is that while our work is supported by GiveWell, our cost-effectiveness model is our own and the thoughts I'm sharing here are my own - I don't speak for GiveWell's team and their views. Our CEA is built off of their past work on acute malnutrition, but the end results + claims are ours.
Generally, the way I think about our model and our $1.6k per life saved estimate is that this is the most accurate + true estimate we have for our program, but that you should probably read this estimate as having higher error bars surrounding it than estimates for current GiveWell top charities. I think there are two primary reasons for this:
Taimaka is a younger charity with a shorter track record. We're extrapolating from a smaller data set than an organization like, say, New Incentives. Our costs could change (up or down) as we grow over the next few years, or things like the baseline conditions in Gombe State (like prevalence or coverage of acute malnutrition and treatment) where we work could change in ways we are not expecting. We try to build some of this into the model, like how we expect costs to change, but we are extrapolating. Our model is accurate to our current program, but things may change over time and the model may need to be revised.
The evidence base for acute malnutrition treatment is more limited than we'd like it to be/for current top charities. There is no direct causal evidence (in the form of RCTs) for the mortality reduction benefits from acute malnutrition treatment, because treatment was developed before RCTs/clinical trials became popular in the 2000s, and now it's considered unethical to withhold treatment that works from kids to study treatment effect on mortality. Instead, GiveWell (and Taimaka by extension) model mortality reduction through a meta-analysis of historical data that provides information on mortality rate by anthropometric status, combined with data on change in anthropometric status from before/after treatment. This is just a worse evidence base than the RCTs we have for bed nets for instance, and so brings in higher error bars.
We're working on some studies now to supplement this evidence base, but it's going to be a while before those bear fruit.
Hopefully this is helpful! In summary: we take this figure seriously, and stand by our modeling. We haven't put our thumb on the scales anywhere to make this number lower, it's the true result of a good faith effort to adapt GiveWell's model of other acute malnutrition treatment programs to our own. That said, expect higher error bars in this than you would in models for current top charities, both because of vagaries in Taimaka being younger + because of limitations in what we currently know about acute malnutrition treatment. If you're willing to accept that higher level of risk, I think Taimaka is a great donation option to do a lot of good, potentially even more cost-effectively than other places. Happy to have a call to chat this through in more detail if you'd like, feel free to shoot me an email! (My first name at taimaka.org).
I'll pick up on specifically the Liberia thread, because I think it provides an interesting context to explore!
My (again, very rough) understanding is that much of the Liberian CHW program was funded/run externally to Liberia (lots of international donor dollars from bilaterals, implementation details were handled by partners like Last Mile Health or others depending on the state). I think a lot of this has gone off the rails post-USAID, essentially revealing this wasn't really a domesticated program (which doesn't mean it is inherently bad, just worth being clear-eyed about if true).
Assuming that understanding is correct, I think this is somewhat instructive: Liberia may be a good example of where proCHWs have worked, but it is not a great example of where 'lobby the government with a 5 year catalytic grant and then the government will pay for it forever' has worked. As we think about what specific interventions we might want to get funded in this space, what we're analyzing with CEAs, etc., I think it will be important to accurately model what we really want. Do we want:
i) to do a pure policy change play
ii) to directly fund an NGO to do proCHWs
iii) some mixture of both
and so on - there's a ton of options here!
My guess is in the right context any of these could be cost-effective (or at least they are equally probable to be so), but that ensuring we have evidence for each part of the theory of change we attempt to enact will be vital.
Thanks for posting this Madeleine, it is great to see people from outside the traditional EA global health space engaging here!
This isn't an area I'm super familiar with, but I'll try to throw in some questions/thoughts to perhaps draw out the argument a bit, because I think this is valuable to think about!
Let's assume for the sake of discussion here that the creation of a CHW program in an area where core CHW-delivered care (like vaccination, malaria bed nets, SMC, vitamin A, deworming, etc.) is completely unavailable is cost-effective at a typical EA bar.
I think it is interesting that the recommended thing to get funded notably slightly different/more indirect, which is to fund policy change to get governments to pay for the creation of more professional CHW programs. I think I'd be really interested in hearing more about the evidence base behind this recommendation (e.g., the systematic review you linked pertains more to academic/NGO interventions designed to improve CHW performance, rather than efforts to improve government rollout of CHWs). Questions I'd be really interested in hearing some more about:
I think it is then interesting to revisit our assumption at the top here. The counterfactual we're talking about here is probably not zero treatment to CHW treatment. It probably looks more like a reasonably competent government rolls out CHWs in an area that has some existing primary healthcare services - in this situation, how many more people get treatment? At what marginal additional cost? Is that marginal benefit worth that cost?
Anyway in closing - you guys should put together an EA-style CEA of this! I think that'd be the best way to make this case.
Always happy to answer questions! Thanks for your support + belief in what we do! Means a lot (which sounds very "charity language" but really is true for us and for the other people running charities hanging out around here).
Hi! I'm Justin - I run Taimaka. We're an EA org, but pretty quiet on the forum - keep meaning to get around to writing up something about our work, but hasn't happened yet, so this is a good excuse to say hello!
This is a good question, and our cost-per-life-saved figure is also obviously a bold claim, so I'll share a bit about our thinking here. One disclaimer I'll make for clarity is that while our work is supported by GiveWell, our cost-effectiveness model is our own and the thoughts I'm sharing here are my own - I don't speak for GiveWell's team and their views. Our CEA is built off of their past work on acute malnutrition, but the end results + claims are ours.
Generally, the way I think about our model and our $1.6k per life saved estimate is that this is the most accurate + true estimate we have for our program, but that you should probably read this estimate as having higher error bars surrounding it than estimates for current GiveWell top charities. I think there are two primary reasons for this:
Hopefully this is helpful! In summary: we take this figure seriously, and stand by our modeling. We haven't put our thumb on the scales anywhere to make this number lower, it's the true result of a good faith effort to adapt GiveWell's model of other acute malnutrition treatment programs to our own. That said, expect higher error bars in this than you would in models for current top charities, both because of vagaries in Taimaka being younger + because of limitations in what we currently know about acute malnutrition treatment. If you're willing to accept that higher level of risk, I think Taimaka is a great donation option to do a lot of good, potentially even more cost-effectively than other places. Happy to have a call to chat this through in more detail if you'd like, feel free to shoot me an email! (My first name at taimaka.org).