We have added the cash transfer cost , to 91$ after the beneficiaries reported consuming it all on regular medical treatment, when a new regular drug was included in their treatment, pen-V
Hi, the project is still very small, we found a charity that was already doing this and had identified the family, possibly because they were known to be in dire situation (begging) in their community. Other families have been identified through community mechanisms after.
In the future, we can work with health centers to reach to those who are accessing treatment with their consent, even if they are accessing intermittently. Targeting should not be expensive.
We generally don't see a problem with giving the cash and there would be additional complexity if we have to oversee the treatment process, but your concern is reasonable.
we agree. Small, cheaper projects that focus on one thing tend to work better. We understand big ambitious projects can have more potential impact and that "impact" is difficult to measure. E.g. if a peace-building project can truly achieve peace, nothing beats that, is just that most peace-building projects are just workshops with people that have no say on whether war or peace will happen.
We prefer avoiding the word impact entirely and talking of effects (intended, visible), this misses some of the biggest opportunities and lots of indirect benefits.
At least a few of the projects could have benefitted from breaking them down into smaller projects and considering them separately in each category. We were going down that road at the beginning but then we thought that wasn't entirely fair to the other participants. Because many organizations were not fully understanding with how they were going to be evaluated, giving one 3 chances was an unfair advantage.
I think at least one of these projects could have been a finalist on the economic growth section, should it have been broken down in this way, but if the organization applying thought that was a human rights project then we have to compare it on the effects in that category (which we phrased as likelihood of protecting people from abuse, this was the hardest as there is less work on EA on this subject), and then it did not do much but spent lots of money. In any case, this is only relevant for the contest, we tried to have a fair one but that wasn't the ultimate objective. The contest was only relevant to us as a way to identify cost-effective projects that we would like to support further, it was clear this wasn't one of them.
Despite our energetic writing, we may have been carried away. We had very limited tools and information. It would be more accurate to say the first, no evidence of effects.
In fact, we did not have the tools or data to look rigorously into all projects and their intended and unintended effects.
We had two layers:
In the first layer we assume projects do exactly what organizations claim they do, and just establish a possible output per dollar (well in this case, per franc CFA). If they have usable output or outcome data, we use that, if not we may even use research on an equivalent program (eg. the GBV example). In each category, it is easy to compare which output per dollar is cheaper, still with some working assumptions.
From there, the difference between organizations was quite huge, and we had some budget to do data collection for the possible best 6 (thanks EA Infrastructure fund). There we just try to confirm the effect claimed by interviewing beneficiaries of the assistance. In two cases the effect claimed wasn't visible at the time of data collection and that gave us two finalists (economic, because in the other project most beneficiaries did not remember participating, and health, because in the other project participants appeared to be worse off than before the project), in the human rights category there were two very similar projects as finalists and one had slightly stronger effects.
We were clearly biased for small budgets, so the overall winner had a big advantage because it was literally an intervention for 1 family, we think this may still be accurate anyway, and it is plausible that there are great opportunities to do good at small scale in developing countries, particularly through cash.
We had also limitations in comparing across sectors, but more or less the 3 finalists got the same (a badge, a framed award, some feedback they can use with potential donors, and subscription to an online newsletter to funding opportunities. We recognized the human rights final position was more tight and we added the runner-up to the newsletter). We decided to do more for the winner because we thought it was the only one meeting cost-effectiveness expectations and we can't find much better in Cameroon, but that was out of the contest.
Going back to your question, if I have to guess, I am sure these projects may have effects that we did not get to see. I am unsure these effects are achieved in a cost-effective manner because they are buried in so much else.
There is a level of confidentiality because organizations shared this in good faith and not for public dissemination.
Moreover, any numerical values were very speculative and just for the purpose of comparison. We can only confidently share a broad analysis like the one in this post.
Thank you very much! we will repeat the analysis at the end of 2025 with 2025's data.
We have added the cash transfer cost , to 91$ after the beneficiaries reported consuming it all on regular medical treatment, when a new regular drug was included in their treatment, pen-V
This is sensible and we agree. We will take it into consideration.
Hi, the project is still very small, we found a charity that was already doing this and had identified the family, possibly because they were known to be in dire situation (begging) in their community. Other families have been identified through community mechanisms after.
In the future, we can work with health centers to reach to those who are accessing treatment with their consent, even if they are accessing intermittently. Targeting should not be expensive.
We generally don't see a problem with giving the cash and there would be additional complexity if we have to oversee the treatment process, but your concern is reasonable.
I hope this is useful
Hi, we have published a report of this intervention in GlobalGiving: Reports on Help children with sickle cell syndrome - GlobalGiving
we agree. Small, cheaper projects that focus on one thing tend to work better. We understand big ambitious projects can have more potential impact and that "impact" is difficult to measure. E.g. if a peace-building project can truly achieve peace, nothing beats that, is just that most peace-building projects are just workshops with people that have no say on whether war or peace will happen.
We prefer avoiding the word impact entirely and talking of effects (intended, visible), this misses some of the biggest opportunities and lots of indirect benefits.
At least a few of the projects could have benefitted from breaking them down into smaller projects and considering them separately in each category. We were going down that road at the beginning but then we thought that wasn't entirely fair to the other participants. Because many organizations were not fully understanding with how they were going to be evaluated, giving one 3 chances was an unfair advantage.
I think at least one of these projects could have been a finalist on the economic growth section, should it have been broken down in this way, but if the organization applying thought that was a human rights project then we have to compare it on the effects in that category (which we phrased as likelihood of protecting people from abuse, this was the hardest as there is less work on EA on this subject), and then it did not do much but spent lots of money. In any case, this is only relevant for the contest, we tried to have a fair one but that wasn't the ultimate objective. The contest was only relevant to us as a way to identify cost-effective projects that we would like to support further, it was clear this wasn't one of them.
Despite our energetic writing, we may have been carried away. We had very limited tools and information. It would be more accurate to say the first, no evidence of effects.
In fact, we did not have the tools or data to look rigorously into all projects and their intended and unintended effects.
We had two layers:
In the first layer we assume projects do exactly what organizations claim they do, and just establish a possible output per dollar (well in this case, per franc CFA). If they have usable output or outcome data, we use that, if not we may even use research on an equivalent program (eg. the GBV example). In each category, it is easy to compare which output per dollar is cheaper, still with some working assumptions.
From there, the difference between organizations was quite huge, and we had some budget to do data collection for the possible best 6 (thanks EA Infrastructure fund). There we just try to confirm the effect claimed by interviewing beneficiaries of the assistance. In two cases the effect claimed wasn't visible at the time of data collection and that gave us two finalists (economic, because in the other project most beneficiaries did not remember participating, and health, because in the other project participants appeared to be worse off than before the project), in the human rights category there were two very similar projects as finalists and one had slightly stronger effects.
We were clearly biased for small budgets, so the overall winner had a big advantage because it was literally an intervention for 1 family, we think this may still be accurate anyway, and it is plausible that there are great opportunities to do good at small scale in developing countries, particularly through cash.
We had also limitations in comparing across sectors, but more or less the 3 finalists got the same (a badge, a framed award, some feedback they can use with potential donors, and subscription to an online newsletter to funding opportunities. We recognized the human rights final position was more tight and we added the runner-up to the newsletter). We decided to do more for the winner because we thought it was the only one meeting cost-effectiveness expectations and we can't find much better in Cameroon, but that was out of the contest.
Going back to your question, if I have to guess, I am sure these projects may have effects that we did not get to see. I am unsure these effects are achieved in a cost-effective manner because they are buried in so much else.
thank you very much this is appreciated. We did not know we were doing much useful honestly :D, the reaction to this post is very encouraging
There is a level of confidentiality because organizations shared this in good faith and not for public dissemination.
Moreover, any numerical values were very speculative and just for the purpose of comparison. We can only confidently share a broad analysis like the one in this post.
thank you I appreciate this