Thanks Shubham! a) yes, agreed that delivery will be crucial (and may well be harder than the development). We haven't done a study ourselves, however are basing our work on several other studies:
the Annual Status of Education Report found that 2/3 of rural students had access to a smartphone. I presume (and am backed by my own experience) that this is higher in urban areas.
Our literature review found several studies of EdTech in India, though most of these involved the researchers/school providing the hardware, rather than using parents' phones.
b) My organisation, DEVI Sansthan, has partnered with Teach for India in the past, and we're aware of Pratham, Labhya, LLF, CSF etc's work. We are ourselves quite well grounded in the frustrating intricacies of the Indian education system, through rolling out the paper-based ALfA program in thousands of schools. But it's a good point that if we involve some other organisations in the development of the program, then they could be very helpful in distributing it (in a sense, this is what we are hoping with Google's ReadAlong).
Thanks Ozzie! I agree that it's a bit hard to take $5 per income doubling at face value. I was tempted to revise some of my figures (eg reduce the 1/4 probability of 'success') to come up with a more 'realistic' figure (though arguably this undermines the very process of cost efficacy analysis if we post-hoc change the inputs to reach an output we want/find plausible). I fully agree that the uncertainty bars are large. Addressing your points in more detail: 1. It's true, David & Edward have both made massive contributions to the coding for free. That said, both have reported that use of AI has consistently helped them achieve goals faster & easier than they had thought. 2. We were budgeting $3 per user for distribution costs (row 41 on first sheet of spreadsheet) which is roughly what my organisation's paper-based intervention currently costs. But it obviously depends a lot on whether word-of-mouth helps it take off, or if we need to continue promoting it widely. 3. Agreed that the AI landscape is highly uncertain; for this reason we are only modelling out 5 years and not assuming any benefits post that period (perhaps we should reduce this to 3 years). More broadly, I think if/when we do reach AGI, it will require a massive rethink of everything - including most global health & development initiatives - and most cost-efficacy analyses will go out the window. Whether returns to education would increase or decrease in a post-AGI world is a fascinating question. 4. Agreed, we have not yet built a Hindi version of the app; once we have this will be a top priority to measure learning gains. The effect sizes are derived from both others' studies of comparable EdTech and my organisation's paper-based ALfA program. As written above, I have similar doubts about whether we can hold the user's attention with the app - will require significant work. Fair call to be skeptical until we get some results. If/when we do manage to build the app and pilot it, will post the results here. Thanks, Tom
Love the design here! One doubt I have is that in India, where I live, the air quality is worst during winter - a time of year that you wouldn't want a ceiling fan on. Would be interested if you have any thoughts on an alternative to use in winter.
Thanks for sharing this idea! I want to encourage you that it may be much more cost-effective if you do include health benefits.
I appreciate that you left health benefits out of your analysis because of this GiveWell meta-analysis which found that distribution of clean-burning stoves did not have major health benefits. However, my reading of this study is that many of the distribution programs suffered from low usage of the new, non-traditional stoves; hence the indoor air pollution did not decrease much. In your intervention, if people did start soaking beans that seems virtually certain to reduce the amount of time required to cook -> reduce indoor air pollution -> improve health.
I live and work in North India, where there are also still substantial issues with indoor air pollution. I've started a small program to help subsidize people switching from biomass stoves to gas stoves. Using fairly similar assumptions and methods to yours, my rough cost-effectiveness analysis found that subsidizing one family's gas connection saved (over a 3 year period) 0.3 DALYs, which is probably more important than the $100 or the 3 tonnes of CO2e which it also saved.
Thanks Shubham!
a) yes, agreed that delivery will be crucial (and may well be harder than the development). We haven't done a study ourselves, however are basing our work on several other studies:
b) My organisation, DEVI Sansthan, has partnered with Teach for India in the past, and we're aware of Pratham, Labhya, LLF, CSF etc's work. We are ourselves quite well grounded in the frustrating intricacies of the Indian education system, through rolling out the paper-based ALfA program in thousands of schools. But it's a good point that if we involve some other organisations in the development of the program, then they could be very helpful in distributing it (in a sense, this is what we are hoping with Google's ReadAlong).
Thanks again!
Thanks Ozzie!
I agree that it's a bit hard to take $5 per income doubling at face value. I was tempted to revise some of my figures (eg reduce the 1/4 probability of 'success') to come up with a more 'realistic' figure (though arguably this undermines the very process of cost efficacy analysis if we post-hoc change the inputs to reach an output we want/find plausible). I fully agree that the uncertainty bars are large.
Addressing your points in more detail:
1. It's true, David & Edward have both made massive contributions to the coding for free. That said, both have reported that use of AI has consistently helped them achieve goals faster & easier than they had thought.
2. We were budgeting $3 per user for distribution costs (row 41 on first sheet of spreadsheet) which is roughly what my organisation's paper-based intervention currently costs. But it obviously depends a lot on whether word-of-mouth helps it take off, or if we need to continue promoting it widely.
3. Agreed that the AI landscape is highly uncertain; for this reason we are only modelling out 5 years and not assuming any benefits post that period (perhaps we should reduce this to 3 years). More broadly, I think if/when we do reach AGI, it will require a massive rethink of everything - including most global health & development initiatives - and most cost-efficacy analyses will go out the window. Whether returns to education would increase or decrease in a post-AGI world is a fascinating question.
4. Agreed, we have not yet built a Hindi version of the app; once we have this will be a top priority to measure learning gains. The effect sizes are derived from both others' studies of comparable EdTech and my organisation's paper-based ALfA program. As written above, I have similar doubts about whether we can hold the user's attention with the app - will require significant work.
Fair call to be skeptical until we get some results. If/when we do manage to build the app and pilot it, will post the results here.
Thanks,
Tom
Love the design here!
One doubt I have is that in India, where I live, the air quality is worst during winter - a time of year that you wouldn't want a ceiling fan on. Would be interested if you have any thoughts on an alternative to use in winter.
Thanks for sharing this idea! I want to encourage you that it may be much more cost-effective if you do include health benefits.
I appreciate that you left health benefits out of your analysis because of this GiveWell meta-analysis which found that distribution of clean-burning stoves did not have major health benefits. However, my reading of this study is that many of the distribution programs suffered from low usage of the new, non-traditional stoves; hence the indoor air pollution did not decrease much. In your intervention, if people did start soaking beans that seems virtually certain to reduce the amount of time required to cook -> reduce indoor air pollution -> improve health.
I live and work in North India, where there are also still substantial issues with indoor air pollution. I've started a small program to help subsidize people switching from biomass stoves to gas stoves. Using fairly similar assumptions and methods to yours, my rough cost-effectiveness analysis found that subsidizing one family's gas connection saved (over a 3 year period) 0.3 DALYs, which is probably more important than the $100 or the 3 tonnes of CO2e which it also saved.
So I think this is a promising idea!