Thanks for writing this up; super interesting, and it does seem like a cluster RCT would be a good idea. Two questions:
1) You're seeking additional funding, which understandably leads you to highlight ways in which Zzapp is promising. Would you mind also highlighting what you view as being the biggest downsides of the intervention (i.e., if it were to wind up being less cost-effective than LLINs, why do you think that would be)?
2) You say one of your goals is to develop a cost-effective solution for rural areas. But why not just focus on evaluating/scaling this up in urban areas, where it looks most promising?
Hi thanks for your comment 1) Bed nets are a good example, but other interventions like house spraying, seasonal drugs for toddlers, and investing in diagnostics are also relevant. A separate post may be needed to discuss the best combination of interventions for different conditions. In brief, compared to bed nets, our interventions require strong implementation capabilities and government commitment, making the process more complex. Zzapp aims to simplify implementation, but it remains a challenging task. Kind of "deep" implementation.
2)That's a great question! As malaria rates are higher in rural areas of Africa than in urban areas, we are still deciding whether to focus or expand. Our decision will be based on our internal resources, demand from countries, and potential impact.
This looks like excellent work, a very logical intervention and where you’ve put a lot of effort into putting together the data to attract serious funding for a scale-up.
One thing I would like to know: in urban areas, I presume access to medical care is higher, and so I am wondering whether the death rate, as well as incidence, may be lower. I see that you achieved a 52% reduction in cases, and I am wondering if you have data, or will be gathering data, on the effect on deaths due to malaria?
That's a valid point. Morbidity and mortality rates are indeed lower in cities, and I attempted to reflect this in my calculation. However, if someone has more accurate data, I would be happy to update my calculations. Designing an experiment around death is extremely challenging because it requires 200 times more area to achieve the same statistical power.
I have a second question. You compared before/after intervention malaria rates for the treated vs. control districts, and found that the multiplier was 52.5% lower in the treated areas. Do we have information on how this compares to historical data? Also, were the districts randomly selected for the treatment vs. control group, or was it chosen on a convenience basis?
I am thinking about the possibility that the treated and control districts may have significantly different base rates of malarial increase at the seasonal time points chosen for the before and after measurements, since there are only 7 districts and it sounds like they may be ecologically and demographically heterogeneous.
Having looked at the original paper, I found a partial answer in table 3:
Before the intervention, the treatment district had a malaria rate 3.3 times higher than the control district. After the intervention, the treatment district had a malaria rate 1.6 times higher than the control district. There were large differences in the levels of malaria incidence between the two districts before the intervention.
As far as I can tell, there has not been an attempt to rule out the possibility and size of any systematic differences in how malaria fluctuates or how it is measured between the treatment and control districts. To address this, historical data showing the average multiplier in these districts during the same time of year in previous years when the treatment was not applied could be used to compare with the current base rates.
If historical data is available for these districts, it seems like it ought to be possible to examine that historical data prior to rolling out a larger-scale $6 million RCT.
If I am making mistakes in this analysis, please let me know and I will correct my comment. Thank you!
First of all, a small clarification - we are seeking $6M for various purposes; the cost of the RCT should be about $1.5M. To the main point: historical data from the Ministry of Health on Anopheles mosquitoes supports the same conclusions but was not included in the publication (history of malaria data per district has not been shared with us by the government). As highlighted in the paper, the intervention was a pilot and NOT a clustered randomized control trial (cRCT), though it was the Ministry of Health (and not us) who selected the intervention and control areas.
In other words, we do need a cRCT to fully validate our method, but the existing evidence is definitely strong enough to justify spending $1.5M on rolling out such an cRCT.
Hope this answers your questions. Let me know if something is still unclear.
Thanks for this approachI really like it I really like it. I also might message you to follow up on your house identfication system - our org OneDay Health could use that kind of ability!
One important point here which hasn't been mentioned, one of which is that relatively few people die of malaria in Urban areas - they have access to healthcare and are treated promptly. Here in my town of Gulu, malaria mortality from people who live in town approaches zero (I almost never hear of people die who live in town), while in rural areas without good access to healthcare in the village mortality remains high.
Any intervention reducing malaria incidence in urban areas will therefore prevent far fewer deaths than the same reduction of incidence in rural areas, perhaps an order of magnitude less - you can't just use general mortality rates and match them with prevalence to estimate deaths in Urban areas.
Cost effectiveness of doing anything for malaria in Urban areas needs massive discounting compared with rural interventions - I'm too lazy to look into the research to see exactly how much
Interested to hear your thoughts on this and any rebuttal
Thank you for sharing your comment with us. We have considered it in our analysis, but we cannot confirm the accuracy of the morbidity and mortality ratios we have used. We may be able to provide the exact calculation next week, if time permits. It is worth noting that we have assumed that semi-urban areas may be a significant contributor to mortality. These areas differ from Gulu in two important ways - they are more rural and have more water bodies, and they are less developed, which means they may have fewer clinics.
Hi Arnon, thanks for sharing your work here. I'm always excited to hear about new ideas for fighting malaria.
I notice that the study you published was created by ZzappMalaria staff itself. Are you planning to partner with outside researchers to conduct the proposed follow-up study? For example, GiveDirectly has partnered with researchers at a number of top universities to study various aspects of their program. When a charity studies its own program there can be more conflicts of interest than when the study is conducted by disinterested academics.
Hi Ian, I totally agree, super important to engage leading scientists (and the company is working together with leading scientists on some collaborations and publications).
In this specific operation (which was not designed as a study but rather as an operational pilot) the authors are from Sao Tome's Ministry of Health and from the company, so we do have here a level of external control, but again, we should aim for more.
I'm Arnon Houri Yafin, CEO of ZzappMalaria. I'm writing about our larviciding pilot published in Malaria Journal (Vigodny et al. 2023), and this post is my interpretation of the results.
ZzappMalaria: Twice as cost-effective as bed nets in urban areas
TL;DR: Zzapp Malaria’s digital technology for planning and managing large-scale anti-malaria field operations attained results that are twice as cost-effective as bed nets in reducing malaria in urban and semi-urban settings.
Call to action
Use our solution
Fund us
How to save more than 140,000 people annually for a cost per person that is lower than bed nets?
In 2021, 627,000 people died from malaria, more than 80% of whom were children. The number of people who live in areas at high risk for malaria is approximately 1.1 billion - which comprises the population of sub saharan Africa (excluding South Africa) and high burden malaria countries such as Odisha in India. According to estimates, more than 75% of these people live in urban and peri urban areas for which our solution costs less than US$0.7 per person per year. The cost to protect all these people is therefore:
0.7 * 1.1B * US$0.7 = US$0.54B.
The malaria incidence in urban areas is lower than in villages so it is assumed that the urban and peri urban population accounted for 45% of the malaria death:
0.45 * 627,000 = 282,100 people.
Assuming that our solution can reduce malaria by 52.5% (based on a peer-reviewed article we recently published in Malaria Journal; elaboration below), our intervention can save:
282,100 * 0.525 = 148,100 people.
As would be elaborated below, we believe the actual numbers of both cost and of effectiveness will dramatically improve over time.
What is Zzapp’s solution and how does its technology work?
TL;DR: We believe in actively targeting disease-bearing mosquitoes, and use digitization to manage large-scale field operations focusing on treatment of mosquitoes breeding sites.
ZzappMalaria harnesses entomological knowledge and data analyzed from satellite imagery and collected by field workers to optimize large-scale larviciding operations. In such operations, the stagnant water bodies in which Anopheles mosquitoes breed are treated with an environmentally benign bacteria that besides mosquitoes and blackflies does not harm any other animals (not even other insects) and is approved for use in sources of drinking water. Although larviciding has led to malaria elimination in many countries in the 20th century, it is not easy to implement since it requires planning, management and monitoring of large teams working kilometers away from each other. To complicate things, effective larviciding requires that a large proportion of water bodies are detected and treated on a regular basis (sometimes weekly).
We developed a system comprising a planning tool, mobile app and dashboard designed to overcome these challenges. The system extracts from satellite images the location of houses and demarcates the general area for the intervention. It then recommends where to scan for water bodies, and helps in implementation through a designated GPS-based mobile app that allocates treatment areas to workers, monitors their location in the field to ensure they cover the entire area, and keeps track of schedules for water body treatment. All information is uploaded to a dashboard, allowing managers to monitor the operation in real time. Specifically designed for sub-Saharan Africa, our house-detection algorithm was developed to detect both modern houses and traditionally built huts. Our app was built to work on inexpensive smartphones and in areas with weak internet infrastructure. Its interface is simple and intuitive, even for workers with basic reading skills and limited or no digital experience.
Screenshots from the Zzapp mobile application. Left: Map view during mapping activity which helps fieldworkers keep track of progress, and ensure the entire area is surveyed in search of water bodies. Middle and right: Sample questions from the questionnaire workers complete for every water body reported
Why do we think this approach is effective?
TL;DR: Because in a large-scale larviciding operation we attained an impact of mosquito population reduction of 74.9% and an impact on malaria reduction of 52.5% for a cost of approximately US$0.86 per person protected (PPP) nationwide, and US$0.44 PPP in urban areas
Recently, together with the ministry of health of the west African country of São Tomé and Príncipe (STP), we completed a large-scale larviciding operation covering three districts with a total area of 240 square kilometers and in which approximately 166,000 people live. The country’s four other districts, in which approximately 62,000 people live, were not included in the intervention, and therefore served as a control. To establish the impact, we compared between malaria cases before the intervention and after it, both in the intervention and in the control areas. In the intervention areas, malaria cases were multiplied by 1.7 and in the control area cases were multiplied by 3.57 which equals an overall impact of 52.5% on malaria cases compared to the control (i.e., prevention of 52.5% of malaria cases). This effectiveness is comparable to – or even slightly better than – what has been reported in studies measuring the effectiveness of bed nets (45%). Even more exciting, our cost in urban areas was significantly lower: US$0.44 per person protected (PPP) per 6 months compared to US$0.695 PPP for bed nets.
Left: Map of the main islands of São Tomé and Príncipe, with the districts of Água Grande, Lobata and Mé-Zóchi in which the operation took place highlighted. Right: Map showing the coverage obtained in mapping activities and the resulting distribution of water bodies in the intervention area.
Breakdown of cost drivers and the distinction between urban and rural areas
TL;DR: population density makes larviciding significantly more cost-effective in urban areas; we developed an AI-generated heat map that estimates larviciding costs per locality
The cost of larviciding in urban areas is considerably lower than in rural areas due to the higher population density (which means fewer water bodies per person), shorter traveling distances (which means less fuel expenses and less working hours), improved infrastructure and lack of agriculture (which means fewer and smaller water bodies). The following illustrations show a breakdown of the total expenditure and a comparison between the expenditure in the rural areas and the urban areas.
Overall breakdown of the cost of the operation
Comparison of the main cost drivers between the rural and the urban areas
In this operation, we defined each locality as urban or rural based on the number of structures per sq km >1,500), which was calculated using Google’s Open Buildings dataset. To automatize the definition of urbanity for future operations, we programmed our AI to factor relevant variables – for example, population density, distance from the headquarters, level of agricultural use (analyzed with machine vision from satellite images) – and create a heatmap that displays the estimated cost for each location.
Density of buildings (the number of houses per operational unit, based on Google’s OpenBuildings)
Cost per person per locality: the cost is determined by factoring population density, abundance of water bodies, and the distance from the operational center
In recent years there is growing interest in urban malaria, mostly due to the growing rate of urbanization in sub-Saharan Africa, which is often regarded as the world's fastest urbanizing region and whose urban population is expected to reach 50% by 2030. In light of this, our tool is expected to assume an important role in assisting policymakers who wish to allocate their anti-malaria budgets between larviciding and other interventions in the most cost-effective way.
Room for improvement
TL;DR: Managerial improvements, automation and digitization can further reduce costs and increase effectiveness of anti-malaria field operations.
In the operation in STP, our cost for the urban area was US$0.44 per person protected (PPP) per 6 months. During the operation we reviewed different management methods, such as providing lunch to fieldworkers, awarding cash handouts to outstanding employees and optimizing working schedules – which improved workers’ productivity by 26%. Based on this, we believe that our annual costs in similar operations in the future will be significantly lower than US$0.7 PPP in urban areas.
In addition to operational optimization, and the expected reduction of costs if applied at scale, improvement may be reached by the incorporation of additional technologies, for example drones. Although we did not use drones in STP, we know (from an operation in Zanzibar incorporating drones in which we took part; publication pending) that they can be very useful in covering large areas and that they are better at detecting water bodies in rural areas.
Source of uncertainty
We derived the numbers from a pilot, using the country's ongoing data. The pilot is not a randomized controlled trial and the statistics are based on comparison of a small number of districts. In order to substantiate the generalization of the results, there is need for additional and larger trials that will be conducted across various climate regions and with different rates of malaria prevalence. However, given the fact that larviciding is a well established method, and that the Zzapp system is proven to effectively manage it, the risk is low.
Scale of the problem we tackle
Malaria is a global problem that poses a risk to nearly half of the world's population, and which, in 2020, caused sickness to an estimated 241 million people and killed more than 627,000 people. We are limiting the discussion here to malaria in urban or peri-urban areas in Sub-Saharan Africa, where approximately 74.9% of the population of 1.1 billion lives.
What still needs to be done?
Zzapp is now seeking US$6M for:
(1) running a randomized controlled trial to test its current solution;
(2) developing a cost-effective solution for rural areas;
(3) developing our next gen product: an AI-based system that optimizes the best mix of integrations – larviciding and other methods – per village.
Acknowledgements
I would like to express my sincere gratitude to Edo Arad, Niki Kosteno, and Yonatan Cale for their valuable contributions in writing this post.
This is the third in a sequence of posts taken from my recent report: Why Did Environmentalism Become Partisan?
Summary
Rising partisanship did not make environmentalism more popular or politically effective. Instead, it saw flat or falling overall public opinion, fewer major legislative achievements, and fluctuating executive actions.
Public Opinion...
This post presents the executive summary from Giving What We Can’s impact evaluation for 2025. At the end of this post we share links to more information, including the full report and...
Thanks for writing this up; super interesting, and it does seem like a cluster RCT would be a good idea. Two questions:
1) You're seeking additional funding, which understandably leads you to highlight ways in which Zzapp is promising. Would you mind also highlighting what you view as being the biggest downsides of the intervention (i.e., if it were to wind up being less cost-effective than LLINs, why do you think that would be)?
2) You say one of your goals is to develop a cost-effective solution for rural areas. But why not just focus on evaluating/scaling this up in urban areas, where it looks most promising?
Hi thanks for your comment
1) Bed nets are a good example, but other interventions like house spraying, seasonal drugs for toddlers, and investing in diagnostics are also relevant. A separate post may be needed to discuss the best combination of interventions for different conditions.
In brief, compared to bed nets, our interventions require strong implementation capabilities and government commitment, making the process more complex. Zzapp aims to simplify implementation, but it remains a challenging task. Kind of "deep" implementation.
2)That's a great question! As malaria rates are higher in rural areas of Africa than in urban areas, we are still deciding whether to focus or expand. Our decision will be based on our internal resources, demand from countries, and potential impact.