Seven things that surprised us in our first year working in policy - Lead Exposure Elimination Project

Following the interest in our post announcing the launch of Lead Exposure Elimination Project (LEEP) eight months ago, we are now sharing an update unpacking seven findings that have surprised us in our experiences so far. We hope these will be relevant to others interested in policy change or starting a new project or charity.

For those who are not familiar with LEEP, we are a Charity Entrepreneurship-incubated NGO advocating for lead paint regulation in countries with large and growing burdens of lead poisoning from paint. The reasons we focus on reducing lead exposure are outlined in our introduction post. In short, we believe the problem to be neglected and tractable, and that the intervention has the potential to improve lives with a high level of cost-effectiveness.

1. The speed of progress with government has been less of a limiting factor than expected

In our first target country, Malawi, we had a number of uncertainties about how quickly progress could be made. We were unsure if we would be able to get in touch with the relevant government officials, if they would be willing to engage, and whether our advocacy would lead to action in a reasonable timeframe. We found that stakeholders were far more willing to engage than we had expected. Even without existing connections or introductions, government officials replied to our emails and agreed to meetings. Beyond getting initial meetings, the tractability of achieving change was also higher than expected. After we carried out a study demonstrating high levels of lead in paint, the Malawi Bureau of Standards agreed to begin monitoring and enforcement for lead content of paint (using pre-existing but unimplemented standards), and have since confirmed that they have begun. This change occurred within three months of beginning advocacy in Malawi - significantly faster than our expected timeframe of 1-2 years.

We also found a surprising willingness to cooperate from the local paint industry. Since presenting to the paint manufacturers our findings of lead in paint and the benefits of switching to non-lead, they have engaged with us and with our support identifying non-lead alternative ingredients. We will be carrying out a repeat paint study in a few months to measure how this progress relates to levels of lead paint available on the market.

There are a number of factors that we think contributed to this faster traction and high level of stakeholder engagement. One is the new country-specific data that we were able to generate through a small paint sampling study. We believe that this data provided an effective opener to communications and also convincingly demonstrated that lead paint is a problem in Malawi. Our government contacts confirmed that this Malawi-specific evidence was key for their decision to take action. Generating new country-specific data through small-scale local studies seems to be an effective advocacy strategy that may be cross-applicable to other areas of policy.

Other reasons why stakeholder engagement has been greater than expected might be specific to lead paint regulation advocacy. For example, lead paint regulation is not particularly expensive for governments to implement or for paint manufacturers to comply with, reducing the barrier to action for both stakeholder groups. Also, there is a strong and established evidence-base for the harms of childhood lead poisoning, increasing consensus on the issue. As well as this, there is a growing awareness of a global movement towards lead paint regulation, including examples of neighbouring countries and the support of respected international bodies such as the WHO. This may facilitate action because the change can be seen as inevitable. 

Although progress in Malawi has been far faster than expected we are unclear how broadly this will generalise. It could be that Malawi is a uniquely tractable location and the factors that accelerated progress might not cross apply (e.g. the Bureau of Standards’ willingness to engage), or it might be that certain factors that made progress faster will meaningfully cross apply (like the ease of running paint studies).

2. The presence of legally-binding regulation is often insufficient

This was not a major surprise, but something we had not fully appreciated when starting LEEP. We initially expected our main focus would be advocating for new lead paint regulation in countries with no legally-binding regulation. However, our experience in Malawi demonstrated that the presence of legally-binding lead paint standards does not equate to low levels of lead paint on the market. In Malawi there was simply no monitoring or enforcement for lead in paint because it was not known to be a problem, and there were no major incentives for manufacturers to produce non-lead paint. From conversations with experts in other countries and a few studies that have been carried out after the introduction of regulation there are likely many countries where, for a variety of reasons, lead paint regulation is not being complied with or effectively enforced. 

This has updated us towards working in countries where we expect high levels of lead paint use irrespective of whether there is already regulation in place. It has also highlighted to us the importance of follow-up paint studies and ongoing engagement in our target countries. Even if we successfully advocate for new regulation or for better enforcement and compliance, we will need to continue operations until there is convincing evidence of reduced lead paint on the shelves.

3. There are some surprising low hanging fruit in the area of lead paint regulation advocacy

We were surprised to see some easy-to-overcome bottlenecks in the area. For example, the Lead Paint Project in Madagascar (run through the Ministry of Environment and Sustainable Development) had identified that evidence of lead in paint would be necessary to make progress in their advocacy for lead paint regulation. They had been seeking support for a study into the lead content of paint in Madagascar but had been unable to secure funding, despite the costs of this type of study being relatively low (around $4000). Fortunately, LEEP is able to provide this small amount of funding, as well as a study protocol and support. This absence of data on the lead content of paint is a major gap globally, with more than 80 countries lacking any data on whether their paint contains high levels of lead, including our other current target countries Zimbabwe and Botswana. In Malawi this simple lack of data also seemed to be a primary bottleneck, as evidence of lead in paint was sufficient for the Bureau of Standards to commit to implementing lead paint regulation. 

The presence of this low-hanging fruit can be partly explained by the extreme neglectedness of the area, both in terms of attention and funding. It has been estimated that only around $3 million is spent per year globally on lead paint regulation advocacy (GiveWell).

4. Cost-effectiveness can look very promising even in a small country

Our preliminary modelling of LEEP’s intervention to bring about the implementation of lead paint regulation in Malawi suggests very promising cost-effectiveness. This is despite Malawi being a relatively small country (19.5 million) and is largely driven by the low costs of the intervention for both LEEP and the Malawi government. Currently, we estimate the cost-effectiveness in Malawi to be $12/DALY averted (or roughly $360 per life saved), exceeding comparable GiveWell top charities. In Malawi we estimate that the intervention will avert 76,000 DALY-equivalents over the next 20 years, or 2533 lives saved equivalent.  This estimate depends on various assumptions and should be interpreted cautiously. This writeup explains the model assumptions and uncertainties and links to the model itself. Furthermore, there is uncertainty in applying these estimates to other circumstances, as traction may not be this smooth in other locations. However, these results are very promising and these estimates will become more certain as we target more countries and gather further evidence.

5. Remote advocacy can be viable, especially during COVID 

One assumption we had initially was that much of our advocacy work would centre around in-person meetings. This could slow our work, as we would be limited by the time and resources needed to travel to target countries or recruit in-country staff to engage in advocacy. However, in Malawi we found that government officials, local experts and advisors, and paint industry representatives preferred to meet over the phone or video call rather than in person. This preference is likely mainly due to the COVID-19 pandemic, but even when in-person meetings are safer the precedent might make it more feasible to hold stakeholder meetings remotely in future. We found that the preference for remote meetings made it easier to meet with stakeholders in different cities and countries. We did not notice significant downsides  and were able to make substantial progress with this approach. Of course, there can be many other advantages to having staff in-country, but in some cases the feasibility of remote meetings could significantly reduce costs for policy advocacy organisations. If this trend lasts it could mean reduced costs for policy focused organisations and an increased ability to quickly scale with international staff. This also might provide strong opportunities for newer or smaller organisations aiming to work in policy change without large start-up budgets.

6. Experts have been very generous with their time and advice even though we are new to the field 

As a charity start-up new to the field and to the contexts we are working in, we have reached out to many experts, including academics, lead NGOs, and policy experts, both in our target countries and internationally. We have been continually surprised by and grateful for how consistently generous they have been with their time, advice, and encouragement. This, together with the support of Charity Entrepreneurship, has made LEEP’s first few months run much more smoothly than expected. Many other new Charity Entrepreneurship-incubated organisations have had a similarly supportive experience, and we hope this is encouraging to others considering starting a new project or organisation.

7. There is a challenging fundraising period between seed funding and larger grants, even for quite successful projects in the global health space

A common idea in the health space is that there is a ‘valley of death’, basically a time of particular challenge for NGOs to get funding. This is somewhere between seed grants and being established enough to, for example, get governmental or large foundation funding. We have noticed this trend as well and even though we are not largely worried about longer term funding as we have interest from larger-scale funders, we expect they will take considerable time to evaluate and grant funding. Some funders have a specific number of years you have to be registered (often three) or a certain fairly large minimum budget before they accept applications. Filling this gap and allowing organisations to cross the bridge between being an early and more established organisation seems like a uniquely impactful way to donate for effective altruists, particularly those interested in health or policy.

If anyone is interested in supporting LEEP, here’s how you can help:

Feedback

We always value feedback and suggestions, especially from EA-minded thinkers and at this early stage. Please feel free to post your questions or comments below or reach out to us at [email protected].

Advisors

Specifically, we are currently looking to connect with experts involved in policy or advocacy in Zimbabwe, Botswana, or Mali who could provide guidance for our approach in these counties. We are also keen to speak to advisors who have done policy work in a remote context.

Funding 

Although we have funding to continue operations at a minimum scale thanks to generous donors in the EA community we think short-term unrestricted funding will be our main barrier to scaling in the next 24 months. We have an unfilled funding gap of ~$200,000 for Year 2. Filling half of this funding gap would allow us to conduct activities in three target countries, while filling the entire gap would allow us to scale-up to two further target countries. We would love to be in contact with any donors who might be interested in supporting LEEP. You can donate here or email us at [email protected]