Hide table of contents

The Mission Motor supports animal advocacy organizations in developing and implementing Monitoring, Evaluation, and Learning (MEL) systems. From mid-2023 to 2024, we worked with 27 organizations, providing both short-term support (19 organizations, up to 3 months) and longer-term support (8 organizations, 4+ months).
Together with the organizations, we've completed 31 MEL deliverables, including theories of change, monitoring frameworks, and data collection tools. 

We also support the broader movement through regular webinars and practitioner meetups.

Below we share our findings from our 18 months supporting animal advocacy organizations. The insights shared come from tracking organizational changes, conducting interviews and a survey, and analyzing our different support approaches.

MEL Can Drive Substantial Program Changes - 
With the Right Support

We have found that organizations will make meaningful changes based on MEL insights when they have adequate support and tools. Among nine organizations we worked with extensively, we tracked 16 implemented changes to processes and programs and 13 changes in progress:

  • One group completely redesigned their meat reduction program after data showed their target audience was unlikely to change. This led to focusing on more receptive demographics.
  • Another organization shifted significant resources to outreach after their theory of change analysis revealed it was a neglected but crucial component of their strategy.
  • A third group eliminated leafleting from their program after reviewing evidence of its effectiveness.

Organizations reported that The Mission Motor’s hands-on guidance in data collection and interpretation helped enable these changes. As one of the supported organizations mentioned “MEL has been something we’ve wanted to do for a long time. The tailored, personal support from The Mission Motor has been huge. Having an expert's input at the start saved us so much time and helped us avoid reinventing the wheel.

The appreciation for this type of support was also reflected in a 9/10 average satisfaction rating. However, we acknowledge that social desirability bias will likely strongly influence this feedback.

Key Challenges

We have also found a few barriers to reaching our impact. Below we listed some of the predominant concerns. 

1 - MEL is Perceived as Complex and Time-Consuming

When we first started, we were concerned the biggest barrier may be attitude towards MEL and its value. In our initial year and a half, however, resistance to MEL support has not been a significant barrier to our work. We have generally found organizations open to the concepts of MEL. All of the five organizations interviewed already had positive attitudes toward evidence-based decision-making. 

The real barriers groups cited were:

  1. Perceived complexity of MEL tools: 4/5 organizations interviewed found MEL challenging due to its complexity and resource demands.
  2. Resource constraints, especially staff time: 2/5 organizations noted they were unable to implement lessons due to timing constraints within their organizations.

This insight has significant implications for our MEL support: focusing on simplification and efficiency may be more important than persuasion about MEL's value.

To note, there is a strong selection bias as we have likely worked predominantly with the ” low-hanging fruit” organizations. This is however in line with our strategy which prioritizes working more extensively with the most invested groups.

2- Sustained Impact Will Require Systematic Integration

While organizations readily made changes with direct support, maintaining MEL practices independently proved challenging. We see some encouraging signs, with 2 of the 29 ongoing or completed changes being made without our prompting, but still this number suggests sustained MEL practice remains a significant challenge. We expect that the key elements to achieve long-term MEL practice are:

  • MEL components integrated into regular operations
  • Clear ownership of MEL responsibilities within the team
  • Leadership actively supporting evidence-based decision-making

Looking Forward

Over 2025, we will be investing in understanding these key challenges better. We'll be simplifying our MEL tools—starting with Theory of Change processes—to make them more accessible while maintaining their effectiveness. We're also exploring how AI can assist in streamlining our work, and are improving our internal MEL system to ensure consistent, high-quality support. Additionally, we're developing a measurement tool to better track organizations' MEL development.

Want to learn more about our work? Read our full 2023-2024 years in review.

Want to Learn More About MEL?

  • Get in touch to discuss MEL support for your animal advocacy organization.
  • Follow us on LinkedIn to join our MEL webinars.  The topics covered are generally relevant for all types of organizations, also outside the animal space, and open to all.

Claude Sonnet 3.5 was used to produce the first draft of this post, which was subsequently edited by Thomas Billington and reviewed by Nicoll Peracha.

Comments5


Sorted by Click to highlight new comments since:

That's amazing news! Thank you for sharing your insights, they are very valuable to us :)

You're welcome. Let us know please if you have any questions :)

Thanks for the update, Nicoll and Tom!

From mid-2023 to 2024, we worked with 27 organizations, providing both short-term support (19 organizations, up to 3 months) and longer-term support (8 organizations, 4+ months).

Are you aiming to work with the most cost-effective animal welfare organisations? I think you would have to spend more to increase by 1 % their cost-effectiveness than that of a random organisation helping farmed animals. However, I believe the most cost-effective animal welfare organisations are way more cost-effective, such that you would have to spend less to achieve the same absolute increase in cost-effectiveness (which is the product between the initial cost-effectiveness, and relative increase in it you caused). I would say there is lots of variation in the cost-effectiveness of animal welfare organisations:
 

  • I estimated the Shrimp Welfare Project (SWP) has been 412 and 173 times as cost-effective as broiler welfare and cage-free campaigns.
  • I estimated Veganuary in 2024 and School Plates in 2023 were 1.20 % and 19.4 % as cost-effective as cage-free campaigns.
  • I estimated the Fish Welfare Initiative's (FWI's) farm program from January to September 2024 was 1.55 % as cost-effective as cage-free campaigns.
  • I estimated Sinergia Animal's meal replacement program in 2023 was 0.107 % as cost-effective as their cage-free campaigns.

Hi Vasco, Great question we've been looking into for a while now. We indeed use cost-effectiveness as one factor to decide which organizations to support more intensively. We also look at other factors.

The research base (and practicing MEL) is not yet very well developed for animal interventions. Also, interventions that are cost-effective now, might not be in a few years from now if e.g. the context changes. Besides the evidence base not being robust (yet), it is also more difficult to assess the longer-term effects of interventions.

MEL can contribute to building an evidence base for interventions and to know when to pivot or scale. It is therefore important for The Mission Motor to not only support interventions that are assessed as being cost-effective and impactful now, but also to help collect data on existing, or novel interventions without a firm evidence base yet, that have the potential to be impactful.

What we've landed on is to use a set of proxies primarily for organizations we support longer-term. As said, (potential) cost-effectiveness is a factor, next to other factors such as the ability to grow in MEL capacity (can we still contribute?) and organizational characteristics such as learning attitude, capacity to work on MEL, and a certain stability.

Let me know if you have input or questions plse! We'll be evaluating this system, and probably updating it regularly.

Thanks for the context, Nicoll!

MEL can contribute to building an evidence base for interventions and to know when to pivot or scale. It is therefore important for The Mission Motor to not only support interventions that are assessed as being cost-effective and impactful now, but also to help collect data on existing, or novel interventions without a firm evidence base yet, that have the potential to be impactful.

I very much agree with the 1st sentence above. On the other hand, I think the vast majority of animal welfare organisations lacks the potential to become 10 % as cost-effective as SWP. So I believe being highly selective about which organisations to work with would still be good.

Curated and popular this week
 ·  · 7m read
 · 
Recently, @Lizka and @Ben_West🔸 published A shallow review of what transformative AI means for animal welfare. The main conclusion of this review was that animal welfare interventions should be heavily temporally discounted due to the possibility of transformative AI on short timelines. A reaction I had when reading this piece was that things tend to happen very slowly in animal agriculture, and even big wins like a corporate welfare commitment can take years before a specific animal is concretely better off. I therefore thought it might be useful to look at some of the main animal welfare interventions and assess how quickly they can help animals in the best case scenario.  A conclusion from this analysis is that animal interventions vary significantly from how quickly they start to have impact, with some interventions having impact almost immediately, some having predictable impact within some period of time (which can be up to a few years), and some having impact at some unspecified point in the future. Optimizing for speed to impact might be a new kind of frame under which animal advocates can evaluate and prioritize interventions.  These are just some preliminary thoughts that I wanted to get out there in the spirit of "shallow reviews" (also given the analysis itself is about the importance of speed). I'd welcome additional thoughts / feedback / pushback. Lowering / shifting meat demand Many animal welfare interventions achieve impact through lowering demand for animal products. In this category, I include things like starting a plant-based meat company, or doing vegan advocacy.  While it seems clearly possible to have wins in this area extremely quickly, there will often be a delay on having impact because of the structure of the supply chain for animal products.  For the simplest example, a beef cow in the US is generally 18-22 months old when they are slaughtered. This means that, to some extent, the beef supply in the US for the next 18-22 months h
 ·  · 6m read
 · 
Note: This post was crossposted from the Open Philanthropy Farm Animal Welfare Research Newsletter by the Forum team, with the author's permission. The author may not see or respond to comments on this post. ---------------------------------------- > How I decided what to say — and what not to I’m excited to share my TED talk. Here I want to share the story of how the talk came to be, and the three biggest decisions I struggled with in drafting it. The backstory Last fall, I posted on X about Trump’s new Secretary of Agriculture, Brooke Rollins, vowing to undo state bans on the sale of pork from crated pigs. I included an image of a pig in a crate. Liv Boeree, a poker champion and past TED speaker, saw that post and was haunted by it. She told me that she couldn’t get the image of the crated pig out of her head. She resolved that if she won prize money at her next poker tournament she’d give 20% of it away to help factory-farmed animals. She won $2.8 million. And she not only donated 20% of it, she also started posting to her many followers about factory farming, invited me on her podcast … and then invited me to speak at TED. (She was a guest curator at this year’s conference.) This was a huge opportunity. I don’t think the main TED stage has ever had a talk solely about factory farming before. (TED’s head Chris Anderson told me later that he regretted that TED hadn’t tackled the topic until now.) So I really didn’t want to mess it up. I knew what I wanted to convey: the moral urgency that we address factory farming. But I didn’t know how best to convey it. In particular, I struggled with three questions: how to talk about a moral atrocity, what my big idea would be, and what to ask for. Everyone likes an origin story. Thankfully my parents still had this stereotypically-New Zealand photo of me on our small sheep farm growing up. I was always more into the picnics than the farm work. Photo: Gilberto Tadday / TED. How to talk about a moral atrocity? Perha
 ·  · 50m read
 · 
In 2021, a circle of researchers left OpenAI, after a bitter dispute with their executives. They started a competing company, Anthropic, stating that they wanted to put safety first. The safety community responded with broad support. Thought leaders recommended engineers to apply, and allied billionaires invested.[1] Anthropic’s focus has shifted – from internal-only research and cautious demos of model safety and capabilities, toward commercialising models for Amazon and the military. Despite the shift, 80,000 Hours continues to recommend talented engineers to join Anthropic.[2] On the LessWrong forum, many authors continue to support safety work at Anthropic, but I also see side-conversations where people raise concerns about premature model releases and policy overreaches. So, a bunch of seemingly conflicting opinions about work by different Anthropic staff, and no overview. But the bigger problem is that we are not evaluating Anthropic on its original justification for existence. Did early researchers put safety first? And did their work set the right example to follow, raising the prospect of a ‘race to the top’? If yes, we should keep supporting Anthropic. Unfortunately, I argue, it’s a strong no.  From the get-go, these researchers acted in effect as moderate accelerationists. They picked courses of action that significantly sped up and/or locked in AI developments, while offering flawed rationales of improving safety.   Some limitations of this post: * I was motivated to write because I’m concerned about how contributions by safety folks to AGI labs have accelerated development, and want this to be discussed more. Anthropic staff already make cogent cases on the forum for how their work would improve safety. What is needed is a clear countercase. This is not a balanced analysis. * I skip many nuances. The conclusion seems roughly right though, because of overdetermination. Two courses of action – scaling GPT rapidly under a safety guise, starting a ‘