Comments
That's amazing news! Thank you for sharing your insights, they are very valuable to us :)
Can Monitoring and Evaluation Drive Organizational Change? Lessons from The Mission Motor
3 min read · Mar 3, 2025
33
That's amazing news! Thank you for sharing your insights, they are very valuable to us :)
You're welcome. Let us know please if you have any questions :)
The Mission Motor supports animal advocacy organizations in developing and implementing Monitoring, Evaluation, and Learning (MEL) systems. From mid-2023 to 2024, we worked with 27 organizations, providing both short-term support (19 organizations, up to 3 months) and longer-term support (8 organizations, 4+ months).
Together with the organizations, we've completed 31 MEL deliverables, including theories of change, monitoring frameworks, and data collection tools.
We also support the broader movement through regular webinars and practitioner meetups.
Below we share our findings from our 18 months supporting animal advocacy organizations. The insights shared come from tracking organizational changes, conducting interviews and a survey, and analyzing our different support approaches.
We have found that organizations will make meaningful changes based on MEL insights when they have adequate support and tools. Among nine organizations we worked with extensively, we tracked 16 implemented changes to processes and programs and 13 changes in progress:
Organizations reported that The Mission Motor’s hands-on guidance in data collection and interpretation helped enable these changes. As one of the supported organizations mentioned “MEL has been something we’ve wanted to do for a long time. The tailored, personal support from The Mission Motor has been huge. Having an expert's input at the start saved us so much time and helped us avoid reinventing the wheel.”
The appreciation for this type of support was also reflected in a 9/10 average satisfaction rating. However, we acknowledge that social desirability bias will likely strongly influence this feedback.
We have also found a few barriers to reaching our impact. Below we listed some of the predominant concerns.
When we first started, we were concerned the biggest barrier may be attitude towards MEL and its value. In our initial year and a half, however, resistance to MEL support has not been a significant barrier to our work. We have generally found organizations open to the concepts of MEL. All of the five organizations interviewed already had positive attitudes toward evidence-based decision-making.
The real barriers groups cited were:
This insight has significant implications for our MEL support: focusing on simplification and efficiency may be more important than persuasion about MEL's value.
To note, there is a strong selection bias as we have likely worked predominantly with the ” low-hanging fruit” organizations. This is however in line with our strategy which prioritizes working more extensively with the most invested groups.
While organizations readily made changes with direct support, maintaining MEL practices independently proved challenging. We see some encouraging signs, with 2 of the 29 ongoing or completed changes being made without our prompting, but still this number suggests sustained MEL practice remains a significant challenge. We expect that the key elements to achieve long-term MEL practice are:
Over 2025, we will be investing in understanding these key challenges better. We'll be simplifying our MEL tools—starting with Theory of Change processes—to make them more accessible while maintaining their effectiveness. We're also exploring how AI can assist in streamlining our work, and are improving our internal MEL system to ensure consistent, high-quality support. Additionally, we're developing a measurement tool to better track organizations' MEL development.
Want to learn more about our work? Read our full 2023-2024 years in review.
Claude Sonnet 3.5 was used to produce the first draft of this post, which was subsequently edited by Thomas Billington and reviewed by Nicoll Peracha.
Thanks for the update, Nicoll and Tom!
Are you aiming to work with the most cost-effective animal welfare organisations? I think you would have to spend more to increase by 1 % their cost-effectiveness than that of a random organisation helping farmed animals. However, I believe the most cost-effective animal welfare organisations are way more cost-effective, such that you would have to spend less to achieve the same absolute increase in cost-effectiveness (which is the product between the initial cost-effectiveness, and relative increase in it you caused). I would say there is lots of variation in the cost-effectiveness of animal welfare organisations:
Hi Vasco, Great question we've been looking into for a while now. We indeed use cost-effectiveness as one factor to decide which organizations to support more intensively. We also look at other factors.
The research base (and practicing MEL) is not yet very well developed for animal interventions. Also, interventions that are cost-effective now, might not be in a few years from now if e.g. the context changes. Besides the evidence base not being robust (yet), it is also more difficult to assess the longer-term effects of interventions.
MEL can contribute to building an evidence base for interventions and to know when to pivot or scale. It is therefore important for The Mission Motor to not only support interventions that are assessed as being cost-effective and impactful now, but also to help collect data on existing, or novel interventions without a firm evidence base yet, that have the potential to be impactful.
What we've landed on is to use a set of proxies primarily for organizations we support longer-term. As said, (potential) cost-effectiveness is a factor, next to other factors such as the ability to grow in MEL capacity (can we still contribute?) and organizational characteristics such as learning attitude, capacity to work on MEL, and a certain stability.
Let me know if you have input or questions plse! We'll be evaluating this system, and probably updating it regularly.
Thanks for the context, Nicoll!
I very much agree with the 1st sentence above. On the other hand, I think the vast majority of animal welfare organisations lacks the potential to become 10 % as cost-effective as SWP. So I believe being highly selective about which organisations to work with would still be good.