Wrapping up on our first operational year, we thought it would be beneficial to share an update on our work and projected impact so far. Following a stakeholder evaluation earlier in the year and completing the internal monitoring and evaluation of our pilot programme, we think there are also some key lessons that could be useful to members of the effective animal advocacy and effective altruism communities.
For anyone unfamiliar with our work, we encourage you to read our introductory post. More information on our organisation can also be found on our website. In summary, Animal Ask is a non-profit that supports existing animal advocacy organisations with research and consultancy. We help them prioritise between different legislative or corporate campaigns through in-depth, systematic research of a wide variety of potential campaigns, assessing their impact, to ensure that advocacy resources are spent on the highest priority opportunities.
We have spent our first year completing a pilot programme aimed at testing both our research process and organisational model. This pilot provided three organisations with extensive research towards their upcoming campaigns. Specifically, we supported Sinergia Animal (an international animal protection organisation working in countries in the Global South), Advocates for Animals (the UK's first law firm dedicated to animal protection law), and Sentience Politics (an anti-speciesist political think tank based in Switzerland with the goal of reducing the suffering of all sentient beings).
Alongside this, we provided assistance to multiple groups through our Ask Consultancy programme, as well as completing foundational research into different methods of evaluating animal welfare.
Animal Ask’s First Year Summary
These are top line summaries of activities from our launch in September 2020 to the present day.
Ask Prioritisation Research
- 144 possible interventions, or asks, were taken through our extensive research rounds across the three projects, resulting in eight extensive reports.
- We have modelled the expected impact of these three collaborations by monitoring and evaluating updates in the decision making of the partner organisations, our predictions of likely counterfactuals without our involvement, and the expected value of the campaigns we recommended. Given the update in decision making we have observed, we estimate that our collaboration increased the expected value of their campaigns by ~30% - Advocates for Animals, ~50% - Sinergia Animal, and ~70% Sentience Politics.
- In expectation, between the three projects this should result in ~2.3 million equivalent animal lives saved and 10s of millions affected. However, we do approach these projections with skepticism as we continue to evaluate the value of our work, and with the impact of these campaigns yet to be determined.
- All three groups indicated that they are actively using our research to inform their ask decision-making. Updates on this will be released when they have finalised their strategic directions and we can more accurately report the difference in value of the campaign due to our involvement. Sentience Politics has already taken one of the interventions we suggested forward, an intervention their post-collaboration interview suggests they would not have considered independently.
We are happy to share our models with those who may be interested in further detail. We will consider requests on a case by case basis due to the sensitive nature of the data.
Ask Consultation Research
- Three extensive reports for the Conservative Animal Welfare Foundation; ‘Welfare at the time of killing’, ‘Ending cages for laying hens’, and ‘Non-stun slaughter’. All three reports were presented to the UK government and are contributing towards potential policy changes within their respective areas.
- We partnered with the Aquatic Life Institute, completing research to form the foundations of their Benchmark Report, due to be publicly released in the coming months.
- We are currently working with Aaron Ross from The Humane League on establishing a robust framework for new groups and groups looking to expand to a new area of farmed animal advocacy. This published framework will reduce the duplication of efforts and assist organisations across the globe with their ask prioritisation.
Foundational Research
We published the following research paper: ‘Measuring Animal Welfare’. This includes a brief discussion of the philosophical foundations of welfare and theories of well-being, an examination of the empirical research into the reliability and validity of indicators of animal welfare, and a review of existing overall welfare assessments.
Organisation Development
Thanks to grants from both Animal Charity Evaluators and the EA Animal Welfare Fund, we hired a Research Scientist and a Research Analyst earlier this month. Both new team members are now actively contributing to both the Ask Prioritisation and Ask Consultation services.
Six lessons learned from our first year
We have learned a great deal in our first 12 months of operation. From experimenting with how to successfully conduct research in this area to attempting to monitor and evaluate our efforts, we hope that the top takeaways below are useful. As always, if you have any thoughts or feedback, please reach out to us - we would love to hear from you.
1. Monitoring and evaluation for meta-orgs is even more difficult than expected
A continuous area of learning and improvement for our organisation over the past year has been monitoring and evaluating our impact. As with all meta-orgs, our theory of change depends on the actions of other organisations or individuals. Thus, tracking our impact relies on our ability to predict and monitor the decision-making and behaviour of the organisations we support. Currently, we track this by assessing the organisation's priorities before and after our collaboration using surveys and interviews. However, even in this near best-case scenario for assessing the impact of research organisations, it’s still difficult to know the impact of our work with a high degree of certainty.
The main area of uncertainty is in assessing the counterfactuals of the groups we work with. Most of the groups we have worked with had no strong initial leaning towards the asks they would likely work on prior to our collaboration. Therefore, even though they have updated based on our research, we are unsure what the counterfactual would have been without our involvement. Without our support, the organisations may have spent some time doing their own more resource-constrained research and spoken to advisors in the field. This could have led them to a wide variety of asks that we researched or possibly another idea that we missed. As we are unable to observe this scenario, we are left to estimate possible outcomes depending on the asks they showed the most initial interest in, and our estimates of their relative value.
The main generalisable learning we have made in this area, other than a high level of skepticism of the impact of meta organisations, is that survey feedback alone from groups or individuals is not enough to track impact. We currently ask the groups how useful they have found the research for their decision-making. So far, all groups have reported that they have found the reports useful. However, we have found that this doesn't track how likely it is that the report has affected the asks they plan to work on. This suggests that this sort of feedback is better used as a measure of satisfaction with the quality of work, rather than as a proxy for decision-making change, and thus impact.
As a more direct measure, we are experimenting with directly asking groups to forecast how likely they would have been to select an equally impactful ask without our involvement. However, we can see ways in which this question could be biased or inherently difficult to answer as well.
For other groups or individuals who are considering working with or within meta organisations, we encourage the following:
- Establish clear falsification criteria and fully consider what evidence would look like for each stage of your theory of change
- Consider the biases or challenges with methods you could use to assess these, as individuals might attribute actions they would have done anyway to your involvement.
2. Targeted research is often applicable to a much wider audience
When evaluating the impact of our early collaborations, we recognised that the scope of the research could appeal (and to a certain extent should be intended to be useful) to a much wider audience than just the organisation we are collaborating with. We want our resources to have the greatest impact possible and due to this, assessing a wider intended audience has informed and reframed our research process.
In our most recent partnership with Animal Equality (October 2021), we have decided to run two tracks of research. During the prioritisation round of research,, we previously discarded asks that were potentially high-impact but did not fit the partner organisations’ criteria . Moving forward, we have decided to continue to pursue these potentially high-impact opportunities even if these asks fall outside of the partner’s scope. Due to this, we expect to conclude our findings with:
- Asks that are high-impact and fall specifically within the scope of Animal Equality’s organisational framing and vision.
- A secondary list of asks that performed particularly well through the process that, while not a good fit for Animal Equality, are in their own right high-impact opportunities.
It is our intention to present these high-impact opportunities to other organisations, including those outside of the animal advocacy space, as potentially high impact interventions, potentially doubling the impact we can have from our overall resource.
3. Experts in this area of work are incredibly generous with their time and resources
Across our three collaborations, we interviewed 38 experts who all offered their time and expertise free of charge. During the research rounds for our Ask Prioritisation we reach out to academics, animal welfare specialists, intervention specialists, industry leaders, and policy makers in the relevant fields. We have been delighted with the responses from these generous individuals. Often, they have given us more than the requested hour of their time to talk to us about the asks we are considering and have followed up with further reading on the relevant topics. As the Lead Exposure Elimination Project discussed in their post ‘Seven things that surprised us in our first year working in policy’, this collaborative approach has enabled us to increase the impact of our work to date. We echo their hope that this can be encouraging to others considering starting a new project or organisation.
4. The scope and interest for largely unexplored asks remains …unexplored
During the Ask Prioritisation programme, the first research round begins with the creation of a list of all possible interventions. This includes any that are within reason and within the scope and country or species focus of our partner organisation. We aim for this list to be as extensive as possible and spend time researching less popular interventions to see how they fair through our process. Through all three of the pilot projects, we identified potentially high impact asks that look very promising with strong second order effects, but that are fairly niche. We found groups to be generally hesitant to pursue asks of this nature. This was generally due to lack of alignment with the organisation’s positioning or due to groups feeling concerned that a less public-friendly theory of change may be more difficult to communicate to their supporters and stakeholders. Examples of this include insect asks and asks that include animals towards the edge of the moral circle, both areas groups were hesitant to work within.
Due to this apprehension, there are multiple impactful asks that remain unexplored, despite their probable high-impact based on sheer animal lives, tractability, or ease of implementation. This has also highlighted the need for more exploratory research into new or niche ask ideas that lie outside of groups’ typical expertise and focus. In the coming months we will be devising and publishing a list of these potentially high impact, niche opportunities and reaching out to organisations directly to actively pitch these largely unexplored areas.
5. There are many important foundational questions in animal advocacy that are still left to the intuition of decision makers
During our extensive research process, many overarching questions arise that could affect the broader decision-making between different asks. These foundational questions remain unanswered by the animal advocacy movement as a whole. Without specific research into these questions, this has left important decisions to be largely informed by the intuition of a handful of organisation decision makersOne specific example that arose during our partnership with Sinergia Animal, whilst exploring corporate asks for fish, was the question around how much to feasibly ask for. Previous examples of two approaches come from the cage-free ask and the broiler ask (European Chicken Commitment and Better Chicken Commitment). The cage-free ask targets one specific improvement that is both important for the welfare of the animal and, crucially, easily communicated: taking hens out of cages. The broiler ask is a composite ask made up of multiple elements that are important for the welfare of the chicken but seemingly more difficult to communicate due to the nature of the more complex composite. Disregarding their different contexts, the following question arises: is it more impactful to campaign for one important tractable improvement or multiple different elements simultaneously? Even with the already more extensive broiler ask, additional elements could have been added. What is the cut-off before the ask begins to lose value from a loss of traction due to its complexity? Currently, the question of accurately trading off the addition of a new element against the overall tractability of the ask remains unclear.
We plan to spend time researching foundational questions like this in order to better understand the trade-offs involved in corporate asks of this nature. Similarly to the niche ask list, we will be circulating a list of fundamental foundational questions for feedback and completing this research periodically between Ask Prioritisation projects. If any other organisations or individual researchers are interested in or are currently looking into similar questions, we would be excited to hear from you.
6. There is less room for prioritisation within legal advocacy
We were initially optimistic about the value of prioritisation research in many different domains, including legal advocacy. However, after evaluating our partnership with Advocates for Animals where we provided research into possible judicial reviews, we found that we provided much less value to the organisation than during our other projects. There are two reasons for this.
Firstly, the barrier to entry for testing an ask is relatively low within this context. Unlike some other forms of advocacy, where research can more confidently identify what asks seem tractable, confidently assessing the strength of a legal case can only be accomplished by domain experts. Even then, accurately forecasting which cases have a sufficiently high probability of success to have good expected value is something many of these experts will be unable to do. Fortunately, the cost of launching the initial application for a case is relatively low, making this a viable method for testing the strength of different cases. This makes the initial application a more efficient method for gathering information than additional research. Therefore, a more impactful strategy could be to outline a range of possible cases and launch applications near simultaneously, barring concerns with having multiple simultaneous cases.
The second reason that prioritisation research was of lower value in this area was the restriction in the range of potential cases that could be brought to court. Cases can only be brought against entities that are being accused of breaching existing laws, making the number of potential asks much smaller than for other advocacy methods. In addition, generating ideas for cases depends on having detailed knowledge of existing laws and an awareness of current practices. This makes longer term exploration of opportunities within a specific jurisdiction alongside existing cases a much better strategy than short-term idea generation.
Given these two challenges, we are more skeptical of assisting groups in prioritising between legal cases in future. However, we will still offer consultation support, such as research on the number of different animals in their country, and the trade-offs or second order effects that could occur if the case is successful (e.g tail biting if tail docking rates are reduced) as these are areas where we have research experience and expertise.
Animal Ask’s Asks
Fundraising
In our first year of operation we were funded solely by a seed grant from Charity Entrepreneurship. As we look to continue our impact-driven support service for the movement, we currently have an unfilled funding gap of ~$135,000 within our two-year budgeting plan. This could increase to ~$400,000 if our pending and predicted funding opportunities are not successful. Filling this funding gap would allow us not only to continue our work for the rest of 2021/22, but to scale-up by hiring or finding a consultant research analyst with more legal experience to the team in 2022.
With your donation, we can continue to increase the value of campaigns within the farmed animal welfare space by ensuring that interventions are robustly researched, resulting in organisations putting their precious and limited resources towards the most impactful opportunities for animals.
We would love any donors interested in supporting our work to get in touch at [email protected].
Alternatively, if you wish to contribute to our Giving Tuesday fundraiser with the opportunity to get your donation doubled, please follow this link for further information.
Work with us
If any groups are interested in working with us and having access to this additional research resource through either our Ask Prioritisation or Ask Consultation programmes, please get in touch - we would love to hear from you.
Volunteering
In the coming weeks we will be advertising for a volunteer research coordinator and a volunteer communications coordinator. If you feel you could have the experience and time to support our efforts in this low-time commitment, high-impact opportunity, please keep an eye on our website and consider joining our mailing list.
Hey Animal Ask team and Amy, thanks a lot for the blog, really insightful!
I thought of a super time-consuming way of evaluating impact: have the team do their own research and then do the same research yourself and compare results. Maybe if you do this on a relatively small project, you will see the objective difference in results. Of course, you still have counterfactuals to deal with as you will eventually go with only a couple of intervention choices.
Also in my own consultancy work, I've found that it takes years sometimes to track results, so I guess it will be the case with you too. Looking forward to reading your post about progress in a year's time!
Thanks so much for your important work for the animals and I hope to collaborate on something soon!
Hey Sofia, Great idea. Groups have usually indicated they would spend <10% of the time we spend researching without our involvement, so this seems like a more viable idea than one may expect. There are some reasons this may not entirely cross-apply to the rest of our work. Such as concerns with groups anchoring too much to their more shallow research, which usually results in more optimistic assessments (Optimizer's Curse). Or possibly a selection effect with the groups that are willing to do this being more likely to make better decisions. We are tracking the asks other similar organisations are using in the regions or areas we have worked in to. This gives us some sense of this, but a more direct experiment of this kind could be valuable. Particularly if we ran it with a few groups using different advocacy methods. We will look into the idea more as well as some of the other ways we could amend our pre/post surveys before we partner with the next group!