EA-aligned animal advocacy is often criticized for measurability bias. In this post, I argue that the representation-based epistemic safeguards are unavailable in animal advocacy, and as a result, measurability is doubly important in our domain.
What do pro-measurement people say?
Intuition and abstract reasoning are very poor predictors of charity performance. Furthermore, charities differ 100x in effectiveness, therefore measuring the cost-effectiveness of charities enables us to do much more good.
We need careful scientific studies and preferably RCTs to figure out which interventions work. Through measurement, we will have a transparent accountability mechanism which will help us identify cases of failure and stop funding things that don't work.
What are the usual criticisms of measurability bias?
- Not all important outcomes are measurable. How do you quantify the value of a social movement or a shift in public discourse?
- External measurement misses out on the perspectives and lived experiences of the affected communities.
- Academic credentials sought after by measurability-focused people are deeply entrenched in unjust power structures and therefore biased. Oppressed groups don't have enough power to get a favorable paper published out of Harvard University. The politics of research in prestigious institutions pollutes the reliability of academic research.
- Measurement often creates perverse incentives, leading organizations to focus on what's easily counted rather than what creates meaningful change.
What are representation-based alternatives to measurable outcomes?
For people who dislike the focus on measurable outcomes, there are alternative epistemic safeguards like standpoint epistemology. Proponents of this approach claim that oppressed groups know best what's in their interests. So you're just supposed to find out which groups in society are powerless, and transfer them power.
As long as you ensure that the organizations you fund are led by members of the beneficiary groups and incorporate democratic mechanisms, you are in safe territory. People know what's good for themselves. So it's fine even if you don't actively measure all the outcomes of the organisations you support, because you have other epistemic safeguards.
You can't do standpoint epistemology with non-human animals
The challenge for animal advocacy is that animals cannot politically organize or participate in democratic decision-making. The representation-based epistemic safeguards that work for human-centered movements are simply unavailable when advocating for non-humans. You cannot have organizations led by chickens, nor can chickens participate directly in strategy meetings or provide feedback on campaigns.
Some advocates bemoan how "nonprofit industrial complex" becomes a self-serving machine. I take this concern very seriously, and that's why I place so much emphasis on measurable outcomes. Without measurable outcomes, how do we know when someone failed? All we're left with is people forming opaque opinions about other people. If we want to prevent people bullshitting and networking their way into getting funded, we need stronger accountability mechanisms than that.
Balancing measurement with movement building
What I agree with in the critiques is that we need valid, reliable and generally useful scales on pro-animal beliefs and attitudes. It's helpful to go beyond focusing on "number of animals affected" to build a powerful movement and track long-term social change.
While immediate welfare improvements can and should be measured, we also need tools to track the growth of moral concern for animals and the development of institutional capacity that will benefit animals in the long term.
