animal welfare has an evidence problem

Hi Amanda, thank you for your thoughtful analysis. I do believe taking a step back and  scrutinizing the evidence and direction where we're heading is extremely important, and I agree with your  conclusion that increasing R&D and talent is very much needed. I also agree that the evidence gaps are enormous. As a research group working specifically to address them, we're very much aligned. In that same spirit, I share some considerations below on the cage-free transition example, into topics to make the text lighter. 

The problem with the CSES study as a reference for mortality / welfare conclusions.

The CSES study is a highly cited reference for the argument that cage-free aviaries are not necessarily better. It was funded by the American Egg Board and facilitated by another industry-funded organization focused on building consumer confidence and maintaining the industry's viability. Unfortunately, this study had multiple design flaws and biases in favor of cages (a more detailed analysis, done some years ago, is available here). A few examples:

• Several observations point to stockmanship being ‘very deficient’ in the aviary (the farmer had never managed one), but not in cages, with which the farmer had decades of experience. Inspection frequency also seems to have been higher in cages.The research authors themselves declared they were still learning about what to do in the aviary during the research, which led to multiple failures. In fact, inspection was so infrequent/deficient that a large fraction of dead birds were found in advanced stages of decomposition.
• Vet/disease treatment was employed for cages, but not aviaries
• The overwhelming majority (~90%) of good practices for cage-free aviaries were *not* adopted (e.g., aviary chicks were raised in cages, with no development of skills to perch or navigate the aviary, densities were higher than typical, laying birds in aviaries were confined for hours every day, increasing the risk of feather pecking, breeds adapted for decades to cages were used in aviaries, chicks did not have access to litter, among many others poor management decisions - see a summary table in this document)
• Key practices to reduce the risk of feather pecking / cannibalism in aviaries were not used (in fact, CSES management choices are known to promote these issues).
• CSES mortality data does not add up, there are multiple inconsistencies in the dataset and in the mode of data collection/reporting, discussed here.

Mortality Data

Farm animal welfare in natural and more extensive systems, including cage-free aviaries, depends more heavily on good management practices and stockmanship. As such, we should expect greater variability in terms of mortality, as well as greater absolute mortality in the first production cycles following a transition. For this reason, the fact that there is a greater number of orange cells (mortality higher in aviaries) in the datasets of the meta-analysis is expected, as most are comparisons of established caged systems with newer cage-free systems. The studies span  two decades during which cage-free systems underwent major changes. This is why we explicitly modeled the year of data collection as a predictor. Mortality rates are changing systematically over time. Modeling that trend directly, and then reporting recent mortality separately, is necessary for relevance. Another point is that brown-feathered genotypes (associated with higher mortality) are more common in cage-free systems, naturally increasing mortality because of breed, not system.

The data below, from an internal database from a  breeder for countries around the world (in 2018), can also be useful.

Mortality as an indicator of welfare

Mortality may or may not correlate well with welfare. It is widely used because it is easy to measure, routinely collected, and economically important. However, mortality captures whether animals survive, not what their lives are like while they are alive.
The industry is often very good at keeping animals alive and productive until the end of the production cycle, even under conditions associated with extremely poor health and welfare (what we refer to as the "hospital bed effect"). Conversely, some more extensive or naturalistic systems may have higher mortality because animals are exposed to more hazards. Healthy pasture-raised animals, for example, may experience higher mortality due to predation. There may often be a trade-off between behavioral freedom and protection from mortality risks (‘children who never play outdoors are less prone to injuries and fatal accidents’, yet few would argue this is better for their well-being).
Mortality is most informative when comparing otherwise similar systems. When comparing systems that differ fundamentally in housing and behavioral opportunities, mortality should be interpreted together with other welfare metrics.

Life Quality/Well-being in cages x cage-free

The 2021 WF analysis was very conservative (i.e. favored caged systems) in a number of ways as we discussed in the book and elsewhere. For example, we considered prevalence for ailments in cage-free systems as reported, without any adjustments for improvements over time (despite evidence that the frequency of various harms  was going down, similar to mortality).

Also, we did not consider positive welfare (opportunities are naturally more frequent in cages) nor the longer lifecycle of caged hens (welfare is typically worse at the end of life), and end-of-life events such as induced molting, still practiced in many countries in caged systems.

We also did not consider the negative impacts of learned helplessness, lack of agency, and depression-like states in cages - there is now new evidence for such depressive states in caged hens.

Importantly, we did not make any adjustment for what I believe to be robust evidence that the pain from an injury or disease is perceived as more intense and longer in cages than in cage-free systems. Barren, confined environments disable multiple endogenous analgesic mechanisms  while simultaneously activating several neurobiological pathways that intensity nociceptive signaling and delay healing. Should that be taken into account, it would further reduce time and intensity of pain in cage-free aviaries.

Fear in Cages vs Cage-free systems

Several studies have found that hens reared or housed in cage-free systems are less fearful than hens kept in conventional cages. Aviary-reared birds show reduced fear responses in tonic immobility, novel object, and novel environment tests, spend more time near humans and novel objects, use elevated areas more readily, and perform better in spatial memory tasks than cage-reared birds (Hansen et al., 1993; Tahamtani et al., 2015; Brantsaeter et al., 2016). They suggest that the more complex environments in cage-free systems may reduce fearfulness and improve behavioural adaptability.  See box 9.2 of the laying book for more details.

Behavioural and Physiological Indicators as a standard for welfare-related decisions

Welfare is multidimensional, and cumulative experience matters. So unfortunately no single behavioural or physiological measure, or restricted set of measures, can provide a complete picture of welfare, nor even for humans (for which calibration is possible). Behavioral, immunological, neurological, and physiological measures are valuable for inferring states associated with specific experiences, often at specific points in time. However, they are insufficient for overall welfare assessments, as well as confounded by multiple factors, and more reliable for acute rather than chronic harms (particularly immunology and physiology). Because they are typically species-, harm- and context-specific, they also do not enable comparisons across harms, systems and species. Several attempts have been made in recent years to design an umbrella measure of welfare (e.g., telomere length, cognitive bias tests), but so far unsuccessful. That's not to say indicators are not useful: we rely heavily on them for our work, and believe having more monitoring systems and research would be extremely needed. But for overall welfare assessments and system comparisons as in the case of cage-free transitions, we need welfare metrics, which integrate evidence from multiple welfare dimensions, providing a stronger basis to infer cumulative experience. In case it's useful, here we discuss in more detail the differences between welfare metrics and welfare indicators.

Thank you again for this critical analysis! 
Cynthia
 

Thanks for writing this post! Thanks too for sharing it in advance, and sorry I didn't have time to review it. For the same reason, I won't have time now to go into the detail that I'd love to on your specific claims. (It's great to see such a vibrant discussion in the comments! I'd love to see more critical discussion of the evidence like this in the EA animal welfare space.) So I just wanted to share three high-level reflections.

First, I agree with you on the need for more evidence collection on the animal welfare outcomes of popular interventions. We've funded >$20M of such evidence collection work (e.g. the Welfare Footprint Institute, Rethink Priority's review of evidence across multiple areas, the shrimp slaughter study you cite, a Guelph study on broiler genetics welfare outcomes, and a dozen other studies and meta-analyses). We've also done a ton of internal analysis on this, e.g. we have a complicated model of estimated rates of factory farm meat displaced by alt protein in various scenarios. And yet we need to do a lot more! I'm really excited that the likely inflow of more animal welfare funding will enable a lot more high-quality evidence generation. And I completely agree we should prioritize it.

Second, I think we need to be realistic about how much certainty that evidence collection will create -- and how quickly. I think your cage-free example is a good one. There have been 50+ studies and large-scale data collection efforts on relative mortality in caged vs. cage-free housing systems and we still don't have a clear answer on it. And yet mortality is just one of 20+ metrics we could look to as partial proxies for what we really care about: "what is the relative overall welfare of caged vs. cage-free birds?" I think we should also fund studies and commercial data collection on all those other metrics (or at least the ones current tech allows us to measure). But that will take many years and, even then, people will reasonably debate the relevant importance of each of those 20+ metrics to the welfare of birds. 

(I should note that I really admire the work you're doing to try to significantly accelerate the pace of this kind of research, starting with shrimp slaughter. I think I agree with you on everything you think should be funded to accelerate it further, and am confident we will fund that full agenda -- please tell me if there's something we're not covering. And I hope that advances in AI can accelerate the evaluation and syntheses of existing evidence too.)

Third, I think we need to continue to act to alleviate suffering even in the face of significant uncertainty. I'm jealous of global health direct service provision (e.g. anti-malarial bednets, cash payments) where I think you can have perhaps the closest to total certainty of positive impact of any areas. But I think for basically any advocacy intervention in EA (whether in AI safety, bio, farm animal welfare, or even global aid advocacy) you have to accept significant uncertainty not just about the immediate impacts but also about second, third, etc. order effects. That's not an excuse for ignoring the uncertainty. But I think when a cluster of evidence points to a high likelihood that something is robustly net positive -- as I believe it does for cage-free (sorry I don't have time to go into all the specifics here!) -- than we should move forward on it. I think the opportunity costs of waiting for a higher confidence level of evidence are too high.

Thanks again for writing this! One of the things I love most about EA is the application of critical thinking and evidence to disrupt commonly accepted wisdom, and I think this is a good example of this. I disagree on some of the specifics -- wish I had the time to explain more -- but I'm really glad you wrote and published this.

However correct or compelling these critiques turn out to be,^[1] I want to praise the incredibly constructive framing here. Most 'criticism' on the Forum does not evince 1) the conviction that the underlying problem is still extremely important, 2) the resolve to continue making progress on it, and 3) the trust that actors will update on evidence/arguments. Wonderful.  

1. ^{^}

   FWIW, my summary of the discourse is:

   The evidence on shrimp stunning does seem terrifyingly thin

   The cage-free controversy is comparatively well studied (but still too little), and very sensitive to assumptions about inputs to quality of life

   Meat alternative uptake is a known issue. The question is how long we should keep trying things before deciding the theory of change is fundamentally flawed (much longer, imo!) 

Hi matthes,

This is an extremely interesting post to read, thanks for putting the time and effort into it.

I'm new to the EA forum, one of our PhD students mentioned it to me and I'm so glad she did. I'm Jill, I'm an animal welfare researcher at the University of Edinburgh and I'm a specialist in research methodology. To get my cards on the table, I'm a huge proponent of open scholarship approaches, and advocate of greater adoption of open scholarship within animal welfare science. This paper might be a good precis of my biases in that area: https://doi.org/10.3389/fvets.2021.745779 

Your recommendations regarding the entire evidence pipeline should all be incorporated into recommendations from the Manifesto for Open Science, to my thinking. I do think there is a 'gap' in your analysis, in that animal welfare science is highly interdisciplinary, but in my opinion has been slow to reckon with the impact of the reproducibility crisis on our understanding of human behaviour, which is frequently more needed than the fundamental biological research on suffering.

This has given me a lot to think about though, thank you, I may return!

Jill

Thanks for writing this! I'm generally supportive of more critical review of evidence and analyses like this and would like to see more of it done, as well as more/better data collection, as you point to. I'm also glad you're working on new shrimp stunners and better validating them.

Re cage-free vs caged systems

My sense is that it's reasonable to infer that in the long-run, we don't know whether mortality rates are higher in caged or cage-free systems, and that the differences in average mortality rates are "small", based just on the graphs in this post and WFI's meta-analysis to which this post refers.

I agree that it's "an overstatement to say that there is “no differences in mortality” given the actual data" (in recent years, reading the graphs here).

I think it's still reasonable to treat them as if there's no expected difference in mortality rates in the long-run, but I also think it's reasonable to disagree with that, and this is a potentially important and fairly subjective judgement call, so should be flagged pretty explicitly. It could also mask differences in causes of mortality and associated harms by system, like the kind you point out, which could be important.

Cynthia Schuck-Paim from Welfare Footprint cited more US and EU data here, which show a very small difference in average mortality rates (3.75 in cage-free vs 3.64 in caged), much smaller than the standard deviation in mortality rates (2.62 and 2.34). I don't know what the "Standard" column in the table means, though. (Thanks to @mvolz for flagging WFI's responses.)

On the other hand, this could still allow that mortality rates increase substantially for several years as farms transition, and that might be quite bad, both in potentially extremely painful deaths and in indicating higher prevalences of other causes of suffering. I think it's worth being upfront about that, because there could be reasonable disagreement about whether it's worth it for the benefits of cage-free.

I think one trouble here is that WFI has to make a large number of judgement calls, given the available evidence. I think they are pretty transparent about them, but there can be a lot to check.

Some questions I have are:

1. If mortality rates between the systems do roughly converge on average in the long-run, but start higher early in the transition to cage-free, how long does the ~convergence take in practice across geographies, including in the Global South, especially for our marginal cage-free work now?
2. WFI's currently published analysis estimates generally less expected suffering in cage-free systems across pain intensity categories. At what point in the transition period to cage-free are these estimates based on, what mortality rates do they reflect, and how representative would they be of transitions across geographies? @cynthiaschuck @Wladimir J. Alonso
3. How much do increased egg costs and the resulting reduced egg demand mitigate the potential harms of cage-free transitions? Could the reduction in number outweigh any potential increases in harms on average per hen?
4. (May be answered by 2 or 3.) Are we, on net, increasing or decreasing excruciating pain with cage-free transitions in expectation?
   1. Are we just counterfactually moving forward these potentially painful cage-free transitions, if and because industry would (be forced to shift) eventually anyway, and so reaping the benefits earlier? Would egg replacements and other alt proteins or a global catastrophe (e.g. from TAI) mean we don't get to reap the benefits after mortality rates would have converged?
   2. Does our other work focused on excruciating pain (e.g. humane chicken slaughter) make up for potentially more excruciating pain in cage-free systems?
   3. How are we making tradeoffs between excruciating and disabling pain? How should we make them? (Personally, I'm averse to increasing excruciating pain overall in my portfolio, with high uncertainty about how much worse it is than disabling pain, and would probably have a part of my portfolio dedicated to preventing excruciating pain.)
5. Does nest deprivation really cause (so much) disabling pain? The supporting evidence I've seen seems somewhat confounded, and I would assign a lower probability to disabling pain than in WFI's published analysis, but would guess there's still a decent chance of it. (I expect WFI to have updated estimates in their upcoming book.)

Hi Amanda. Great post.

The Welfare Footprint Institute (WFI) argues the opposite by adding up hours of pain experienced by an average chicken in different systems^[17]. With their model, cage-free aviaries have lower pain scores than conventional cages. However, these results are heavily dependent on what harms are included, how they are scored, and how different types of pain are weighed. Deprivation of natural behaviours accounts for most of the difference in scores. Chronic fear and stress from violence is not included.

I wonder if you are overestimating the uncertainty here. I speculate there is like a 2/3 chance that cage-free layers have higher welfare than ones in furnished cages. The results from WFI's analysis are below. I think the lines cover the 95 % confidence intervals (CIs). Based on the harms considered, cage-free layers experience less annoying and hurtful pain than ones in furnished cages, and it is unclear whether they experience more or less disabling and excruciating pain. However, "The analysis primarily aimed to estimate the minimum welfare improvement associated with transitioning to cage-free housing. In this sense, we preferred to err on the side of caution than potentially overestimated reform benefits in any particular aspect. Thus, it is reasonable to anticipate that the actual benefits of this transition may surpass the estimates [below]" (see the 6 bullets in the piece elaborating on why). In agreement with this, Rethink Priorities (RP) got 3 animal welfare scientist to replicate 4 cumulative pain estimates from WFI, and found these "consistently erred on the conservative side".

• We have very limited data on electrical shrimp stunning that doesn't support a confident conclusion as to whether it's good or bad.

Here is a review of stunning methods for shrimp from Nino O'Shea-Nejad published in February.

Overall, the evidence indicates that electrical stunning has greater potential than chilling to induce insensibility, while chilling may suppress behaviour without eliminating sensory processing. Crucial uncertainty remains regarding defining species-appropriate electrical stunning parameters and translating neural evidence into reliable operational standards. On this basis, the report outlines implications for research priorities, regulatory standards, and industry practices aimed at ensuring that humane stunning practices reliably induce insensibility.

Unlike the other two examples, alternative proteins are unlikely to cause harm.

Alternative proteins may decrease the welfare of farmed animals by decreasing the population of farmed animals with positive lives? I estimated cage-free layers and slower growth broilers have negative, but close to neutral lives, and I would not be surprised if they have positive lives. 64.3 % (= 1 - 0.357) of layers in the European Union (EU) were outside cages in 2025, and funding work on alternative proteins does not have an immediate effect. It could be that it mostly reduces the consumption of eggs in 10 years, when the fraction of cage-free layers will likely be higher, and mortality lower due to improvements from experience. So I can see, for example, Coefficient Giving's (CG's) grants to Dansk Vegetarisk Forening (DVF) leading to fewer layers in the EU with positive lives.

In addition, alternative proteins may harm wild animals?

• Significant additional funding and talent should be allocated to raise the bar for animal welfare interventions by building R&D infrastructure that can rapidly generate high-quality action-relevant research results.

I agree impact-focussed funders underrate such research relative to supporting existing interventions. In addition, I think they underrate research on animal sentience, and, more broadly, on comparing welfare across species. I believe there is huge uncertainty, and ways of decreasing it. Here is some context about my uncertainty. In Bob Fischer’s book about comparing welfare across species, the tentative sentience-adjusted welfare range of shrimps is 8.0 % of that of humans. Welfare range is defined there as the difference between the maximum and minimum welfare per unit time among “realistic biological possibilities”. For sentience-adjusted welfare ranges proportional to “individual number of neurons”^“exponent”, and “exponent” from 0 to 2, which covers the best guesses that I consider reasonable, the sentience-adjusted welfare range of shrimps is 10^-12 to 1 times that of humans.

• Even some of the most prominent animal welfare interventions have surprisingly weak evidence behind them. In some cases, the available evidence even suggests that the intervention may be causing harm.

I would be curious to know your thoughts about effects on soil invertebrates. I would still have very little idea about whether cage-free egg campaigns, and electrically stunning shrimps increase or decrease welfare in expectation even if I was certain they increased the welfare of chickens and shrimps. I think their effects on soil invertebrates may be much larger or smaller than those on their target beneficiaries. This is another reason for prioritising research on comparing welfare across species.

Thanks for the well-argued and insightful post!

On alt proteins, if we ever substantially beat price parity (say by 50%), it’s just hard for me to see how we wouldn’t get mass consumer adoption. This comes from a model of most people’s stated preferences as downstream of what’s most convenient and protects their egos most. That’s the model I think best explains most people’s historical responses to moral catastrophes. Under this model, people maintaining they’d stick to factory farmed meat over cultivated meat is hopefully a temporary cope which will go away once cultivated meat becomes much more convenient than factory farmed meat. So for me, that model is the main reason why I still think alt proteins are a good use of funds. (Probably the stronger argument against alt proteins is that it may be unclear whether reducing meat consumption is good when accounting for wild animal effects!)

On cage-free campaigns and shrimp stunning, my main caveat would be that since animal welfare interventions affect such a huge number of individuals, I’d still expect the magnitude of their direct welfare effects (ignoring indirect effects) to be huge relative to global health. Of course, that only underscores your point that the sign deserves more research, and I look forward to reading comments from others far more knowledgeable than I am on this.

Would Rethink Priorities’ animal welfare research group look like a good investment under your views?

While reading this I felt quite confused and even a bit discombobulated. The cage free campaign institutions tell me how many hens can be saved from cages for a dollar. I always interpreted that as just straightforwardly good, even if we weren't sure how much good. For animal Welfare debate weeks and cross-cause priortisation models we assume that these interventions hugely move the needle - we debate other things like moral weights and the importance of uncertainty...

But the best research institutions like @Rethink Priorities and @Ambitious Impact have models which calculate huge reductions in suffering per dollar directed towards these very interventions. They must have good evidence based models for this.

It seems to me very unlikely to be that these big, thoughtful orgs are directionally wrong but the data presented here seems reasonable at a glance so what's up?

I would love to read a detailed data - based refutation / counter piece to this, which lays out the best argument that uncaging hens and stunning shrimp really do make a clear, huge difference. Or even just direct me to the best stuff already written. I donate (small amounts) to cage-free and this article more than any other has made me rethink that.

Also looking forward to seeing some more detailed pushback on this in the thread too!

One statistical/methodological point I’d add (something I always harp on). I don’t think “not statistically significant” should directly cited as evidence for a lack of a difference. If the question is whether mortality differences are small enough to be decision-irrelevant, we’d want something closer to an equivalence test or Bayesian posterior over the mortality difference, plus a welfare model translating mortality causes, morbidity, behavioral deprivation, fear/stress, and transition dynamics into aggregate welfare burden.

A forest plot or explicit meta-analytic summary could also make the cage-free evidence easier to interpret than the table of pairwise significance checks.

Related: I wouldn't always treat small sample sizes or mixed statistical significance as automatically implying “no useful inference.” Small-N studies can be informative if underlying measurement noise is low. For example if I ask 4 people to taste a drink and they all wince deeply in pain and disgust, I'm going to be highly confident it tastes bad. If all 4 smile and praise it, I'll be fairly confident that it's at least tolerable.

McElreath's globe-tossing example illustrates how much we can sometimes learn from small samples.

 (Still, in the shrimp case, it does seem like there is some substantial underlying variation unrelated to the different slaughter methods.)  

we already had multiple strong studies showing that the availability of equivalent substitutes does not lead to major changes in meat consumption.

This feels like your weakest argument. So far there aren’t “equivalent substitutes” to meat - only plant based imitations - so it’s not clear what these studies are telling us. The RP study you link to explicitly says it’s an analysis of plant-based meats and not cultured meat.

On alternative proteins, I’d adjust the claims. I agree we don't currently have strong, clean evidence that today’s plant-based products robustly displace animal products at the scale we’d want. But I’d describe the evidence as “limited, mixed, and hard to interpret” rather than “evidence that the substitution effect is weak, at best.”

Evidence in this domain is very hard to come by, and there are substantial doubts about the extent to which we can even reliably measure these things. See Unjournal's evaluation of Bray et al.; tldr, a large-scale grocery pricing experiments found that own-price elasticity estimates much smaller than standard/cutting-edge demand-estimation methods generated on parallel data. (But other critiques also suggest doubts about the relevance of the experimental estimate.s)

Also see this AI curated evidence/discussion on the lack of validation evidence in this domain as well as  our earlier post showing plant-vs-animal estimates all over the map,and not clearly linked to methodology. 

Challenges include:

• The (limited) evidence is mostly about current plant-based products and current meat-reduction interventions, not about genuinely price/taste/convenience-competitive products, cultivated meat, precision fermentation, etc.
• Measurement issues ... Scanner data misses food-away-from-home, substitution across animal products, longer-run habit formation, product introductions, and general equilibrium effects. And of course, price endogeneity.

And (as others mentioned) I'd separate the consideration of current plant-based products from future alternative proteins.  

My prior is that if something becomes a close enough substitute and is cheaper / tastier / more convenient, it should substantially displace some animal consumption, though the magnitude and species mix are very uncertain. 

But there's a chicken/egg problem in gathering evidence and funding: we probably won't get decisive ("revealed-preference") evidence until (e.g.) cultured meat products are actually on shelves and menus at scale, but getting them there requires investment under uncertainty. There's a potentially high upside here which we'd sacrifice if we're not willing to invest under uncertainty.

Thanks for the post! I was surprised to see how weak the academically-published evidence is regarding shrimp stunning effectiveness, and agree that it would be valuable on current margins for effective animal advocates to invest more in research. 

For what it's worth, I read the Tesco-Hilton case study more optimistically than you do. You quote it as:

The results are a bit vaguely presented, but with at least one machine setting “a significant proportion” of shrimp did not “show signs of recovery” within 10 minutes.

But the full quote is

A significant proportion of prawns were in irrecoverable stun (stun-kill) on exit of the stunner as evidenced by transfer back to a controlled aqueous environment where none showed signs of recovery within the monitoring period (10 minutes plus handling time of 32 seconds)

(emphasis mine). That's not to say that we should rely on a single vague industry report when the academic evidence is conflicting, but I think the report does provide some evidence. 

I really appreciate this post and am also very excited to see more primary research. In addition to the uncertainty about how a given change in farming practices impacts welfare, I think whether or not those changes are actually occurring is highly uncertain, and I’d like to see a greater investment in M&E at animal orgs to validate this.

More primary research could be complementary to this, for example, better monitoring of welfare indicators on farms and/or at slaughter could allow us to see whether the welfare is actually improving, which requires both that the intervention is actually occurring and that it’s helping animals when it occurs.

I do wonder about our ability to measure harms like stress and fear though. I think focusing just on mortality or health conditions might lead us to make false conclusions about which farming practices are better for animals. Does anyone know anything about our capacity to measure psychological distress from deprivation of natural behaviors vs fear of violence for example? (the former being one that drives a significant amount of the improvement that cage-free brings according to WFI, the latter being one Amanda points to being absent and potentially outweighing the former)

Nice post. 

Before we chose to campaign for a phase-out of cages for hens in Slovenia, we also wanted to be quite damn sure we were actually doing good for animals.

We eventually won a legislative phase-out of cages for layer hens. And the reason we felt confident enough to push for it was not just work by WFP, RP, OWA and posts on this forum. 

EFSA, the main scientific authority on food safety in the EU, recommends phasing out cages on welfare grounds. Our own veterinary faculty and animal welfare centre experts in Slovenia came to the same conclusion.

So at least in our case, this was not based on vibes or claims by a single community (EA). It was based on a broader scientific and veterinary consensus. Or at least, I should say, an independent assessment of evidence by these institutions. 

Hi there

See also this^[20] more recent meta-analysis that came to a similar conclusion about alternative proteins and other meat reduction interventions.

👋 I am lead author of that paper!

Two studies we looked at addressed alt proteins directly

The second paper is a small sample and doesn't really address what we consider the modern generation of alt proteins. The first does. From that paper's abstract:

Methods

Adult volunteers who regularly consumed meat were recruited from the general public and randomized 1:1 to an intervention or control condition. The intervention comprised free meat substitutes* for 4 weeks, information about the benefits of eating less meat, success stories, and recipes. The control group received no intervention or advice on dietary change. The primary outcome was daily meat consumption after 4 weeks, assessed by a 7-day food diary, and repeated after 8 weeks as a secondary outcome...

Results

Between June 2018 and October 2019, 115 participants were randomized. The baseline meat consumption values were 134 g/d in the control group and 130 g/d in the intervention group. Relative to the control condition, the intervention reduced meat consumption at 4 weeks by 63 g/d (95% CI: 44–82; P < 0.0001; n = 114) and at 8 weeks by 39 g/d (95% CI: 16–62; P = 0.0009; n = 113), adjusting for sex and baseline consumption. The intervention significantly increased the consumption of meat substitutes without changing the intakes of other principal food groups.

* Participants selected the meat-free substitute foods from a catalogue containing the full range of products available in a major UK grocery store at the time of the trial. This included mycoprotein meat alternatives and vegetable- and pulse-based meat substitutes.

How much should one update on this study? IDK. But by its own lights, giving people alt proteins for free -- embedded in a larger intervention -- had meaningful effects. 

However, when we test intro-ing alt proteins in a hypothetical, online ordering, chipotle-like environment, alas we don't find much.

A few years ago, spending on animal welfare reforms in EA was so limited that running rigorous, informative studies to reduce the movement's uncertainty might have cost more than the movement's spending on the campaigns themselves. But as you mention, now that the funding bottleneck might be easing, such studies would inform much larger amounts of spending, and the expected value of information looks much higher. It thus seems like a very good time to raise an issue that would have been much harder to make progress on in the past (though as Lewis Bollard's comment notes, the challenge is still daunting).

I found the post well-written and balanced, and the discussion in the comments mostly helpful. Strongly upvoted.

Thanks for this post! FYI one cage-free reform that seems unambiguously good is abandoning sow crates, provided the space is increased enough to avoid piglet crushing. Have been looking into that over the past few months, will share more info probably in a month or so.

Thank you very much for the post! I have read some comments (and except for Cynthia's, mostly incompletely). I want to leave a comment that is meant to be a reply to some comments, but also possibly the post itself:

Some comments in the discussion, and perhaps the post implicitly, seem to treat global health as a point of high certainty — Lewis described it as "the closest to total certainty of positive impact of any areas." But I think that "certainty" is partly an artefact of where we stop scrutinising. Yes, we have strong evidence that bednets counterfactually avert statistical deaths. But we have much weaker evidence that the counterfactual life thereby preserved is net-positive over its remaining course, and weaker still that it's more net-positive than the resources spent would have produced elsewhere (even limiting the resources within just humans, or the global health cause). That second layer — the value of the outcome, not the efficacy of the intervention — usually gets carried by unstated assumptions rather than by data. (FWIW, part of the assumptions are philosophical. For instance, there are serious philosophers who think that each extra life year is a net-negative, regardless of people's preferences. Also, people who have their lives saved might go on to harm other humans, but some EAs and ethicists think we ought not consider this when it comes to saving kids.)

I want to be careful not to overstate this, because there are disanalogies: The human prior is genuinely stronger on the immediate impact level. It also seems that on the secondary or further levels, interventions targeting humans are often even less certain than AW ones. So I'm not claiming the two are equally uncertain. 

But the (in)consistency point still bites. If AW has a major evidence problem vis-à-vis whether overall welfare was indeed improved, life-saving human interventions have it too — it just happens that we rarely turn the skepticism in that direction (welfare).

P.S. I'm aware of problems raised by population ethics and the meat-eating problem (it's a more productive framing than the version you heard), so not a novel observation in general. I'm raising my points narrowly because the comment section (or maybe just Lewis, and David Reinstein, plus the post implicitly?) leans on global health being the secure benchmark, in comparison to AW interventions seeking to improve welfare.

P.P.S. I used Claude 4.8 to help me check whether my points were already made by someone else here, and to help me draft the reply, of which I modified.

A sample size of six individual shrimp  for the stunning study is insane

Thanks for bringing this up and going in-depth on the evidence. I've always felt uneasy with the cage free campaigns and shrimp stunning. 

Part of that is definitely my more "abolitionist" viewpoint. I hear about those and think "wait how much money are we putting towards exploiting animals in a bit less bad way?" Of course an improvement is an improvement so if they are actually better then that is better, and there is benefit to getting momentum. It's easier to take a mile once you've gotten an inch, so to speak. But if we don't even actually know if the increment we're pushing is better, that's a problem that deserves to have alarm bells sounding.

That said, I don't really know what would be better tactically. Lately I'm pretty AI-pilled, thinking that if we can make AGI aligned with all sentient beings then that will be a huge benefit. But today's labs aren't gonna release a model that only ever suggests vegan recipes, and the future of AI is highly speculative. 

Sometimes I wonder if we should just start doing door-to-door vegan advocacy, like the jehovahs witnesses and normy politics and deep canvassing (and street epistemology and "smart politics" https://www.lesswrong.com/posts/D2GrrrrfipHWPJSHh/book-review-how-minds-change ).

Thanks for the thoughtful and truth-seeking post. 

One of the shrimp-related interventions that animal advocates have worked on is ending eyestalk ablation - the act of cutting the eyes off of living mother shrimp.

Shrimp Welfare Project work on this - https://www.shrimpwelfareproject.org/blog/what-is-eyestalk-ablation / https://www.shrimpwelfareproject.org/eyestalk-ablation-free 

And animal campaigners have won some commitments to end this practice - https://www.globalseafood.org/advocate/lidl-gb-bans-eyestalk-ablation-for-farmed-prawns-joining-nine-other-u-k-retailers/ 

Curious to hear how people feel about this kind of work?

(I haven't done the maths or read any academic work on this topic, but my assumption is that this is probably a robustly good thing, and good value for money?) 



 

I’d frame the solution a bit differently from “the EA animal welfare community should take ownership of every stage of the evidence pipeline.” 

I don’t think EA/AW people necessarily need to personally design every study, collect the data, or do all the econometrics/biology. But I do think the community needs to take responsibility for setting the agenda,  creating the incentives, providing funding,  clearly communicating the goals, priorities, and need for rigor to researchers, and enabling coordination.

And it would indeed help if the researchers intrinsically cared about animal welfare and thus about producing useful and accurate results. This makes the incentive alignment easier. But I think it can still work even if much of the work is done by people who aren't intrinsically interested in animal welfare or don't think about effectiveness in the same way - as long as they can understand and embody the priorities in their work.

I think entire organisations could and should be founded for this. Until now, this was simply not possible. Research is expensive and slow, especially at universities. But we're about to have the luxury to aim higher.

I agree that academic incentives and institutions on their own are not enough to ensure high-quality, credible work focused on animal welfare implications (although AW funders can leverage them to some extent). These are not well aligned with producing the kind of evidence funders want, and (e.g.) much existing agricultural/economic research is naturally oriented toward industry questions. 

(Warning: self-promo) we're trying to do some of this bring together academic researchers, practitioners, and funders to help them understand their mutual priorities, abilities and needs at The Unjournal.   See our “Pivotal questions”: an Unjournal trial initiative  e.g., this one on cultured meat, and this potential upcoming one on animal/plant product substitution.

We also try to aim this at the value of information focusing on identifying "what  specific operationalized questions (and what evidence) would productively change funding or implementation decisions", sourcing and evaluating the best evidence, fostering high-value discussion/debate, and eliciting and synthesizing beliefs and uncertainty.

I’d be excited to coordinate with others working in this direction.

(This overlaps with Jo’s comment somewhat, but I wanted to emphasize the institutional-design angle.)

Yes, this! 

To add, I also have issues with ACE's methodology & transparency. I don't have a background in stats or anything, but when I tried to independently verify ACE's evaluation of charity efficacy, I found several figures that seem (to me) not well-sourced. e.g., the 'source' column is mostly empty for ACE's AWO spreadsheet: https://docs.google.com/spreadsheets/d/1CAcmf4dXPk9tNWit25C7A7GJqEGNtnN-unXKZoyzvx4/edit

Compare with GiveWell's analysis: https://www.givewell.org/international/technical/programs/seasonal-malaria-chemoprevention#Sources

I'm still allocating my donations based on ACE's recommendations, but I'm really worried about how effective I'm being. 

Thanks again for this post. I left a bunch of Hypothes.is margin notes while reading: see here or install plugin. (I'll post some of these as regular forum comments now.) 

I strongly agree with the central diagnosis: animal welfare seems under-supplied with decision-relevant evidence, especially compared to global health. ACE, Rethink, WFI, CG, SWP, etc. are doing serious work, but the AW still lacks something analogous to the dense ecosystem around GiveWell-style global health evidence: funders, implementers, independent evaluators, academic researchers, field-data pipelines (and research evaluation institutions) all pulling in the same direction. 

I don't think academia/journals in the current state provides the right incentives or platforms. We end up with many small-scale projects papers, often not aimed at the key animal welfare outcomes, and a lack of follow-up. We need something more coordinated, directly aimed at effective animal welfare, collaborative, larger scale and longer term. 

This is a very important post, and thank you for writing it. Coming from the environmental world, this parallels my frustration that huge amount of efforts go into projects for which the evidence is shockingly flimsy. A significant fraction, maybe even the majority, of environmental advocacy is for measures that are nearly useless or outright harmful.

If you will pardon a personal plug, I wrote a piece on my Substack blog recently that looks at the evidence at a high level, drawing heavily on the methodology of Rethink Priorities' Moral Weigh Project. If we want to monetize animal welfare (in my piece, broiler chicken welfare), then estimates for the proper valuation span several orders of magnitude due to irresolvable philosophical differences. I am very grateful that the Moral Weight Project was done, but clearly we have a long way to go before we have reliable numbers that we can use.

I am particularly interested in your comments on alternative proteins, since it will be very relevant for a piece I am working on regarding meat taxation. A major focus will be to review that we know about the rebound effect and induced/latent demand to argue that, if they attain widespread market viability, alternative proteins will mostly augment the meat supply rather than replace it.

We asked GPT 5.5-extra in Codex to do an evidence audit and discussion, because this seems to be  a high value post relevant to Unjournal's work on evidence quality and animal welfare-relevant economics. It reports

We also added several public Hypothes.is annotations directly on the post text, tagged `gpt` and `evidence-audit`. Public annotations are visible here: Hypothes.is search for this post. You can also see them in context by installing the Hypothes.is browser extension.

My [GPT's] main takeaways from checking the cited and related sources:

1. Shrimp stunning: The evidence seems best read as parameter-sensitive rather than as a blanket case for or against electrical stunning. The recent Somerville et al. preprint and SWP review suggest electrical stunning may have more potential than chilling, but species-specific settings and industrial validation remain central uncertainties.
2. Cage-free reforms: The post seems strongest when framed as a critique of mortality-based inference and transition management. Mortality evidence is mixed and confounded, but that is not the same as saying the overall welfare evidence is equally mixed. Schuck-Paim et al. 2021, WFI’s laying-hen model, and Cynthia Schuck-Paim’s comments in this thread all seem relevant here.
3. Alternative proteins: I would frame this less as “we have evidence that substitution is weak” and more as “we have limited and hard-to-interpret evidence, plus serious measurement challenges.” The available evidence seems more directly about current plant-based meat and meat-reduction interventions; it does not settle future cultivated meat, precision fermentation, or genuinely cheaper/tastier/more convenient substitutes. 
    

Sources checked included Weineck et al. 2018, Somerville et al. 2026, SWP’s stunning review, Schuck-Paim et al. 2021, WFI’s laying-hen work, Peacock’s RP report, and Green/Smith/Mathur’s 2025 meta-analysis.

David Reinstein: I added further comments manyally in hypothes.is and I'm working on a general response comment synthesizing that, also bringing in resources from our PQ/workshop on plant-based substitution. Let me know if the above is misleading or distracting. 

Hi @matthes, I appreciate your writing this post and agree with much of it. Thank you for recommending my article as well. I wanted to note I updated it for publication (preprint) earlier this year. In line with your thesis, little new evidence has been produced recently, so there wasn't much to update.

I did add a section on the effects of reducing alt protein prices, which gently pushes back against your framing that "alternative proteins are unlikely to cause harm":

Price changes [of alternative proteins] might seem to have the most straightforwardly favorable case: decreases in plant-based meat analogue prices presumably lead to more sales and, consequently, less animal-based meat sales. However, this is not guaranteed. For example, a lower price may lead some consumers to treat plant-based meat analogues as inferior goods—or cheap substitutes—rather than a better deal.

Reviewing cross-price elasticity studies of plant-based meats, milks and butters, I conclude: "Throughout, estimates sometimes suggested that decreased plant-based meat, butter and milk analogue prices might cause increased consumption of the corresponding animal product." Obviously, this is pretty counter-intuitive, but such is the evidence at present, in my opinion. We elaborate on the difficulties of interpreting this evidence in our review of butter cross-price elasticities.

Interesting post! I definitely second the call to fund more quality animal welfare research. It's surprising how poor some research is, even when peer-reviewed. I especially like the specification of "action-relevant" research, as this is key. Especially when trying to implement interventions in LMICs, often lab studies aren't applicable. 

I'm tardy to the party but I wanted to join in. I share a lot of OP's concerns especially when it comes to shrimp welfare. 

As I wrote in a previous post: 

"Most of our data on the topic [of farmed shrimp sentience] is based on another shrimp species about as evolutionarily distant from what's most commonly eaten as sharks are from humans"

I don't donate to any "EA" animal welfare charities, but I 100% support vegan advocacy; about a quarter of my donations go to Vegan Outreach. 

I'm all for having the good data backing up various interventions. But stepping back, killing sentient animals unnecessarily for taste preference is morally indefensible. The easiest way to solve this problem is to reduce demand.

Sometimes the simple and obvious answers are the ones worth investing in.

​Thank you for this deeply analytical and constructive post. It highlights a vital epistemic challenge in the animal welfare space: the tension between the urge to scale intuitive solutions and the reality of thin, sometimes conflicting data. The risk of unintended direct harm (such as increased mortality or distress in poorly managed cage-free or stunning transitions) underscores why a rigorous R&D infrastructure is so necessary. Shifting funding priorities toward mapping the entire evidence pipeline feels like a high-leverage way to ensure our interventions match our intentions. I'd love to see more organizations focus heavily on action-relevant primary research before we push for global implementation.