Hi Amanda, thank you for your thoughtful analysis. I do believe taking a step back and  scrutinizing the evidence and direction where we're heading is extremely important, and I agree with your  conclusion that increasing R&D and talent is very much needed. I also agree that the evidence gaps are enormous. As a research group working specifically to address them, we're very much aligned. In that same spirit, I share some considerations below on the cage-free transition example, into topics to make the text lighter. 

The problem with the CSES study as a reference for mortality / welfare conclusions.

The CSES study is a highly cited reference for the argument that cage-free aviaries are not necessarily better. It was funded by the American Egg Board and facilitated by another industry-funded organization focused on building consumer confidence and maintaining the industry's viability. Unfortunately, this study had multiple design flaws and biases in favor of cages (a more detailed analysis, done some years ago, is available here). A few examples:

• Several observations point to stockmanship being ‘very deficient’ in the aviary (the farmer had never managed one), but not in cages, with which the farmer had decades of experience. Inspection frequency also seems to have been higher in cages.The research authors themselves declared they were still learning about what to do in the aviary during the research, which led to multiple failures. In fact, inspection was so infrequent/deficient that a large fraction of dead birds were found in advanced stages of decomposition.
• Vet/disease treatment was employed for cages, but not aviaries
• The overwhelming majority (~90%) of good practices for cage-free aviaries were *not* adopted (e.g., aviary chicks were raised in cages, with no development of skills to perch or navigate the aviary, densities were higher than typical, laying birds in aviaries were confined for hours every day, increasing the risk of feather pecking, breeds adapted for decades to cages were used in aviaries, chicks did not have access to litter, among many others poor management decisions - see a summary table in this document)
• Key practices to reduce the risk of feather pecking / cannibalism in aviaries were not used (in fact, CSES management choices are known to promote these issues).
• CSES mortality data does not add up, there are multiple inconsistencies in the dataset and in the mode of data collection/reporting, discussed here.

Mortality Data

Farm animal welfare in natural and more extensive systems, including cage-free aviaries, depends more heavily on good management practices and stockmanship. As such, we should expect greater variability in terms of mortality, as well as greater absolute mortality in the first production cycles following a transition. For this reason, the fact that there is a greater number of orange cells (mortality higher in aviaries) in the datasets of the meta-analysis is expected, as most are comparisons of established caged systems with newer cage-free systems. The studies span  two decades during which cage-free systems underwent major changes. This is why we explicitly modeled the year of data collection as a predictor. Mortality rates are changing systematically over time. Modeling that trend directly, and then reporting recent mortality separately, is necessary for relevance. Another point is that brown-feathered genotypes (associated with higher mortality) are more common in cage-free systems, naturally increasing mortality because of breed, not system.

The data below, from an internal database from a  breeder for countries around the world (in 2018), can also be useful.

Mortality as an indicator of welfare

Mortality may or may not correlate well with welfare. It is widely used because it is easy to measure, routinely collected, and economically important. However, mortality captures whether animals survive, not what their lives are like while they are alive.
The industry is often very good at keeping animals alive and productive until the end of the production cycle, even under conditions associated with extremely poor health and welfare (what we refer to as the "hospital bed effect"). Conversely, some more extensive or naturalistic systems may have higher mortality because animals are exposed to more hazards. Healthy pasture-raised animals, for example, may experience higher mortality due to predation. There may often be a trade-off between behavioral freedom and protection from mortality risks (‘children who never play outdoors are less prone to injuries and fatal accidents’, yet few would argue this is better for their well-being).
Mortality is most informative when comparing otherwise similar systems. When comparing systems that differ fundamentally in housing and behavioral opportunities, mortality should be interpreted together with other welfare metrics.

Life Quality/Well-being in cages x cage-free

The 2021 WF analysis was very conservative (i.e. favored caged systems) in a number of ways as we discussed in the book and elsewhere. For example, we considered prevalence for ailments in cage-free systems as reported, without any adjustments for improvements over time (despite evidence that the frequency of various harms  was going down, similar to mortality).

Also, we did not consider positive welfare (opportunities are naturally more frequent in cages) nor the longer lifecycle of caged hens (welfare is typically worse at the end of life), and end-of-life events such as induced molting, still practiced in many countries in caged systems.

We also did not consider the negative impacts of learned helplessness, lack of agency, and depression-like states in cages - there is now new evidence for such depressive states in caged hens.

Importantly, we did not make any adjustment for what I believe to be robust evidence that the pain from an injury or disease is perceived as more intense and longer in cages than in cage-free systems. Barren, confined environments disable multiple endogenous analgesic mechanisms  while simultaneously activating several neurobiological pathways that intensity nociceptive signaling and delay healing. Should that be taken into account, it would further reduce time and intensity of pain in cage-free aviaries.

Fear in Cages vs Cage-free systems

Several studies have found that hens reared or housed in cage-free systems are less fearful than hens kept in conventional cages. Aviary-reared birds show reduced fear responses in tonic immobility, novel object, and novel environment tests, spend more time near humans and novel objects, use elevated areas more readily, and perform better in spatial memory tasks than cage-reared birds (Hansen et al., 1993; Tahamtani et al., 2015; Brantsaeter et al., 2016). They suggest that the more complex environments in cage-free systems may reduce fearfulness and improve behavioural adaptability.  See box 9.2 of the laying book for more details.

Behavioural and Physiological Indicators as a standard for welfare-related decisions

Welfare is multidimensional, and cumulative experience matters. So unfortunately no single behavioural or physiological measure, or restricted set of measures, can provide a complete picture of welfare, nor even for humans (for which calibration is possible). Behavioral, immunological, neurological, and physiological measures are valuable for inferring states associated with specific experiences, often at specific points in time. However, they are insufficient for overall welfare assessments, as well as confounded by multiple factors, and more reliable for acute rather than chronic harms (particularly immunology and physiology). Because they are typically species-, harm- and context-specific, they also do not enable comparisons across harms, systems and species. Several attempts have been made in recent years to design an umbrella measure of welfare (e.g., telomere length, cognitive bias tests), but so far unsuccessful. That's not to say indicators are not useful: we rely heavily on them for our work, and believe having more monitoring systems and research would be extremely needed. But for overall welfare assessments and system comparisons as in the case of cage-free transitions, we need welfare metrics, which integrate evidence from multiple welfare dimensions, providing a stronger basis to infer cumulative experience. In case it's useful, here we discuss in more detail the differences between welfare metrics and welfare indicators.

Thank you again for this critical analysis! 
Cynthia
 

Hi Vasco, yes, this is something that will be part of the Welfare Footprint of the Egg. We're now including the analyses of plausible scenarios within each system, including scenarios where best practices are used, as well as scenarios of complete failure, which will give you an idea of the variability you're mentioning.

Hi Vasco, thank you for raising this.

My general sense is that option A leads to a greater welfare increase. Not only based on what we measured, but also on recent evidence that pain is likely more intense and longer in cages (even furnished), as discussed here, as well as recent evidence of depressive-like states in cages (e.g., here).

So my answer is that even if furnished cages are less harmful than battery cages, I do not think that advocacy for 'cages' is worth pursuing.

Aaron, this is a great idea. I strongly agree that bringing research closer to commercial farms is essential if we want findings that actually reflect what happens in practice, with much of what is produced in research settings (even those that try to mimick commercial practice) suffering from what we call the 'healthy farm effect'. Commercial data capture the full messiness of real production, which is why it is so valuable.

External validity is not the only thing missing in welfare science. Most of what we know about animal welfare at commercial scale comes from single visits to farms, essentially a photograph of what happens. We need the video (longitudinal research). We need to know when different welfare problems start, how long they last, and how many animals are affected. In short, we need an epidemiology of animal welfare, and commercial farms are the only place where that can happen.

That said, for this to work, I believe a few things are needed:

• Independence between funding and publication. In my experience working with industry-funded groups, there are often contractual or simply informal pressures (e.g. anticipation of future fundign) that discourage publishing unfavorable results . Agreements need to explicitly guarantee the right to publish regardless of outcome.
• Independent analysis. Data collection can happen on farms, but analysis should be conducted or audited independently to reduce potential biases.
• A clever design of incentives. Farmers should benefit from participating, but incentives shouldn't promote selective reporting.
• Standardization. The power of research on commercial farms will come from collecting consistents data and indicators across many farms, enabling large scale analyses with the proper statistical power and validity. This needs pre-defined protocols, clear definitions of variables and methods, preregistration where possible, and transparent data access. Without this, farm data risk being too messy or selectively reported.

With these safeguards, I believe research on farms will be of immense value. Farms are already generating huge amounts of data as you mentioned, one challenge now is creating the proper systems to use it .

Thank you for your thoughtful analysis. This is helpful for us to understand areas that must be improved or made clearer. This is especially important as we will be soon publishing  the full Welfare Footprint of the Egg (various systems), where we analyse over 120 experiences (diseases, injuries, deprivations, imbalances – nearly everything we could identify), in different housing systems, from birth to slaughter (more info here), for layers and breeders.

On your specific points on the direction of the results: there are various ways in which the existing analysis was conservative (i.e. favored caged systems). For example, in estimating the prevalence of keel fractures and other ailments in cage-free systems, we considered prevalences as they were reported, which typically was in the first few cycles of experience with cages. Evidence indicates that these prevalence rates go down as management experience increases (examples in the Prevalence Chapter), yet we preferred not to make that assumption and use the data as it was. Also, we did not take into account positive welfare (greater in cage-free settings) and more diffuse experiences, like fear, helplessness and boredom (greater in cages). Nor did we consider the flow-through effects of frustrations from behavioural deprivations beyond the period corresponding to the time budget of engagement, or practices like induced-molting (more likely in cages), or the longer cycles of caged-hens (with end disproportionally worse at the end)

Importantly, there is substantial evidence indicating that Pain is more intense (and healing delayed), even for the same injury/disease, in cages than in cage-free systems. We did not consider these modulatory effects, but they are likely present. That said, we’ll look deeper into the references you mentioned.

Unfortunately, many existing comparisons of cage and cage-free cite the CSES studies, to which the analysis from Fulton are part of. These studies were funded by the American Egg Board and facilitated by another industry-funded organization focused on building consumer confidence and maintaining the industry's viability. While management in the caged systems was good and based on decades of experience, the cage-free systems were implemented for the first time, and did not adopt nearly any of the good management practices required. For example, during the laying phase, birds in the aviary were confined for many weeks before accessing the floor litter (something that makes injurious pecking much more likely). Also, insufficient space allowance and perch space in the aviary led to crowding, collisions, and failed landings, likely contributing to the high rate of keel injury and mortality. The authors themselves declared publicly they were still learning about what to do in the aviary systems during the research, which led to many failures. These design and management failures likely substantially inflated the negative outcomes in CF systems, including mortality (mortality data is also inconsistently reported in their publications, with some mortality - e.g. during placement - apparently excluded from caged systems).

Some info that may be useful:

• Keel injuries: so far, the most consensual explanation is the earlier onset of laying, before the keel bone is ossified. This affects both cage and cage-free systems.
• Nest deprivation:  the justification for the ranges of durations and probabilities of distress intensities is provided in this chapter. These inferences are grounded in multiple lines of evidence (reviewed in the chapter). As such, they remain open to revision if new data or alternative interpretations of the evidence emerge. If you believe that the evidence supports a different intensity, please share your reasoning. This kind of scientific exchange is precisely what helps us approach values that are likely closer to reality.
• Mortality: the lack of differences in mortality is seen in many datasets, with and without beak trimming (the control mentioned in the 2021 meta-analysis is for Week’s data, not for the meta-analysis data). Industry data also shows that the best cage free systems have lower mortality than the best caged systems.  

More generally, both in the prior analysis, and in the forthcoming book, a major issue is data scarcity. Therefore, we inevitably rely on estimating uncertainty ranges for parameters like duration and prevalence. Inter-rater agreements should be made available together with future estimates, but what we have seen so far shows reasonable levels of agreement among WFI and independent academic raters/estimators.

As we build more comprehensive analyses, we’d be keen to have our estimates scrutinized as you did, so thank you!

Hi Vasco, thanks for this follow up. I believe we cannot rule out the possibilities you mentioned. That is, the answer could be yes to both questions. Even with humans, there is high variability in preference for intense but short aversiveness, as compared to moderate but longer aversion. 

One solution we have been adopting lately is to simply 'add up' times in the different categories (with no weighting or conversion). Since the 'boundaries between intensity categories' are acknowledged to be uncertain (i.e., we assign probabilities to each intensity category), adding up the time spent in different intensities follows naturally from this uncertainty. For instance, hours in Hurtful, Disabling, and Excruciating pain can be added together to report total time in 'moderate to intense pain,' or hours in Disabling, and Excruciating pain can be added together to report total time in 'intense pain'. 

Hi Vasco, that's a good question, but I do not think we can say with certainty that the ratio is preserved across species. The ratio may differ across species for a number of reasons, such as species-specific differences in the subjective perception of time. In fact, the relationship between the aversiveness of pain and its intensity may itself change dynamically for a 'same' species, depending on the duration of the experience. If on the one hand it's true that the 'criteria' defining each intensity category is universal (the same for all species), we cannot know for sure whether Excruciating pain in a shrimp is the same as Excruciating pain in a human being.
 

Thank you for this thought-provoking article, Vasco! In situations where we have clear, certain benefits in one area (here, knowing that use of slower-growing chicken breeds clearly reduces suffering for those chickens, even if their lives are longer) and a high degree of uncertainty on multiple dimensions in another (here the welfare consequences for arthropods), my inclination is that it's often more effective to focus on the suffering we are confident we can alleviate, and avoid  the non-negligible risk of relevant unintended negative consequences in the uncertain case (while simultaneously promoting further research on the latter effects). 

Hi CB, thanks a lot for your comment, I think it represents a main concern of many people. I'll break my thoughts in two parts

(1) AI use in shrimp farming and similar situations.

In this case, I understand what AI-monitoring did was to enable farmers to optimize feed use enormously (shrimp grew larger, mortality was reduced, and feed was not wasted), as well as water quality monitoring. This could be seen as negative for welfare, as it facilitates farming in high stocking densities, makes shrimp farming more profitable and could reduce prices, though this price effect is complex since the same AI technologies will likely make alternative proteins cheaper too, making the net effect on consumption less certain.

However, consider the actual conditions shrimp face. Without AI, feed distribution was uneven, leading to competition, stress, malnutrition and starvation for a large fraction of animals (mortality without AI was higher), as well as longer exposure times to poor water quality, and higher incidence of toxicities (hence respiratory distress, gill damage, skin damage) that come associated with it. This leads to suffering and higher mortality rates. So it's possible (though this should be measured) that even in higher-density environments, AI use can maintain better welfare than lower-density farms with poor feed and water quality management. Importantly , if shrimp feed relies on fishmeal and fish oil, optimizing feed reduces the number of wild fish needed, so each pound of shrimp has a smaller welfare footprint in terms of wild fish captures.

The industry trajectory also matters. Aquaculture is already moving toward higher-density and intensified farming with or without AI. So I believe the relevant comparison isn't between AI farming and a low-density or extensive scenario, but between AI-farming and conventional (intensive) high-density farming without AI. 

(2) On AI leading to greater income/prosperity, potentially increasing consumption of animal foods.

I see greater incomes and prosperity as extremely positive to reduce human suffering, but animal suffering as well. While rising incomes historically increased meat consumption, the relationship is not linear, in that as societies become more prosperous (on top of being an extraordinary thing in itself), they often can afford being more concerned with environmental and ethical issues. It's particularly in wealthier nations that we see a trend towards reduced meat consumption, stronger welfare legislation, increased interest in plant-based alternatives, and the means needed for the development of innovations like cultivated meat and other substitutes of animal protein. And again, the same technologies making animal farming more efficient are simultaneously making alternatives more competitive and affordable. I believe that the key isn't if AI increases income (something to be celebrated), but how to channel greater incomes toward ethical food systems.

Hi Vasco! Yes, it would be very interesting to collaborate on this. Right now we do not have the resources (in terms of people, and time) to do it ourselves, but we would gladly collaborate with anyone leading this effort. One possibility would be running WTP tests with people, from various demographics, to determine the extent to which they would pay to trade Disabling Pain by Hurtful Pain, or Excruciating Pain by Disabling Pain and so on (having the understanding of these intensities well explained and calibrated with examples, past experiences, etc, within a clear set of criteria).  This would help understand some level of equivalence (though from a "human" perspective) between the categories, but also generate rational WTP numbers for any estimates of Cumulative Pain (e.g., if cage-free campaigns avert X hours of Disabling Pain per hen, and people are willing to pay on average 1-10 dollars to avert one hour of this pain, you can in theory calculate the extent to which they would pay more for improved welfare, run CBA analysis, apply these to funding decisions, etc)