Abstract: There is an ongoing debate about the quality of the evidence that meta-analysis provides. But both critics and defenders of meta-analysis generally assume that the core purpose and contribution of meta-analysis is to tell us what the evidence really says about the existence and magnitude of causal relationships, to extract conclusions that agree from datasets that do not. I argue that while delivering such information about cause-and-effect relationships is the most common use of meta-analysis, this is only an application of the tool, and that its primary epistemic role of meta-analysis is something else entirely: to help us explore and understand variation among populations of studies.
Thanks, Caleb – here's the technical paper detailing the methods Jared used in the re-analysis, and here's a more accessible paper introducing some closely related sensitivity analyses. The second is paywalled, but if you'd like, I can send you a copy if you message me with your email address. You can play with the latter method with this online calculator.
It would be nice if research on changing the consumption of animal-based foods estimated not only the changes by type of animal-based food, but also by living conditions. I estimate broiler welfare and cage-free campaigns increase welfare per living time by 92.9 % and 80.4 %. So I think shifting from eating chicken meat from broilers in a conventional scenario to ones in a reformed scenario, and from eating eggs from hens in conventional cages to ones in cage-free aviaries is much better approximated by a 100 % reduction in the consumption of chicken meat and eggs than by no change, which is what may be naively inferred by observing no changes in the consumption of eggs or chicken meat. My sense is that the research is at an early stage, where changes in consumption mostly refer to the total consumption of animal-based foods, or not even this.
I agree, although in many food-service settings other than grocery stores (e.g., restaurants and dining halls), there often are not multiple welfare options within a given animal product. Although overall societal demand could ultimately change the welfare options in those settings, those effects are longer-term and hard to estimate. However, we do have an upcoming project using grocery store data (which I forgot to write about...), and we'll explore whether we have the right data to look at welfare options. Thanks for suggesting!
Greater uncertainty about whether the lives of farmed animals are positive or negative implies a stronger case for improving their conditions (which is always good) instead of decreasing their population (which decreases welfare if their lives are positive).
Yes, but interpreting the latter part (decreasing total welfare by reducing the population) as bad hinges on total utilitarianism, and would otherwise be interpreted as neutral. If one puts low credence on total utilitarianism, then the possible downside of entrenching CAFOs with net-negative lives (which, as you allude, is bad on pretty much any ethical view) could ultimately outweigh the possibility that having lots of minimally net-positive lives is good (which is true primarily on total utiliarianism). I acknowledge that there is a lot of meta-ethical uncertainty here. The unfalsifiability of it all is so annoying.
On the mission statement: Ultimately, yes, our goal is to end farmed animal suffering, and while we are in favor of interventions that improve animal welfare (and I personally am a THL donor), our own research focus is on displacing demand that necessitates CAFOs in the first place. I enjoyed reading your net-positive lives post earlier, and it was a significant update for me. I do, however, think that estimating the threshold between a net-positive and net-negative life seems really hard even for humans (and maybe even for oneself!), let alone other species, so I would be very wary of entrenching CAFOs on the assumption that the lives are net-positive. I am also not personally sold on total utilitarianism, so I am not sure that I want a repugnant-conclusion situation with CAFOs, even if we were certain the lives were minimally net-positive.
On the animal-welfare modeling: What you suggest would be the gold-standard approach, but we need to strike a balance between complexity and comprehensibility/credibility for implementation at Stanford and other universities. We are going to start with much simpler arguments about the total number of animals affected. This is already a big jump, since currently, the university occasionally considers welfare standards (e.g., cage-free) but never the number of animals (hence, much more chicken is now served than in the past to reduce GHG emissions). There are also limitations on the granularity of food procurement data the suppliers give us (e.g., fish species are often unclear). We do plan to do our own sensitivity analyses using more rigorous weights.
Thanks for sharing this candid account of your journey, Seth! How fortunate for the HSF Lab that your path landed you with us. Even though the trip was roundabout, it gave you a breadth and depth of skills, and worldliness, that doesn't come rolled up inside a PhD diploma.
Start by doing work you're invested in, that you're proud of, and people may notice.
Agreed, though if their model isn't correctly specified to identify the causal effect on meat (which I agree is tough here), then presumably the effects on plant-based sales would also be suspect.
Maybe of interest: "What is meta-analysis good for?"
Thanks, Richie – causal inference is definitely tricky and subtle! Thanks for sharing "The Effect". I had not seen this.
Thanks, Caleb – here's the technical paper detailing the methods Jared used in the re-analysis, and here's a more accessible paper introducing some closely related sensitivity analyses. The second is paywalled, but if you'd like, I can send you a copy if you message me with your email address. You can play with the latter method with this online calculator.
I agree, although in many food-service settings other than grocery stores (e.g., restaurants and dining halls), there often are not multiple welfare options within a given animal product. Although overall societal demand could ultimately change the welfare options in those settings, those effects are longer-term and hard to estimate. However, we do have an upcoming project using grocery store data (which I forgot to write about...), and we'll explore whether we have the right data to look at welfare options. Thanks for suggesting!
Yes, but interpreting the latter part (decreasing total welfare by reducing the population) as bad hinges on total utilitarianism, and would otherwise be interpreted as neutral. If one puts low credence on total utilitarianism, then the possible downside of entrenching CAFOs with net-negative lives (which, as you allude, is bad on pretty much any ethical view) could ultimately outweigh the possibility that having lots of minimally net-positive lives is good (which is true primarily on total utiliarianism). I acknowledge that there is a lot of meta-ethical uncertainty here. The unfalsifiability of it all is so annoying.
Thanks, Vasco!
On the mission statement: Ultimately, yes, our goal is to end farmed animal suffering, and while we are in favor of interventions that improve animal welfare (and I personally am a THL donor), our own research focus is on displacing demand that necessitates CAFOs in the first place. I enjoyed reading your net-positive lives post earlier, and it was a significant update for me. I do, however, think that estimating the threshold between a net-positive and net-negative life seems really hard even for humans (and maybe even for oneself!), let alone other species, so I would be very wary of entrenching CAFOs on the assumption that the lives are net-positive. I am also not personally sold on total utilitarianism, so I am not sure that I want a repugnant-conclusion situation with CAFOs, even if we were certain the lives were minimally net-positive.
On the animal-welfare modeling: What you suggest would be the gold-standard approach, but we need to strike a balance between complexity and comprehensibility/credibility for implementation at Stanford and other universities. We are going to start with much simpler arguments about the total number of animals affected. This is already a big jump, since currently, the university occasionally considers welfare standards (e.g., cage-free) but never the number of animals (hence, much more chicken is now served than in the past to reduce GHG emissions). There are also limitations on the granularity of food procurement data the suppliers give us (e.g., fish species are often unclear). We do plan to do our own sensitivity analyses using more rigorous weights.
Have you considered having an open, competitive process to submit talk abstracts, similarly to academic conferences?
Congratulations, Bob!
Thanks for sharing this candid account of your journey, Seth! How fortunate for the HSF Lab that your path landed you with us. Even though the trip was roundabout, it gave you a breadth and depth of skills, and worldliness, that doesn't come rolled up inside a PhD diploma.
Start by doing work you're invested in, that you're proud of, and people may notice.
This is fantastic advice.
Yes, good point
Agreed, though if their model isn't correctly specified to identify the causal effect on meat (which I agree is tough here), then presumably the effects on plant-based sales would also be suspect.