AGI & Animals Symposium (Thursday 5-7pm UK)

I think the debate motion bundles together several distinct mechanisms by which human flourishing under AGI could translate to animal welfare and I’m interested in which ones folks put the most weight on. I've tried to identify mechanisms that might connect human and animal welfare under AGI, each of which could hold in some possible worlds and fail in others. This list isn't a claim about what I think is most probable, since I'm highly uncertain. Some mechanisms (non-exhaustive list) might be:

Expanding moral circle: as AGI makes humans become more secure and prosperous, humans may extend moral concern outward to more groups. I think this is possible, but wealthy societies have industrialised animal agriculture and increased reported animal welfare concern simultaneously, and the concern doesn’t prevent poor animal welfare.

• How strong do you think the empirical relationship between prosperity and animal welfare concern actually is? And will concern translate to meaningful change?

More resources: AGI-driven wealth could mean wealthier people could direct more resources for animal welfare. Global spending on improving animal welfare is currently tiny compared to the global meat industry, so more resources could make a meaningful difference.

• Do you think rising incomes shift food consumption patterns toward higher-welfare products, or are attitudes to food sticky enough that the pattern persists even under significant income growth?
• Do resources get distributed to those who would direct them to animal welfare or do they get concentrated somewhere else?

Technological co-benefits: AGI solving human problems could also solve the barriers to replacing animal agriculture. I’m unsure how AGI-optimised factory farming plays out against other food systems that might come about with AGI.

• How do you expect AGI-optimised conventional farming to compete against AGI-optimised alternative proteins, or some other food system? Which gets there first, and does that create a lock-in dynamic?
• How much can AGI help with non-technical barriers, like regulatory and political constraints?

Institutional improvement: AGI creates better, more rational institutions for humans and the benefits get extended to animals.

• Do you expect that new institutions reshaped by AGI would still need to explicitly include animal welfare considerations in the objectives, or could they emerge as a byproduct?
• If AGI concentrates institutional power in the hands of a small number of actors, does that make pro-animal institutional reform more or less likely?
• How sticky are today’s laws/regulations?

Moral AGI: a sufficiently capable AGI reasons from first principles, weights animal suffering heavily, and acts on it unprompted. Unlike moral circle expansion, which requires humans to change their values, this could bypass human values. I think this is possible but worry about AGI instead being well-aligned with today’s human values, which I don’t think would benefit animals.

• Do you think rigorous moral reasoning from first principles tends toward weighting animal welfare heavily? 
• How much does it matter that animal welfare is underrepresented in AI alignment frameworks today? 

Note: anything I comment during the symposium is my personal view and not necessarily the views of my employer :)

Nice little Claude summary of the debate so far, which might help identify the missing points:

The debate centres on whether human-aligned AGI would automatically benefit animals, or whether animal-specific interventions are needed.

The pessimistic case is well-represented. Jim Buhler argues we have no good reason to assume AI safety work helps animals — saving humans preserves factory farming, and the claim that empowered humans would improve wild animal welfare rests on untenable assumptions. Simon Eckerström Liedholm (Wild Animal Initiative) estimates only ~30% probability of good animal outcomes conditional on good human outcomes, largely because the most likely alignment path locks in current human values, which permit enormous animal suffering. Hannah McKay (Rethink Priorities) argues that cultivated meat won't be automatically solved by AGI — regulatory, political, and consumer barriers form a sequential chain where the combined probability of resolution is low.

The bridge position comes from Aidan Kankyoku, who thinks it probably (~70%) goes well for animals but that this isn't sufficient certainty to neglect animal-specific alignment. He argues animal welfare is now functionally a subsidiary of the "Make AI Go Well" movement.

MichaelDickens contributed three posts: a taxonomy of alignment research by animal-friendliness, a cost-effectiveness model finding alignment-to-animals only marginally more cost-effective than general alignment, and a meditation on how current alignment paradigms (unlike CEV) give him roughly 50/50 odds on animal outcomes.

The discussion thread (~58 comments) skews disagree, though with real spread. The most common argument for disagreement is historical precedent: technological and economic progress has been bad for animals so far, with factory farming as the central exhibit. Value lock-in is the second recurring worry — that alignment to current human values would freeze in a set of preferences that are largely indifferent to animal suffering (SimonM_, Babel, Dylan Richardson, Tristan Katz). Several voters also flag the risk of spreading wild animal suffering to new planets. On the agree side, the strongest argument is economic: post-scarcity conditions erode factory farming's viability because alternatives become cheaper (OscarD, Erich_Grunewald, Brad West, JDBauman). A few voters (Ronak Mehta at 100% Agree, Ligeia, Artūrs Kaņepājs) argue that a genuinely superintelligent system would recognise animal sentience as morally relevant. A notable cluster sits at or near 0% Agree not because they're confident things go badly, but because they think the question is unanswerable given the number of branching futures (NickLaing, Seth Ariel Green, Jim Buhler). Peter Wildeford offers a useful split: on a causal reading (alignment mechanisms also help animals) he's pessimistic; on an evidential reading (conditional on good human outcomes, what world are we in?) he's somewhat more optimistic.

My position statement (20% disagree with the statement "If AGI goes well for humans, it'll go well for animals")^[1]

If I accept conventional assumptions in EA Animal welfare^[2], AGI will be negative for animals in expectation. On the other hand, AGI being good for humans makes it worse for animals in expectation. However, both rogue AGI and human-friendly AGI seem positive for animals in most scenarios: it just happens that the "bad" scenarios seem much worse than the "good" scenario.

Why is that? AGI, whether rogue or human-aligned, may not decide to keep other planets free of biological animals (though it seems like a bigger risk for human-aligned AGI). And EA Animal Welfare advocates generally believe that the likelihood that wild animal welfare is negative makes such spreading of biological animals too risky.

A small chance of this decision being made outweighs the positives. This seems very unlikely with rogue AGI (0.1%, perhaps much less), but it could still dominate the scales in my view. An AGI that is more human-friendly seems at least one order of magnitude more likely to terraform other planets.^[3]

That said, this doesn't flip the sign of AI safety work. This judgment is lightly held; digital minds (human-like or animal-like) are a larger portion of welfare patients in expectation; and I have no idea of what the counterfactuals are. Thus, I don't treat this as an action-guiding beliefs.

To caveat, I think terraforming is still relatively unlikely in human-friendly scenarios because biodiversity becomes less instrumentally valuable post-AGI, so memes that would favor the existence of wild animal populations would lose in popularity. Even in human lock-in scenarios, the values that control AGI won't favor deep ecology.

How about farmed animals? Even in precision Livestock Farming's best and worst cases, suffering in factory farms shifts by a few orders of magnitude at most.^[4] AGI makes the end of factory farming through developing alternatives more likely, though I'm more convinced by "biological food systems become unnecessary or unrecognizable" than "clean meat wins". In the vast majority of scenarios, wild animals would be the most numerous moral patients.^[5]

1. ^{^}

   Probably 0% on reflection because aliens could count as animals, but it's less indicative

2. ^{^}

   Farmed animal welfare is negative, wild animal welfare is negative, "good" and "bad" relate to expected total welfare

3. ^{^}

   Though what that looks like is still underdefined.

^{^}

However, precision livestock farming offers massive near-term risks and opportunities for farmed animals, and interest in this area appears justified.

^{^}

Human-friendly AGI could decide to only keep animals under human control, but that would probably not lead to massive animal populations.

Show all footnotes

This will be more of a loose collection of weakly held takes and what I see as cruxes for this question than a firm position.

• I think that for farmed animal welfare, the crux is "If all the technical barriers to cultivated meat / brainless chickens / something that can outcompete meat in the market are solved so that we reach the theoretical optimum for suffering-free meat or meat alternatives, will we still be in too much of an economic stable state due to economies of scale for regular meat to get outcompeted?"
• i.e. I have a strong intuition that the theoretically optimal way to produce meat or something that is experientially superior to it does not require raising sentient beings in horrific conditions, so I think whether things go well for farmed animals is mostly a function of economic or cultural lock-in
• For wild animal welfare, I worry significantly more about lock-in of e.g. the naturalistic fallacy that systems are good as they are and we should abstain from any interventions. Whether this dominates long-term animal welfare concerns depends on the relative numbers of wild vs farmed vs other (pet?) animals. I think wild animal welfare does become a tractable problem post-AGI, and the question will be whether there is will to solve it, and whether those with the will to solve it are able to (i.e. how much does this require global agreement/coordination that won't be possible due to lock-in?)
• Cruxes related to this: To what extent will animal advocates be enabled by AGI, to what extent will they need to be unilateralists, and to what extent could their actions backfire?
• AGI going well for animals seems to depend on how much it amplifies the agency of animal advocates, which is a channel that could be closed off due to power concentration (i.e. we may not see the world where AI is focused on problems like cultivated meat or wild animal welfare in certain worlds). That said, I would not classify power-concentrating outcomes that leave some humans without meaningful agency over the world as "good for humans".
• In general, I think this question as framed is going to produce a lot of false disagreement due to diverging definitions of "AGI going well for humans". For me, things like lock-in of current values, power concentration, and a loss of agency over the future are firmly in the bucket of "AGI did not go well for humans", which potentially cuts out a lot of the failure modes for animals.
• Crux: Can AI be robustly aligned to animal preferences in principle? There are a lot potential concerns around specification gaps (e.g. you optimize for a proxy of animal welfare, not the real thing) for the long-term future of AI and animals. That said, I don't think this is something a sufficiently powerful AI could not solve with sufficiently advanced technology. If preference / welfare is a physical structure in the brains of animals, a hypothetical AI could determine exactly what to optimize for without needing some external channel, though it might still need to calibrate its understanding of what structures correspond to preference based on e.g. behaviors that communicate preferences or analogies to creatures for which this is clearer.
• Crux: how many actors have terminal preferences for suffering? agency may be amplified for animal advocates, but it could also be amplified for malevolent actors.

Also, some arguments I do not quite buy:

• Stability of regulations, culture, and consumer preferences keeping cultivated meat or something similar off the table. I think post-AGI, a lot of things that are not baked into the trajectory of technology or the preferences of agents will get eroded. For instance, I do not expect particular state regulations over cultivated meat to have much stability into the long-term future. The exception here is if these things get locked in -- I am not immediately sure how likely this is, but I have a harder time imagining things like a terminal consumer preference for slaughtered meat or current regulations getting locked in than some broad moral intuition that humans are a higher moral priority because they are "smarter", conservationism vs welfarism w.r.t. to wild systems, etc.
• That because meat consumption is on the rise globally, post-TAI trends will look the same. I think this hinges too much on AI being a normal technology. If we had a strong reason to believe that it was physically impossible to produce meat / something better without suffering, this would change my mind, but I do not think this is true. I would put >85% probability on factory farming not existing 100 years post-AGI because I think optimizing for something consumable is more likely to optimize away the suffering than to keep it around by happenstance. I could see things getting worse near-term due to precision livestock farming until a phase shift later this century that makes vat meat economically competitive at scale. 

My position statement (50% agree with the statement "If AGI goes well for humans, it'll go well for animals")

• As a suffering-focused ethicist who generally rejects moral aggregation across individuals (I am most sympathetic to painism), I have a higher bar for “AGI going well for humans” for humans than many others do; it’s not clear to me that previous technological advances went well for humans
  • Agricultural revolution’s “luxury trap”: going from hunting-gathering to farming allowed humans to consolidate unprecedented wealth and power, but at the cost of the wellbeing/welfare/rights of very many humans
  • Perhaps similar arguments can be made for the industrial and digital revolutions
  • Even AGI Omelas is not an instance of AGI going well
• “AGI going well" necessarily leaves many humans the stated preference to help animals (which might look like "abolishing animal exploitation and solving wild animal suffering"), and it certainly gives us the means and opportunity to do so
• I happen to think that AGI going well for humans is unlikely, even by the lights of someone who is more upside-focused
  • We're on track for creating something that is more intelligent than us (better at understanding the world and achieving goals within it) – and probably something with awareness, autonomy, agency, and the capacity for recursive self-improvement and self-replication – without understanding how it works, how to make it do what we want, or what it is we even want it to do
• So, between normative and empirical claims, I believe a world in which AGI goes well for humans is a very small fraction of the possibility space
• And when I try to think about what this AGI-going-well-for-humans world looks like, mostly I don’t really know, but it seems likely that in this world:
  • We retain and develop our moral wisdom (the most fundamental tenet of which is plausibly “non-maleficence and compassion towards all sentient beings”)
  • And we have the means to enact this moral wisdom
  • So, we abolish animal exploitation and solve wild animal suffering
  • Thus, AGI goes well for animals as well as humans!

If you had to allocate a marginal $500,000, would you put it towards animal-specific alignment work (like the ideas in this list) or general alignment work?

Thanks for your contributions to the discussion @Hannah McKay🔸 , @Jo_🔸 , @Lee Wall , and @Alistair Stewart!

I have to head off at 7, but you are welcome to keep commenting, as is anyone else who sees this comment.

Anyone can post a comment, which our guests and other participants can respond to. These comments might be questions, the answer to which might change your mind on the debate statement, or crucial considerations that you are uncertain about, and might be able to make progress on in this conversation.