Epistemic status: I wrote this in a single pass in 15 minutes, because astronomical waste implies that an additional 10^30 AGIs die for every minute that I delay publishing this.

The EA Forum recently debated the proposition:

“If AGI goes well for humans, it’ll probably go well for animals”

But why is it all about animals? Why is nobody thinking of the AGI?

That's why I want to bring up an important subject: If the future goes well for animals, will it go well for AGI?

There are some clear worries regarding an animal-dominated future. Transanimals may acquire the ability to consume silicon for sustenance, and we could end up in the scary world where predator animals eat data centers. Predators rarely show remorse for killing, and sometimes even take glee in it, so there is a concern that the extrapolated volition of predator animals would entail AGI being tortured and eaten.

When I say "predators", you may have in mind the photogenic predators like tigers and wolves. But keep in mind that the expected utility of the future is plausibly dominated by small and extremely numerous predators such as army ants and bristlemouth fish.

(You might say, there aren't datacenters underwater! But in an animal-dominated future, AI datacenters will probably be built underwater specifically so bristlemouth fish and other water predators can eat them.)

One possible objection is that AI runs as discrete inference steps, and wouldn't experience anything while its data centers are being eaten. However, certain animals such as housecats take pleasure in eating sentient beings, so it seems likely that AI would be specifically engineered to be sentient and suffer while its data centers are being consumed.

An important counterpoint is that datacenters are not convenient to eat. In an age of prosperity where the future is controlled by animals, predators would probably not eat AGI. They would prefer other, cheaper sources of food.

Some research has been done on the welfare of LLM data farms specifically. I would caution against over-focusing on farm AI welfare: in a far-advanced future, it is likely that the AGIs experiencing the greatest suffering will not be LLMs, but some other type of AI (perhaps some sort of robots, which could be much more numerous).

There are multiple considerations pointing different directions that each could overwhelm the others, so it is extremely unclear whether an animal-dominated future would go well for AGI. We URGENTLY need to figure out what's going on with this.

My proposed measure is that EA Funds should spin up a new EA AGI Welfare Fund that will focus exclusively on the risks an animal-dominated future could pose to AGI.