Thanks for flagging this, and for generally engaging here. That's a sobering and generally depressing piece... It seems like Buck and I are definitely gesturing at similar things. This passage in particular stuck out to me:
But I don't think that's how most people actually work. For most people, if they have a choice between an object-level belief that is core to their identity and a meta-level principle like "believe what you'd believe if you were smarter and more informed," they will choose the object-level belief. Tell a devout Christian that superintelligent AI analysis suggests their faith is unfounded, and they won't abandon their faith, they'll abandon the AI (perhaps just for the competing AI that is just as good, and happy to tell them otherwise). Tell them that their children will be more successful at reaching reflective equilibrium if exposed to diverse viewpoints, and they'll question your definition of success, not their approach to parenting.
I was envisioning a similar dynamic playing out with meat consumption. You can argue to someone that a superintelligent AI analysis says that that cultivated meat is "the same" as real meat and it's even better in various ways, but if they have a specific attachment to a bucolic / naturalistic vision of animal husbandry ("my meat came from happy animals living in a pasture and had one bad day") then they might just cling to that. Perhaps in a superabundant world, AI will even be able to provide this to them.
I don't understand why you (and Ben / Lizka) think we shouldn't focus on farmed animals in a post-TAI world, can you explain a little more?
It seems to me that regardless of how weird the world gets (which I'm on board with), if AGI is aligned, then humans will still be around. And if the post-TAI humans are mostly the same as the pre-TAI humans, what makes you confident that they wouldn't want meat from farmed animals?
Looking at current human preferences around animal products, there's a strong "naturalistic" push - people want their animal products to come from environments that are as "natural" as possible (e.g. outdoor access, no hormones or antibiotics, etc). It seems like lots of people think cultivated meat is weird and gross, and could feel the same way about any other kind of technology that looks significantly different than traditional animal production. Perhaps TAI will be able to convince people to not feel this way, but it seems just as likely to me that this preference will be amplified and animal protein production will look more similar to the current day than you're anticipating.
You call out the excerpt about the political pushback to cultivated meat, and I agree with you that this probably isn't what will cause cultivated meat to fail post-TAI. More likely in my mind is that people won't want cultivated meat (as evidenced by the fact that they currently don't want it), or any other super tech-y seeming solution to protein production. So it seems to me that thinking about what farming might look like post-TAI at least deserves a spot on the list of possible strategies for for having positive impact on animals in light of ASI.
A couplet different potential mechanisms could help farmed animals:
More abstractly, people generally care about welfare so it will be one of the things that an aligned AGI optimizes for. However, it wont be optimal for animals because AGI won't be directly optimizing for welfare. For example, most people don't think it's wrong to eat meat, and we might still not want do things like beneficial vaccines or genetic edits.
Wild animals, less clear though!
This seems like an interesting and important point, thanks for writing.
I might nitpick the way you're characterizing the terrestrial animal case though. Layers and broilers may technically be the same species, but I think they're different enough that a lot of the same considerations apply from the fish case. For example, you mention cage-free as the paradigm scalable intervention, but actually they only apply to layers, which I'm guessing constitute less than 5% of global terrestrial animals (most are broilers). Applying a similar intervention to broilers (BCC) has been less successful than for layers.
Speaking just about the US, I would say there are actually four groups of poultry worth considering. The following are their population sizes and days spent on form from this population per year (US):
This is much more homogenous than aquatic animals, but it's not quite as homogenous as you made it seem!
Academic studies are definitely slow, but 3-5 years strikes me as extremely slow, even for academia.
I'm generally on board with what you're describing but I wonder whether there's also opportunities to work better with academia? Like if you're the one providing the funding, you might be able to negotiate with them and keep them accountable to timelines. There's probably also lots of variability between academics so you can identify the ones that are capable of executing well and quickly, and then work with them.
I've seen AI-based animal communication technologies starting to be involved in some EA events / discussions (e.g. https://www.earthspecies.org/ ). I'm worried these initiatives may be actively negative, and I'm wondering if anyone has / will articulate a stronger defense of why they're good?
The high-level argument I've heard is that communicating with animals will make humans be more empathetic towards them. But I don't see why this would be the most likely outcome:
A similar argument is that communication would allow us to see that animals are actually intelligent, but again I don't see why this is necessarily the case. If their thoughts are things people would generally consider crude, it's possible people would become more confident in their lack of intelligence (despite still deserving moral consideration).
More importantly, a large effect of being able to communicate with animals is that they'll become more useful to humans. If animals had political power or legal rights, this might open the door to mutually beneficial trade. But in reality, they don't have these things, so it seems more likely that this would allow humans to exploit these species more easily. They reason chickens, cows, and pigs are in such a bad state is because they're very useful to humans, and I'm worried animal communication technologies will subject more species to similar fates.
It doesn't seem to me like AI could radically change the economics of higher welfare products / alternative proteins without other broader transformative effects, or at least if it does, it seems like it would be for a pretty brief period of time before things get very weird. So I think "past X years" ( to use the framework of your comments) there should be heavy discounting and I wouldn't recommend a save and invest strategy. So to address your four types of interventions:
Organisations using Rethink Priorities’ mainline welfare ranges should consider effects on soil nematodes, mites, and springtails.
I appreciate you championing this view Vasco, despite all the pushback. I found your reasoning pretty convincing, and it seems to me like if it's wrong, it will be because of more general philosophical problems with utilitarianism or expected value reasoning.
Wow that's super interesting, and not what I would have expected. I appreciate you making this explicit.
My first reaction is that you and I are talking to different groups of people. I frequently encounter the claim that AI will "solve cultivated meat" and so we shouldn't worry about farmed animals as long as AI doesn't kill everyone. That's what I was mainly reacting to here. I don't work in AI safety though, so I wouldn't be surprised if the group of people you talk to has thought about it more than the group of people I talk to.
My second reaction is that I don't understand how biological humans disappearing is different than the disastrous x-risk scenarios that I associate EAs as trying to avoid. Like it seems like EAs are worried about AI scheming in ways that would cause them to make a power grab against humanity's interest. If instead humans disappear for other (perhaps less violent) reasons, how is that then consistent with AI safety?
Maybe the answer is that humans would be succeeded by some other form of life that are sufficiently similar to humans that it would be better for that form of life to exist than squiggles?
If that's the claim (which I actually find plausible), then I agree animals would no longer be farmed. However, it seems like this would fall into the bucket of "not actionable unless you work directly on AI," so it seems like it might be practically useful to act as if this wasn't going to be the case?
(Also, this claim feels at odds with what I understood your perspective to be from the shallow review you did a while ago. I haven't had a chance to go back and more carefully read that piece, and maybe on a closer reason it will all look consistent. But I'm just flagging that I still can't fully model your perspective)