You're splitting this the same way I meant to, though the quote you point out wasn't as clear as it could have been: in strictly verifiable domains, AIs will eventually be so much better than humans that having humans involved will only lead to worse decisions. But the line between verifiable and unverifiable is gradual and fuzzy. My expectation in practice is that humans will develop such a strong habit of deferring to AIs that even if there are still humans "in charge" of certain things in the sense that they could implement one decision or another, they will not effectively differentiate the domains in which they by their own lights should maintain informed control, and will end up giving over complete control.

Keeping humans in the loop is a bigger question! I treated "humans eventually wind up not in the loop" as a premise for this post in part because I think it's the default and in part because my primary audience of people focused on anima welfare probably don't have the means to help keep humans in control, but maybe that's giving up too easily.

I have observed the same sharp distinction as @ElliotTep: EAs not focused on animal welfare accept that TAI will end factory farming with something like Matt Reardon's 99% certainty, while most focused on animal welfare (myself included) feel the risk of factory farming being perpetuated into the future is unacceptably high. This is probably a mix of several factors:

• Animal welfare people having some bias to think our work is still important
• Animal welfare people having a lower threshold for unacceptably high risk of this particular outcome
• EAs outside animal welfare having less firsthand experience of frustration at the ways humans today seem to favor meat from tortured animals over cultivated meat
• Animal welfare people having thought less about what extremely weird posthuman futures look like and how strongly TAI might pull in that direction
• EAs outside animal welfare being overly confident about the extremely weird posthuman future

I'm focused on animal welfare and have been for 12 years. But I've also read my Greg Egan and think it is less likely than not that biological humans will continue to exist into the far future. But I still believe that ensuring AI ends factory farming rather than perpetuating it should be a top focus area for EA. I might try to think through this and write something better, but off the cuff reasons:

1. It's possible biological substrates are necessary for consciousness/sentience. My credence on this is maybe 15% which I consider very low given our cluelessness about consciousness. If silicon platforms can't support consciousness, then Von Neumann probes will probably carry gene sequences to other planets to seed biological civilizations. Low probabilities of galaxy-scale factory farming still look very important!
2. Maybe the weird posthuman future will get the rest of the lightcone and leave earth for biologicals. In that case it could be a thousand years before digital minds colonize the nearest star system. That's a lot of time where factory farming could still represent a share of conscious experience within an order of magnitude of today's. Any model that discounts future beings even just due to uncertainty about our ability to affect their welfare seems likely to make factory farming remain a top priority.

Adding to what Other Aidan said, I think it's a mistake to think of the point on the spectrum from most to least helpful interventions where the real impact crosses from "very slightly helpful" to "very slightly harmful" as especially significant. If intervention A is worth 10,000 units, B is 1 unit, and C is -10 units, the difference between A and B matters much more than between B and C.

Congrats/good luck on soon entering the workforce!

I think OP is a valuable contribution to an ongoing discussion, but I'd encourage you not to update too aggressively on any one post. There are good reasons that some of the most rigorous organizations in the space (CG) have made cage-free a top priority and ex ante you should think it unlikely that they all converged without good reasons– see e.g. @Vasco Grilo🔸's comment above.

I strongly agree that the animal welfare implications of AI should be owned at least as much by the AI safety space as by the animal welfare space, not that there needs to be a hard distinction between the two but there is obviously some declarative truth to it. Animal suffering is among the greatest lock-in risks.

I'm worried that many people outside the AW space believe the end of factory farming is a foregone conclusion. At one forum in SF in February mixing leaders from AI safety and AW, many AIS folks came away at least partly convinced by the AW folks that this cannot be taken for granted. AIS needs to take seriously the inside view of AW leaders that AI will not necessarily solve FAW by default, not to mention WAW.

Fair, the semantics of "going well" ultimately don't matter so I think this is clear.

Most animals are wild animals, so the answer to this question should focus on them.

I can imagine a future where most animals are farmed animals. I'm not saying it's particularly likely, but if humans spread to other planets, I think we're more likely to take factory farming with us than take nature with us. Farmed animals should be part of this convo.

From @Tristan Katz :

Most animals are wild animals, so the answer to this question should focus on them.

disagree

agree

Does WAW dwarf FAW in expectation? Or is FAW still important to consider in this discussion?

I can imagine a future where most animals are farmed animals. I'm not saying it's particularly likely, but if humans spread to other planets, I think we're more likely to take factory farming with us than take nature with us. Farmed animals should be part of this convo imo.

How am I supposed to predict whether the solution will be good for animals if I have no idea what that solution will look like?

That about sums it up haha.