Yeah that seems correct and I think that my post tomorrow is probably guilty of this. I'd even say I'm doing it intentionally to try to pull the best new founders into this area. I think the actual outlook for helping animals is just terribly bleak across the board and that's probably not going to inspire people to join welfare tech incubators.
Bubble in the sense of railroad bubbles successfully building railroads we still use, dot-com bubble laying fiber optic cable, etc. It could be a perfectly wise action for us to direct most of our spare energy/resources in this direction for a while.
The early research behind leafleting was low-quality in a way that should have been noticeable, and a better-calibrated version of the movement would not have spent so much time and money on it. Whereas the cage-free bubble was a necessary way of getting a massive worldwide campaign off the ground. I'd like use to have higher standards for our fads than we did for leafletting but I think the fad phenomenon is inevitable for a movement as constrained on talent and resources as we are.
I think the bubble framing is a useful and accurate description of welfare tech excitement. I just don't think that's a bad thing. From a longer post on this topic I'm publishing tomorrow:
From the outside, it might seem like the EA wing of the animal advocacy movement goes through fads. That’s because it does. In each cycle, funders get excited about a shiny new object, and put out a call for established organizations and new startup founders to rush into that space by making it clear funding will be available for anyone who does. Vegan leafleting was one such fad, followed by corporate cage-free campaigns, then cultivated meat and alternative proteins in general. Politics is the current fad, and welfare innovation will be the next one.
This can be frustrating for people working on an intervention that never gets popular. But that doesn’t mean it’s actually suboptimal. Some of our fads have worked out, and others haven’t; so it goes.
Animal advocates have limited resources, both money and people, spread thin across researchers, campaigners, and grantmakers. It’s probably the case that we needed to put all of our spare energy into cage-free campaigning to get the ball rolling there, and indeed, while the cage-free fight has been a big success, it’s hardly over despite all the energy invested. I’m glad that increasing our political capacity has been a major influence for the last few years and that it will continue to get attention.
But I’m also glad that there’s about to be a major infusion of money and attention into welfare innovation. And because of that, it’s quickly becoming clear that the pace of this innovation will be limited not by funding, but by talent.
You're splitting this the same way I meant to, though the quote you point out wasn't as clear as it could have been: in strictly verifiable domains, AIs will eventually be so much better than humans that having humans involved will only lead to worse decisions. But the line between verifiable and unverifiable is gradual and fuzzy. My expectation in practice is that humans will develop such a strong habit of deferring to AIs that even if there are still humans "in charge" of certain things in the sense that they could implement one decision or another, they will not effectively differentiate the domains in which they by their own lights should maintain informed control, and will end up giving over complete control.
Keeping humans in the loop is a bigger question! I treated "humans eventually wind up not in the loop" as a premise for this post in part because I think it's the default and in part because my primary audience of people focused on anima welfare probably don't have the means to help keep humans in control, but maybe that's giving up too easily.
I have observed the same sharp distinction as @ElliotTep: EAs not focused on animal welfare accept that TAI will end factory farming with something like Matt Reardon's 99% certainty, while most focused on animal welfare (myself included) feel the risk of factory farming being perpetuated into the future is unacceptably high. This is probably a mix of several factors:
Animal welfare people having some bias to think our work is still important
Animal welfare people having a lower threshold for unacceptably high risk of this particular outcome
EAs outside animal welfare having less firsthand experience of frustration at the ways humans today seem to favor meat from tortured animals over cultivated meat
Animal welfare people having thought less about what extremely weird posthuman futures look like and how strongly TAI might pull in that direction
EAs outside animal welfare being overly confident about the extremely weird posthuman future
I'm focused on animal welfare and have been for 12 years. But I've also read my Greg Egan and think it is less likely than not that biological humans will continue to exist into the far future. But I still believe that ensuring AI ends factory farming rather than perpetuating it should be a top focus area for EA. I might try to think through this and write something better, but off the cuff reasons:
It's possible biological substrates are necessary for consciousness/sentience. My credence on this is maybe 15% which I consider very low given our cluelessness about consciousness. If silicon platforms can't support consciousness, then Von Neumann probes will probably carry gene sequences to other planets to seed biological civilizations. Low probabilities of galaxy-scale factory farming still look very important!
Maybe the weird posthuman future will get the rest of the lightcone and leave earth for biologicals. In that case it could be a thousand years before digital minds colonize the nearest star system. That's a lot of time where factory farming could still represent a share of conscious experience within an order of magnitude of today's. Any model that discounts future beings even just due to uncertainty about our ability to affect their welfare seems likely to make factory farming remain a top priority.
Adding to what Other Aidan said, I think it's a mistake to think of the point on the spectrum from most to least helpful interventions where the real impact crosses from "very slightly helpful" to "very slightly harmful" as especially significant. If intervention A is worth 10,000 units, B is 1 unit, and C is -10 units, the difference between A and B matters much more than between B and C.
Congrats/good luck on soon entering the workforce!
I think OP is a valuable contribution to an ongoing discussion, but I'd encourage you not to update too aggressively on any one post. There are good reasons that some of the most rigorous organizations in the space (CG) have made cage-free a top priority and ex ante you should think it unlikely that they all converged without good reasons– see e.g. @Vasco Grilo🔸's comment above.
If you end up reading it I'd particularly value your feedback on how I'm striking this balance.
Yeah that seems correct and I think that my post tomorrow is probably guilty of this. I'd even say I'm doing it intentionally to try to pull the best new founders into this area. I think the actual outlook for helping animals is just terribly bleak across the board and that's probably not going to inspire people to join welfare tech incubators.
lol I actually held it back a week because of some infohazard concerns and I'm kicking myself for not beating you to the punch
Bubble in the sense of railroad bubbles successfully building railroads we still use, dot-com bubble laying fiber optic cable, etc. It could be a perfectly wise action for us to direct most of our spare energy/resources in this direction for a while.
The early research behind leafleting was low-quality in a way that should have been noticeable, and a better-calibrated version of the movement would not have spent so much time and money on it. Whereas the cage-free bubble was a necessary way of getting a massive worldwide campaign off the ground. I'd like use to have higher standards for our fads than we did for leafletting but I think the fad phenomenon is inevitable for a movement as constrained on talent and resources as we are.
I think the bubble framing is a useful and accurate description of welfare tech excitement. I just don't think that's a bad thing. From a longer post on this topic I'm publishing tomorrow:
You're splitting this the same way I meant to, though the quote you point out wasn't as clear as it could have been: in strictly verifiable domains, AIs will eventually be so much better than humans that having humans involved will only lead to worse decisions. But the line between verifiable and unverifiable is gradual and fuzzy. My expectation in practice is that humans will develop such a strong habit of deferring to AIs that even if there are still humans "in charge" of certain things in the sense that they could implement one decision or another, they will not effectively differentiate the domains in which they by their own lights should maintain informed control, and will end up giving over complete control.
Keeping humans in the loop is a bigger question! I treated "humans eventually wind up not in the loop" as a premise for this post in part because I think it's the default and in part because my primary audience of people focused on anima welfare probably don't have the means to help keep humans in control, but maybe that's giving up too easily.
I have observed the same sharp distinction as @ElliotTep: EAs not focused on animal welfare accept that TAI will end factory farming with something like Matt Reardon's 99% certainty, while most focused on animal welfare (myself included) feel the risk of factory farming being perpetuated into the future is unacceptably high. This is probably a mix of several factors:
I'm focused on animal welfare and have been for 12 years. But I've also read my Greg Egan and think it is less likely than not that biological humans will continue to exist into the far future. But I still believe that ensuring AI ends factory farming rather than perpetuating it should be a top focus area for EA. I might try to think through this and write something better, but off the cuff reasons:
Adding to what Other Aidan said, I think it's a mistake to think of the point on the spectrum from most to least helpful interventions where the real impact crosses from "very slightly helpful" to "very slightly harmful" as especially significant. If intervention A is worth 10,000 units, B is 1 unit, and C is -10 units, the difference between A and B matters much more than between B and C.
Congrats/good luck on soon entering the workforce!
I think OP is a valuable contribution to an ongoing discussion, but I'd encourage you not to update too aggressively on any one post. There are good reasons that some of the most rigorous organizations in the space (CG) have made cage-free a top priority and ex ante you should think it unlikely that they all converged without good reasons– see e.g. @Vasco Grilo🔸's comment above.