(Not an AI welfare/safety expert by any stretch, just adding my two cents here! Also I was very piqued by the banner and loved hovering over the footnote! I've thought about digital sentience, but this banner and this week really put me into a "hmm..." state)
My view leans towards "moderately disagree." (I fluctuated between this, neutral, and slightly agree.) For context, when it's AI safety, I'd say "highly agree." Thoughts behind my current position:
Why I'd prioritize it less:
- I consider myself longtermist, but I have always grappled with the opportunity costs of highly prioritizing more "speculative" areas. I care about high EV areas, I also grapple with deprioritizing very tangible cause areas with existing beings that have high EV too. When I looked at the table below, I'd lean towards giving more resources towards AW versus making AI welfare right now a priority.
- I also think about how, if we divert more resources into AI welfare, I worry about the ramifications of diverting more EAs into a very dense, specialized sector. While this specialization is important, I am concerned that it might sometimes lead to a narrower focus that doesn't fully account for the broader, interconnected systems of the world. In contrast, fields like biosecurity often consider a wider range of factors and have a more integrative perspective. This more holistic view can be crucial in addressing complex, multifaceted issues, and one reason I would prioritize AI welfare less is the opportunity costs towards areas that may be more holistic (not saying AI welfare has no reason to be considered holistic)
- I have some concerns that trying to help AI right now might make things worse since we don't fully know yet what's being done now that can make things riskier? (Nathan said something to this effect in this thread).
- I don't know to what extent AI welfare is irreversible compared to unaligned AI
- It seems less likely for multiplanetary civilizations to develop with advanced AI, reducing likelihood of AI systems across the universe, which reduces my prioritizing AI welfare on a universal scale
Why I'd still prioritize it:
- I can't see myself prescribing a 0% chance AI would be sentient, and I can't see myself prescribing less than (edit:) 2% of resources and talent in effective altruism to something wide-scale I'd hold a possibility of being considered sentient, even if it might be less standard (i.e. more outside average moral circles) because of big value creation, just generally preventing suffering, and potentially preventing additional happiness, all of which I'm highly for.
- I think exploratory and not very tapped-in work needs to be done more, and just establishing enough baseline infrastructure is important for this high EV type of cause (assuming we would say AI will be very widespread)
- I like the trammell's animal welfare analogy
Overall, I agree that resources and talent should be allocated to AI welfare because it's prudent and can prevent future suffering. However, I moderately disagree with it being an EA priority due to its current speculative nature and how I think AI safety. I think AI safety and solving the alignment problem should be a priority, especially in these next few years, though, and hold some confidence in preventing digital suffering.
Other thoughts:
- I wonder if there'd ever be a conflict between AI welfare and human welfare or the welfare of other beings. Haven't put much thought here. Something that immediately comes to mind is if advanced AI systems would require substantial energy and infrastructure, potentially competing with human needs. From a utilitarian point of view, this presents a significant dilemma. However, there's the argument that solving AI alignment could mitigate these issues, ensuring that AI systems are developed and managed in ways that do not harm human welfare. My current thinking is that there's less likely potential for conflict between AI and human welfare if we solve the alignment problem and improve the policy infrastructure around AI. I might compare bioethics to historical precedents, showing that ethical alignment leads to better welfare outcomes
- Some media that have made me truly feel for AI welfare are "I, Robot," "Her," Black Mirror's "Joan is Awful," and "Klara and the Sun”!
Ok--at Toby's encouragement, here are my thoughts:
This is a very old point, but to my mind, at least from a utilitarian perspective, the main reason it's worth working on promoting AI welfare is the risk of foregone upside. I.e. without actively studying what constitutes AI welfare and advocating for producing it, we seem likely to have a future that's very comfortable for ourselves and our descendants--fully automated luxury space communism, if you like--but which contains a very small proportion of the value that could have been created by creating lots of happy artificial minds. So concern for creating AI welfare seems likely to be the most important way in which utilitarian and human-common-sense moral recommendations differ.
It seems to me that the amount of value we could create if we really optimized for total AI welfare is probably greater than the amount of disvalue we'll create if we just use AI tools and allow for suffering machines by accident, since in the latter case the suffering would be a byproduct, not something anyone optimizes for.
But AI welfare work (especially if this includes moral advocacy) just for the sake of avoiding this downside also seems valuable enough to be worth a lot of effort on its own, even if suffering AI tools are a long way off. The animal analogy seems relevant: it's hard to replace factory farming once people have started eating a lot of meat, but in India, where Hinduism has discouraged meat consumption for a long time, less meat is consumed and so factory farming is evidently less widespread.
So in combination, I expect AI welfare work of some kind or another is probably very important. I have almost no idea what the best interventions would be or how cost-effective they would be, so I have no opinion on exactly how much work should go into them. I expect no one really knows at this point. But at face value the topic seems important enough to warrant at least doing exploratory work until we have a better sense of what can be done and how cost-effective it could be, only stopping in the (I think unlikely) event that we can say with some confidence that the best AI welfare work to be done is worse than the best work that can be done in other areas.
When telling stories like your first paragraph, I wish people either said "almost all of the galaxies we reach are tiled with some flavor of computronium and here's how AI welfare work affected the flavor" or "it is not the case that almost all of the galaxies we reach are tiled with some flavor of computronium and here's why."
The universe will very likely be tiled with some flavor of computronium is a crucial consideration, I think.
To my mind, the first point applies to whatever resources are used throughout the future, whether it’s just the earth or some larger part of the universe.
I agree that the number/importance of welfare subjects in the future is a crucial consideration for how much to do longtermist as opposed to other work. But when comparing longtermist interventions—say, splitting a budget between lowering the risk of the world ending and proportionally increasing the fraction of resources devoted to creating happy artificial minds—it would seem to me that the “size of the future” typically multiplies the value of both interventions equally, and so doesn’t matter.