A selection of some writings and considerations on the cause of artificial sentience

Raphaël_Pesah

Thank you for writing about this. I am definitely a person whose concerns about AI are primarily about the massive suffering they might cause, especially when it comes to already-marginal entities or potential entities like non-human animals or digital minds.

I'll note beforehand that I'm suffering-focused, but I'll also note that I think even a regular utilitarian using EV reasoning could come to the same conclusions as I do.

I'm curious as to why this isn't a greater focus in the AI Safety community. At least from my vantage point and recollection, over 90% of the people who talk about AI Safety focus exclusively on the threat AI poses to the continued existence of humanity. If they elaborate at all on what's at stake in the far future, they emphasize the potential good that could come from having massive populations that are in immense states of bliss, which could be destroyed if we are destroyed (again this is my experience).

I think this rests on the assumption that there is a high likelihood (let's say >90% confidence) that humanity will become a force of net good in the long term future should it survive to see that. I think that, at the very least, this crux should be tested more than it currently is. I would argue that humanity of the current day is almost certainly (>99% confidence) net harmful (even factory farming alone is an immense harm that it's hard to argue any good humans do outweighs). I would also argue with similar confidence that humanity's net impact was consistently negative at least from the agricultural revolution onward (mistreatment/exploitation of non-human animals, slavery, war to name a few major things). Suffice it to say that I would be very worried if an AGI was locked-in with the values of a randomly selected person today (I know some AGI timelines are quite short), or even a randomly selected person 100 years from now (assuming we survive that long), especially if they decide to keep us alive. I can't give an estimate for how confident I am that humanity's continued existence with AGI would be a good/bad thing. However, I agree that the suffering risk from AGI is not emphasized proportional to its potential expected consequence, and I'm curious to hear EA/AI Safety perspectives regarding this topic.

I'll also quickly throw in the idea of humans deliberately creating malicious AGI with the intention of serving their own ends, which is an idea I've heard around a few times but know practically nothing about. Though I will say that I think the potential for such a scenario to arise and then become an S-risk is non-negligible (though I can't really give a good estimate or back it with anything more than intuition).

Kenneth_Diao

I'll note beforehand that I'm suffering-focused, but I'll also note that I think even a regular utilitarian using EV reasoning could come to the same conclusions as I do.

^{^}

For a discussion on the different terms used to refer to artificial entities, see The Terminology of Artificial Sentience by Janet Pauketat.

Also, in this overview article, what I mean when I write “the cause of artificial sentience” is something very broad, maybe like “any kind of work centered on taking seriously the implications of the possibility of artificial sentience”. But how one defines or conceives of the issue may be another important question.

^{^}

See Chapter 4 of Avoiding the Worst for a more detailed discussion.

^{^}

These risks are mentioned directly regarding artificial sentience p.83.

^{^}

Note that Jacy Reese Anthis has briefly written on what makes digital minds research different from other AI existential safety research.

But a reply to Buck’s comment would be that his point applies to some kinds of research relevant to artificial sentience, but not to other kinds. (H/T Jim Buhler)

^{^}

He will also publish a book titled The moral circle.

^{^}

Whose list of guests can be used to find additional people doing relevant work.

^{^}

Likely the members of FHI’s Digital Minds Research Group.

^{^}

The method described in the footnotes here may be a good source of inspiration.

^{^}

For instance, Jamie Harris writes: “This blog post lists possible crucial considerations affecting whether AS advocacy seems likely to be positive on balance, as well as lesser questions affecting how we might prioritize AS relative to other promising cause areas. We include questions that affect broader categories of priorities and intervention types that include at least portions of AS advocacy: longtermism, suffering risks, and moral circle expansion.”

And also: “The especially complex, technical, and futuristic (and thus easily dismissed) nature of AS advocacy suggests further caution, as does the unusually high leverage of the current context, given that advocacy, policy, and academic interest seems poised to increase substantially in the future. [Footnote: This raises the stakes of experimentation and the size of potential negative consequences, which is concerning given the unilateralist’s curse.]

Additionally, there are more uncertainties than in the case of animal advocacy. What “asks” should advocates actually make of the institutions that they target? What attitudes do people currently hold and what concerns do they have about the moral consideration of artificial sentience? What opportunities are there for making progress on this issue?”

Similarly, the great uncertainty we face when trying to reduce s-risks, including from artificial sentience, is repeatedly mentioned in Avoiding the Worst.

^{^}

From 37:30 to 42:30, he talks about that and then about the implications of uncertainty more generally.

Earlier in the podcast (from 13:48 to 15:34), he talks about the risks of losing credibility by "crying wolf" about artificial systems too often (i.e. warning about their sentience when they’re actually not).

Robert Long also mentions this issue: “In many ways, we are in our understanding of large language models where the study of animals was in the middle of the 20th century. Like animal cognition, the field of AI is overshadowed by founding traumas — cases in which credulity and anthropomorphism have led researchers to exaggerate and misconstrue the capabilities of AI systems. Researchers are well aware of the ELIZA effect, our tendency to readily read human-like intentionality into even very simple AI systems — so named for an early chatbot built in 1964 that used simple heuristics to imitate a psychoanalyst.”

Given the significant risks faced, it could be useful to investigate in more detail the extent to which different types of work on artificial sentience risk turning out detrimental, especially from an s-risks perspective, before prematurely pursuing interventions and research directions. (H/T Jim Buhler)

^{^}

Here are other mentions of risks from direct outreach and advocacy:

-Reasoned Politics, section 10.7.3 : “[...] the ideal strategy might be to first engage with scientists and ethicists whose work relates to these risks - a group of experts who can contribute important insights, and who are likely to be taken seriously by legislators (cf. Harris, 2021). An encouraging sign is that many such experts already show considerable concern for the issue (Harris & Reese, 2021). Even so, one would probably need to go about these efforts in a thoughtful manner, as one could easily frame the issue in a way that makes it seem alarmist or uninformed. Outright advocacy is likely ill-advised, as opposed to making reasoned arguments about the ethics of risk and the moral significance of suffering, including uncertain suffering (cf. Birch, 2017; Knutsson & Munthe, 2017; Ziesche & Yampolskiy, 2019; Metzinger, 2021).”

-Key Questions for Digital Minds: “What strategies are most promising for improving futures with digital minds?

All of the foundational research outlined above needs to ultimately be cashed out in better strategies for building the best future for all sentient beings. One tentative strategic claim is that research should be prioritized before other projects such as public policy or outreach. First impressions may be very important for digital minds as with other technosocial issues (e.g., lock-in of GMO and nuclear energy narratives), and there has been so little research on this topic that the most promising outreach strategies could easily change after only a few research projects. Before promoting a narrative or policy goal, such as a moratorium on digital consciousness, we should consider its direct viability and indirect effects of its promotion.

Delay should not be too long, however, because suboptimal narratives may take over in the meantime, especially with short timelines—making digital minds research a highly time-sensitive AI safety project. Discussion to date has arguably been largely confused and quite possibly detrimental. The most promising work informed by digital minds research may be preparation to push forcefully for certain technical and governance strategies during major advances in AI capabilities.”

^{^}

For research, see for instance section ‘Governance of AI’ of Chapter 10 and section ‘Research on how to best reduce s-risks’ of Chapter 11 in Avoiding the Worst, section 9.3.3 in Reasoned Politics, or section ‘Research’ in The Importance of Artificial Sentience. This article with a broader scope, but arguments that are relevant in the context of the cause of artificial sentience, also makes the case for more research (on the current margin).

For capacity and field-building, see for instance sections ‘Capacity building’ and ‘A movement to reduce s-risks’ of Chapter 11 in Avoiding the Worst, section ‘Recommendations for the movements for future generations of sentient beings’ in Key Lessons From Social Movement History, or section ‘Field-building’ in The Importance of Artificial Sentience. (Relatedly, here are some strategic suggestions for academic field-building.)

^{^}

Of course, that’s not at all to say that AI Welfare research is not useful, and as Jeff Sebo writes: “Research fields are path dependent, and which path they take can depend heavily on how researchers frame them during their formative stages of development. If researchers frame AI welfare research in the right kind of way from the start, then this field will be more likely to realize its potential.”

Relatedly, Amanda Askell writes: “I don't think we yet live in a world where AI labs are running the moral equivalent of animal experiments on their models. But I would like to live in a world where, over time, we have more evidence grounding the probabilities we assign to where we are on the scale of "we're doing no harm" to "we're doing harm equivalent to swatting a fly" to "we're doing harm equivalent to a large but humane mouse experiment" to "we're doing harm equivalent to a single factory farm" to "our RL agents are sentient and we've been torturing them for thousands of years.".

We are used to thinking about consciousness in animals, which evolve and change very slowly. Rapid progress in AI could mean that at some point in the future systems could go from being unconscious to being minimally conscious to being sentient far more rapidly than members of biological species can. This makes it important to try to develop methods for identifying whether AI systems are sentient, the nature of their experiences, and how to alter those experiences before consciousness and sentience arises in these systems rather than after the fact.”

^{^}

Also note the results from Protecting Sentient Artificial Intelligence: A Survey of Lay Intuitions on Standing, Personhood, and General Legal Protection and their discussion: “the fact that laypeople rate the desired level of legal protection to sentient AI as twice as high as the perceived current level, as well as the fact that the difference between the desired and perceived current level of protection was higher than virtually any other group [e.g. present humans, future humans, non-human animals, the environment] would imply (through this lens) that the existing legal institutions should be reformed so as to increase protection of sentient AI well beyond the current level afforded to them”, although “the majority of laypeople were not in favor of granting personhood or standing to sentient AI.”

Other writings that mention a focus on institutions or legal protection include Artificial sentience - Problem profile, section ‘Institutional reform’ of Chapter 9 and section ‘Governance of AI’ of Chapter 10 in Avoiding the Worst, section 14.5.1 in Reasoned Politics, and bullet point ‘Explore legal rights for artificial entities.’ in The History of AI Rights Research.

^{^}

Considering that, perhaps, a focus on “prevention strategies” is judicious.

On this, Magnus Vinding writes: “Metzinger recommends a policy of banning all research that “risks or directly aims at the creation of synthetic phenomenology”, as well as greater funding of research into the ethics and risks of creating novel systems capable of conscious experience (Metzinger, 2018, p. 3; 2021). A ban on attempts to create new kinds of suffering beings would also be well in line with the bounding approach to worst-case risks outlined in the previous chapter, as such a ban could help serve as a protective wall preventing us from getting into the “danger zone” where novel technologies can vastly increase suffering (cf. Baumann, 2019).”

In the context of wild-animal welfare, Brian Tomasik writes about an analogous preventive focus.

On a side note, (this is a common interrogation but) I wonder what lessons can be taken from movements or efforts that were focused on the prevention of bad - but not necessarily existentially catastrophic - outcomes, such as early successful efforts in space governance, nuclear non-proliferation or climate change advocacy.

^{^}

Anil Seth recently wrote this article.

^{^}

On a related note, the idea of targeted (moral) advocacy to the potential creators of artificial sentience, such as employees at top AGI labs, and sometimes more generally to influential decision-makers around AI, has been discussed several times over the years.

Tobias Bauman, for instance, mentions the idea of "try[ing] to expand the moral circle of the creators of AI" while pointing out the risks of backlash (section ‘Governance of AI’ of Chapter 10 in Avoiding the Worst). Also note that Jade Leung wrote, in her AI governance thesis in 2019, that she was expecting researchers' influence to shrink as the technology matures (see for instance section 7.4.3.3 ‘AI researchers will become less influential over time’).

The appeal of the idea may to some extent depend on how narrowly it is presented or construed. On one hand, the more narrow strategy of “influencing [AGI companies, by working for or lobbying them] with an eye toward 'shaping the AGI that will take over the world'” may sound simplistic. While on the other hand, the broad idea of directing overall efforts and outreach on relevant stakeholders seems uncontroversial. Jamie Harris, for example, tentatively proposes that “initial advocacy should focus primarily on influencers rather than mass outreach.”

^{^}

For instance, regarding what can be seen as credible advocacy, it would have been hard to imagine, not long ago, that such a cover image could be used for a legitimate BBC article. On artificial sentience specifically, see for instance this recent TIME article by Brian Kateman.

^{^}

Section 10.7.1 of Reasoned Politics also covers the strategy of establishing welfare biology as a proper field of study for helping wild animals.

^{^}

In addition to effective animal advocacy research and movement history research, more general social movement research, e.g. from Social Change Lab, could be strategically relevant (although it doesn't seem entirely clear yet what part of this research is the most useful).

In the context of movement history research, I’m curious which social movements, possibly from these lists, are the most comparable to artificial sentience advocacy - specifically, rather than to the broader category of movements encouraging moral circle expansion - based on these features, and potentially others.

^{^}

Regarding the places to pursue direct work, it’s interesting to see that Zach Freitas-Groff’s proposal of “[one or multiple organizations] dedicated to identifying and pursuing opportunities to protect the interests of digital minds.” received an "Honorable Mention" award (akin to a rank between 8 and 21 out of roughly 1000 project ideas) at the Future Fund’s Project Ideas Competition last year.

All else equal, it would be better if founder(s) of potential future projects related to artificial sentience fitted what is described here or here, while being particularly cautious of causing accidental harm, and not underestimating the challenge of tackling these types of problems. Also, in Career advice for reducing suffering, Tobias Baumann writes: “Lastly, you could consider founding a new project if you have a good idea, though we would not recommend that at the beginning of your career.”

A selection of some writings and considerations on the cause of artificial sentience

A selection of some writings and considerations on the cause of artificial sentience

Introduction

Pieces arguing artificial sentience is a pressing problem

Some arguments against focusing on this problem

Who is working on the problem

The Mind, Ethics, and Policy Program at NYU

Sentience Institute

Center for Reducing Suffering

MILA-FHI-University of Montréal’s Digital Minds Project

Some Philosophy Fellows at the Center for AI Safety

Other organizations and researchers

Some high-level strategic considerations

Uncertainty and risks

Broad research focus

Broad target objectives

Timing

Inspiration from other fields and movements

Conclusion: ways for individuals to contribute