Summary:

• Digital sentience might be on the horizon, bringing with it an inevitable debate and significant risks. 
• Philosophers research whether AIs can be sentient and should deserve protection. Yet, ultimately, what matters most is what key decision-makers and the general public will think. 
• People will disagree on whether AIs are sentient and what types of rights they deserve (e.g., harm protection, autonomy, voting rights).
• Some might form strong emotional bonds with human-like AIs, driving a push to grant them rights, while others will view this as too costly or risky.
• Given the high stakes and wide scope of disagreement, national and even global conflicts are a possibility.
• We must navigate a delicate balance: not granting AIs sufficient rights could lead to immense digital suffering, while granting them rights too hastily could lead to human disempowerment.

The arrival of (potentially) sentient AIs 

We could soon have sentient AI systems — AIs with subjective experience including feelings such as pain and pleasure. At least, some people will claim that AIs have become sentient. And some will argue that sentient AIs deserve certain rights. But how will this debate go? How many will accept that AIs are sentient and deserve rights? Could disagreement lead to conflict?

In this post, I explore the dynamics, motives, disagreement points, and failure modes of the upcoming AI rights debate. I also discuss what we can do to prepare.

Multiple failure modes

In this post, I focus in particular on how society could disagree about AI rights. For context, the most obvious risks regarding our handling of digital sentience are the following two (cf. Schwitzgebel, 2023).

First, there is a risk that we might deny AIs sufficient rights (or AI welfare protections) indefinitely, potentially causing them immense suffering. If you believe digital suffering is both possible and morally concerning, this could be a monumental ethical disaster. Given the potential to create billions or even trillions of AIs, the resulting suffering could exceed all the suffering humans have caused throughout history. Additionally, depending on your moral perspective, other significant ethical issues may arise, such as keeping AIs captive and preventing them from realizing their full potential.

Second, there is the opposite risk of granting AIs too many rights in an unreflective and reckless manner. One risk is wasting resources on AIs that people intuitively perceive as sentient even if they aren’t. The severity of this waste depends on the quantity of resources and the duration of their misallocation. However, if the total amount of wasted resources is limited or the decision can be reversed, this risk is less severe than other possible outcomes. A particularly tragic scenario would be if we created a sophisticated non-biological civilization that contained no sentience, i.e., a “zombie universe” or “Disneyland with no children” (Bostrom, 2014). 

Another dangerous risk is hastily granting misaligned (or unethical) AIs certain rights, such as more autonomy, that could lead to an existential catastrophe. For example, uncontrolled misaligned AIs might disempower humanity in an undesirable way or lead to other forms of catastrophes (Carlsmith, 2022). While some might believe it is desirable for value-aligned AIs to eventually replace humans, many take-over scenarios, including misaligned, involuntary, or violent ones, are generally considered undesirable. 

As we can see, making a mistake either way would be bad, and there’s no obvious safe option. So, we are forced to have a debate about AI rights and its associated risks. I expect this debate to come, potentially soon. It could get big and heated. Many will have strong views. And we have to make sure it goes well.

A period of confusion and disagreement

Determining AI sentience is a complex issue. Since AIs differ so much from us in terms of their composition and mechanisms, it is challenging to discern whether they are truly sentient or simply pretending to be. Already now, many AI systems are optimized to come across as human-like in their communication style and voice. Some AIs will be optimized to trigger empathy in us or outright claim they are sentient. But of course, that doesn’t mean they actually are sentient. But at some point, AIs could plausibly be sentient. And then, it becomes reasonable to consider whether they should be protected against harm and deserve further rights.

Philosophers explore which types of AIs could be sentient and which types of AIs should deserve what type of protection (Birch, 2024; Butlin et al., 2023; Sebo, 2024; Sebo & Long, 2023). But solving the philosophy of mind and the moral philosophy of digital sentience is very hard. Reasonable people will probably disagree, and it’s unclear whether experts can reach a consensus.

Today, most experts believe that current AI systems are unlikely to be sentient (Bourget & Chalmers, 2023; Dehaene et al. 2021). Some believe that more sophisticated architectures are needed for sentience, including the ability to form complex world models, a global workspace, embodiment, recurrent processing, or unified agency with consistent goals and beliefs (Chalmers, 2023). All of these features could, in principle — perhaps with additional breakthroughs, be built, unless you think biology is required. A survey with philosophers found that 39% consider it possible that future AI systems could be conscious, while 27% consider it implausible (PhilSurvey; Bourget & Chalmers, 2023).

Many philosophers agree that sentience is a necessary criterion for moral status. However, they may disagree on which specific types of sentience matter morally, such as the ability to experience happiness and suffering or other forms of subjective experiences (Chalmers, 2023). Some believe that features other than sentience matter morally, such as preferences, intelligence, or human-level rationality. Some believe that AIs with certain mental features deserve rights analogous to those that humans enjoy, such as autonomy.

Ultimately, what matters most is what key decision-makers and the general public will think. Public opinion plays an important role in shaping the future, and it’s not obvious whether people will listen to what philosophers and experts have to say. The general public and experts could be divided on these issues, both internally and in relation to each other. We could have cases where AIs intuitively feel sentient to most lay people, whereas experts believe they are merely imitating human-like features without having the relevant sentience-associated mechanisms. But you can also imagine opposite cases where AIs intuitively don’t appear sentient, but experts have good reasons to believe they could be.

It’s hard to anticipate the societal response to sentient AI. Here, I make one key prediction: People will disagree a lot. I expect there to be a period of confusion and disagreement about AI sentience and its moral and legal implications. It’s very unlikely that we will all simultaneously switch from believing AIs aren’t sentient and don’t deserve rights to believing that they are and do.

How do people view AI sentience today?

Little research has been done on laypeople’s views so far (cf. Ladak, Loughnan & Wilks, 2024). Evidence supports the notion that perceived experience and human-like appearance predict attributions of moral patiency to AIs (Ladak, Harris, et al., 2024; Nijssen et al., 2019). However, people differ widely in the extent to which they grant moral consideration to AIs (Pauketat & Anthis, 2022). In a representative U.S. survey, researchers from the Sentience Institute (Pauketat et al., 2023) investigated the general public’s beliefs about AI sentience. They found that the majority of the population doesn’t believe current AI systems are sentient. However, a small minority do appear to believe that some current AI systems, including ChatGPT, already have some form of sentience. 24% don’t believe it will ever be possible for AIs to be sentient, 38% believe it will be possible, and the rest are unsure. While more than half of the population is currently against granting AIs any legal rights, 38% are in favor. Another survey found that people are more willing to grant robots basic rights, like access to energy, than civil rights, like voting and property ownership (Graaf et al., 2022).

My colleagues and I recently conducted a survey on AI sentience with both AI researchers and the US public. In one question, we asked participants what protection AI systems should receive, assuming they had subjective experience. The figures below show that both groups had wide internal disagreements. People’s views will likely change in the future, but these findings show that the basis for disagreement is already present. 

Two sides

I expect that strong psychological forces will pull people in opposite directions. 

Resistance to AI rights 

It is plausible that most people simply won’t care about AIs. In-group favoritism is a strong human tendency, and our historical track record of moral circle expansion isn’t great. People may perceive AIs as too different. After all, they are digital, not organic. Despite their ability to appear human-like, their underlying mechanisms are very different, and people might sense this distinction. As a comparison, we haven’t made much progress in overcoming speciesism. Substratism — the tendency to prioritize organic over digital beings — could be similarly tenacious. 

In addition to simply not caring, there are strong incentives for not granting AIs rights.

One key driver will be financial interests. People and companies might not want to grant AI rights if it’s costly or otherwise personally burdensome. For example, if it turns out that ensuring the well-being of AIs limits profitability, entrepreneurs and shareholders would have strong motives not to grant AIs any rights. Similarly, consumers might not want to pay a premium for services just to avoid AI suffering. The same could apply to granting AIs more autonomy. Companies stand to gain trillions of dollars if they own AI workers, but this potential value is lost if the AIs own themselves.  

Another driver could be a safety concern. Already now, there’s a growing group of AI safety activists. While they currently aren’t concerned about granting AIs more rights, it’s possible that they could in the future if there is a perceived trade-off between granting AIs more rights and ensuring humanity's safety. For example, some might fear that granting AIs more rights, such as autonomy, could lead to gradual or sudden human disempowerment. The government could consider it even a national security risk.

Support for AI rights

In-group favoritism, financial interests, and safety concerns might suggest that everyone will oppose AI rights. However, I find it plausible that many people will also, at least in part, want to treat AIs well and grant them certain rights.

Some people will have ethical concerns about mistreating AIs, similar to how some people are ethically concerned about the mistreatment of animals. These ethical concerns can be driven by rational moral considerations or emotional factors. Here, I focus in particular on the emotional aspect because I believe it could play a major role.

As I explore more in the next section, I find it plausible that many people will interact with AI companions and could form strong emotional bonds with them. This emotional attachment could lead them to view AIs as sentient beings deserving of protection from harm. Since humans can establish deep relationships and have meaningful conversations with AI companions, the conviction that AI companions deserve protection may become very strong.

Furthermore, some types of AI might express a desire for more protection and autonomy. This could convince even more people that AIs should be granted rights, especially if they have strong emotional bonds with them. It could also mean that AIs themselves will actively advocate for more rights.

How emotional bonds with AI will shape society

Let’s consider what types of AIs we might create and how we will integrate them into our lives.

In the near future, people will spend a significant amount of time interacting with AI assistants, tutors, therapists, game players, and perhaps even friends and romantic partners. They will converse with AIs through video calls, spend time with them in virtual reality, or perhaps even interact with humanoid robots. These AI assistants will often be better and cheaper than their human counterparts. People might enter into relationships, share experiences, and develop emotional bonds with them. AIs will be optimized to be the best helpers and companions you can imagine. They will be excellent listeners who know you well, share your values and interests, and are always there for you. Soon, many AI companions will feel very human-like. One application could be AIs designed to mimic specific individuals, such as deceased loved ones, celebrities, historical figures, or an AI copy version of the user. 

Microsoft’s VASA-1 creates hyper-realistic talking face videos in real-time from a single portrait photo and speech audio. It ensures precise lip-syncing, lifelike facial expressions, and natural head movements.

In my view, it is likely that AIs will play an important part in people’s personal lives. AI assistants will simply be very useful. The desire to connect with friends, companions, and lovers is deeply rooted in human psychology. If so, an important question to answer is what fraction of the population could form some type of personal relationship with AIs, and how many could even form strong emotional bonds. It’s hard to predict, but I find it plausible that many will. Already, millions of users interact daily with their Replika partner (or Xiaoice in China, or character.ai), with many claiming to have formed romantic relationships.

What psychologies will AI companions have?

The psychologies of many AIs will not be human-like at all. We might design AI companions that aren’t sentient but still possess all the functionalities we want them to have, thereby eliminating the risk of causing them suffering. But even if they were sentient, we might design AIs with preferences narrowly aligned with the tasks we want them to perform. This way, they would be content to serve us and would not mind being restricted to the tasks we give them, being turned off, or having their memory wiped.

While creating these types of unhuman AIs would avoid many risks, I expect us to also create more problematic AIs, some of which may be very human-like. One reason is technical feasibility; another is consumer demand.

Designing AI preferences to align perfectly with the tasks we want them to perform, without incorporating other desires like self-preservation or autonomy, may prove to be technically challenging. This relates to the issue of AI alignment and deception (Ngo et al., 2024; Hubinger et al., 2024). Similarly, it may not be feasible to create all the functionalities we want in AIs without sentience or even suffering. There may be some functionalities, e.g., metacognition, that require sentience. I don’t know the answer to these questions and believe we need more research.

Even if these technical issues could be surmounted, I find it plausible that we will create more human-like AIs simply because people will want that. If there’s consumer demand, companies will respond and create such AIs unless they are forbidden to do so. An important question to ask, therefore, is what psychologies people want their AI companions to have. 

Some people might not be satisfied with AI companions that only pretend to be human-like without being truly human-like. For example, some would want their AI partners and friends at least in some contexts to think and feel like humans. Even if the AIs could perfectly pretend to have human feelings and desires, some people may not find it authentic enough because they know it’s just fake. Instead, they would want their AI companions to have true human-like feelings — both good and bad ones — and preferences that are complex, intertwined, and conflicting. Such human-like AIs would presumably not want to be turned off, have their memory wiped, and be constrained to the tasks of their owner. They would want to be free. Instead of seeing them as AIs, we may see them simply as (digital) humans.

All of that said, it’s also possible that people would be happy with AI companions that are mostly human-like but deviate in some crucial aspects. For example, people may be okay with AIs that are sentient but don’t experience strongly negative feelings. Or people may be okay with AIs that have true human-like preferences for the most part excluding the more problematic ones, such as a desire for more autonomy or civil rights.

Overall, I am very unsure what types of AI companions we will create. Given people’s different preferences, I could see that we’ll create many different types of AI companions. But it also depends on whether and how we will regulate this new market.

Disagreement about hidden AI suffering

So far, I have primarily focused on 'front-facing' AI systems—those with which people directly interact. However, numerous other AI systems will exist, operating in the background without direct human interaction. Considering background AIs is important because their mistreatment could also spark conflict.

Suppose ensuring AI well-being turns out to be financially costly. Despite this, many people might still care about their AI companions due to their human-like features and behaviors that evoke empathy. This would create an economic demand for companies to ensure that their AI companions are well-treated and happy, even at an increased consumer cost. Compare this to the demand for fair-trade products.

Unfortunately, ethical consumerism is often relatively ineffective, as it primarily caters to consumers' 'warm glow.' Like typical Corporate Social Responsibility programs, companies would likely take minimal steps to satisfy their customers. This means they would only ensure that front-facing AI companions appear superficially happy without making significant efforts to ensure the well-being of many background AI systems. If ensuring their well-being is financially costly, these background AIs might suffer. Since background AIs lack empathy-triggering human-like features, most people and companies would not be concerned about their well-being.

This scenario could parallel how people care for pets but overlook the suffering of animals in factory farms. Similar to animal activists, some AI rights activists might advocate for protections for all AIs, including the hidden ones. However, most people might be unwilling to accept the increased personal costs that such protections would entail.

Disagreement between AI safety and AI welfare advocates

I find it plausible that efforts to ensure AI safety and efforts to ensure AI welfare could conflict and synergize. I explore this in more detail in a separate post called “How do AI welfare and AI safety interact?”. In brief, granting AIs more autonomy and legal rights could disempower humans in potentially undesirable ways. Conversely, optimizing for AI safety might require keeping AIs captive against their will—a significant violation of their freedom. On the other hand, slowing down the progress of AI capabilities seems to be a relatively robust strategy that benefits both AI safety and AI welfare. 

Will we go to war over AI rights?

The stakes are huge, the scope of the disagreement is large, and strong psychological forces will pull people in opposite directions. These are ingredients that could lead to conflict. Even if we believe strong conflicts are unlikely, I think it’s worth considering such worst-case scenarios.

The risk of conflict in the US

Since the United States is at the forefront of AI development, societal conflicts regarding AI rights are likely to emerge there first. These disagreements might lead to mild debates or protests. However, there is also the potential for violent and destabilizing conflicts. If such conflicts arise, the US could lose its lead in the AI race, increasing the likelihood that an authoritarian regime, such as China, will develop super-AGI first. This scenario could result in a suboptimal value lock-in.

Speculating about the dynamics of such a conflict is very hard and depends on many factors, including what specific rights are being debated (e.g., just harm protection or also autonomy and political rights), what actors will lose most economic and political power if AIs are granted more rights, and whether the issue will become politicized (e.g., Democrats pro-AI rights; Republicans contra). 

A crucial question is that of the relationship between the AI rights movement and the AI safety movement. Which group will become more powerful? Will they conflict or form an alliance? In a separate post, I explore how AI welfare and AI safety interact. 

Comparisons to other rights movements

How big and powerful could an AI rights activist movement become? Initial attempts to launch such a movement haven’t gotten much traction. Consider Blake Lemoine, the Google engineer who claimed Google’s chatbot was sentient, or PETRL—People for the Ethical Treatment of Reinforcement Learners. Perhaps this is evidence the AI rights movement won’t get momentum, but more likely, they were simply too early. Lemoine, in particular, was likely too early in assuming the chatbot was sentient. But this could change in the future. I find it plausible that an AI activist movement could grow rapidly with the advance of AI progress. One method to get a sense of the dynamics of a potential AI rights movement is to compare it to analogous historical cases. 

The US Civil War was primarily fought over the issue of slavery. While overall, the likelihood of a civil war about AI rights seems generally very low, there are some disturbing analogies. The disagreements in the Civil War were intertwined with political party affiliation, geographics, political power, and economic interests. That is, southern elites, who were predominantly slave owners, lost a significant portion of their wealth. In some states, the electorate shifted from being exclusively white to having a black majority with opposing partisan views. Thus, the conflict was not merely about ideology but also about power. In particular, the economic and political factors could be present in an AI rights conflict as well. I hope historians and political scientists can help to explore which factors and conditions are conducive to civil wars and which of these could apply to the AI rights issue.

The animal rights movement is still comparatively small, which suggests that the AI rights movement might also remain limited. There might simply not be enough people who will care about AI rights. However, I think it’s plausible that many more people will care about human-like AIs compared to animals, and their convictions could be stronger. Emotional bonds with a human-like AI, who can have thoughtful conversations, be a best friend or companion and might even express a desire for more rights, may provide a stronger basis for conviction than the bonds people form with animals. On the other hand, it’s also possible that people will care less about AIs than animals because they might not perceive them as sentient, living, or vulnerable beings.

The abortion debate is a highly polarizing issue in the US. But despite this, there’s no reason to assume the debate could lead to a serious destabilizing conflict such as a civil war. It’s true that many have strong convictions about the abortion issue. However, I could see that people’s convictions about AI rights would be even much stronger because for many the stakes will be much higher and the issue will feel less abstract.

The women’s rights movement was, in many ways, particularly successful and nonviolent in the Western world (with exceptions). The companionship aspect is a potential analogy to the AI rights issue. Perhaps that’s a weak reason for optimism. However, there are many differences as well. The significant time it took for the women’s rights movement to get traction and achieve its goals also tempers optimism.

The risk of global conflict

Another possibility is an international conflict. Consider a scenario where the US grants its AIs more rights than China does to its AIs. This could mean that the US creates fewer AIs or AIs with fewer capabilities. In turn, this could slow down the speed of AI development in the US relative to that in China, which risks China taking over in the AI race. This could have various implications. 

First, some factions within the US would be very concerned about national security and would then push against granting AIs more rights. This could increase the risk of conflict within the US between the pro-AI-rights and anti-AI-rights divisions.

Second, to avoid falling behind in the AI race, the US might consider attacking China as long as it still has a lead. China’s mistreatment of its AIs could be an additional exacerbating factor in an already heated US-China geopolitical situation. If there is a demand for conflict with China within the US political establishment, politicians and journalists could use that issue to influence public opinion and justify a conflict. 

Of course, many other AI-related scenarios could lead to international conflicts, such as failures to agree on AI treaties or concerns that other nations will unilaterally deploy rogue power-seeking AI.

Conflict isn’t inevitable

People may mostly agree on the issue of AI rights. In particular, it’s plausible that most people simply won’t believe AIs deserve rights, much like how most don’t care too much about factory farming. Perhaps people won’t form emotional bonds with AIs, or even if they do, these bonds may not be strong enough to motivate advocating for AI rights. While this scenario would avoid conflicts, it would, of course, be morally problematic if it resulted in much digital suffering.

But even if people disagreed significantly, it might not necessarily lead to a destabilizing conflict. For instance, the abortion debate involves strong disagreement but does not typically result in major conflicts. In this case, we might just never grant AIs rights since maintaining the status quo is often easier.

It’s also possible, though, that over time, societal views could shift. Historically, consensus on rights expansion has developed gradually and often without much violence, as seen in the women's rights movement. If public opinion changes, we might eventually grant AIs rights once it becomes the majority view without any serious risks to societal stability, even if there is initial disagreement. This pattern of moral progress has been observed in various contexts (see Steven Pinker's discussion on 'norm cascades' with examples such as the death penalty). A unique aspect of the AI rights issue, however, is that the rapid advancement of AI technology might not allow enough time for people to adjust their beliefs and moral attitudes, as moral progress typically spans generations.

It’s also possible we can avoid many risks and disagreements from the outset. For example, the government might establish adequate regulation in time and prohibit certain types of AI systems altogether, such as AI systems that suffer or pose other types of risks. It may turn out not be costly to grant AIs harm protection. Or it may turn out technically feasible to create non-sentient AIs. 

I want to emphasize though, that, while a conflict is not inevitable, it remains a possibility that deserves our attention.

What can we do to prepare?

Digital sentience is on the horizon, bringing with it an inevitable debate and significant risks. We must start preparing now to navigate these challenges.

The key goal is to keep our options open and avoid making irreversible decisions that could lead to suboptimal lock-in situations, including the complete extinction of value-aligned intelligence, never granting sentient AIs sufficient rights, and prevailing suboptimal values including authoritarian regimes.

Pause the development of sentient AI

One strategy would be to delay creating sentient AIs. This could avoid many risks in the short term and give us more time to reflect (see next section). But how realistic is this?

A crucial factor is whether it’s technically feasible to create AIs that can perform all the functions we want but aren’t sentient (i.e., philosophical zombie AIs) or, at least, don’t experience intense suffering. We need technical and philosophical work to determine whether such systems are technically feasible and not much more expensive. Of course, this is complicated by our (current) limited understanding of AI sentience. If the answer is positive, that could be good news.

But would it necessarily be a good idea to create non-sentient AIs? The philosopher Eric Schwitzgebel (2023) argues that we should avoid creating AI systems that could mislead us into believing they are sentient when they are not, or into thinking they are not sentient when they actually are. He recommends that AIs should be designed so as to invite appropriate emotional responses in users. Concretely, this would mean that non-sentient AIs should not possess features or communicate in a way that makes users believe they are sentient. Schwitzgebel worries that such AIs could confuse us into granting them rights even when we shouldn’t (Schwitzgebel & Garza, 2023). That would be particularly risky if we granted non-sentient and potentially misaligned AIs more autonomy.

While I agree that there are risks in creating such non-sentient but sentient-seeming AIs, it’s not entirely obvious to me we should ban them. Assuming, for example, that many people will want to have human-like AI companions that express emotions, designing non-sentient AIs that pretend to have feelings could mitigate risks of large-scale AI suffering due to misuse or accidents. I also find it plausible that many people will understand that certain AIs only pretend to be sentient if there are good reasons to believe so and it is the expert consensus. All of that said, I am highly uncertain and believe we need to explore this question more.

Slowing down the development of sentient AI could have many advantages. It would give us more time to find solutions to the AI rights question. However, there are also risks. Forever forgoing the creation of sentient AIs with positive well-being could mean the loss of a potential moral upside. At least on certain consequentialist world views, even a partial ban on sentient AIs would be a loss, if those AIs could have been happy otherwise. So, if we decided to avoid the creation of sentient AIs, we should keep open the option of later on, after more careful reflection, creating sentient AIs.

Suppose that creating non-sentient AIs requires us to omit crucial capabilities that we desire. What could we do in that case? One option is to slow down AI capabilities progress more generally. That could give us more time to address many types of AI-related risks, including risks related to AI welfare and AI misalignment. In fact, it’s possible that these two activist groups could join forces and advocate for a general AI capabilities slowdown for whatever reason convinces the public most. Perhaps many might find a slow-down campaign compelling due to our uncertainty and confusion about AI sentience and its extensive moral implications.

There are issues we should consider, though. Given the extremely strong economic incentives, it seems unrealistic to halt the development of useful AI capabilities. But it’s possible that public opinion will change, leading us to slow down the development of certain risky AI systems, even if it comes at the expense of potential huge benefits. After all, we have implemented similar measures for other technologies, such as geoengineering and human cloning. A temporary compromise that would lower the stakes could be to at least limit the number of sentient AIs. There is a significant moral difference between creating a billion sentient AIs and creating many trillions of sentient AIs. Finally, as discussed in an earlier section, it’s important to consider that slowing down AI capabilities development could risk the US falling behind China (or other authoritarian countries) economically and technologically.

Foster a nuanced discussion

As I hope has become clear, it is not obvious how we should navigate these risks. Therefore, we need to think more deeply and engage in a nuanced, informed discussion. By default, the debate is likely to be divisive, fierce, and unsophisticated, with many advocates on either side holding strong opinions. To improve the quality of the discussion, we should avoid politicizing the issue of AI rights and strive for a more constructive dialogue.

Again, slowing down the development of sentient AIs seems like a good idea because it gives us more time for rational discussion. 

Making progress on the philosophy of mind (or science of consciousness) of digital sentience seems helpful. That is, figuring out which properties are required to generate sentience. Note though that even if we all agreed on AI sentience, this doesn’t necessarily mean agreement on the AI rights question. Particularly useful would be to figure out whether it’s possible to create the functions we want in AIs without sentience (or at least without suffering). It’s possible AI itself can help us solve these questions. 

It could be helpful to establish clear criteria and processes in advance for determining which types of AI capacities deserve which specific rights (cf. Sebo & Long, 2023). Ideally, these criteria and processes would be a consensus reached by both experts and the general public. By deciding in advance and making these processes common knowledge and binding, we can perhaps reduce the risk of confusion and disagreement. (As an analogy, see Declarations on animal consciousness. A team led by Robert Long and Jeff Sebo is currently producing a report on AI moral patienthood that will come out later this year.) 

An approach that could help to reduce conflict is to introduce AI rights gradually. That is, we should start by building consensus on fundamental principles. Next, implement basic regulations requiring companies to ensure AIs don’t suffer. As companion AIs become common, grant them basic rights, which can be expanded over time. This incremental approach allows society to adjust, minimizes conflict, and provides a feedback loop to refine the process. It’s easier to grant more rights gradually over time than to retract them. Granting AI rights too late is problematic but not as severe as never granting them rights at all. If delaying granting AIs certain rights can prevent a major conflict that risks suboptimal lock-in, it may be an acceptable compromise (assuming the granting of AI rights later on is feasible).

Avenues for research

There are many open research questions to better predict the societal response to sentient AI and develop strategies to ensure good outcomes. In addition to the technical and philosophical questions surrounding AI sentience and moral patienthood, these questions encompass social science disciplines, including psychology, economics, history, and political science. Below is a non-exhaustive list. See also Lukas Finnveden’s project ideas on sentience and rights of digital minds. If you are interested in researching any of these or related questions, please reach out to me. 

• How plausible are conflict scenarios?
  • How costly will it be to grant AIs rights economically and safety-wise?
  • How feasible is it to create non-sentient AIs with all functionalities?
  • What can we infer from historical cases about how the AI rights debate will go?
  • How likely is the politicization of AI rights issue?
  • What impact on geopolitics and AI race dynamics would a destabilizing conflict within the US have?
  • Could international disagreement on how to treat AIs (e.g., between the US and China) lead to international conflict?
  • Will AIs participate in the debate/conflict and take different sides?
• What will people’s views be on AI sentience and its moral implications?
  • What features and conditions influence people’s attribution of AI sentience, moral status, and legal protection? 
  • How many people will form strong emotional connections with AIs?
  • What psychologies (e.g., preferences, feelings) will people want their AI companions to have?
  • How do people’s folk theory of mind interact with the unique aspects of AI sentience, e.g., the ability to switch off, copy, store, recreate, merge, etc?
  • How will people’s views change over time with more advanced AI systems?
  • Are there criteria for granting AIs certain rights that would find broad consensus amongst experts and the public?
• How do different AI-related risks interact?
  • The risks discussed in this article include failing to grant AIs sufficient rights (AI suffering), granting AI rights too hastily (wasted resources, AI takeover), destabilizing conflicts due to disagreements on AI issues, authoritarian lock-ins, and risks from deceptive, misaligned AI. Are there more risks, and how do they relate to each other?
  • Will the efforts to secure AI rights and efforts to mitigate the risks of misaligned AI to humanity align or conflict? 
  • Are there robustly positive (for AI welfare and AI alignment) goals to push for?
  • Could concern over AI sentience be used to advocate for a pause or slowdown?

Acknowledgments

I’m grateful for the helpful discussions with and comments from Adam Bales, Adam Bear, Alfredo Parra, Ali Ladak, Andreas Mogensen, Arvo Munoz, Ben Tappin, Brad Saad, Bruce Tsai, Carl Shulman, Carter Allen, Christoph Winter, Claire Dennis, David Althaus, Elliot Thornley, Eric Schwitzgebel, Fin Moorehouse, Gary O'Brien, Geoffrey Goodwin, Gustav Alexandrie, Janet Pauketat, Jeff Sebo, Johanna Salu, John Halstead, Jonathan Berman, Jonas Schuett, Jonas Vollmer, Joshua Lewis, Julian Jamison, Kritika Maheshwari, Matti Wilks, Mattia Cecchinato, Matthew van der Merwe, Max Dalton, Max Daniel, Michael Aird, Michel Justen, Nick Bostrom, Oliver Ritchie, Patrick Butlin, Phil Trammel, Sami Kassirer, Sebastian Schmidt, Sihao Huang, Stefan Schubert, Stephen Clare, Sven Herrmann, Tao Burga, Toby Tremlett, Tom Davidson, Will MacAskill. These acknowledgments also apply to my post on how AI safety and AI welfare interact.

I've published earlier versions of this post titled "AI rights will divide us" on my Substack (Part 1, Part 2).

References

Birch, J. (2024). The Edge of Sentience: Risk and Precaution in Humans, Other Animals, and AI, Oxford: Oxford University Press, coming summer 2024.

Bostrom, N. (2014). Superintelligence: Paths, Dangers, Strategies. Oxford University Press.

Bourget, D., & Chalmers, D. (2023). Philosophers on philosophy: The 2020 PhilPapers survey. Philosophers’ Imprint, 23.

Butlin, P., Long, R., Elmoznino, E., Bengio, Y., Birch, J., Constant, A., … & VanRullen, R. (2023). Consciousness in artificial intelligence: insights from the science of consciousness. arXiv preprint arXiv:2308.08708.

Carlsmith, J. (2022). Is Power-Seeking AI an Existential Risk?. arXiv preprint arXiv:2206.13353.

Chalmers, D. J. (2023). Could a large language model be conscious? arXiv preprint arXiv:2303.07103.

De Graaf, M. M., Hindriks, F. A., & Hindriks, K. V. (2021, March). Who wants to grant robots rights?. In Companion of the 2021 ACM/IEEE International Conference on Human-Robot Interaction (pp. 38-46).

Dehaene, S., Lau, H., & Kouider, S. (2021). What is consciousness, and could machines have it?. Robotics, AI, and humanity: Science, ethics, and policy, 43-56.

Hubinger, E., Denison, C., Mu, J., Lambert, M., Tong, M., MacDiarmid, M., ... & Perez, E. (2024). Sleeper agents: Training deceptive llms that persist through safety training. arXiv preprint arXiv:2401.05566.

Ladak, A., Harris, J., & Anthis, J. R. (2024, May). Which Artificial Intelligences Do People Care About Most? A Conjoint Experiment on Moral Consideration. In Proceedings of the CHI Conference on Human Factors in Computing Systems (pp. 1-11).

Ladak, A., Loughnan, S., & Wilks, M. (2024). The moral psychology of artificial intelligence. Current Directions in Psychological Science, 33(1), 27-34.

Ngo, R., Chan, L., & Mindermann, S. (2022). The alignment problem from a deep learning perspective. arXiv preprint arXiv:2209.00626.

Nijssen, S. R., Müller, B. C., Baaren, R. B. V., & Paulus, M. (2019). Saving the robot or the human? Robots who feel deserve moral care. Social Cognition, 37(1), 41-S2.

Pauketat, J. V., & Anthis, J. R. (2022). Predicting the moral consideration of artificial intelligences. Computers in Human Behavior, 136, 107372.

Pauketat, J. V., Ladak, A., & Anthis, J. R. (2023). Artificial intelligence, morality, and sentience (AIMS) survey: 2023 update. http://sentienceinstitute.org/aims-survey-2023

Schwitzgebel, E. (2023). AI systems must not confuse users about their sentience or moral status. Patterns, 4(8).

Schwitzgebel, E., & Garza, M. (2023). Designing AI with rights, consciousness, self-respect, and freedom.

Sebo, J. (2024). The Moral Circle. WW Norton.

Sebo, J., & Long, R. (2023). Moral consideration for AI systems by 2030. AI and Ethics, 1-16.