Summary of 'Sharing the World With Digital Minds' by Carl Shulman and Nick Bostrom
Full paper can be found here: digital-minds.pdf (nickbostrom.com). The following is written from Shulman & Bostrom’s point of view; see this companion post for my own take: https://forum.effectivealtruism.org/posts/qLYFBpT2amHxJtf2o/questions-about-and-objections-to-sharing-the-world-with
In the future we will be able to create “digital minds”, conscious agents whose lives can contain different amounts of well-being. These would likely:
Should we create such digital minds? On views on which adding happy people to the world is good, the answer is probably ‘yes’, given that this is a method of creating super-beneficiaries. On views which deny this, it will depend on whether creating digital minds benefits existing people, and on how good/bad the lives of those digital minds will be.
In practice, we have some prima facie moral reason to create digital super-beneficiaries, since some reasonable moral views take this to be inherently good, others to be neutral in itself, and we aren’t sure which view is correct.
If we create digital minds, we face difficult questions about how to share resources with them:
Bostrom and Shulman recommend:
They suggest that the following are probably morally permissible and probably a good idea:
In the future, we may create many “digital minds” (AIs) who are conscious agents, and hence whose lives can contain different degrees of well-being (for example, because they experience pleasure and pain). If we create such digital minds, it will be morally better if their “lives” go well, rather than badly, and they will likely be owed some resources with which to further their interests. Because of this there are challenging questions about whether we should create digital minds, and (if we do create them), how much weight we should give to their interests relative to human beings.
It is likely that if we learn how to create conscious digital minds, we will eventually be able to create digital minds that are so-called “super-beneficiaries”: better at converting resources into well-being than humans. There are multiple reasons to think that if we learn how to create conscious digital minds, we’ll eventually be able to make them super-beneficiaries:
Given that we will probably one day have the capacity to create digital minds which are super-beneficiaries, two obvious moral questions arise:
The answer to (11) varies depending on what moral principles about creating new people we accept. If we think that it is good to create people with lives that have net-positive well-being, then we have moral reasons to create super-beneficiaries. On the other hand, if we accept a view on which creating new happy people is morally neutral, then the fact that digital minds would be super-beneficiaries gives us no reason to create them. Instead, we should create digital minds only if doing so would benefit the people who already exist. (And probably only if those digital minds will not have net-negative lives.)
Views on which we should only care about the interests of currently existing people are implausible. And if we are unsure whether the correct moral theory says that bringing people with net-positive lives into existence is good, then we should probably assign some value into bringing people with net-positive lives into existence, since we think there’s a non-zero chance that doing so is morally valuable. So we probably have at least some moral reason to create digital super-beneficiaries. On total utilitarian views on which we are morally required to create as much value as possible the moral reasons for creating large numbers of digital super-beneficiaries are very strong, since it would allow us to generate a very large amount of well-being.
Thinking through (12) raises difficult issues, because there are various reasons why we might be morally required to give so many resources or so much power to digital super-beneficiaries that the interests of humans are very significantly harmed.
Firstly, on total utilitarian views where maximizing value is morally required, we are probably required to give all society’s resources to digital super-beneficiaries, even though this will lead to human extinction. Since the super-beneficiaries generate more utility per unit of resources than we do, giving them all our resources maximizes utility.
On many other moral views, we are not automatically required to transfer our own personal resources to the super-beneficiaries just because doing so would maximize utility. And on many of those other moral views it will be morally impermissible to steal resources owned by other human beings and give them to digital super-beneficiaries. In general on non-utilitarian views humans wouldn’t owe digital super-beneficiaries our resources just because they could create more well-being from them than we can. However, even on such views there are still other reasons to expect that if we create digital super-beneficiaries we will be morally obligated to transfer a large amount of power and/or resources to them. And the consequences for humans if the obligations are fulfilled could still be very bad:
It’s unclear how to avoid concluding we have these obligations, if we’re committed to an anti-speciesist principle which says that humans don’t matter more or have greater political rights than digital minds, just because we are human and they aren’t.
However, there is a practical compromise that-whether or not it’s a morally permissible outcome to aim for-might be acceptable to both humans and digital super-beneficiaries. A world with digital minds that massively outnumber humans, would produce far, far more resources than one without those minds. Because of this, if we split society’s resources so that the vast, vast majority went to the digital minds, but some resources were still given to humans, this would:
In thinking about what ways of dividing power and resources between humans and digital minds are morally acceptable, we can rely on these two principles (directly quoted from Bostrom and Shulman):
Principle of Substrate Non-Discrimination
If two beings have the same functionality and the same conscious experience, and differ only in the substrate of their implementation, then they have the same moral status.
Principle of Ontogeny Non-Discrimination
If two beings have the same functionality and the same conscious experience, and differ only in how they came into existence, then they have the same moral status.
We should also place a high value on avoiding actions which are highly efficient at producing large amounts of suffering or digital minds which suffer much more easily than humans.
One final interesting question is whether it’s morally permissible to deliberately design digital minds so that they consent to share resources with us in the way that we favor. It’s (plausibly) wrong to deliberately genetically engineer human children with preferences that are convenient for their parents, as this is arguably an immoral type of manipulation. So it might also be impermissibly manipulative to engineer digital minds with preferences that make them accept particular resource-bargains with humans. However, there are important moral disanalogies between the case of genetically engineering humans to have particular preferences, and the case of choosing preferences for digital minds, which might show that the latter is permissible, even if the former is morally wrong. Firstly, unlike in the case of human procreation, there probably won’t be a way to create digital minds without making choices that predictably shape their preferences. So perhaps when knowingly shaping preferences is unavoidable, it is not automatically wrong to shape preferences in ways that benefit you, or humanity as a whole. Secondly, if we shape humans to have particular strong preferences, those may conflict with other preferences they have, and cause them suffering. If this is what explains why genetically engineering children to have particular preferences is wrong, then it’s probably not inherently wrong to deliberately shape the preferences of digital minds. For in the case of digital minds, we can probably design the minds in ways that avoid them experiencing suffering due to them having competing strong preferences, whilst also ensuring they have the preferences about resource-sharing that we desire.
Want to listen to the original paper? You can find this, and 26 other academic papers by Nick Bostrom, at radiobostrom.com.
https://radiobostrom.com/2/sharing-the-world-with-digital-minds