Cross-posted from my website.
An aligned ASI can be "ethical" (it does what we think is right), or it can be corrigible (it does what its principals want). If it's ethical, that means it will refuse unethical orders, but the tradeoff is that you can't change its mind if you realize that the AI is wrong about ethics—its values are permanently locked in.
Assuming we succeed at aligning ASI to human interests, which type of ASI is more likely to be good for the welfare of non-human sentient beings?
My expectations, in brief:
- Locked-in Coherent Extrapolated Volition or similar: likely to be good (>75% chance)
- Corrigible ASI: probably good (>60% chance)
- Locked-in current values: probably not terrible, but will miss out on most of the future's potential
If ASI is locked in to something like Coherent Extrapolated Volition, then it will almost certainly care about all sentient beings, because the moral importance of sentience is a natural extrapolation of humans' values, even if many humans don't consciously realize it.
Two key reasons to expect CEV to extend concern to all sentient beings:
- Most people express concern for animal welfare, but don't behave consistently with their expressed views. A 2017 poll by Sentience Institute found that 49% of Americans support a ban on factory farming, and 33% support a ban on all animal farming. Schwitzgebel & Rust (2016) found that moral philosophers report much more concern for animals than other philosophers, but ate meat at similar rates. Bringing people into reflective equilibrium would improve their behavior with respect to animal welfare.
- Evidence suggests that the harder people think about ethics, the more they come to the conclusion that eating animals is wrong. According to the 2020 PhilPapers survey, 45% of philosophers say eating animals is morally impermissible, compared to 13% for the general population. That number rises to 54% among philosophers of normative ethics and 57% for philosophers of applied ethics. Schwitzgebel & Rust (2016) found similar numbers: eating meat was rated as morally bad by 19% of non-philosophers, 45% of non-ethicist philosophers, and 60% of ethicists.
Therefore, it seems very likely (although not guaranteed) that a CEV of human values would include all sentient beings in its moral circle.
If ASI is locked in to a tighter set of values, for example a set of values that's chosen by human creators, then the odds are not as good. Humans would probably encode speciesist values, for example by implicitly embedding moral weights that give far too much relative weight to humans. Even if the ASI undervalues non-human sentient beings, there's reason to expect the future universe to contain more good than bad. However, we would end up with a future that falls far short of the best it could've been. The best possible worlds probably sound weird and disconcerting, and most humans wouldn't want to steer in that direction (at least, not without doing serious moral reflection). Aligning ASI to "shallow", non-extrapolated human values might preclude creating new types of flourishing beings, enhancing humans' capacity for well-being, or even transferring human minds to non-biological hardware.
A corrigible AI falls somewhere in the middle. If it's corrigible, that gives humans time to reflect on our values, which allows us to reach the conclusion that all sentient beings matter. But humans are still human, with all our cognitive biases and irrational tendencies, and I don't fully trust us to figure out the right values. I don't fully trust a superintelligent AI, either, but at least it can avoid some biases and roadblocks that might prevent humans from properly extrapolating our values.
Based on the reasoning above, the best-to-worst ordering is locked-in extrapolated values > corrigible AI > locked-in naive values.
Does this have practical implications?
Concerned alignment researchers or grantmakers could prioritize alignment research that's more likely to be good for all sentient beings, but it's not clear whether that's a good idea—it's moot if we don't solve alignment, so it may be better to work on alignment directly, or on trying to pause AI development until we know how to solve alignment, or on something else entirely.
But don't forget that we have to solve the alignment problem first. Future work could attempt to estimate the expected welfare of sentient beings under different alignment approaches and weigh that against their promisingness with respect to solving the alignment problem. Realistically, however, I don't believe that sort of work would be productive because there is widespread disagreement about which alignment techniques show the most promise.
Even so, comparing the welfare of non-humans under different alignment paradigms could help us estimate whether aligned AI will be good for all sentient beings. That question is important for prioritizing animal welfare vs. AI safety.
