One concern I've had with aligning AI to aggregated human values is that we risk preserving our moral disagreements, biases and mistakes rather than overcoming them. If moral realism is true, and moral facts are discoverable, then there may be something more reliable to converge towards than our current preferences.
Even as our ontologies evolve, we might become progressively better at tracking those facts. This makes indirect normativity attractive to me - using advanced AI explore the landscape of value - to help us discover where we're mistaken, reduce moral uncertainty and make moral progress.
However, under motivational externalism discovering what's morally right and being motivated to act accordingly are two different problems.
Nice post! It seems similar to the moral alignment approach I explored in AI Alignment to Moral Realism.
One concern I've had with aligning AI to aggregated human values is that we risk preserving our moral disagreements, biases and mistakes rather than overcoming them. If moral realism is true, and moral facts are discoverable, then there may be something more reliable to converge towards than our current preferences.
Even as our ontologies evolve, we might become progressively better at tracking those facts. This makes indirect normativity attractive to me - using advanced AI explore the landscape of value - to help us discover where we're mistaken, reduce moral uncertainty and make moral progress.
However, under motivational externalism discovering what's morally right and being motivated to act accordingly are two different problems.
Thanks Ahmed - very interested in joining - also could help facilitate if that's a bottleneck.
Any ball-park dates?
Will there be any more of these?