Really love this Lizka!

I think that to some extent you're proposing smashing the "defect" button in a prisoner's dilemma and hoping the other side doesn't do the same.

I've been pondering this. I think your button-smashing characterisation is basically accurate, and it is a leap of faith that those who engage in civil disobedience make: an appeal to the conscience of society, the jury etc..

You're right to say that one way to think about universalisability is "if it's okay for me to break the law to achieve what I consider to be a moral goal here, why can't everyone break the law to achieve their own moral goals?". But another way to think about universalisability is to go "if I were the one in Ridglan / Unit 731 / Willowbrook, what actions would I support to end my suffering?"

I don't know whether it would be illegal for parents to break their children out of Willowbrook, but for the purposes of this question assume it was.

What if "protecting innocent sentient beings from torture" is a higher moral priority than "living together in a society with people of greatly differing moral views"?

I'm sceptical that the distinction between flawed democracy and dictatorship is clean enough to justify civil disobedience on behalf of others only in the latter (if this is what you're saying). Would you support rescuing American children from deliberate infection with hepatitis at Willowbrook in the 1960s?

The clearest such historical cases are ones where a disenfranchised group of people broke laws that directly enforced their own exclusion from political participation or basic legal personhood. These cases are self-limiting (and thus pass reasonable tests of universalizability) since the principles justifying such lawbreaking achieve their own obsolescence once participation is granted.

I worry this approach excludes the most vulnerable (those who cannot meaningfully participate in political life, like human babies and animals), and focuses on less fundamental rights: I think protection from torture is more urgent than legal personhood.

Why would women be justified in engaging in civil disobedience to get the vote for themselves, but not be justified in engaging in civil disobedience to rescue babies from Josef Mengele?

Thanks for organising Toby!

I guess the causal mechanism I'm thinking of here is:

1. Most humans feel at least a little sad when they see a baby gazelle being eaten alive by hyenas
2. AGI is so powerful that humans can order it to do things like "stop baby gazelles being eaten alive whilst retaining the beauty of nature and the complexity of ecosystems" and then it'll just go away and do it somehow

Maybe this is foolish and naive on my part! And maybe I'm wrong to think our moral preferences/intuitions will be so robust to the disruption of AGI, even if AGI goes well for us.

Some really cool points here Lee, and I mostly agree with you I think.

Crux: how many actors have terminal preferences for suffering? agency may be amplified for animal advocates, but it could also be amplified for malevolent actors.

This could be very important. I'm not sure what it means for AGI to go well for humans if some of those humans have terminal preferences for suffering / are sadistic. If the AGI protects the rest of us from the sadists, is AGI going well for the sadists?

EDIT: as well as sadists, we can consider humans who think animal agriculture, testing etc. has enough aesthetic/historical/cultural value that it's worth continuing to do it in a post-AGI world of abundance.

I need to think about b) more. I see arguments in both directions.

I don't think I can properly imagine what it's like to be tortured or eaten alive, and yet the thought of each happening to me or someone else makes me feel some combination of horror, fear, upset and compassion. And the idea of suffering more intense than torture or being eaten alive (if future artificially sentient beings have wider welfare ranges than we do) is terrifying to me.

But if I could never suffer worse than a pinprick, maybe I would stop caring about the most intense forms of suffering. Concerning stuff.

What kinds of values will humans have post-AGI, if AGI goes well for us? We don't need to be scope-sensitive utilitarians to want to adopt even radical preferences like ending animal exploitation and solving WAS, no? (Most humans don't like factory farming or the idea of cute animals being eaten alive.)

This makes sense. I would worry about the purely corrigible AGI being used by actors in such a way that we never get to instil the correct/good/post-long-reflection values in AGI/ASI down the line.