Comments
76
AI Use Note: Main body text entirely human written. Claude (Opus 4.8) helped develop models of animal life histories in the appendix.
Cross-posted from Good Structures.
Executive Summary
* Animal advocates sometimes make claims like “there are X of this animal...
An entity powerful enough to threaten all humans on earth would most likely also be powerful enough to defeat the MAD mechanism.
Suppose we have 10 different MAD mechanisms. Wouldn't the defense become impossible at some point? Wouldn't the AI think at one point, "It is simpler to just complete the task without killing anybody"?
EDIT: okay, if the task given to AI is "calculate the 2^256 th digit of Pi", then perhaps it'd rather risk the easier task of "before they certainly interrupt my Pi calculations, defeat 50 MAD threats and kill all people, then hopelessly proceed"
Perhaps, but now you're substituting one risk for another, which is the risk that the MAD mechanisms trigger by accident, causing unintentional destruction of the earth.
Yeah, thanks for replying. I did now realize that its more complex than that.