Comments
To give you an overly specific answer, presumably the AGI could realize the existence of the trigger and just keep the lights on while surrounding them with with paperclips?
Curated and popular this week
This is the third in a sequence of posts taken from my recent report: Why Did Environmentalism Become Partisan?
Summary
Rising partisanship did not make environmentalism more popular or politically effective. Instead, it saw flat or falling overall public opinion, fewer major legislative achievements, and fluctuating executive actions.
Public Opinion...
I think right now EAs might be making a significant mistake by paying insufficient attention to the political realm. As EAs we tend to figure out what’s most impactful for us to work on and focus hard. That’s great! But there are various actions that are ‘non-delegatable’ - the extent to which an individual can do the action is limited (like voting, going to a protest, making hard money contributions to particular campaigns). It might be useful if we were all more in the habit of doing variou...
This post presents the executive summary from Giving What We Can’s impact evaluation for 2025. At the end of this post we share links to more information, including the full report and...
Recent opportunities to take action
An entity powerful enough to threaten all humans on earth would most likely also be powerful enough to defeat the MAD mechanism.
Suppose we have 10 different MAD mechanisms. Wouldn't the defense become impossible at some point? Wouldn't the AI think at one point, "It is simpler to just complete the task without killing anybody"?
EDIT: okay, if the task given to AI is "calculate the 2^256 th digit of Pi", then perhaps it'd rather risk the easier task of "before they certainly interrupt my Pi calculations, defeat 50 MAD threats and kill all people, then hopelessly proceed"
Perhaps, but now you're substituting one risk for another, which is the risk that the MAD mechanisms trigger by accident, causing unintentional destruction of the earth.
Yeah, thanks for replying. I did now realize that its more complex than that.