EA Forum Bot Site
EA Forum

Hide table of contents
Comment Permalink
Answer by Yellow (Daryl)2
0
1

Hi Jeremy

I think an AGI would be much better at creating arguments for why humanity should not be eliminated. If an agi is incapable of creating these arguments itself I wonder if it capable enough to destroy humanity.

I think the thing to worry about more is that an agi correctly determines that humanity, or most of humanity needs to be destroyed (ex. Agi cares about all life and all humans murder there face mites so they must be stopped). But is that really all that bad?

Reply
See in context
by Jeremy
1 min read2 answers 2

8

AI safetyExistential riskDecision theory
Frontpage

I'm imagining an AGI weighing the pros and cons of destroying humanity. If it finds persuasive arguments against in its training data (or via search), maybe it would decide not to? 

Should we crowdsource arguments in favor of AGI preserving humanity?

Feel free to post about why this might or might not work, but I am most interested in hearing reasons why it (or this post) could backfire. If someone has convincing reasons, or this post is heavily downvoted, I will delete it.

I haven't come across anyone else attempting to make these arguments (to a future AGI, about why it should preserve humanity), but I would also be interested to hear about them if you know of any.

8

0
0

Reactions

0
0
New Answer
New Comment

2 Answers sorted by
Top

Yellow (Daryl)

2
0
1

Hi Jeremy

I think an AGI would be much better at creating arguments for why humanity should not be eliminated. If an agi is incapable of creating these arguments itself I wonder if it capable enough to destroy humanity.

I think the thing to worry about more is that an agi correctly determines that humanity, or most of humanity needs to be destroyed (ex. Agi cares about all life and all humans murder there face mites so they must be stopped). But is that really all that bad?

Geoffrey Miller

0
0
0

Jeremy - I think it will be important to develop compelling arguments about why it's worth preserving humanity -- not so much to convince an ASI not to kill us all, but to remind ordinary citizens why AI X risk is bad, and to persuade transhumanists/posthumanists/Singularity-enthusiasts that replacing humanity ASAP with machine intelligences is a ghoulish, psychopathic, & repulsive ideology.

The trouble is, the standard 20th century arguments for preserving humanity in the face of AI are wearing a bit thin. A lot of those arguments involved sentimental appeals to human creativity, e.g. 'We should preserve humans because humans make wonderful art, music, literature, science, and humor that AI won't be able to match'. Well, AI is getting increasingly better than average humans at all of these domains, so appeals to human creativity won't be very compelling. 

What other arguments can we offer?

Comments2
Sorted by
New & upvoted
Click to highlight new comments since:
harfe2
1
0

I haven't come across anyone else attempting to make these arguments (to a future AGI, about why it should preserve humanity), but I would also be interested to hear about them if you know of any.

googling "arguments for the preservation of humanity ai" yields https://www.lesswrong.com/posts/azRwPDbZfpadoL7WW/an-appeal-to-ai-superintelligence-reasons-to-preserve

see also the discussion in the comments for how valuable this is.

Reply
Jeremy2
0
0

Great thank you. Late to the party, but by less than a month :).

Reply
More from Jeremy
View more
Curated and popular this week
Relevant opportunities
View more