Comments
In this post, I propose the idea that the communication around current AI alignment risk could be improved. I then suggest a possible improvement which is easy for people to adopt individually, and could also be adopted as a whole. This is not meant as an authoritative recommendation, but more of a "putting it out there" discussion topic.
Epistemic statuses:
Certain for my personal experience
Very high for the average person's experience
High for the ineffectiveness of current communication method
Medium-high for the effectiveness of proposed communication method
As a regular person who is new to EA, I would like to share my experience of learning about AI alignment risk. The first thing I came across was the “paperclip maximizer” idea. Then, I read about superintelligence explosion “foom”. These are the top results when searching for “AI existential risk”, so I think it’s a representative sample.
After reading about these two topics, I rejected AI risk. Not in the sense of “I read the pros and cons and decided that the risk was overrated”, but in the sense of “this is stupid, and I’m not even going to bother thinking about it”.
I did this because these scenarios are too far from my model of the world and did not seem credible. Since I had no reason to believe that this line of enquiry would lead to useful learning, I abandoned it.
There are two types of people we want to persuade. Firstly, people with power and influence, and secondly, the public. The first category is important because they have leverage to create outcomes, and the second category is important because it is the main determinant of how favorable the “playing field” is.
I should be much easier to persuade than the average member of the public, because I was interested in learning more about AI risk! But even I bounced off. It will surely be an insurmountable climb for regular people who are disinterested in the topic. This is a disaster of messaging if the goal is to persuade the public.
Is it a good approach for persuading the first category of people, those with power and influence? We might expect it to be more effective, because they have stronger analytical capabilities and would be more receptive to logical analysis. Unfortunately, this misunderstands power and influence. This might work if we’re targeting rich people, because analytical power is correlated with wealth generation. But people with power and influence generally do NOT respond well to analytic arguments. Instead, they respond to arguments focusing on power and influence, which is completely unaddressed by the paperclip foom approach. This line of persuasion may even backfire, by reducing its perceived credibility.
What’s vexing is that persuading people by talking about paperclip foom is clearly correct. As Carl Sagan said, “extraordinary claims require extraordinary evidence”. Taking the outside view, AI alignment risk is an extraordinary claim, because it’s so alien to our general understanding of the world. So if that’s the case, we need to provide extraordinary evidence.
Unfortunately, extraordinary evidence consists of claiming catastrophic outcomes and then showing how they could come about. This is ineffective because the reader is required to agree to a huge inferential distance. In most cases, the reader will not even read the evidence or evaluate the evidence before dismissing it.
This is an “Always Be Closing” approach. Make a big ask, if you encounter resistance, you can then retreat slightly and make a sale. A small ask such as “AI alignment is potentially dangerous” is feeble, instead, let’s alarm people by announcing a huge danger, getting them to pay attention. Attention, Interest, Desire, Action!
It’s a pity that this approach crashes and burns at the Interest stage.
Another sales maxim is “don’t sell past the close”. Just do enough to get the sale - in this case, the “sale” is to get people thinking that AI alignment is a serious problem that we should care about. What would this approach look like?
Recently Nintil posted his thoughts about AGI. His approach basically claims that the “Always Be Closing” maximalist approach is unnecessary. This is encapsulated by a quote within the blogpost:
“Specifically, with the claim that bringing up MNT [i.e. nanotech] is unnecessary, both in the "burdensome detail" sense and "needlessly science-fictional and likely to trigger absurdity heuristics" sense. (leplen at LessWrong, 2013)”
Nintil makes a case that AGI is a concern even without the “needlessly science-fictional” stuff. I found this interesting and worth serious consideration, because it didn’t require a large inferential leap and so it was credible. “Here are some assumptions that you probably already have. I’ll show how this leads to bad outcomes” is a much more effective approach, and it caused me to re-evaluate the importance of AI alignment.
AI alignment persuasion does not require paperclip foom discussions, and these are likely counterproductive. Since an effective explanation exists, I think it would be helpful to focus on this line when evangelizing AI risk.
You may be tempted to discount my opinion because I am an average person, but I think my viewpoint is useful because the extremely high intelligence of the median effective altruist leads to some blindspots. A recent example in the news is the failure to elect Carrick Flynn, an effective altruist running for a congressional seat in Oregon. It wasted $10 million and backfired as “a big optical blunder, one that threatened to make not just Bankman-Fried but all of EA look like a craven cover for crypto interests.”
Being highly intelligent makes it harder to visualize how a regular person perceives things. I claim that my perspective here is valuable for EA, since we’re considering the question of how to persuade regular people. Please give this serious consideration!
Finally, this would also help other EA cause areas. All effective altruists contribute to the EA commons by working hard on various causes. Maximalist AI alignment discussion is burning the commons because it makes EAs look “weird”, which reduces the influence of EA as a whole. Therefore, not just AI alignment EAs, but all EAs have a stake in moving to a more effective communications method for AI risk.
I'm always keen to think about how to more effectively message EA ideas, but I'm not totally sure what the alternative, effective approach is. To clarify, do you think Nintil's argument is basically the right approach? If so, could you pick out some specific quotes and explain why/how they are less inferentially distant?
Hi, I'm the author of Nintil.com (We met at Future Forum :)
Essentially, an essential rule in argumentation is that the premises have to be more plausible than the conclusion. For many people, foom scenarios, nanotech, etc makes them switch off.
I have this quote
Basically, you can't tell people "Nanotech is super guaranteed to happen, check out this book from Drexler". If they don't agree with that, they won't read it, there is too much commitment. Instead, one should just start from premises the average person agrees with (speed, memory, strategic planning) and get to the "AI risk is worth taking seriously". That is a massive step up from them laughing at it. Then one can argue about timelines and how big of a risk it is, but first one has to bring them into the conversation, and my arguments accomplishes (I hope) that.
This makes a lot of sense, thanks so much!
I think I agree with this point, but in my experience I don't see many AI safety people using these inferentially-distant/extreme arguments in outreach. That's just my very limited anecdata though.
Excellent! This is the core of the point that I wanted to communicate. Thanks for laying it out so clearly.
Great! Yes. The key part I think is this:
My view is that normal people are unreceptive to arguments that focus on the first three (advanced nanotechnology, recursive self-improvement, superhuman manipulation skills). Leave aside whether these are probable or not. Just talking about it is not going to work, because the "ask" is too big. It would be like going to rural Louisiana and talking at them about intersectionality.
Normal people are receptive to arguments based on the last three (speed, memory, superior strategic planning). Nintil then goes on to make an argument based only on these ideas. This is persuasive. The reason is that it's easy for people to accept all three premises:
Thanks, this makes sense! Yeah, this is why many arguments I see start at a more abstract level, e.g.
I might have entered at a different vector (all online) so I experienced a different introduction to the idea! If my experience is atypical, and most people get the "gentle" introduction you described, that is great news.