Summary
In 1979, a nuclear power plant in Three Mile Island (TMI), Pennsylvania experienced a meltdown. In response, the public became much more concerned with the safety of nuclear power and successfully demanded increased regulation. TMI seems potentially instructive for how the American public might respond to an AI “warning shot”, i. e. a situation in which AI visibly causes significant but not catastrophic damage. In particular, TMI suggests that
- The public is capable of being quickly and strongly influenced by warning shots (95%).
- The public’s response to a warning shot is influenced more by pre-existing discussions about the risk (including fictional accounts), media coverage of the accident, and the public’s attitude towards the actors responsible for the warning shot than by the magnitude of the accident (90%).
- Confusion about the relevant technology may lead the public to fail to recognize the threat prior to the warning shot, but once the threat is clear, it may cause the public to become more afraid (50%).
- Capitalizing on warning shots is much more likely if the actors who aim to do so are part of a broader political coalition (85%).
- The legislative response is likely to be ineffective by default, particularly given that the technical problems involved in regulation are complicated (80%). However, industry may be able to respond more effectively through internal regulation in order to avoid further reputational damage.
These takeaways suggest that if the longtermist AI governance community (henceforth referred to as “the AI governance community”) wants to effectively influence AI policy in response to an AI warning shot, it should attempt to influence the media environment prior to the warning shot (perhaps through the release of realistic seeming fiction about AI takeover scenarios), utilize/play into public suspicion towards big tech, form allies with other groups who support AI regulation, and think hard in advance about what policy responses would actually reduce AI risk.
Epistemic Status: This post was written as part of the Stanford Existential Risks Initiative's Summer Research Fellowship, and is the product of ~50 hours of writing and research. Moreover, n = 1, so this post overall provides relatively weak evidence for its claims; the probabilities assigned are more based on my priors than on the evidence from Three Mile Island. That being said, I am quite confident in the straightforwardly historical claims made.
Acknowledgements: Thank you to Matthew Gentzel and Ben Snyder for their feedback
Introduction
Currently, one of the biggest barriers to governments taking major political action to reduce existential risk from AI is that neither voters nor government officials take the risk very seriously. This could possibly change in response to a “warning shot,” i. e. an event in which an AI system causes a major accident but does not precipitate existential catastrophe. Thus, it seems important to answer the question of how the public might respond to a warning shot if it occurs, and how we might channel this response into effective political action to reduce AI risk.
In order to answer these questions, I examine a previous case of a warning shot-like event which led the public to successfully demand legislation to reduce future risks: the Three Mile Island (TMI) nuclear accident. In March, 1979, a nuclear power plant in Three Mile Island, Pennsylvania experienced a partial meltdown. While a total meltdown was ultimately averted and, according to most estimates, the damage was relatively minor, TMI prompted increased public concern about the safety of nuclear power and led to major efforts to regulate it. Thus, TMI plausibly serves as a good case study for what factors make a warning shot generate major public and legislative response.
Limitations
TMI is far from a perfect analogy to an AI warning shot. Firstly, the risks posed by AI are very different to those posed by nuclear power. In particular, risk from nuclear meltdowns may be easier to understand and certainly seems more “realistic” to the public than risk from misaligned AI. Moreover, U. S. culture and politics have changed in important ways since 1979. For example, in the more partisan environment of US politics today, it is possible that the question of how to respond to an AI warning shot might become a partisan issue. Finally, particularly given that the AI industry will likely grow significantly in the next decade prior to a warning shot, it might be much more influential than the nuclear industry was at the time of TMI and effectively oppose legislation. Nevertheless, TMI is one of few cases of a widespread public response to a major accident involving technology in recent history. Thus, insofar as we want to have some kind of baseline for what responses to warning shots look like, studying the case of TMI seems valuable. With this in mind, I present 7 lessons that I think we can learn from the example of TMI for how the public might respond to an AI warning shot, and how we might take advantage of this response.
Lessons from Three Mile Island
Lesson #1: Warning shot events can have large effects on public opinion
Some people believe that the public is insufficiently attentive to the state of the world to respond in any strong manner to warning shots (see: COVID and pandemic preparedness measures). However, the case of TMI suggests that this is false. Prior to the incident, public opinion was generally favorable towards nuclear energy, with about 60% of the public supportive of building nuclear power plants in the US and about 30% opposed. Immediately after the accident, the public became about evenly split on the issue, with about 45% on each side. While support for nuclear power seems to have briefly rebounded after a few months, this rebound then quickly reversed partially owing to a second wave of coverage in response to the release of a public report on TMI. Moreover, in addition to shifting views on nuclear power, TMI seems to have caused the public to pay more attention to the issue of nuclear power in general, as the percent of people answering that they did not have an opinion about whether more nuclear power plants should be built in the US dropped from over 20% to under 10% immediately following the accident. Finally, TMI prompted a significant legislative response. In particular, immediately after the accident, Jimmy Carter formed a commission to investigate the accident and to make recommendations about what could be done to reduce the probability of future accidents. The commission then released a report which ultimately resulted in significant reforms to the agency which regulated nuclear power, the Nuclear Regulatory Commission (NRC). While these reforms were not very effective, they did involve a significant restructuring of the NRC. Thus, TMI suggests that as irrational as the public can be, it can quickly update in response to major technological accidents and pressure public officials into taking major actions aimed at reducing future risks.
Lesson #2: Which warning shots grab the public’s attention is not mostly explained by the objective “size” of the warning shot
While the evidence presented above suggests that warning shot events can get the public concerned about an issue, the public’s broader pattern of response to nuclear energy accidents suggests that which specific accident grabs its attention may be more a matter of chance and media coverage than of the actual size of the issue. TMI was not the first nuclear power accident in the US; throughout the 1950’s, ‘60’s, and early ‘70’s, there were several other accidents involving nuclear power that failed to generate nearly as much public attention as TMI. While TMI was in important ways “bigger” than these other accidents, it was in other ways “smaller” insofar as some of the past accidents caused deaths while TMI did not immediately lead to any. Similarly, the differences in the public response to nuclear energy accidents between countries does not seem to be well explained by how they are actually affected by the accident. For example, different European countries’ publics had significantly different responses to the Chernobyl nuclear meltdown. However, the size of these responses was unrelated to the extent to which the country in question was negatively impacted by the radiation which Chernobyl generated. This suggests that it may be difficult to predict what specific accident or kinds of accidents will draw public attention to AI risk.
Lesson #3: Fiction and media coverage can greatly influence public response to a warning shot
If the public’s strong reaction to TMI was not primarily due to the nature of the accident itself, then what caused it? While public response to an event is determined by a huge number of variables, two key ones in the case of TMI seem to have been the prior presence of fiction which primed the public to be concerned about nuclear meltdowns and the media coverage of the event itself. Some have argued that a major reason for lack of public concern about AI risk is that people see AI takeover scenarios as science fiction rather than as something that could happen in real life. However, the case of TMI suggests that fiction can sometimes have the opposite effect, causing the public to be concerned about the technological accidents which it presents. In particular, the movie “The China Syndrome” was released less than two weeks prior to TMI. The China Syndrome presented the fictional story of the near-meltdown of a nuclear power plant and the efforts of the company responsible for the accident to cover it up. It was widely viewed by the public, starring major actors such as Jane Fonda and Jack Lemmon, and explicitly aimed to call attention to the issue of nuclear safety, dramatizing the possible negative effects of nuclear meltdowns and portraying nuclear power companies as corrupt. Moreover, even before TMI, at least some commentators took the concerns raised by the movie seriously. While other commentators viewed the concerns which the movie raised as overblown, it would prove critical to the public response to TMI. In their coverage of TMI, media commentators often drew parallels to The China Syndrome. For example, in its cover story immediately following the accident, Time magazine stated that the public statements given by the public spokesman for the plant sounded “as if they were taken right out of the script for the film The China Syndrome,” and Newsweek coverage of the event included images taken from the film. Confirming the importance of the film, the report issued by the NRC months after the accident stated that its effects on the public perception of TMI had been “immeasurable.”
A second major reason why the public seems to have reacted in the way that it did was that media coverage of the accident was extensive and dramatic. Partially, this was due to the fact that the media had little else to cover during the week in which TMI occurred. However, TMI also happened at a time when the media was already paying increased attention to nuclear power issues due to the release of the China Syndrome, Jimmy Carter’s efforts to pass his energy policies, and generally increasing concern about the safety of nuclear power throughout the 1970’s. As a result, TMI received lots of TV coverage, with 40% of all evening news coverage being devoted to it during the first week of the crisis. Moreover, this coverage was in many cases quite dramatic. For example, Walter Cronkite of CBS evening news stated during his coverage of TMI that “The world has never known a day quite like today,” and ABC News showcased sentimental stories of mothers evacuating with their children. Between the release of The China Syndrome and the coverage of the event, the public was heavily influenced by media to respond fearfully to TMI.
Lesson #4: The public is more responsive to warning shots if it does not trust the authorities responsible for them
Another major reason that the public responded strongly to TMI was that it distrusted the nuclear power industry and other authorities, partially due to the conflicting reports which they issued about the accident. Polls from the 1970’s reveal that public skepticism towards the nuclear industry and its regulators pre-dated the crisis, perhaps owing to Americans’ generally skeptical attitudes towards authorities. However, TMI itself increased this distrust. During the crisis, spokespeople for the plant were confused about what was happening. Nevertheless, in response to media pressure, they made various reassuring positive statements which they later had to retract once they learned more information. For example, Jack Herbein, the vice president and spokesperson for Metropolitan Edison (the company operating the power plant), first claimed that the accident was not serious, and the utility spokesperson Dave Klucsik stated that “[t]here is absolutely no risk of a meltdown.” However, soon afterwards, the NRC declared that a meltdown was possible, and media figures revealed that they had been told by people working at the plant that the crisis was more severe than their public statements had indicated. Officials continued to give contradictory and confusing statements as the crisis continued. As a result, the public began to suspect that industry and government were both conspiring to conceal information from them. This lack of trust and information caused further panic, with many local residents citing anxieties over conflicting information as their reason for evacuating the area during the crisis. Moreover, media coverage after the event continued to push this narrative, with The Washington Observer-Reporter running a headline stating that “Three Mile Island Meltdown Was Near” and other media outlets covering accusations that the radiation released during TMI had had more damaging health effects than authorities had claimed.
Lesson #5: Lack of understanding is not necessarily a barrier to public concern, and in fact can in some cases increase the public's level of fear
One reason that some suspect that the public may not become concerned with AI risk even if a warning shot occurs is that it does not understand the technical problems involved in AI alignment. However, in the case of TMI, the public’s lack of technical knowledge does not seem to have made it any less concerned about nuclear energy. Due to the inconsistency of the reporting, as well as the complex nature of nuclear power, much of the public seems to have been confused about exactly what was happening during the meltdown. However, this led many to infer that the situation was worse than it was in actuality. For example, 36% of those who responded to an NYT poll believed that the accident would produce a mushroom cloud explosion. Confusion about radiation seems to have particularly contributed to public panic. Many viewed radiation as a “mysterious and dangerous” force due to its ability to invisibly cause disease and death, which heightened their fear. Similarly, some commentators framed nuclear energy as an unnatural power which humans should not tamper with in response to TMI, invoking both Christian and classical rhetoric against playing God or opening the “Pandora’s box” of nuclear power. Thus, while lack of understanding can sometimes cause the public to be unconcerned about a risk, it can also lead to heightened perception of the risk once it is noticed, particularly if it seems spooky or alien.
Lesson #6: Capitalizing on warning shots is easier if you have allies
While there are many factors that increase the probability of legislative action in response to a warning shot, one important one seems to be whether the actors pushing for legislative action have political allies. In the case of the US, the anti-nuclear energy movement made allies with social movements that were originally much larger than it, such as the broader environmental movement and the anti-nuclear weapons movement, and was organized by protestors with connections to the civil rights and women’s liberation movements. The anti-nuclear energy movement in Finland seems to have adopted a similar strategy. In Germany, the movement was initially coolly received by existing political parties but eventually found a major ally in the newly founded Green Party. In contrast, in countries such as France, the anti-nuclear power movement was not similarly part of a larger coalition and did not have a political party supporting it. Even though nuclear power had similar levels of public support across the different countries, the French anti-nuclear power movement failed to halt the construction of new nuclear power plants, while it largely succeeded in Germany, Finland and the United States.
Lesson 7: The legislative response to a warning shot involving a technically complicated technology may be ineffective by default, though the private sector may respond more effectively
While TMI prompted significant responses from legislators and regulators, these responses seem like they did not have actually done much to reduce the risk of nuclear meltdown. While the rate of development of nuclear reactors in the US slowed considerably in the 1980’s, this seems to have been significantly if not entirely due to pre-existing economic trends and increased regulation during the 1960’s and ‘70’s rather than the legislation caused by TMI. Furthermore, internal reports by the NRC suggest that the agency was not very effective at regulation even after the reforms, as it often failed entirely to inspect plants and, when it did, lacked the technical knowledge to understand what changes were necessary. However, despite the lack of successful legal regulation, TMI prompted the nuclear power industry to informally regulate itself which was more effective due to the greater technical knowledge of people working in the nuclear power industry.
Conclusion
Several facts about the TMI incident suggest that there are reasons to believe that, conditional on there being an AI warning shot, the public will become concerned about AI risk. Broadly, TMI serves as a proof of concept that the public can quickly become concerned with the safety of a technology and demand regulation of it in response to major accidents. Moreover, an AI warning shot seems likely to share some of the characteristics that seem to have contributed to the public’s strong response to TMI. Firstly, just as there was increased coverage of safety problems with nuclear power in the years and months leading up to TMI, there has recently been increasing attention to the negative effects of AI such as algorithmic discrimination and polarization due to social media algorithms, which will likely only increase over time. Additionally, just as public concern with the safety of nuclear energy post-TMI was increased by feelings of distrust towards regulators and the nuclear power industry, the broad distrust of “big tech” by the contemporary public seems likely to prime it to respond strongly to an AI warning shot. Finally, just like radiation, risk from AI is currently poorly understood by the public but, once it is made salient, is alien-seeming and threatening in a way that seems likely to further scare people.
However, even though these factors suggest that there is a substantial chance that the public becomes concerned about AI in response to a warning shot, further efforts are necessary to increase the odds that this reaction occurs and that it is translated into effective legislative action. The critical factor for the public's response to an AI warning shot which the AI governance community can attempt to influence is media coverage. If the analogy to TMI holds, both the immediate media response to an AI warning shot as well as the media environment prior to the warning shot, such as fictional accounts of AI risk and political discourse about the potential harms of AI, would be very important for how the public would react to an AI warning shot. Thus, aiming to influence media narratives about AI, as well as perhaps releasing fictional media such as an episode of a TV show (e. g. an episode of Black Mirror) or a movie which depicts a realistic seeming AI takeover situation might be effective pathways to increase the odds of a strong public response.
In addition to increasing the odds that the public becomes concerned in response to a warning shot, the AI governance community also needs to put itself in a position to actually influence legislation if the public becomes concerned about AI. To that end, the successes of the Finnish and US anti-nuclear movements which forged a broader coalition opposed to nuclear power are instructive. In the case of AI risk, forming a coalition with actors who seem like they could possibly favor stricter regulation of the AI industry such as those concerned with bias and discrimination and AI unemployment seems potentially worthwhile. If the AI governance community chooses to pursue political allies, then it should be mindful about who those are, as picking allies also means picking enemies. For example, if it were to align itself with the aforementioned movements, it would run the risk of being perceived as a left-wing movement and thus alienating conservatives. However, particularly given that it is probably smaller than the anti-nuclear movement was before TMI, the AI governance community seems unlikely to successfully influence policy in response to a warning shot by itself.
Perhaps most importantly, even if put into a position of influence, the AI governance community will also need to have effective concrete policy proposals ready if the legislative response to a warning shot is to reduce AI risk. Without such carefully researched proposals, regulation is likely to be ineffective by default, because effective regulation of AI requires even more technical knowledge than effective regulation of nuclear energy which seems to have been beyond the capacity of the NRC. Indeed, given the potentially dim prospects for such regulation, it could also be wise for the AI governance community to attempt to influence AI developers to institute non-legal regulations in response to a warning shot than to influence the US government, as developers are likely to have more technical knowledge and be closer to the actual problem than regulators.
Zooming out, I think that major legislative action to reduce AI risk is unlikely without a warning shot, as neither the public nor politicians seem likely to be convinced by abstract arguments that AI could be very dangerous without concrete examples. Thus, to the extent that reducing AI risk through legislation is a viable strategy, understanding the dynamics of the public’s response to a warning shot and how that response affects legislation is important. While this post has begun to address these questions, further research on this seemingly neglected question is necessary.
