[New Paper] Prioritizing Risks from AI: A Delphi Study of 272 Experts

Peter Slattery 🔸

[New Paper] Prioritizing Risks from AI: A Delphi Study of 272 Experts

Comments 1

Sorted by

New & upvoted

Vasco Grilo🔸

Hi Peter.

TL;DR: We ran a Delphi study with 272 international AI experts to prioritize 24 AI risk domains from the MIT AI Risk Domain Taxonomy.

I wonder how much the probabilities would change if they were about all sources of risk instead of just AI. If they would not change much, could it be that AI has not increased the risks, but simply became associated with all risks because it is in the process of being integrated into practically everything that is relevant for assessing the risks you covered?

Comments

[New Paper] Prioritizing Risks from AI: A Delphi Study of 272 Experts — EA Forum

AI evaluations and standards

TL;DR: We ran a Delphi study with 272 international AI experts to prioritize 24 AI risk domains from the MIT AI Risk Domain Taxonomy. In a business-as-usual scenario, experts judged a more than 10% chance of catastrophic outcomes (i.e., ‘more than 1 million human deaths or more than a USD 100B in financial loss or civilizational-scale intangible impacts’) from 18 of our 24 AI risk domains over the next five years.

They also identified a responsibility gap: AI users and affected stakeholders are most vulnerable, while general-purpose AI developers and governance actors are seen as most responsible for reducing the risks.

Below are three of the key findings and related visualizations.

Key finding 1: Experts judge that many risks could cause catastrophic outcomes under current trajectories

18 of 24 risks were judged to have a more than a 10% chance of causing catastrophic outcomes (which could include more than one million deaths, more than $100 billion in financial losses, or other harms) by 2030 under a business as usual scenario.

AI's Worst Risks figure showing which risks experts believed had the highest chance of the worst outcomes — Figure: Experts’ mean catastrophic risk probability under business as usual and with pragmatic mitigations. Note: “Business as usual” assumes organizations and governments continue their existing practices but do not implement additional AI-specific risk mitigations; “Pragmatic Mitigations” assumes organizations and governments make pragmatic, cost-effective efforts to address AI risks.

Key finding 2: Those most vulnerable to AI risks are not those most responsible for addressing them

According to experts, general-purpose AI developers and governance actors such as governments, regulators, and standards bodies hold primary responsibility for addressing AI risks. In contrast, AI system users and affected stakeholders such as members of the public are most vulnerable to AI risks.

This mismatch means that those who are most responsible for addressing AI risks are not those who are most vulnerable, leading to misaligned incentives in addressing the most important AI risks.

AI's Responsibility Gap figure showing that those most responsible are not the most vulnerable, while those most vulnerable are least responsible — Figure: Experts assessed who is vulnerable to AI risks and who is responsible for addressing them.

Key finding 3: Information, finance & insurance, and national security are the most vulnerable sectors

Across most risks, experts identify information, finance & insurance, and national security as the most vulnerable sectors. The results also show how vulnerability differs across sectors and risk categories.

Who's exposed to AI risk figure showing sector vulnerability across AI risk categories — Figure: Expert consensus on sector vulnerability for AI risks.

A 7-minute overview of the study and findings ↓

Read our paper and explore the interactive results here.

Disclosure: I used an LLM to help generate the first draft of content in this post. I then rewrote and reviewed the text, and I endorse the final version. We also used LLMs as part of the research and communication, for instance, in generating the images and interactives.