TL;DR
- Much of AI safety depends on changing what governments, labs, engineers etc do. That makes it, in large part, a behaviour change problem.
- There are decades of evidence on how to change behaviour, including at the level of institutions and systems, and AI safety mostly isn’t drawing on it.
- Two common mistakes are relying on raising awareness (the knowledge deficit model) and assuming x-risk interventions can’t be evaluated.
- The field needs social science expertise in strategy, grantmaking and evaluation, as well as routes in for social scientists who want to help.
I’m a behavioural scientist. I was Deputy Director of Experimental Research at the Yale Program on Climate Change Communication, and I now lead impact measurement across 15 country offices as Head of Impact at an international NGO. I’ve had a few conversations recently with (fantastic, talented!) people in AI safety, that have made me realise how much the field is missing from social science. For example, several people have repeatedly suggested the need for new interventions focused on public education. Their reasoning is that if people knew more about AI safety they would take action, and so would governments. A couple of other people were reluctant to evaluate the x-risk interventions they were working on, because they assumed that they couldn’t be evaluated. These are understandable assumptions. Conservation held them for a long time too, and social scientists have been working on both problems for decades.
Currently it feels like there are three tracks in the field: technical, governance, and “generalist” (which in practice means ops). I’d argue that much of AI safety depends on changing what governments, labs and publics do, and there is a whole field that studies how to do that. Irving & Askell gets cited as proof the field values social science, but it basically just described a handful of roles for computational cognitive scientists, as Vael Gates has pointed out. Emilio recently proposed a mentored research fellowship to bring more social science talent into AI safety, which I strongly hope gets funded. As he says, the world needs researchers to “study AI’s effects on people and institutions”. AI safety also needs social science expertise to reach its own goals.
Behaviour change for AI safety
So many societal problems are at their heart a behaviour change problem - this is a reckoning that biodiversity conservation went through recently, and I think it applies in part to AI safety also. I’ve argued before that conservation should learn from how public health and international development approached behaviour change, and AI safety can learn from all three. Don’t get me wrong - there are 100% massive technical problems we still need to solve in AI safety! But operationalising those solutions will mean targeting specific actors and behaviours. For example:
- governments: legislate, fund enforcement, coordinate internationally
- lab leadership: adopt safety frameworks and keep to them when it’s costly
- engineers: build safety culture, raise concerns
- compute providers and investors: due diligence, conditions on access
- publics: support regulation and vote on it; choose which products to use
Each one of the above behaviour change challenges has barriers and levers that can be diagnosed, designed for and tested.
An example: whistleblowing
Let’s take one example, the work of AIWI, who support whistleblowers in AI labs. We can apply the COM-B model (Michie, van Stralen and West 2011) to understand why people do or don’t whistleblow. This model posits that there are three essential components for behaviour change to occur: the capability (the physical and psychological ability to change), the opportunity (a physical and social environment that is conducive to change), and the motivation (positive personal beliefs and desires towards change). For whistleblowers, relevant barriers might span:
- Capability: people aren’t sure what counts as a legitimate concern, which channels exist, or what protection they’d have.
- Opportunity: there are very few trusted, AI-specific places to get advice, and the internal channel may be part of the problem.
- Motivation: career and equity risk, loyalty to colleagues and the mission, and the very human reasoning that someone more senior must already know.
Interventions that use behavioural models tend to be more successful. Combined with primary research, a model like this would help identify the best ways to support potential whistleblowers.
The knowledge deficit model
Unfortunately, it’s a super common mistake for people to rely on the knowledge deficit model when it comes to behaviour. This is basically the assumption that people just don’t know enough, and if we give them the correct information, they’ll act. A wealth of evidence shows that awareness and knowledge rarely translate into behaviour on their own. One great example is D.A.R.E., a programme where police officers taught schoolchildren about the dangers of drugs. It was the most widely used school drug prevention programme in the US, but meta-analyses found it had little to no effect on drug use. Knowing was not enough!
Sadly, I have many more examples, including a (really cool) intervention improving learning, but not behaviour. Education can have an impact, but combining multiple interventions (like education, prompts, and feedback) tends to be more effective at creating positive behaviour change. For AI safety, this means explainer-heavy strategies need theories of change that go beyond awareness. An AI risk explainer video should be judged on what viewers do afterwards, as well as what they learn.
Message testing is one place where this is starting to change. I’ve run message tests myself, including an experiment with almost 58,000 people in 23 countries, which found that climate messages increased support for climate action. In AI safety, Seismic Foundation and colleagues tested AI risk messages with over 1,000 people, and found that risks to jobs and children mobilised people more than existential risk did. Another survey experiment with 1,200 Americans found that fear raised perceived risk the most, but a hopeful message showing that regulation could work built more support for regulation. That same Seismic paper also notes that the public broadly backs strong AI regulation, but ranks it low against other issues. The concern is already there, and it isn’t turning into action.
This is a promising start, but most message tests measure attitudes, knowledge and intentions, which often don’t lead to behaviour. In one of my experiments, messages encouraging people to talk to friends about plant-based eating raised their intentions to do so, but didn’t move them to the next stage of preparing to act. Wherever possible, tests should include a behavioural outcome as well, like signing a petition, contacting a representative or sharing information.
Institutions and social movements
The previous paragraphs focused more on individual behaviour, but social science also has a lot to say about institutional and systems change. For AI safety, that means looking beyond individuals to the institutions and incentives around them. Most AI governance work happens inside governments and think tanks. That work is valuable, but there is also a large research literature on how social movements create change from outside. AI safety already has groups running protests and public campaigns, and they can draw on a solid evidence base. Experiments show that extreme protest actions tend to reduce public support for a movement, because observers see them as immoral. But that backlash can be short-lived: after Animal Rising disrupted the Grand National, the negative effects on public attitudes had faded six months later, and the protest boosted media attention and mobilisation. A radical faction can also increase support for moderate organisations in the same movement, by making them look more reasonable in comparison (a positive radical flank effect). Media coverage matters too, as the way outlets covered the Grand National protest was linked to how much people supported the activists. Overall, the picture is more nuanced than either side of the “is disruption good?” debate tends to assume.
Evaluation and unknown unknowns
There’s a general problem of the “unknown unknowns” - when you don’t have prior expertise, it’s easy to assume something can’t be done at all. This is particularly true for evaluation. I’ve written before about the need for stronger evaluations of AI safety talent programmes. I suggested a regression discontinuity design, but there are tons of other methods that could be used more broadly in the field (including for policy and advocacy work). These include contribution analysis, process tracing (including Bayesian process tracing, which lets you put numbers on how much each piece of evidence should update you), outcome harvesting and theory-based evaluation. I’ve used these kinds of methods myself, looking at why demand for ivory fell in Japan. The advocacy evaluation literature is a good place to start.
A lot of AI safety work focuses on x-risk, and it may seem like that makes it impossible to evaluate because well, either we go extinct or we don’t! However, every intervention should have a rigorous, specific theory of change, with the intervening steps and causal mechanisms clearly set out. We can test each of these steps to build up an evidence base for how well we’re reducing x-risk overall. Otherwise, funding goes to things nobody is learning from, and avoidable basic mistakes get repeated.
For example, take a campaign pushing frontier labs to adopt safety frameworks. The theory of change might be that public and investor pressure leads a lab to publish a framework, the framework contains specific, checkable thresholds, and the lab then sticks to them when a model crosses one. Each step leaves evidence: whether frameworks appear after campaign activity, whether their thresholds are concrete or vague, and whether deployments are delayed or modified when evaluations flag risks. Animal advocacy learned this the hard way with cage-free commitments: securing a pledge and getting it implemented are two separate outcomes that need tracking separately.
Building on the theory of change point above, participatory systems mapping is another underused tool. It brings stakeholders together to define the problem, agree the system’s boundaries, and map causal links to find leverage points. Applied to frontier lab governance, for example, it would show where interventions reinforce or undercut each other, which no single organisation’s theory of change can. It can also surface the gap between what stakeholders say they’re aiming for and their more pressing priorities (Stroh 2015). A lab may be publicly committed to safety, for instance, while its most pressing concern is staying at the frontier. I haven’t come across an AI safety organisation that has run one with outside experts. If yours has, I’d love to hear how it went, and if you’re thinking about trying one, get in touch.
What I’d like to see
A few concrete changes would help:
- Social scientists on grant panels and strategy reviews.
- Behavioural expertise involved from the design stage of comms and advocacy strategies.
- Evaluation built in from the start.
- Hiring for disciplinary expertise, beyond hiring social scientists into ops roles.
To sum up, the field’s hardest problems include human ones, and there are decades of evidence on how to work on them. Please start bringing social scientists into the conversation. If you’re designing an intervention and want a behavioural science sounding board, message me!
I am an evaluator myself, living in a non-frontier developing country, and again I see this is as more than interesting and offers a rather "common" and "touching the grass" point of view.
Yes the worldview around longtermism and existential risk are valid and accepted within EAs yet still "far fetched" for people outside the community.
I think we need more bridges, or to be more humble, we need more planks to bridge the importance of AI Safety to the "laypeople" to convince that Safety is not the interest of those super small and elite circles in the frontier AI labs and higher education institutions.