Some underrated reasons why the AI safety community should reconsider its embrace of strict liability

Introduction

It is by now a well-known fact that existing AI systems are already causing harms like discrimination, and it’s also widely expected that the advanced AI systems which the likes of Meta and OpenAI are building could also cause significant harms in the future. Knowingly or unknowingly, innocent people have to live with the dire impacts of these systems. Today that might be a lack of equal access to certain opportunities or the distortion of democracy but in future it might escalate to more concerning security threats. In light of this, it should be uncontroversial for anyone to insist that we need to establish fair and practically sensible ways of figuring out who should be held liable for AI harms. The good news is that a number of AI safety experts have been making suggestions. The not-so-good news is that the idea of strict liability for highly capable advanced AI systems still has many devotees. 

The most common anti-strict liability argument out there is that it discourages innovation. In this piece, we won’t discuss that position much because it’s already received outsize attention. Instead, we argue that the pro-strict liability argument should be reconsidered for the following trifecta of reasons: (i) In the context of highly capable advanced AI, both strict criminal liability and strict civil liability have fatal gaps, (ii) The argument for strict liability often rests on faulty analogies and (iii) Given the interests at play, strict liability will struggle to gain traction. Finally, we propose that AI safety-oriented researchers working on liability should instead focus on the most inescapably important task–figuring out how to transform good safety ideas into real legal duties. 

AI safety researchers have been pushing for strict liability for certain AI harms 

The few AI safety researchers who’ve tackled the question of liability in-depth seem to have taken a pro-strict liability for certain AI harms, especially harms that are a result of highly capable advanced AI. Let’s consider some examples. In a statement to the US Senate, the Legal Priorities Project recommended that AI developers and deployers be held strictly liable if their technology is used in attacks on critical infrastructure or a range of high-risk weapons that result in harm. LPP also recommended strict liability for malicious use of exfiltrated systems and open-sourced weights. Consider as well the Future of Life Institute’s  feedback to the European Commission, where it calls for a strict liability regime for harms that result from high-risk and general purpose AI systems. Finally, consider Gabriel Weil’s research on the promise that tort law has for regulating highly capable advanced AI (also summarized in his recent EA Forum piece) where he notes the difficulty of proving negligence in AI harm scenarios and then argues that strict liability can be a sufficient corrective for especially dangerous AI. 

The pro-strict liability argument

In the realm of AI safety, arguments for strict liability generally rest on two broad lines of reasoning. The first is that historically, strict liability has been applied for other phenomena that are somewhat similar to highly capable advanced AI, which means that it would be appropriate to apply the same regime to highly capable advanced AI. Some common examples of these phenomena include new technologies like trains and motor vehicles,  activities which may cause significant harm such as the use of nuclear power and the release of hazardous chemicals into the environment and the so-called ‘abnormally dangerous activities’ such as blasting with dynamite. 

The second line of reasoning is that fault-based liability (the alternative to strict liability) has weaknesses that profoundly undermine its usefulness in our attempts to regulate highly capable advanced AI. One frequently cited weakness is the idea that legislators and judges don’t have the necessary technological and risk knowledge to establish the sorts of duties that any fault-based liability regime must outline.  Another is the idea that the unpredictability, opacity and autonomy of advanced AI makes it difficult for anyone to prove fault and causation, something that fault-based liability systems usually require. 

AI safetyists shouldn’t be in a rush to embrace strict liability 

We present three arguments to challenge any embrace of strict liability. 

Argument 1

In the context of AI development, both strict criminal liability and strict civil liability have fatal gaps

For the good of readers without a background in law, it’s important to begin by clarifying that strict liability exists in two varieties: strict criminal liability and strict civil liability. The main factor that differentiates the two is the type of penalty imposed after liability is confirmed. If strict criminal liability is in play, the penalty may be jail time (although fines or other non-custodial sentences are also regularly imposed). In situations where strict civil liability is proved, the penalty is normally a non-custodial sentence, which is often the payment of damages. Unsurprisingly, AI safety researchers seem preoccupied with strict civil liability. However, it's possible that some may be open to endorsing strict criminal liability as well. 

There are two central cases that AI safetyists are typically worried about. The first is a situation in which a malicious actor uses highly capable AI in a manner that results in significant harm. The second is a situation in which some highly capable AI that developers and users honestly think is safe or fine turns out not to be as safe as they imagined. The difference is obvious: In the first scenario there is malicious intent while in the second one there is good-faith decision making. We think that strict criminal liability is inappropriate for either case. In the first central case, any malicious users of highly capable AI can be prosecuted under ordinary criminal law since their malevolent intent can be proved (no wonder we refer to them as “malicious users'') so it would in fact be laughably illogical to even consider strict criminal liability for such a case.  

Strict criminal liability can be used in the second central case but we think that, as some scholars have argued, it would be unreasonable for a legal system to straight-up impose criminal sanctions, fault be damned, in a situation like that. It would be unreasonable particularly because any AI cycle has a number of actors (for example the compute provider, model developer, system deployer and user) and harm could be the result of an oversight by any of them–or, in fact, none of them. Consequently, strict criminal liability is in principle an excessively onerous regime to use for such a case. In any event, we think the adoption of strict criminal liability will almost certainly be a non-starter, as we will discuss further in our third argument. 

As stated earlier, AI safety researchers are more fixated on strict civil liability so let’s now turn our attention to that. The focus on strict civil liability can probably be explained by the fact that it is less onerous (typically no jail term when liability is established) and therefore the penalty (usually the payment of damages) is more defensible. There is also a belief  that strict civil liability can have a deterrent effect or can–at the very least–motivate more caution and care. 

While it is true that having to pay damages seems less onerous than the possibility of jail time, that doesn’t necessarily mean that it’s fair to apply strict civil liability for highly capable advanced AI. Given that there is still no way for model developers to deterministically guarantee a model’s expected behavior to downstream actors, and given the benefits that advanced AI could have in society, we think it is unfair for an actor to be forced to pay damages regardless of any steps they’ve taken to ensure the advanced AI in question is safe. In line with this, we think there is an important lesson to be located in the architecture of discrimination law across the world. In most jurisdictions, anyone accused of indirect discrimination (where the impact of some practice is discriminatory but there is no discriminatory intent) is always allowed to offer justification before the question of their liability is decided. Given that discrimination is universally considered a very serious issue, we think it is instructive that so many societies consider it wrong to hold anyone strictly liable for discrimination if the necessary intent cannot be shown. We think the same should be true for harms caused by AI. 

Beyond that, we also think that AI safetyists may be inflating the promise of strict civil liability. First, let’s dispose of an idea that could be motivating some AI safetyists’ embrace of strict civil liability–That strict civil liability will induce companies to stop doing the things in question, like the development and deployment of highly capable advanced AI. The truth is that in reality strict civil liability does not deliver on that goal, especially when the thing in question might be very lucrative (as is the case with AI development). We’ve not heard or read of any situation where a company stopped doing something because it was concerned about the threat of strict civil liability. That said, there is research which shows that strict civil liability can motivate companies to build more safe products. So we concede that strict civil liability could motivate AI developers and deployers to be more careful. However, clarifying the duties and faults of AI developers through a fault-based liability system can also motivate safety and caution, so what is the point of preferring strict civil liability? A formidable riposte to this point might be the claim that fault-based liability incentivises developers to merely meet the set duty of care while strict civil liability incentivises them to take the greatest caution possible. Still, we think that setting very high duties would likewise force developers to take the greatest caution possible. Fair point, pro-strict liability AI safetyists will say, but setting duties of care and proving fault and causation in the context of AI is very difficult. Besides, legislators, lawyers and judges are not technically equipped to be engaging in such an enterprise. 

We must confess that when we encountered this argument we found it strange. It ignores the fact that legal professionals have in the past been able to develop workable legal duties and standards for some highly technical issues. For example, American courts have been able to figure out a case with complex engineering evidence on the safety of a chainsaw and another case featuring complex medical and actuarial evidence. Furthermore, when done correctly, the task of creating legal duties and standards in highly technical fields usually involves experts who possess the appropriate  technical knowledge, so what’s the big worry? One could respond by claiming that proving the violation of any established duties will still be extremely difficult so we should just endorse strict civil liability. However, that concern can be addressed by tinkering with evidence discovery and disclosure rules (as the Draft EU Liability Directive in fact does) or adopting a regime of fault-based liability with a reverse burden of proof. It doesn’t necessitate the adoption of strict civil liability. 

Even if strict civil liability is taken up for highly capable advanced AI, it seems likely that judges will interpret the law such that fault will still need to be proved in many circumstances, recreating a fault or wrongs-based system in all but word. Consider for example US product liability law. Even though strict liability is apparently the regime that ought to apply there, a claimant is required to first prove that the product causing harm is defective. Essentially, proving defectiveness is proving fault. As some scholars have claimed, this shows that “tort law is filled with strict liability wrongs” meaning that it’s still often the case that a wrong has to be proved for strict civil liability to apply. In our view, AI safetyists might as well save time and focus on figuring out the kinds of faults or wrongs that should trigger liability.  

Of course, the strict civil liability that allows absolutely no way out for a defendant once harm is proved exists. US law, for example, applies that kind of strict civil liability for the so-called ‘abnormally dangerous activities’. However, we think highly capable advanced AI is radically different from abnormally dangerous activities and so it’s hard to defend the use of a similar liability standard in its regulation. We’ll discuss this further in our second argument. 

Argument 2

The analogies used to justify strict liability are flawed in significant respects

As we showed earlier on, the pro-strict liability argument is substantially based on analogies. In effect, AI safety-oriented advocates of strict liability select examples of phenomena governed using strict liability and then explicitly or implicitly draw the implication that highly capable advanced AI has some kind of important similarity to the phenomena and should therefore also be governed using strict liability. This argumentative technique is only compelling if the phenomena being compared are sufficiently similar. We don’t think this standard has been met. 

The common comparator phenomena include trains, motor vehicles, pipelines, nuclear power, the handling of industrial chemicals and the so-called abnormally dangerous activities (which has been defined more specifically in US law). 

Let’s start with the comparisons made with trains, motor vehicles, nuclear power and the handling of industrial chemicals. We’re unconvinced by these comparisons because of a methodological flaw. None of the comparisons begins by reasoning out how the features whose similarity is enough to make an analogy accurate are selected. Is it the primary, most elemental features of the things being compared? Is it the secondary features created through human interaction with the primary features? Is it the primary and secondary features as understood by a certain cadre of experts or is it the primary and secondary features as understood by the general public? To counteract accusations of starting with a conclusion and then scrambling for evidence to support it, it’s critical for this methodology to be clarified. 

It may be useful to look at this in a more practical way. Autonomy seems like a primary feature of the highly capable advanced AI currently being built. None of the comparators typically used shares this feature. Surely, that should matter in any analysis. And although we think that highly capable advanced AI carries significant global catastrophic risk, there is no consensus on this among experts or the public at large–while there is consensus on the sorts of risks posed by the comparators listed above. Again, shouldn’t this matter in any analogizing? And so on and so forth. 

The other common analogy is with “abnormally dangerous activities” as understood in the US. For this one, we’re in luck since US law has been giving meaning to the term for years now. And the truth is, as even Gabriel Weil accepts, it’s super unlikely that the training and deployment of highly capable advanced AI would fit the longstanding legal definition of an “abnormally dangerous activity”. In making our case here we’ll stick to the key features that we think the training and deployment of highly capable AI lacks relative to the activities that have typically been understood as abnormally dangerous. 

To begin with, when you consider the kinds of activities that have been found to be abnormally dangerous in the past (some common ones, for example, are blasting by dynamite or keeping wild animals), you immediately realize that (i) the risks have already materialized in reality and (ii) there is consensus in popular and expert perception of the risks. In the words of some scholars, the activities are not just abnormally dangerous, they are “conspicuously so.” Indeed, it turns out that the common law doctrine from which the concept of abnormally dangerous activities flowed was developed in Victorian England by judges who had experienced the materialization of the risks in question. Other than that, the theoretical basis for courts’ continued recognition of strict liability for abnormally dangerous activities is the idea that it makes sense because the actors in question impose substantial risk in a course of conduct typically undertaken for their own benefit. Unfortunately, the training and deployment of highly capable AI just doesn’t meet these metrics. We’re yet to see any global catastrophic risk from the training and deployment of highly capable AI, and there is no consensus among experts or the general public about the GCR risk posed. Finally, highly capable AI could have enormous societal benefits beyond profiteering for the companies. The training and development of highly capable AI can therefore only sit awkwardly next to abnormally dangerous activities.  

Argument 3

Given the interests at play, strict liability will struggle to gain traction 

To succeed, AI safety advocates can’t just push for the strictest regulation possible. Even if we’re convinced that the strictest regulation is the most fitting, there are other strategic reasons why such a push might be bound to fail. In light of this, we think it’s important to push for proposals that have more than a 40 pc chance of success based on the realities around us. We estimate that strict liability has less than a 15 pc chance of success because of the economic and national security pressures in the world and because there is still no consensus that highly capable advanced AI poses a sky-high level of risk. 

We all know by now that countries like the US and China are in a rush to become global leaders in AI development and use due to the economic and military edge that advanced AI could give them. This ambition is destined to shape their AI regulation decisions. Countries like the US and the UK, for example, have decided to actively pursue a pro-innovation approach that seeks “to mitigate the risks that AI poses without inhibiting AI development and use.” These two countries have emphasized that any regulations put in place–including liability rules–should not inhibit AI development. No surprise then that the UK has explicitly noted its concern that allocating too much responsibility to developers might discourage them from building AI. We expect this view to undergird a hostile climate for any strict liability proposals. Unless some kind of harmful events occur as a result of the development and use of highly capable AI, the chances of strict liability being taken up are very low.  

There is already evidence to confirm our suspicions. Despite recommendations from the European Parliament, EU citizens, academic institutions and consumer organisations, the European Commission’s Draft AI Liability Directive still doesn’t adopt strict liability for harms caused by high-risk systems. Instead, it proposes a lightly adjusted system of fault-based liability. The fact that a bureaucracy frequently chided for being too pro-regulation did not take strict liability surely says something about how long it might remain in the doldrums.

The other reason that strict liability is unlikely to gain traction is the fact that there’s still no expert consensus on how high the risk posed by highly capable advanced AI is. Not only are some experts unconvinced that AI poses a GCR-level risk, some regulators also remain fairly unconvinced. One of the reasons why the European Commission rejected strict liability is because it did not buy the super high risk argument. The consequences that this absence of consensus should have on AI safety strategies has been written about before. In the context of liability proposals, we think AI safety advocates would be better off trying to map out the specific faults that ought to lead to liability because such proposals have a better chance of success (and will also achieve our goals), especially if they also respond to near-term harms. At the end of the day, we must ask ourselves whether we’re more likely to reach the other side by pushing hard against an immovable wall or by finding the most vulnerable part of the wall and then pushing there. Your guess is as good as ours. 

Final point

We’re aware that to some, our piece may seem like an overzealous screed that doesn’t accurately represent what’s going on in AI safety-oriented liability research. To be clear, we aren’t saying that AI safetyists are not trying to figure out the specific duties and faults that ought to trigger liability for the harms caused by highly capable advanced AI. Indeed, LPP’s statement to the US Senate contained a number of recommendations about duties that need to be legislated. We’re saying that in AI safety circles there’s a dalliance with strict liability that isn’t well thought out, and it should be abandoned in favor of more work on how best to design legal duties and standards out of the great ideas being proposed in the community. In other words, let’s invest our time in figuring out what kinds of faults are sufficiently bad, and how they should be framed in law. 

When subjected to careful scrutiny, pro-strict liability arguments in AI safety appear to be insufficiently substantiated, unreasonable, unfair or unrealistic.  This unnecessarily weakens our chances of gaining broad support at a time when that matters a lot. Because the stakes are so high, we should do something while we still can. As for the question of which liability regime ought to be adopted, we generally think that in scenarios where clear malicious intent can’t be proved, a fault-based system with a reverse burden of proof (reverse insofar as it will be upon the sued party to disprove liability) is the best option. The system is already applicable when airlines are sued for accidents and we think it offers great promise when it comes to governing the development and use of highly capable advanced AI.