In today's Time article about Anthropic, Daniela Amodei says about EA,
“The same way that you might say some people overlap with a political ideology in some ways, but don’t have a political affiliation—that’s more how I would think about it”
That's a notable change from her March 2025 comments to Wired:
“I’m not the expert on effective altruism. I don’t identify with that terminology. My impression is that it’s a bit of an outdated term.”
Kelsey Piper’s article on SB 1047 says
I’ve seen similar statements elsewhere too. But after I spent some time today reading through the bill, this seems to be wrong? Liability for developers doesn’t seem to be dependent on whether “critical harm” is actually done. Instead, if the developer fails to take reasonable care to prevent critical harm (or some other violation), even if there is no critical harm done, violations that cause death/bodily harm/etc can lead to fines of 10% or 30% of compute. Here’s the relevant section from the bill:
Has there been discussion about this somewhere else already? Is the Vox article wrong or am I misunderstanding the bill?
I think you are misunderstanding the bill. The key component is this phrase: "for a violation of this chapter"
I.e. this section is about what kind of damages can be recovered, if someone violates any of the procedural requirements outlined in this bill.
The bill basically has two components:
(I am reasonably confident this is true and have read the bill 3-4 times, but I might be getting something wrong. The bill has been changing a lot)
Thanks for your reply! I'm a bit confused—I think my understanding of the bill matches yours. The Vox article states "Otherwise, they would be liable if their AI system leads to a 'mass casualty event' or more than $500 million in damages in a single incident or set of closely linked incidents." (See also eg here and here). But my reading of the bill is that there is no mass casualty/$500 million threshold for liability like Vox seems to be claiming here.
No, there is, that's the definition of critical harm I mention above:
I still don't think you have posted anything from the bill which clearly shows that you only get sued if A) [you fail to follow precautions and cause critical harms], but not if B) [you fail to follow precautions the bill says are designed to prevent critical harms, and some loss of life occurs]. In both cases you could reasonably characterise it as "you fail to follow precautions the bill says are designed to prevent critical harms" and hence "violate" the "chapter".
I mean, what you are saying is literally what I said. There are two ways the bill says the Attorney General can sue you. One, if you developed a covered model that caused more than $500M harm, two if you violated any of the prescribed transparency/accountability mechanisms in the bill.
Of course you need to have some penalty if you don't follow the transparency/accountability requirements of the bill, how otherwise would you expect people to do any of the things the bill requires of them?
To clarify, I agree that that the ways you can be liable mostly fall into the two categories you delineate but think that your characterization of the categories might be incorrect.
You say that a developer would be liable
But I think a better characterization would be that you can be liable
It's possible "to fail to take reasonable care to prevent critical harms" even if you do not cause critical harms. The bill doesn't specify any new category of liability specifically for developers who have developed models that cause critical harm.
To use Casado's example, if a self-driving car was involved in an accident that resulted in a person's death, and if that self-driving car company did not "take reasonable care to prevent critical harms" by having a safety and security protocol much worse than that of other companies, it seems plausible that the company could be fined 10% of their compute/have to pay other damages. (I don't know if self-driving cars actually would be affected by this bill.)
I think the best reason this might be wrong is that courts might not be willing to entertain this argument or that in tort law "failing to take reasonable care to avoid something" requires that you "fail to avoid that thing"—but I don't have enough legal background/knowledge to know.
I think that's inaccurate (though I will admit the bill text here is confusing).
Critical harms is defined as doing more than $500M of damage, so at the very least you have to be negiligent specifically on the issue of whether your systems can cause $500M of harm.
But I think more concretely the conditions under which the AG can sue for damages if no critical harm has yet occurred are pretty well-defined (and are not as broad as "fail to take reasonable care").