Posting something about a current issue that I think many people here would be interested in.
California Governor Gavin Newsom has until September 30 to decide the fate of SB 1047 - one of the most hotly debated AI bills in the world. The Center for AI Safety Action Fund, where I work, is a co-sponsor of the bill. We need your help to encourage the Governor to sign it! You can help by writing a quick custom letter and sending it to his office (see instructions below).
About SB 1047 and why it is important
SB 1047 is an AI bill in the state of California. SB 1047 would require the developers of the largest AI models, costing over $100 million to train, to test the models for the potential to cause or enable severe harm, such as cyberattacks on critical infrastructure or the creation of biological weapons resulting in mass casualties or $500 million in damages. AI developers must have a safety and security protocol that details how they will take reasonable care to prevent these harms and publish a copy of that protocol. Companies who fail to perform their duty under the act are liable for resulting harm. SB 1047 also lays the groundwork for a public cloud computing resource to make AI research more accessible to academic researchers and startups and establishes whistleblower protections for employees at large AI companies.
I believe SB 1047 is the most significant piece of AI safety legislation in the country, and perhaps the world. While AI policy has made great strides in the last couple of years, AI policies have mostly not had teeth – they have relied on government reporting requirements and purely voluntary promises from AI developers to behave responsibly. SB 1047 would actually prohibit behavior that exposes the public to serious and unreasonable risks, and incentivize AI developers to consider the public interest when developing and releasing powerful models.
If 1047 is vetoed, it’s plausible that no comparable legal protection will exist in the next couple of years, as Congress does not appear likely to pass anything like this any time soon.
The bill’s text can be found here. A summary of the bill can be found here. Longer summaries can be found here and here, and a debate between a bill proponent and opponent is here. SB 1047 is supported by many academic researchers (including Turing Award winners Yoshua Bengio and Geoffrey Hinton), employees at major AI companies and organizations like Imbue and Notion. It is opposed by OpenAI, Google, Meta, venture capital firm A16z as well as some other academic researchers and organizations. After a recent round of amendments, Anthropic said “we believe its benefits likely outweigh its costs.”
SB 1047 recently passed the California legislature, and Governor Gavin Newsom has until September 30th to sign or veto it. Newsom has not yet said whether he will sign it or not, but he is being lobbied hard to veto. A veto decision would set back AI safety legislation significantly, and expose the public to greater risk. He needs to hear from you!
How you can help
There are several ways to help, many of which are detailed on the SB 1047 website.
The most useful thing you can do is write a custom letter. To do this:
- Make a letter addressed to Governor Newsom using the template here.
- Save the document as a PDF and email it to leg.unit@gov.ca.gov.
- In writing this letter, we encourage you to keep it simple, short (0.5-2 pages), and intuitive. Complex, philosophical, or highly technical points are not necessary or useful in this context – instead, focus on how the risks are serious and how this bill would help keep the public safe.
Once you’ve written your own custom letter, think of 5 family members or friends who might also be willing to write one. Supporters from California are especially helpful, as are parents and people who don’t typically engage on tech issues. Then help them write it! You can:
- Call or text them and tell them about the bill and ask them if they’d be willing to support it.
- Draft a custom letter based on what you know about them and what they told you.
- Send them a completed letter as a PDF and ask if they’re willing to send it to the Governor’s office using the instructions above.
Organize an event! To make even more of an impact, you should consider hosting an event to organize people to write letters. Please email thomas@safe.ai if you are interested in hosting an event.
Thank you in advance for any help!
Like Buck and Toby, I think this is a great piece of legislation and think that it's well worth the time to send a letter to Governor Newsom. I'd love to see the community rallying together and helping to make this bill a reality!
I think this is a very good use of time and encourage people to do it.
I've done this.
Great idea Thomas.
I've just sent a letter and encourage others to do so too!
Thank you, Toby!
I wrote a quick letter I'm happy with.
(Feel free to DM me for a link tho ofc don't copy anything)
Sent a letter
Sounds like a good thing to do. Is it possible to do that when you're not living in the US ?
You don't have to live in the US to do it. You can help send a powerful message that the entire world is watching California on this issue.
If you're not a CA resident, I would probably discourage writing a letter. Exceptions if you have special expertise or are a donor to Newsom. But in general letters from non-constituents are not persuasive to politicians and often are used against the desired policy, to show their constituents how much pressure from outsiders they're "standing up against".
We have reason to think this is not true in this particular case. It can definitely be true in other cases. See my previous message about this.
So, that comment just says you can do it without living in the US. It doesn't say it's a good idea, and if there are reasons it's sometimes a bad idea, it doesn't engage with those.
If you have reason to think it's not a problem in this case, can you say what that reason is?
Sorry, but from my perspective as someone living outside the US, this whole thing is really not clear to me.
If you don't explicitly say whether you're a CA resident or not, how would they know?
For the website form I filled in, they could look at my IP address and see that I probably didn't fill it in from California; not reliable, but certainly evidence.
If someone sends an email... maybe if there seems to be a real name attached they'd compare that to voter registrations? I dunno if that info is available to them. If so, that also suggests not sending an anonymous or pseudonymous email.
(And maybe I gave a name when filling in the website form, idk.)
I have written a letter
I suspect it should be emphasized that you really shouldn't put much time or effort into your message.
It's been almost 20 years since I worked for someone in government, so I could be wrong, but even then we simply added to the count of people who wrote in in favor (and recorded it with their name in our database) and didn't read the note.
Agree. This really shouldn't take longer than 10 minutes. In this case (can't speak for every case like this), it does matter that the messages are unique and not copy pasted, which is why I didn't provide a letter to copy paste. But it is highly unlikely anybody will read the letter in great detail.
Are you willing to draft some rough bullet points (that you urge people to NOT copy and paste) on SB-1047 that might help people complete this exercise faster?
Also, do you have a sense for how much better a slightly longer letter (e.g. a 1 page note) is as compared to just a 1 paragraph email with something like: "As a [location] resident concerned about AI safety, I urge you to sign SB 1047 into law. AI safety is important to me because [1 sentence]."
FWIW, I did just send off an email, but it took me more like ~30 minutes to skim this post and then draft something. I also wasn't sure how important it was to create the PDF — otherwise I would have just sent an email on my phone, which would again have been a bit faster.
This page has a number of points (it also includes a few other actions that you can take).
One paragraph is definitely fine unless you have personal experience that is relevant (for example, you are an AI researcher, founder, etc.).
Makes sense! It is important to create the PDF, just how these things are supposed to be submitted.
Warning: the Governor's "Contact Us" page (gov•ca•gov/contact) says:
So, if you were planning to send a PDF, I would also send a second email with the letter in the body of the eMail.
@Center for AI Safety Time to change the instructions on your website?
That website is confusing; however, the line about "email attachments" I believe is meant to refer to the website contact form. I think it is attempting to explain why there is no way to attach files in that form. I know it is not an "email" and this message confused me too, but it is in fact common practice to send emails with attachments in the format specified.
"For security, we can’t receive email attachments" most likely means they don't want to be opening files from random petitioners which could install malware on a government computer. So the exact method by which the file is sent wouldn't especially matter. Note the next sentence (emphasis mine): "If you wish to send an attachment, please do so by mail."
(I'm a security guy, and I agree with the California government that avoiding untrusted attachments is a good way to avoid malware. It's true that lots of people open them, but lots of people have poor security practices in general. Another factor is that the California state government can expect to be targeted by more sophisticated cyberattacks than your average Joe or Jane.)
So yeah, I strongly recommend that you edit the instructions here and on your website ASAP.
The instructions you see above come directly from the Governor's office by email to somebody on our team a month ago. They are the instructions that are used for sending support letters and they have been used for multiple years by people who know the governor's office well. They are also used by other organizations for other bills.
I was also confused when I saw that line on the website, since it contradicts the specific instructions and also because it is on a page that does not have an email address. (Why would the Governor's office give people instructions for sending emails on a page that does not list an email address? It doesn't make sense. That's why my hypothesis is that the word "email" is a misnomer and it actually refers to the contact form on that page, which does not have a place for attachments.)
I agree with you that it seems like a poor security practice. But I err in favor of following the explicit written instructions provided to us by the Governor's office.
I see, thanks for the info. Seems confusing. Maybe people could put their letter in both the body of the email and also in an attachment, just to be on the safe side.
Thank you for writing this! I just took the time to write a letter.
Just sent a letter. This post was what pushed me to do it-- I am a CA voter and have never written a letter urging the governor to take action on legislation before, and I believe I wouldn't have done so without 1. reading this post and 2. reading Will MacAskill's comment on this post. So that's counterfactual impact on your part, nice one!
This post pushed me to write a letter. Thanks!
I wrote a letter!
I'm in New Zealand and wrote a letter as well.
I filled out the form at https://www.gov.ca.gov/contact/.
Would sending an email have been better? I'd be interested to know why if so. I was confused by the conflicting "you should attach it as a pdf" / "we can't receive email attachments" messages. (I sent it before this comment.) And creating a pdf would have been annoying. (I don't think I have a low friction way to work with .docx - a link to a google doc that people can copy, edit, and convert to pdf, might be helpful, if a pdf attachment really is better.)
After filling it out I got
I am not Californian or even American, but I never said I was and they never asked or said "only Californians should fill this out" so ¯\_(ツ)_/¯
That said, I was hesitant to do this previously because this seems like the kind of thing that maybe only Californians should be doing, or they'll ask where you live and ignore everyone who doesn't say California or something? So, in case there are other people like me, it might be worth a paragraph on the subject of non-Californians writing in. (Not just "non Californians can do it too", but answering "is it considered prosocial? / is this burning some commons of mutual self-restraint?" and "should you say you're not Californian?" and "will it have as much effect as a Californian?".)
I was also convinced by this and other things to write a letter, and am commenting now to make the idea stay salient to people on the Forum.
I recall previously hearing there might be a final round of potential amendments in response to things Gavin Newsom requests. Was/is that accurate?
Hello! The legislative session is over, so no more changes can be made to the bill. Sometimes that kind of thing does happen, but it happens during the legislative session.