Comment Permalink

Good analysis of this from PauseAI:

I don't want to presume to paraphrase their analysis into one phrase, but if I were forced to, it would seem to be that there was a lot of pressure on Governor Newsom from powerful AI companies and interests, who also threatened to ruin the bill's sponsor Scott Wiener.

Still a pity that he couldn't resist the pressure.

It's kind of pathetic, but this is the reality of politics today. With their money, they really can either make or break a politician, and we voters are not smart enough to avoid being taken in by their negative advertising and dirt-digging.

It's clear that we need a much stronger movement on this. The other reason he was able to veto this bill is that the vast majority of people do not agree that AI poses a major / existential risk, and so they do not insist on the urgent action we need.

Gavin Newsom vetoes SB 1047

by Larks

Sep 30 20241 min read 14

39

AI safetyExistential riskPolicyNews relevant to effective altruismPublic communication on AI safetyUS policy

Frontpage

This is a linkpost for https://www.wsj.com/tech/ai/californias-gavin-newsom-vetoes-controversial-ai-safety-bill-d526f621

The Democrat decided to reject the measure because it applies only to the biggest and most expensive AI models and doesn’t take into account whether they are deployed in high-risk situations, he said in his veto message.
Smaller models sometimes handle critical decision-making involving sensitive data, such as electrical grids and medical records, while bigger models at times handle low-risk activities such as customer service.

39 Reactions

Comments14

Sorted by

New & upvoted

Click to highlight new comments since: Today at 7:35 PM

huwSep 30 202413

For those that have been following this: Is he serious, or is this just lip service and he's blocking it because he was lobbied by people in the tech industry?

LinchOct 1 202411

(Not super well-informed) My guess is that it's 95%+ lip service. He doesn't seem like someone with scruples or object-level opinions about most things other than what gives him power.

LarksOct 1 20245

One new thing to me in that thread was that the California Legislature apparently never overrides the governor's vetoes. I wonder why this is the case there and not elsewhere.

Daniel_FriedrichSep 30 20248

Newsom's press release and veto message include much more detail and suggest "it's too weak" is not the actual reason.

Reasons mentioned:

Discrimination by model size
1. "SB 1047 only applies to large models, giving us a "false sense of security about controlling this fast-moving technology. Smaller, specialized models may emerge as equally or even more dangerous"
"Real risks" are limited to critical decision-making, critical infrastructure etc.
1. "While well-intentioned, SB 1047 does not take into account whether an Al system is deployed in high-risk environments, involves critical decision-making or the use of sensitive data. Instead, the bill applies stringent standards to even the most basic functions - so long as a large system deploys it. I do not believe this is the best approach to protecting the public from real threats posed by the technology."
2. Newsom wants to focus on "specific, known" "demonstrable risks to public safety" "rooted in science and fact", like the deepfake laws he signed.

David Mathers🔸Sep 30 20245

Suggests Newsom is going to be very hostile to any legislation that is designed to deal with X-risk concerns, and that he, frankly thinks they are bullshit. (I personally am also pretty skeptical of X-risk from AI, but I don't want nothing done given how bad the risk would be if it did manifest.)

Manuel AllgaierSep 30 20247

This seems pretty bad news from an AI safety perspective :/

Any chance to override his veto, or get a similar bill passed soon?

LinchOct 1 202412

Some people have talked about trying to get something like that as a ballot initiative, since initial polling suggests that something like this might be very popular with the general public.

MvK🔸Oct 1 20244

Yep, numbers ranged from 60% to 80% support for approving SB 1047, and it was impressively bipartisan, too.

Ebenezer DukakisOct 2 20242

To do a [citizen-initiated] ballot initiative, you stand on the street and ask passerby to sign your petition. Perhaps it would be possible to simultaneously build a mailing list of interested passerby to attend protests and such. That could translate the poll numbers into a stronger street presence.

LinchOct 2 20242

I talked to people who seem to know about this and ~0 modern ballot initiatives are done this way. You need >500,000 signatures, which is a massive logistical undertaking that's not going to be enacted by a couple of volunteers.

Ebenezer DukakisOct 3 20242

About a decade ago, I worked collecting signatures for ballot initiatives in California. I worked with a company which contracted with organizations that financially sponsored the ballot initiatives. At the time I was doing it, the sponsor would usually pay from between $1 to $4 per signature. I would stand in an area with lots of foot traffic and try to persuade passerby to sign my petitions. To maximize profits, the typical strategy is to order petitions from highest-paying to lowest-paying, and try to get any given passerby to sign as many petitions as possible. People who sign your petitions need to be registered voters, so if you're on e.g. a college campus with lots of people who aren't registered voters, you can carry voter registration forms in order to register them. But that strategy is more time-consuming and therefore less profitable. There is a process to randomly verify that petition signatories are registered voters to prevent fraud.

I got into this business because a friend of mine said it could be a good way to make extra cash if you find a good place to stand, and a good way to practice charisma. I got out of it because I wasn't making all that much money, and I was worried that bothering people who were just trying to go about their day was making me callous.

California's current petition system seems like a pretty clear perversion of whatever the designers had in mind. I barely remember having any sort of substantial discussion regarding the policy merits of the petitions I was collecting signatures for. The guy who ran the petition company freely admitted to collecting petitions for initiatives he didn't believe in, and said his most effective tactic for collecting a signatures was to emphasize that "this just puts it on the ballot". There was only one time I ever remember a lady who said "This sounds like bad policy so I'm not signing". I respected the heck out of that, even though her reasoning didn't persuade me. (I believe I was trying to collect her signature for the current top-paying petition related to minutiae of car insurance law.)

So overall, I agree that if you just want to put a petition on the ballot as efficiently as possible, and you have the money needed to hire contractors, then that's a good way to go. But I am also not terminally cynicism-pilled. Based on my insider knowledge of the system, I don't see any reason in principle why a group couldn't use the law more like it was intended.

Yes, collecting half a million signatures is a big project. Imagine 1000 volunteers, each committed enough to collect an average of 500 signatures each. But starting a mass movement is also a big project. So if you want to start a mass movement anyways, you might consider combining those objectives, and using contractors to make up any signature shortfall.

The AI Pause protests I've seen haven't struck me as very effective. I remember in the early days, some EAs were claiming that attending early Pause protests would be high impact, if the protests grew over time. Despite the poll numbers, the Pause protests don't seem to be growing much beyond the core EA/LW audience. If growing those protests is a goal, and a mass movement is considered desirable (a big "if", obviously), maybe it's time to embrace the grind and put in the same sort of leg work you see with e.g. vegan activism.

So -- I'm not suggesting volunteer signature collection as a substitute for professional signature collection, so much as I am suggesting volunteer signature collection as a substitute for doing protests. The Pause movement might see more growth if volunteers split off into groups of two and tried to talk to passerby about AI on an individual basis. AI is a hot topic, much hotter than car insurance law, and the hypothesis that passerby are interested in having 1-on-1 conversations about it may be worth cheap testing. Petitioning could serve as an excuse to start conversations which would ideally end in a signature, a mailing list signup, or a new committed volunteer.

I can share more strategy thoughts if people are interested.

LinchOct 3 20243

I don't think this is the type of thing that armchair theorizing is good for. If you believe it's good to experiment with signature collection and talking to passerby about AI and/or getting more people on PauseAI mailing lists, I encourage you to do so and report back.

Denis Oct 2 20243

Manuel AllgaierSep 30 20242

More context:
TechCrunch article (not paywalled like the WSJ article)
Response from Senator Wiener on Twitter + discussion

Thanks for sharing!