I feel like this post is doing something I really don't like, which I'd categorize as something like "instead of trying to persuade with arguments, using rhetorical tricks to define terms in such a way that the other side is stuck defending a loaded concept and has an unjustified uphill battle."

For instance:

let us be clear: hiding your beliefs, in ways that predictably leads people to believe false things, is lying. This is the case regardless of your intentions, and regardless of how it feels.

I mean, no, that's just not how the term is usually used. It's misleading to hide your beliefs in that way, and you could argue it's dishonest, but it's not generally what people would call a "lie" (or if they did, they'd use the phrase "lie by omission"). One could argue that lies by omission are no less bad than lies by commission, but I think this is at least nonobvious, and also a view that I'm pretty sure most people don't hold. You could have written this post with words like "mislead" or "act coyly about true beliefs" instead of "lie", and I think that would have made this post substantially better.

I also feel like the piece weirdly implies that it's dishonest to advocate for a policy that you think is second best. Like, this just doesn't follow – someone could, for instance, want a $20/hr minimum wage, and advocate for a $15/hr minimum wage based on the idea that it's more politically feasible, and this isn't remotely dishonest unless they're being dishonest about their preference for $20/hr in other communications. You say:

many AI Safety people being much more vocal about their endorsement of RSPs than their private belief that in a saner world, all AGI progress should stop right now.

but this simply isn't contradictory – you could think a perfect society would pause but that RSPs are still good and make more sense to advocate for given the political reality of our society.

I think this is tantamount to saying that we shouldn't engage within the political system, compromise, or meet people where they are coming from in our advocacy. I don't think other social movements would have got anywhere with this kind of attitude, and this seems especially tricky with something very detail orientated like a AI safety.

Inside game approaches (versus outside game approaches like this is describing) are going to require engaging in things this post says that no one should do. Boldly stating exactly the ideal situation you are after could have its role, but I'd need to see and much more detailed argument about why that should be the only game in town when it comes to AI.

I think that as AI safety turns more into an advocacy project it needs engage more with the existing literature on the subject including what has worked for past social movements.

Also, importantly, this isn't lying (as Daniel's comment explains). 

note - everything i say here is based on my own interpretation of the AI scene. I have no special knowledge, so this may be inaccurate, out-of-date, or both

So this post has provoked a lot of reaction, and I'm not going to defend all of it. I do think, however, that debating what is lying/white lying/lying by omission/being misleading is not as important as the other issues. (I think the tone is overly combative and hurts its cause, for instance). But from the perspective of someone like Connor, I can see why they may think that the AI Safety Community (especially those at the larger labs) are deliberately falsifying their beliefs.

On the May 16th Senate Judiciary Committee hearing where the witnesses are under oath^[1], Blumenthal asks Sam Altman about his quote that "development of superhuman machine intelligence is probably the greatest threat to the continued existence of humanity" and leads on to talk about the effect of AI on jobs and employment. Sam answers this by also talking about jobs, but Gary Marcus then calls him out by saying "Sam's worst fear I do not think is employment and he never told us what his worst fear actually is". When asked again, Sam says "my worst fears are that...we cause significant harm to the world"^[2]

I think that's a lie of omission - Sam's worst fear is the extinction of humanity or perhaps some S-risk scenario, and he did not say that. Does this violate Section 1001 of the United States Code? I don't know. I think it'd be good to get more concrete estimates from Sam of what he thinks the risks are, and I think as the CEO of the world's leading AI company, it'd be something the US Senate should interested in.

In a recent podcast, Dario Amodei implied his p(doom) was between 10-25%, though it's not a clear prediction with a timeframe, resolution criteria, or whether it's conditional or unconditional on Anthropic's actions (listen and judge for yourself). But Dario's been aware of xRisk for a long time, so this isn't a new belief, but it didn't make it into his senate testimony either. If my p(doom) was that high, and I was leading one of the world's top AI labs, I'd be making a lot more noise. But, instead, they've recently announced partnerships with and taken money from Google and Amazon with the intent to develop frontier AI and, presumably, commercialise it.

Now, many people see this and go "that doesnt' add up". It's what leads many critics of AI x-risk to go "this is just a galaxy brained marketing strategy", and I hope this comment has given some clarification as to why they think so.

Connor, on the other hand, has been consistently against scaling up AI systems and has argued for this in public appearances on the news, on podcasts, and in public hearings. I don't agree with his pessimism or framing of the positions, but he has authenticity regarding his position. It's what people expect people to act like if they think the industry they work in has a significant chance of destroying the world, whereas I can't make sense of Amodei's position, for example, without guessing additional beliefs or information which he may or may not actually have.

So, tying it back to the intense scepticism of RSPs from the Connor/ctrl.ai/Conjecture side of the AI Safety space is because given a track record like the above they don't trust that RSPs are actually a mechanism that will reduce the risk from developing unaligned AGI. And I can see why they'd be sceptical - if I'd just accepted the promise of $4 billion from Amazon, that's a big incentive to put your thumb on the scale and say "we're not at ASL-3 yet", for example.

The alternative might be titotal's explanation that Sam/Dario have much lower p(doom) than Connor, and I think that's probably true. But that might not be true for every employee at these organisations, and even then I think 10% is a high enough p(doom) that one ought to report that estimate fully and faithfully to your democratically elected government if they ask you about it.

I hope this makes the 'RSP sceptical' position more clear to people, though I don't hold it myself (at least to the same extreme as Connor and Gabe do here), and I'm open to corrections from either Connor or Gabe if you think I've interpreted what you're trying to say incorrectly.

1. ^{^}

   I can't find a recording or transcript of what the oath was, but I assume it included a promise to tell the whole truth

2. ^{^}

   The hearing is here, this exchange kicks off around 35:38

• ARC & Open Philanthropy state in a press release “In a sane world, all AGI progress should stop. If we don’t, there’s more than a 10% chance we will all die.”
• People at AGI labs working in the safety teams echo this message publicly.

Genuine question: Do the majority of people in open Phil, or at AGI safety labs, actually believe the statement above? 

I'm all for honesty, but it seems like an alternate explanation for a lot of people not saying the statement above is that they don't believe that the statement is true. I worry that setting this kind of standard will just lead to accusations that genuine disagreement is secret dishonesty.  

'There are very few people that we have consistently seen publicly call for a stop to AGI progress. The clearest ones are Eliezer’s “Shut it All Down” and Nate’s “Fucking stop”.

The loudest silence is from Paul Christiano, whose RSPs are being used to safety-wash scaling.'

I'm not blaming the authors for this, as they couldn't know, but literally today, on this forum, Paul Christiano has publicly expressed clear beliefs about whether a pause would be a good idea, and why he's not advocating for one directly: https://forum.effectivealtruism.org/posts/cKW4db8u2uFEAHewg/thoughts-on-responsible-scaling-policies-and-regulation

Christiano: "If the world were unified around the priority of minimizing global catastrophic risk, I think that we could reduce risk significantly further by implementing a global, long-lasting, and effectively enforced pause on frontier AI development—including a moratorium on the development and production of some types of computing hardware. The world is not unified around this goal; this policy would come with other significant costs and currently seems unlikely to be implemented without much clearer evidence of serious risk. 

A unilateral pause on large AI training runs in the West, without a pause on new computing hardware, would have more ambiguous impacts on global catastrophic risk. The primary negative effects on risk are leading to faster catch-up growth in a later period with more hardware and driving AI development into laxer jurisdictions.

However, if governments shared my perspective on risk then I think they should already be implementing domestic policies that will often lead to temporary pauses or slowdowns in practice. For example, they might require frontier AI developers to implement additional protective measures before training larger models than those that exist today, and some of those protective measures may take a fairly long time (such as major improvements in risk evaluations or information security). Or governments might aim to limit the rate at which effective training compute of frontier models grows, in order to provide a smoother ramp for society to adapt to AI and to limit the risk of surprises."