Quick takes

Tentative thesis: China is unlikely to accept a subordinate position in AI capabilities to the US, just as the US is unlikely to accept a subordinate position in AI capabilities to China or anyone else.  This thesis suggests that there are two likely paths for the future international AI (non)regulation: 1) a continuing race between the US and China, perhaps joined by some late entrants, possibly with some rules established (e.g. mutual ban on autonomous weapons or something like that), or 2) a “pause or stop deal” that will establish some sort of ceiling on capabilities, which will be equal for China and the US. What I think is far less likely is a deal in which China would accept lower AI capabilities than the US. I simply don’t see a good reason why Chinese leaders would accept subordinate position. They know that China has the ability to catch-up to the US in various technological domains, as evidenced by the fact that in, like, 1990, China was behind in more or less everything, and now they are pushing the technological frontier in many areas.  Some estimates floating around the web suggest that, if the US would stop AI development now, China could reach current US capabilities within months. That is maybe overly optimistic/pessimistic depending on where you stand, but I very much doubt that catch-up would take more than a decade. And Chinese government, being patriotic about the abilities of the Chinese nation, probably will not have an absurdly pessimistic estimate. Moreover, the Chinese regime is in many ways oriented around this idea of catching up (this is a big difference between China and EU). It is a central plank of the official historical narrative of the People’s Republic of China (see for example preamble to their constitution: https://english.www.gov.cn/archive/lawsregulations/201911/20/content_WS5ed8856ec6d0b3f0e9499913.html) that the century between First Opium War and the Communist victory in the Chinese civil war in 1949 was the century

Help The Unjournal prioritize AI economics and governance research (https://uj-prioritization-dashboard.netlify.app/ai-governance/) The Unjournal (https://info.unjournal.org/) is prioritizing research on AI economics, governance, social and economic impacts, and risks from increasingly capable AI. (NB: We're not covering technical/CS/ML/AI-safety research.). We want to commission expert public evaluations and help synthesize, disseminate, and curate this work. Much of this research seems 1) important/impactful/influential, (2) involves considerable 'firepower' from prominent researchers and practitioners, (3) not 'obviously true', and (4) time-sensitive: relevant for funding, policy, and research-steering now, but not after the "six months to six years" required for the traditional (economics) journal process Our preliminary shortlist page (https://uj-prioritization-dashboard.netlify.app/ai-governance/) includes paper summaries, AI-generated audio walkthroughs, and an interface for giving feedback. We also invite suggestions fr other research (please check the broader list/interface to avoid duplication). Especially looking for your input on: * Usefulness and accuracy:* Is this useful? How could we improve it? What have we got wrong? * Impact: Which research do you use, or think others use or should use? * Scrutiny What claims and approaches need careful experts and practitioners critique? What sorts of experts? If you're that expert, please dm and join our pool. * Participation: Who should be part of this discussion (nominate and nudge)? * Gaps: What's missing (research, claims, etc.)? Timing and next steps (our plan) * Finalize an initial shortlist of at least three research outputs before the end of October, with more depending on interest and funding. * Commission rapid, public, impact-focused evaluations/reviews by experts and practitioners, aiming for a first public evaluation within about two weeks of commissioning. * Over the following two to

AI safety needs people everywhere but quickly stated, current talent bottlenecks to me look like: -- Founders -- Grantmakers -- (technical) Research leads  -- Policy entrepreneurs and implementors (which includes a lot of technical work) -- bets in international coordination and/or cooperation -- All manner of supporting talent -- program leads, ops proper, public outreach, content creators, comms   Most sought-after qualities for talent are: -- context, mission alignment, domain understanding, sophisticated views on AI strategy and threat modelling etc. -- "good judgement", "sound epistemics", "reasoning transparency" and other similar ideas/meta-skills from the EA/rationalist cannon -- a willingness to get shit done/bias for action (rather than be in learning mode, or people who need a lot of management and oversight, or folks with too many preferences/constraints) -- low ego, similar to above -- ambitious folks, since they would be really trying to be their own managers, take on bigger projects, grow themselves and their teams etc.    Finally, even having these, it's not enough to just claim to have these; job-seekers mainly trip up in being able to demonstrate and be legible about having them.

I thought Ezra's feature on recursive self-improvement was really valuable in a number of ways: 1. Generally well-done pedagogically, explaining things in a way that non AI-pilled people can understand and doing a lot of preemptive objection-handling. 2. Making the case that many of the beliefs about x-risks were held for a decade plus, before people involved had the material interests they have now. 3. Explaining well why the perceptions of AI risks diverge between people at the frontier and everyday people using ChatGPT to improve emails. 4. Explaining why recursive self-improvement is such a crucial benchmark and why going marginally slower is not a satisfactory response. 5. Defining a policy ask (control) that is broadly legible and easier to grasp than alignment. 6. Striking a balance of calling different groups out on their respective BS (the AI companies on the impossibility of defining policy responses to RSI, critics who say this is all PR, people who are too certain). Obviously a carefully crafted piece of messaging and positioning, but a pretty useful one.

tl;dr: Stop The AI Race is organizing some rapid-response protest march this Thursday 9am-12pm in San Francisco, calling the Mayor of San Francisco and the board of supervisors to declare an AI Emergency and enforce a pause of frontier AI model development in San Francisco Over the past week, we have seen Dario Amodei and Sam Altman advocate for government intervention in pacing the frontier. However, the US Government has yet to answer their calls. Which is why on Thursday at 9am Stop The AI Race will be organizing another protest march, asking local authorities to step in. We will be marching from OpenAI to Anthropic to City Hall, asking AI company employees to join us in calling on the Mayor of San Francisco and the Board of Supervisors to declare an AI Emergency and enforce a full pacing of frontier AI model development in San Francisco. Schedule: - 9am: Rally at OpenAI - 10-10:45am: March To Anthropic - 10:45am: Rally at Anthropic - 11am-11:30am: March to City Hall - 11:30am: Rally at City Hall Sign up here.

Applications to SPAR Fall 2026 are closing tomorrow Aug 18 EOD Anywhere on Earth. SPAR is the ecosystem's biggest AI safety research program, and it's part-time remote. We still have many strong projects across AI safety, AI policy, and biosecurity with very few applications[1], so please consider applying! This round, we also have 18 non-research/generalist projects that people can apply to, and we have much more mentee capacity than previous rounds; we expect to accept around 500 people into the program. If you have friends who have thought about going into AI safety, spread the word! 1. ^ To be specific, as of 2:25 PM PT, we had around 67 projects with fewer than 20 applications total!

Mildly against the Longtermism --> GCR shift Epistemic status: Pretty uncertain, somewhat rambly TL;DR replacing longtermism with GCRs might get more resources to longtermist causes, but at the expense of non-GCR longtermist interventions and broader community epistemics Over the last ~6 months I've noticed a general shift amongst EA orgs to focus less on reducing risks from AI, Bio, nukes, etc based on the logic of longtermism, and more based on Global Catastrophic Risks (GCRs) directly. Some data points on this: * Open Phil renaming it's EA Community Growth (Longtermism) Team to GCR Capacity Building * This post from Claire Zabel (OP) * Giving What We Can's new Cause Area Fund being named "Risk and Resilience," with the goal of "Reducing Global Catastrophic Risks" * Longview-GWWC's Longtermism Fund being renamed the "Emerging Challenges Fund" * Anecdotal data from conversations with people working on GCRs / X-risk / Longtermist causes My guess is these changes are (almost entirely) driven by PR concerns about longtermism. I would also guess these changes increase the number of people donation / working on GCRs, which is (by longtermist lights) a positive thing. After all, no-one wants a GCR, even if only thinking about people alive today. Yet, I can't help but feel something is off about this framing. Some concerns (no particular ordering): 1. From a longtermist (~totalist classical utilitarian) perspective, there's a huge difference between ~99% and 100% of the population dying, if humanity recovers in the former case, but not the latter. Just looking at GCRs on their own mostly misses this nuance. * (see Parfit Reasons and Persons for the full thought experiment) 2. From a longtermist (~totalist classical utilitarian) perspective, preventing a GCR doesn't differentiate between "humanity prevents GCRs and realises 1% of it's potential" and "humanity prevents GCRs realises 99% of its potential" * Preventing an extinction-level GCR might move u

How impactful would it be to copies of @Garrison's new book Obsolete to elected officials who belong to its political target audience (Dems, particularly left ones) and might not have been responsive to traditional x-risk-centric comms (e.g. IABIED)?