This is a special post for quick takes by Matt Goodman. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Comment Permalink
Why aren't we protesting AI acceleration in the street?
I'm not super up to date with the latest EA thinking on current AI capabilities. The takes I read on social media from Yudkowsky and the like are something along the lines of 'We're at a really dangerous time, various companies are engaged in arms race to make more and more powerful AIs with little regard to safety, and this will directly lead to humanity being wiped out by AGI in the near future'. For people really believe this to be true (especially if you live in San Fransico) - why aren't you protesting on the street?
Some reasons this might work:
- There's lots of precedents of public pressure leading to laws being passed or procedures changed, that have increased safety standards across many industries
- The companies working on AI alignment are based in San Francisco. There's a big EA and rationalist community in SF. Protests could happen outside the HQ of AI companies.
- Stories about silicon valley tech companies get lots of press coverage in mainstream media
- There's a prevailing anti - big tech companies feeling in parts of society that could be tapped into it
- Specifically, there's criticisms of the newest AIs for things like 'training AI models on artists' work, then putting artists out of a job' (Dalle) or 'making it much easier to cheat at university' (ChatGPT). Whilst this isn't directly related to AGI safety, it's the kind of feeling that could be tapped into for the purpose of this protest
- If an AI safety researcher could be interviewed on camera at the march it adds credibility to the march, that experts are concerned
- It adds credibility to the voices of experts warning about AI risk, if they're so worried they're willing to get out on the street to protest about it
Hi Matt, this is a great idea and the closest thing I am aware of is the street-level protests which have been held by the Campaign to Stop Killer Robots. Maybe you and other EAs would consider getting involved in some of their marches? I haven't been able to join any as they weren't local to me, but I've donated to them and maybe you can too or you could march with them? https://www.stopkillerrobots.org/
Objections to 'value of my time arguments'
I often hear EA/ rationalists saying something like 'it's not worth spending an hour to save £20, if your hourly rate of pay is over £20/ hour. I think this is wrong, but I might not understand the argument.
It could be understood as a hypotherical argument, you COULD earn this much in an hour, as a reference point to help you understand the value of your time. This hypothetical reference point isn't really useful, when I have the very real figure of my total balance, and upcoming outgoings to consider, and the factor of whether I can afford to spend on things I enjoy, if I don't save money now.
So, it could be a suggestion that I actually could work an extra hour for money. But I (and almost everyone I know) don't get paid hourly. I have a fixed amount of hours, she working over that doesn't gain me more money. To gain more, I'd need to set up a freelance/ side business, and there's all sorts of initial costs with getting that setup and advertising my services, and reporting my income for tax, and so on and so on.
Lastly, 'you could work an extra hour' doesn't factor in my enjoyment. Working an extra hour would have a negative effect on my mood. I don't want to work more than i already do. By contrast, walking an hour to save an Uber fee would be good for my mood, and health.
Curated and popular this week
·
Regulation cannot be written in blood alone.
There’s this fantasy of easy, free support for the AI Safety position coming from what’s commonly called a “warning shot”. The idea is that AI will cause smaller disasters before it causes a really big one, and that when people see this they will realize we’ve been right all along and easily do what we suggest. I can’t count how many times someone (ostensibly from my own side) has said something to me like “we just have to hope for warning shots”. It’s the AI Safety version of “regulation is written in blood”. But that’s not how it works.
Here’s what I think about the myth that warning shots will come to save the day:
1) Awful. I will never hope for a disaster. That’s what I’m trying to prevent. Hoping for disasters to make our job easier is callous and it takes us off track to be thinking about the silver lining of failing in our mission.
2) A disaster does not automatically a warning shot make. People have to be prepared with a world model that includes what the significance of the event would be to experience it as a warning shot that kicks them into gear.
3) The way to make warning shots effective if (God forbid) they happen is to work hard at convincing others of the risk and what to do about it based on the evidence we already have— the very thing we should be doing in the absence of warning shots.
If these smaller scale disasters happen, they will only serve as warning shots if we put a lot of work into educating the public to understand what they mean before they happen.
The default “warning shot” event outcome is confusion, misattribution, or normalizing the tragedy.
Let’s imagine what one of these macabrely hoped-for “warning shot” scenarios feels like from the inside. Say one of the commonly proposed warning shot scenario occurs: a misaligned AI causes several thousand deaths. Say the deaths are of ICU patients because the AI in charge of their machines decides that costs and suffering would be minimize
·
This is a transcript of my opening talk at EA Global: London 2025. In my talk, I challenge the misconception that EA is populated by “cold, uncaring, spreadsheet-obsessed robots” and explain how EA principles serve as tools for putting compassion into practice, translating our feelings about the world's problems into effective action.
Key points:
* Most people involved in EA are here because of their feelings, not despite them. Many of us are driven by emotions like anger about neglected global health needs, sadness about animal suffering, or fear about AI risks. What distinguishes us as a community isn't that we don't feel; it's that we don't stop at feeling — we act. Two examples:
* When USAID cuts threatened critical health programs, GiveWell mobilized $24 million in emergency funding within weeks.
* People from the EA ecosystem spotted AI risks years ahead of the mainstream and pioneered funding for the field starting in 2015, helping transform AI safety from a fringe concern into a thriving research field.
* We don't make spreadsheets because we lack care. We make them because we care deeply. In the face of tremendous suffering, prioritization helps us take decisive, thoughtful action instead of freezing or leaving impact on the table.
* Surveys show that personal connections are the most common way that people first discover EA. When we share our own stories — explaining not just what we do but why it matters to us emotionally — we help others see that EA offers a concrete way to turn their compassion into meaningful impact.
You can also watch my full talk on YouTube.
----------------------------------------
One year ago, I stood on this stage as the new CEO of the Centre for Effective Altruism to talk about the journey effective altruism is on. Among other key messages, my talk made this point: if we want to get to where we want to go, we need to be better at telling our own stories rather than leaving that to critics and commentators. Since
·
Formosa: Fulcrum of the Future?
An invasion of Taiwan is uncomfortably likely and potentially catastrophic. We should research better ways to avoid it.
TLDR: I forecast that an invasion of Taiwan increases all the anthropogenic risks by ~1.5% (percentage points) of a catastrophe killing 10% or more of the population by 2100 (nuclear risk by 0.9%, AI + Biorisk by 0.6%). This would imply it constitutes a sizable share of the total catastrophic risk burden expected over the rest of this century by skilled and knowledgeable forecasters (8% of the total risk of 20% according to domain experts and 17% of the total risk of 9% according to superforecasters).
I think this means that we should research ways to cost-effectively decrease the likelihood that China invades Taiwan. This could mean exploring the prospect of advocating that Taiwan increase its deterrence by investing in cheap but lethal weapons platforms like mines, first-person view drones, or signaling that mobilized reserves would resist an invasion.
Disclaimer
I read about and forecast on topics related to conflict as a hobby (4th out of 3,909 on the Metaculus Ukraine conflict forecasting competition, 73 out of 42,326 in general on Metaculus), but I claim no expertise on the topic. I probably spent something like ~40 hours on this over the course of a few months.
Some of the numbers I use may be slightly outdated, but this is one of those things that if I kept fiddling with it I'd never publish it.
Acknowledgements: I heartily thank Lily Ottinger, Jeremy Garrison, Maggie Moss and my sister for providing valuable feedback on previous drafts.
Part 0: Background
The Chinese Civil War (1927–1949) ended with the victorious communists establishing the People's Republic of China (PRC) on the mainland. The defeated Kuomintang (KMT[1]) retreated to Taiwan in 1949 and formed the Republic of China (ROC). A dictatorship during the cold war, Taiwan eventually democratized in the 1990s and today is one of the riche
Objections to 'value of my time arguments'
I often hear EA/ rationalists saying something like 'it's not worth spending an hour to save £20, if your hourly rate of pay is over £20/ hour. I think this is wrong, but I might not understand the argument.
It could be understood as a hypotherical argument, you COULD earn this much in an hour, as a reference point to help you understand the value of your time. This hypothetical reference point isn't really useful, when I have the very real figure of my total balance, and upcoming outgoings to consider, and the factor of whether I can afford to spend on things I enjoy, if I don't save money now.
So, it could be a suggestion that I actually could work an extra hour for money. But I (and almost everyone I know) don't get paid hourly. I have a fixed amount of hours, she working over that doesn't gain me more money. To gain more, I'd need to set up a freelance/ side business, and there's all sorts of initial costs with getting that setup and advertising my services, and reporting my income for tax, and so on and so on.
Lastly, 'you could work an extra hour' doesn't factor in my enjoyment. Working an extra hour would have a negative effect on my mood. I don't want to work more than i already do. By contrast, walking an hour to save an Uber fee would be good for my mood, and health.