Quick takes
I'd like to have conversations with people who work or are knowledgeable about energy and security. Whether that's with respect to energy grids, nuclear power plants, solar panels, etc. I'm exploring a startup idea to harden the world's critical infrastructure against powerful AI. (I am also building a system to make formal verification more deployable at scale so that it may reduce loss of control and misuse scenarios.)
I've given workshops on using AIs for productivity/research to various research organizations like MATS. I'm happy to offer a bit of my ti...
Been thinking about morality recently. Here are my current thoughts, take them with a grain of salt because they aren't battle-tested yet.
There are some strong arguments for utilitarianism, but regardless of what is correct theoretically, in practise utilitarianism doesn't work well without some kind of deontological bars.
Continuing with attempting to develop a pragmatic morality, it then become clear that virtue ethics is important too because a) rules are rigid compared to judgement b) decisions aren't independent but also affect how you'll act in the fu...
I quite liked this article by Martha Nussbaum: Virtue Ethics is a Misleading Category. She points out that both the classical utilitarians and Kant talked extensively about virtues. On the other hand, there's great variation among those who call themselves 'virtue ethicists', such that it's not clear if virtue ethics is really a thing.
But the point I want to make is: a good utilitarian has to acknowledge the role of virtue, and I think a lot of modern utilitarians have forgotten this. We want to use utility-calculation to guide our actions, but humans can'...
I wanted to make this poll to see how the community views the speed/x-risk tradeoff. I'm personally 99% x-risk and 1% speed, so I would hard agree. My prediction is most people will agree, maybe a 70/30 split, but I'm curious to see.
I would be willing to delay technological innovation by up to 100 years to significantly reduce existential risk
I think the question is too imprecise phrased to be answered precisely. When would the delay start? Over what time period would it be felt? (e.g. a 100% delay for 100 years is very different than 1% delay over 10,000 years)
I'm thus giving a directional answer assuming we're talking about whether seeking to dramatically reducing technological progress in exchange for safety is a feasible way to make the world a better place. I don't thi...
The recent forecasting is overrated post got me thinking:
Solution Seeking a Problem
When talking about forecasting, people often ask questions like “How can we leverage forecasting into better decisions?” This is the wrong way to go about solving problems.
Intuitively, that seems correct, and I've relied on the expression "when you have a hammer, everything looks like a nail." This got me thinking: is it necessarily the wrong way, or is this a truism?
If I have a legitimately useful and powerful tool, isn't it indeed valuable to look around for problems...
This just came to mind: the reason that it's the wrong way to go about solving problems is that you want to solve the largest problems (well, per resource) and not just solve any random problem. Like, there is a problem that my shoes are currently untied, and I don't want to bend down or spend 10 seconds to tie them, but it's not very important.
So if you want to solve the most important problems, you should start with the problem and then work backwards for what solutions you might wish existed. I think the mere fact that people often talk about forecasting as the solution they are seeking to apply, whether that be Sentinel or whoever, is evidence that things are going wrong.
I have been disappointed by the support some EAs have expressed for recent activist actions at Ridglan Farms. I share others’ outrage at the outcome of the state animal cruelty investigation, which found serious animal cruelty law violations but led to a settlement that still permits Ridglan to sell beagles through July and to continue in-house experimentation. But I personally think the tactics used in the recent open rescues, including property damage and forced entry to remove animals, violate reasonable moral bounds on what actions are p...
State laws are path dependent, and rely very often on common law principles and concepts uncritically applied. That does not equate to democratic legitimacy for every codified version of property and criminal law.
I think we have fundamentally incompatible views on the appropriate frame to apply to balancing questions—I am not at all a utilitarian, and I don’t think you should be either. But I’ll set that aside.
You again seem to conflate lawbreaking with immorality. Please don’t do that. Rosa Parks broke the law. So did the Ridglan rescuer...
You should volunteer at your first EAG! (Especially if you are a student or early career)
- If you don’t have a network in EA, EAG’s can be overwhelming. Volunteering gives you a ready-made, organic network.
- Volunteering is pretty chill - a lot of the shifts aren’t that hard.
- At your first EAG, it’s unlikely that you are using your time so efficiently that a few hours of volunteering would cut into the value of your conference.
We recently published an interview with Matthew Coleman - another entry in our Career Journeys series. Matthew is the Executive Director of Giving Multiplier, a platform that encourages donations to highly effective charities through donation matching. Before this, he completed a PhD in psychology, researching the psychology of altruism.
The interview covers quite a lot of ground, but a few of the things we talked about include:
- The gap between what a career looks like from the outside and what it's actually like day-to-day.
- Advice for people w
Lighting has been getting ridiculously cheaper. And for the most part we seem to be not taking advantage of that positive externality: reducing crime through better lighting. This has been battle-tested as one of the effective ways for public security, see Chalfin, Hansen, Lerner & Parker (2022), an RCT in NYC public housing finding ~36% reductions in nighttime outdoor index crimes from added street lighting. Many, many major cities still haven't copied this at the right levels!
But we're also getting substantially negative externalities of bright light...
Maybe my biggest medium-term worry about transformative AI, other than the takeover stuff, is a constellation of concerns I sometimes abbreviate to "political economy." Right now a large fraction of humans in democracies can live and support their families as a direct result of voluntarily exchanging their labor. It'd take active acts of violence to break from this (pretty good, all things considered) status quo. As a peacetime norm, this is unusually good relative to the history of human civilization.
At some point in the future (in the "good" future...
Thinking of drafting a post on war crimes, trying to answer the following puzzles:
- Why do we have a notion of war crimes at all, given how bad war itself is?
- Why are some things war crimes and not others?
- Why do precursor notions to war crimes appear, independently, in essentially every culture that has fought wars at scale?
- Given that essentially every culture has also broken these norms, sometimes spectacularly, why does the norm always come back, and often come back stronger?
Common answers to these questions seem profoundly misguided. The naive answer, that...
You might like this post I wrote earlier about the bargaining theory puzzle of war. I engaged with the academic literature on the subject pretty significantly, particularly James Fearon, so you might like it. On the other hand Fearon himself mostly reasoned from first-principles rather than conduct a careful historical assessment, so in that regard it might fit your interests less.
The post never got very popular but a few people who read it carefully really enjoyed it. One of the better compliments I've gotten on my writing is when somebody said they were ...
A quick reminder that applications for EA Global: London 2026 close this Sunday (May 10)!
We already have more applications than last year, and this looks set to be our biggest EAG yet (again)! If you've been meaning to apply but haven't gotten around to it, this is your sign.
The admissions bar is more accessible than people often assume. If you're working on or seriously exploring a high-impact problem, you should apply.
This is the EAG I've been most excited to put together yet. I'd love to see you all there.
📍 InterContinental London, The O2 · 29-31 May 2...
Just was watching Dwarkesh/David Reich podcast, fascinating stuff. Looking back at how I was taught taxonomy and anthropological history I find it frustrating. Note that I don't know much about (evolutionary) biology or genetics or the frontier of what genetic-history research so this is my layman attempt to explain why it's generally been puzzling for me how i have had this explained by other people who probably don't understand either, not trying to propose that I understand something david reich doesn't.
My main gripe is that we are taught evolutio...
FYI, next week we will be highlighting the first batch of articles from In Development, @Lauren Gilbert's new global development magazine.
Lauren and most of the authors will be on the Forum to answer your questions throughout the week. More info to come on Monday, but I figured I'd mention in case anyone wanted to read the articles in advance (they are here, and all authors apart from Paul Niehaus will be around to answer questions).
I'm looking forward to the discussion.
Earning to give is lonely and requires repeated decisions. This is bad.
If you're earning to give, you are lucky if you have one EtG team-mate. The people you talk to every day do not have moral intuitions similar to yours, and your actions seem weird to them.
If you do direct work, the psychological default every day is to wake up and do work. You are surrounded by people who think the work is important, and whose moral values at least rhyme with your own.
If you earn to give, most days you do not give (you're probably paid bi-weekly, and transaction costs d...
Somewhat meta point on epistemic modesty, calling it out here because it is a pattern that has deeply frustrated me about EA/rationalism for as long as I have known them:
(making a quick take rather than commenting due to an app.operation_not_allowed error - I'm responding to @Linch's quick take on war crimes)
I guess these are just EA/rationalist norms, but an approach that glosses major positions as being so quickly dismissible strikes me as insufficiently epistemically modest. I would expect such a treatment will fail to properly consider alternativ...
I feel like I've heard this position a lot before, and I have some sympathy for it, but I feel like it implicitly overlooks a lot of what I find valuable about writing EA Forum comments, and it sets an overly high bar.
When one writes academic papers, one is expected to cite relevant previous work. Credit assignation is an important mechanism for tracing the evidence for claims and for assigning credit. Even in academic spheres, I think this is perhaps taken pathologically far (to the point where it probably sometimes is unduly burdensome and vaguely implie...
This is too tangential from the forecasting discussion to justify being a comment there so I'm putting it here:
Forecasting makes no sense as a cause area, because cause areas are problems, something like "people lack resources/basic healthcare/etc.", "we might be building superintelligent AI and we have no idea what we're doing". Forecasting is more like a tool. People use forecasting to address AI, global poverty, and all sorts of more general problems, including ones that aren't major EA focuses.
For instance, we could treat vaccines as a cause area. All ...
The recent work on SAEBER, which applies sparse autoencoders (SAEs) to the screening of dna synthesis printers marks a big step towards effective function based screening.
This allows for printers to be monitored just as a lab technician uses computational gel electrophoresis to separate a messy mixture into clear, readable bands through the use of a specialized gel. SAEs happen to do the exact same thing by taking the muddied activation results of a neural network and projecting them out onto a higher dimensional space until the individual viral motifs can...