James Herbert's Quick takes

Thanks for taking the time to provide this context! 

Quick flag that the FAQ right below hasn't been updated

Not sure how useful this is, and you mentioned you can't speak for the choice of principles, but sharing on a personal note that the collaborative spirit value was one of the things I appreciated the most about EA when I first came across it.

I think that infighting is a major reason why EA and many similar movements achieve far less than they could. I really like when EA is a place where people with very different beliefs who prioritise very different projects can collaborate productively, and I think it's a major reason for its success. It seems more unique/specific than acknodwledging tradeoffs, more important to have explicitly written as a core value to prevent the community from drifting away from it, and a great value proposition.

As James, I also found it weird that what had become a canonical definition of EA was changed without a heads-up to its community.

In any case, thank you so much for all your work, and I'm grateful that thanks to you it survives as a paragraph in the essay.

Thanks for putting it back!

It's really important to me, as I can sometimes find that the (non-EA) charity and government world is a bunch of status-based competition over funding pots that encourages flattery and truth distortions and bitterness.

And, ok, EA can be like that as well, but ideally it isn't - ideally we'd be totally happy for our pet project to get cancelled and the money reallocated to doing a similar thing more efficiently. And also to uphold the people this happens to, recognising their inherent worth as community members and collaborators.

Hi @Agnes Stenlund 🔸 ,

Last week I had a discussion about the core principles with someone at our EA office in Amsterdam. She also liked “collaborative spirit”. I remembered this discussion and decided to check it again and see that you decided to add this in the intro essay. That’s great! Shouldn’t it then also be added on the “core principles” page? (Or am I overlooking something?) 

I don't have a super strong view on which set of guiding principles is better - I just thought it was odd for them to be changed in this way. 

If pushed, I prefer the old set, and a significant part of that preference stems from the amount of jargon in the new set. My ideal would perhaps be a combination of the old set and the 2017 set.

Expanding our moral circle

We work to overcome our natural tendency to care most about those closest to us. This means taking seriously the interests of distant strangers, future generations, and nonhuman animals - anyone whose wellbeing we can affect through our choices. We continuously question the boundaries we place around moral consideration, and we're willing to help wherever we can do the most good, not just where helping feels most natural or comfortable.

Prioritisation

We do the hard work of choosing where to focus our limited time, money, and attention. This means being willing to say "this is good, but not the best use of marginal resources" - and actually following through, even when it means disappointing people or turning down appealing opportunities. We resist scope creep and don't let personal preferences override our considered judgments about where we can have the most impact.

Scientific mindset

We treat our beliefs as hypotheses to be tested rather than conclusions to be defended. This means actively seeking disconfirming evidence, updating based on data, and maintaining genuine uncertainty about what we don't yet know. We acknowledge the limits of our evidence, don't oversell our findings, and follow arguments wherever they lead - even when the conclusions are uncomfortable or threaten projects we care about.

Openness

We take unusual ideas seriously and are willing to consider approaches that seem weird or unconventional if the reasoning is sound. We default to transparency about our reasoning, funding, mistakes, and internal debates. We make our work easy to scrutinise and critique, remain accessible to people from different backgrounds, and share knowledge rather than hoarding it. We normalise admitting when we get things wrong and create cultures where people can acknowledge mistakes without fear, while still maintaining accountability.

Acting with integrity

We align our behaviour with our stated values. This means being honest even when it's costly, keeping our commitments, and treating people ethically regardless of their status or usefulness to our goals. How we conduct ourselves - especially toward those with less power - reflects our actual values more than our stated principles. We hold ourselves and our institutions to high standards of personal and professional conduct, recognising that being trustworthy is foundational to everything else.

Thanks Jari, great questions! Also congrats on your first comment!! 

1. National orgs haven't done much experimentation with programme development, so there isn't much to copy. EA Switzerland recently bolted on an impact cohort to their trad intro programme, which looks promising, and EA UK are going to try and replicate it, so one option would be to just copy that, and I'm considering that. In the past 6 months or so, EA Israel has been trying to build an intro programme from the ground up, but that's more focused on students and young professionals. CEA's virtual programme hasn't been updated significantly in years. CEA's bootcamp or HIP's IAP are interesting candidates for copying BUT they're online and are mostly about helping people switch careers, rather than generally helping people increase their impact/onboarding people to the community.^[1]  

2. We think we might be able to do more. Last year, we experimented with various GTM strats, and got some decent results (140% increase in completions). But our CAC was a bit high, and this might be an indicator we don’t have PMF. We're also worried that, as our target audience has changed, the intro course is less likely to produce the outcomes we're looking for. 

1. ^{^}

   It might be the case that we end up doing something similar regardless because, when we interview our target audience, we discover they just really want career programmes. If that's the case, then we should probably just copy what HIP and CEA are doing (with some adaptations). 

My best guess there is also a large U.S./EU difference here.

I do think you need to differentiate the Bay Area from the rest of the US, or at least from the US East Coast.

I don't think the crux here is whether one ought to micromanage the attendance decisions of external events. It's more about:

1. "Should one give a platform to Hanania?"
2. "If forum users think it's wrong to give a platform to Hanania, is it reasonable for them to express displeasure at Manifest for giving him a platform?"
3. "If someone points out that Hanania has said, by his own admission, horrible things and therefore probably shouldn't be given a platform, is it reasonable to then write a long comment trying to add nuance to the discussion, instead of simply saying, 'yeah, seems right'"

"I think the biases in what positions people feel comfortable expressing publicly clearly go more in the direction of outrage here."

Two comments here. First in terms of comment number on the forum I would say it's pretty balanced. When it comes to voting (not public) I would say pro-platforming-Hananiah sentiment is more heavily upvoted in general

I also don't think "outrage" is an accurate reflection on the sentiment of tge majority of anti-platforming comments. Most comments have seemed like sober arguments about making attendees feel comfortable, the dangers of platforming edgy people who have made comments in the past that many consider racist, or avoiding bad press rather than irrational outage... You can make good arguments against these points but I would hardly label them as outrage.

Agree with the timezone voting thing. I should start posting at EU friendly times ;).

Suggestions for assessing the claim, "forum users are only a subset of the EA community"? Or the claim, "most of them [EAs I know] would think it'd be ridiculous to give a platform to someone like Hanania"?

I don't think there's great evidence for either claim, unfortunately. For the former, I guess we can look at this and observe that forum use is quite unequal between users, which suggests something.

For the latter, I could survey EAs I know with the question, "Do you think it'd be a good idea to invite Hanania to speak at an event?". However, even typing that out feels absurd, which perhaps indicates how confident I am that most EAs I know would think it's a ridiculous idea.

Regarding stigma, my impression is that quite a few people would like to say on the forum, "Giving a platform to Hanania is a ridiculous idea", but don't because they worry the forum will not be receptive to this view. I think this is because people perceive there to be a stigma on the forum against anyone who expresses discomfort at seeing people dispassionately discuss whether it's okay to give a platform to someone like Hanania. 

Maybe this stigma is a good thing. I'm not sure. I like what Isa said: "I want to encourage people not to dismiss that 'wtf' feeling many people have towards him and other speakers as lacking some kind of intellectual rigor or curiosity about the world". 

Regardless, the votes on my quick take suggest the stigma isn't as strong as these people perceive it to be.

One reason to believe that inviting Hanania is unpopular, though far from definitive, is the data we have on political views of EAs. About 70% of EAs identify as either "left" or "centre-left" in the EA survey. Very few identify as "right" or "centre-right". I'd assume, cautiously, the most people who identify as "left" or "centre-left" think inviting Hanania was a bad decision, though I can't be certain of that, as some Hanania supporters do seem to conceptualise themselves as centre-left. But presumably, also, some people who identify as "centre" (and perhaps even "other" or "libertarian") are also not fans of the decision to invite Hanania. 

I agree that there are virtually zero (vocal) pro-Hanania people, and there is definitely an anti-Hanania group and a mixed-on-Hanania one.  

I think a lot of people will be 'boggled' by the number of 'mixed-on-Hanania' people. I wanted to reassure that boggled crowd. 

Yup! The three OOOs are inspired by EA (although a different Dutch foundation should get the credit for the Dutch acronym).^[1]

The main criticism can be found in chapter 8 of the book (only in Dutch for now). The subheading for this chapter gives a clue: "What you can achieve by radical prioritization, and how your moral ambition can be completely derailed." Spoiler: it's SBF. 

The introduction to that chapter closes with the following paragraphs (machine translation):

"I cannot emphasize enough how important this third point is. Rob [Mather] was equally talented through all three phases, but it wasn't until he took a step back and carefully weighed his options that he started to make a huge difference. So don't start with the question: 'What is my passion?' but with the question 'How can I contribute the most?' – and then choose the role that suits you.

Remember: your talent is just a tool, and your ambition is raw energy. The question is what you do with it.

And that also applies to something else. So far I have mainly talked about the waste of talent and ambition. But there is another privilege that we waste on a massive scale: money. In this chapter I will take a step back in time and tell you about my introduction to a young cult that became aware of that. It's a movement that has taken the pursuit of impact to the extreme. A movement that is always looking for the best financial investments with the highest return for as many people and animals as possible.

Their story is about how much you can achieve through radical prioritization, but it also shows how your moral ambition can be completely derailed."

His conclusion to the chapter is much more positive about EA, but it's far from a ringing endorsement. 

I think this is a very interesting case study from the SBF saga. Yes, public polling suggests it didn't damage the reputation of EA as much as some might have feared. However, it has resulted in a loss of support from potential allies, e.g., Bregman. This is a real shame. 

1. ^{^}

   I really enjoy imagining they're talking about objected-orientated ontology. 

You missed the most impressive part of Jan-Willem’s EA CV - he used to co-direct EA Netherlands, and I hear that's a real signal of talent ;)

But yes, I guess it depends on how you define ‘EA project’. They're intentionally trying to do something different, so that's why I don't describe them as one, but the line is very blurred when you take into account the personal and philosophical ties. 

If EA was a broad and decentralised movement, similar to e.g., environmentalism, I'd classify SMA as an EA project. But right now EA isn't quite that. Personally, I hope we one day get there.  

Oh yes I agree. 

Edit: I've written more about how I see EA Netherlands in relation to SMA here.

Oh nice! Congrats with that. Do you know if it was a good use of resources?

I think it's far too early to make judgements about this groups success yet. Hype on social media is different from deep engagement, a vibrant community and billions of dollars of donations which EA has.

The biggest issue here is that Bregman downplays AI.

This is an insubstantial comment but yes I'm also sad they aren't calling themselves SoMA. 

Yeah I see what you’re saying but I guess if you know the answer to the Q ‘is it EA?’ then you have a data point that informs the probability you give to a bunch of other things, e.g., do they prioritise impartiality, prioritisation, open truth seeking, etc., to an unusual degree? So it’s a heuristic. And given they’re a new org it’s much easier to answer the Q ‘is it EA’ than it is ‘is it valuable’.

But I agree, knowing whether it’s actually useful is always far more valuable. Apart from anything else, just because the founders prioritise things EAs often prioritise, it doesn’t mean they’re actually doing anything of value.

I’m not super closely involved, I just know a few of the key people. That being said: a big name is putting his heart and soul into it, they’ve pulled together a big budget, and they’re very open to doing marketing. They’re also a talented bunch, but I think that’s at least partly downstream from the thing being kicked off by a big name.

EDIT: Oh and they are doing something different from EA, so it might just be intrinsically more popular. But I don’t think that’s the main thing going on here.

The critique describes Ord’s proposal as antidemocratic and elitist, but the original quote is more cautious and open-ended. Ord does say that the Long Reflection might take place within intellectual circles, but he also explicitly notes it could involve the wider public. My reading of his position is that he’s deliberately neutral, essentially saying: “It could take many forms - from elite discussions to broad public debates - and I don’t want to commit strongly to any particular form right now”. The only clear stance he takes is that the process should involve humility, care, and efforts to avoid bias.

The critique also states that moral philosophy is at the core of Ord’s proposal, implying Ord is basically saying, “this should be led by moral philosophers” but that’s not quite accurate. Ord suggests that moral philosophy would play a central role, but he equally emphasises the importance of other disciplines. He’s not proposing that philosophers alone decide humanity’s future - he’s advocating careful and serious reflection that draws from many areas of expertise, including moral philosophy. 

Basically, I feel like the author is assuming the worst of Ord. 

You’re right that EA’s current meta budget works out to far more than €50 per “involved person” - but that average includes the highly engaged core: people attending conferences, receiving 1-1 support, travel grants, and significant staff time.

A low-touch “supporter” tier is a different product entirely. If you ask someone for €50/year just to back the mission, receive a newsletter, and gain access to certain events, the marginal cost is minimal: card fees, a CRM entry, an occasional email, maybe a €5 welcome sticker. Even doubling every line item puts the cost at €10–20, leaving €30–40 net per person.

We could keep the high-cost, high-impact activities funded by major donors, while using a supporter tier as a lightweight way for sympathisers to express commitment and reduce funding concentration risk.

Thanks! And yeah, I didn't find it valuable. It's been a while since I read it but I wrote some thoughts here recently. 

+1; except that I would say we should expect to see more, and more high-profile.

AI xrisk is now moving from "weird idea that some academics and oddballs buy into" to "topic which is influencing and motivating significant policy interventions", including on things that will meaningfully matter to people/groups/companies if put into action (e.g. licensing, potential restriction of open-sourcing, external oversight bodies, compute monitoring etc).

The former, for a lot of people (e.g. folks in AI/CS who didn't 'buy' xrisk) was a minor annoyance. The latter is something that will concern them - either because they see the specific interventions as a risk to their work, or because they feel policy is being influenced in a major way by people who are misguided.

I would think it's reasonable to anticipate more of this.

Hmm, I agree that with influence comes increased scrutiny, and the trade-off is worth it in many cases, but I think there are various angles this scrutiny might come from, and I think this is a particularly bad one. 

Why? Maybe I'm being overly sensitive but, to me, the piece has an underlying narrative of a covert group exercising undue influence over the government. If we had more of an outside game, I would expect the scrutiny to instead focus on either the substance of the issue or on the outside game actors. Either would probably be an improvement. 

Furthermore, there's still the very important issue of how appropriate it is for us to try to guide society without broader societal participation.

Thanks! 

I agree that negative articles are inevitable if you get influence, but I think there are various angles these negative articles might come from, and this is a particularly bad one.

The Soros point is an excellent analogy, but I worry we could be headed for something worse than that. Soros gets criticism from people like Orban but praise from orgs like the FT and Politico. Meanwhile, with EA, people like Orban don't give a damn about EA but Politico is already publishing scathing pieces.  

I don't think reputation management is as hard as is often supposed in EA. I think it's just it hasn't been prioritised much until recently (e.g., CEA didn't have a head of comms until September 2022). I can imagine many national organisations such as mine would love to have a Campaign Officer or something to help us manage it, but we don't have the funding. 

Do you have any encouraging examples of progress on 2? Some of the prominent people are incredibly hostile (i.e. they genuinely believe we are all literal fascists and also Machiavellian naive utilitarians who lie automatically whenever it's in our short-term interests) so I'm a bit pessimistic, though I agree it is a good idea to try. What's a good goal to help them accomplish in your view?