Why did the EA organizations find it disappointing? I’m afraid you’re going to say they didn’t like that peer reviewers didn’t agree with them, and therefore they decided the peer reviewers were wrong, and peer review is a waste of time.

Not all EA organizations are consistently producing high-quality work. That’s part of the problem. For instance, the problems with the METR time horizons graph are numerous and severe. Many of them were entirely avoidable, and should have at least been better disclosed. I can’t get over that most of the longer tasks, on which the 2025 segment of the graph depends, don’t have empirically measured human baselines. The baselines are just guesses by the authors. Surely if you don’t even bother to measure data that doesn’t qualify as high-quality? This also wasn’t disclosed until 2026 — a major omission.

What I would recommend to people at this point is to not believe any of METR’s claims, research, or analysis going forward unless and until it can be independently verified by a reliable source. You don’t know if METR’s data is data or just guesstimates. You don’t know that the typical best practices of scientific research have been followed. You don’t know that flaws or shortcomings or limitations that METR is aware of will be disclosed with sufficient emphasis, consistently across all communications.

Very few people outside of EA consider EA’s idiosyncratic ideas to be serious and credible. What is the strategy for gaining credibility outside the EA echo chamber? Right now, it seems to be a media strategy that counts on people not fact checking EA’s messaging. This could work — a lot of misinformation misleads a lot of people a lot of the time — but it also might rightly damage EA’s reputation if people eventually learn EA is not telling them the truth. It’s a risky strategy that depends on being able to fool people, rather than intellectually convince them.

80,000 Hours’ abysmal video on AI 2027 is an example of this. It misinforms its audience about AI experts’ views and insinuates there is a consensus in support of AI 2027’s core claims that doesn’t exist. Either 80,000 Hours knew this and misled its audience anyway, or it didn’t do a proper fact check of its script before producing the video. I was a lifelong fan of 80,000 Hours until that video. Now I no longer trust 80,000 Hours about anything. Not even career advice. I was in the top 1% or 0.1% of biggest supporters of 80,000 Hours. Now I’ve been completely polarized in the opposite direction. This is anecdotal, but, also, most people become angry when they feel as if they’ve been misled. It’s not a stretch to think this strategy could really blow up in an ugly way.

In my opinion, EA is aggressively burning down its reputation and risks being correctly labelled as a purveyor of misinformation. Steps should be taken to at least stop the bleeding.

I don’t know for sure that peer review would help move idiosyncratic EA ideas outside the EA echo chamber. I also don’t know that there isn’t a better strategy for doing so. It just seems like a good idea to me. The economist Tyler Cowen was actually the first person who I heard suggest this. I believe he was talking about AGI/AGI safety. It was on a podcast, either his or someone else’s. I remember he said: publish, publish, publish.

The provider I originally mentioned in this post definitely looks like a shady company that I definitely wouldn’t recommend. I was wrong to mention that company and, in retrospect, the signs were obvious that it wasn’t a trustworthy company. I only gave it a few cursory glances. I’m grateful to Clara for giving it a second look and realizing that both Google Gemini 3.1 Pro (with “Extended thinking”) and I had been duped by some devious SEO.

The trustworthiness of that provider — or indeed any similar company offering a convenient, off-the-shelf service — is beside the point of whether peer review is a good idea or not. There are scam companies selling fake Ozempic online. That has nothing to do with whether genuine Ozempic is a good drug or not.

Reuters recently published a fantastic article on Tesla’s efforts to deploy robotaxis: https://www.reuters.com/investigations/why-teslas-ai-trainers-dont-trust-its-self-driving-tech-or-its-safety-stats-2026-05-28/

The article argues that Tesla’s actual technical progress is far behind what the company claims.

I sense this post has insightful and important things to say, but it’s written in a way that’s so abstract that I find it hard to follow. I’ve read several posts like this on the EA Forum. Posts like this desperately need lots of concrete, specific examples described in plain English. The best would be telling stories (anecdotes) that illustrate a problem you’re trying to talk about.

This part I think I can understand and agree with:

EA community building seems less about helping people have an impact by their own lights, and more about having an impact by the lights of the community builders.

Concretely, I infer this means whether it’s 80,000 Hours’ career advising, EA groups at universities doing outreach, interactions at EA conferences, or whatever else, the EA veterans are ultimately just trying to evangelize to the EA newbies. Which is not what the EA newbies want them to do, nor what the EA veterans are leading the newbies to believe is going to happen. Am I right?

This too:

FTX didn't just happen to EA - it also happened within EA. SBF was supported, celebrated and elevated by many parts of EA leadership. The damage wasn't primarily to EA's brand; it was to the trust people had placed in EA - and that loss of trust was at least partially warranted.

To quote the writer Matt Yglesias:

I understand the various reasons why people can't speak in detail about certain things would I would love to see the issues discussed in "EA and the current funding situation" get revisited in light of the fact that the funding situation has changed a lot. It seems to me that when money became more abundant that didn't just make it easier for certain grant proposals to get approved, but was seen as having big-picture strategic implications. When I go back and read the post, the arguments seem sound to me. But beyond not reflecting knowledge of wrongdoing at FTX (hard to find blame here) they seem to not reflect awareness of Bankman-Fried's overall attitude toward risk which I think should have been knowable. 

I’d add it wasn’t just Sam Bankman-Fried’s attitude toward risk that was knowable. It was also the riskiness of crypto. Shouldn’t people have been expecting there was a decent chance FTX would blow up, even without any financial crimes?

Besides community building and FTX, the third major example in this post is the Centre for Effective Altruism’s communications strategy or brand strategy. The Centre for Effective Altruism describes this in general, abstract terms that leave me guessing what they mean, concretely and specifically. This post responds to the CEA’s description of its strategy in terms that are also abstract and general. 

Who is right and who is wrong? What is the disagreement even about? Is there an actual disagreement or are people just talking past each other? I don’t know. We’d need specific examples of what it would look like to have “honest and positive” messaging versus neutral and purely informative messaging.

I do agree that it feels like EA is dying, but, for all I know, my reasons for feeling that way might be the exact opposite of yours. 

I didn't know that about Open Philanthropy!

If EA organizations commission academic reviews and ignore them, then, yeah, it's pointless. I guess there has to be some underlying belief that academic feedback is epistemically valuable. Or at least an underlying commitment to move ideas out of the EA echo chamber into wider acceptance by doing research that is persuasive to people outside of EA.

I see two discouraging signs. One, an anti-academic prejudice in EA. (Often along with a belief that EA is intellectually or epistemically superior to academia, and possibly the rest of the world, too.) Two, low patience for attempts to persuade people outside of EA about ideas that are popular within EA but unpopular outside it (e.g., a 50%+ chance of AGI within the next decade).

If people in EA want to switch gears from operating EA as an elite enclave (or conclave) to a movement that can influence the world at a large scale, including the policies of large liberal democracies like the United States, this change will be painful. People will have to learn how to go from having the majority opinion (in EA) to the minority opinion (in the world). From having the power to decide which opinions can and can't be expressed (in EA) to fighting to be heard in contexts where others have that power (in the world). This is as much about emotional regulation as it is about intellectual discipline.

Thanks for the comment, David.

Hold on, you're right! They say "Wiley" a lot, but they aren't actually affiliated with Wiley! I think the "Wiley" thing was just an SEO trick! Okay, well now this company definitely seems sketchy, and I wouldn't trust them!

I was looking at this at the same time I was looking at Springer Nature's scientific editing service — which is affiliated with Springer Nature, but it's just editing, not peer review — and ended up thinking it was a similar service. (Google Gemini Pro lied to me/fell for Meritpeer's SEO and told me Metritpeer was Wiley, but it's totally my fault for not fact checking this better when I clicked through to Meritpeer's website.) I'm going to edit my post.

By the way, you're not derailing at all, this is an extremely important and helpful contribution!

The general idea of paying for external expert review or peer review still makes sense, but it would require more doing on the part of EA organizations to make it happen if it's not an off-the-shelf service. Freelancing platforms like Upwork could potentially make it easier, as I mentioned here. I say potentially because I don't know if you could reliably find good peer reviewers on Upwork.

Would you be willing to agree to a bet on this? Anthropic’s revenue has grown at a compound annual growth rate (CAGR) of 570% over the last 3 years. If this trend continued for 1 more year, then Anthropic would hit $200 billion in annualized revenue less than a year from now.

However, Anthropic’s own revenue projection is for $150 billion in 2029. If we infer from Anthropic’s valuation, its investors are implicitly pricing in much slower revenue growth over the next 3 years than a 570% CAGR.^[1]

As an additional data point, an HSBC analyst projected $241 billion in revenue for Anthropic in 2030. Coatue Management predicted $200 billion in revenue in 2031.

So, I propose a bet: if by June 1, 2027, Anthropic has at least $200 billion in annualized revenue, you win. If by June 1, 2027, Anthropic has less than $200 billion in annualized revenue, I win.

I would be happy to bet for a nominal amount like $20 to the charity of the winner’s choice.

I'm also open to shorter-term bets. For instance, I would bet that Anthropic will not hit $125 billion in annualized revenue by the end of 2026 (which is what extrapolation would imply).

1. ^{^}

   A sustained 570% CAGR would imply Anthropic will hit $9 trillion in annualized revenue in 2029. Let’s apply a super conservative revenue multiple, 1.0 (unreasonably low). Let’s also apply a super steep discount rate, 25% (way too high for a normal megacap tech company). Even with these assumptions, we still get a $4.6 trillion valuation for Anthropic. Anthropic’s current valuation is under $1 trillion.

Hm, interesting! Thanks for weighing in!

My wild guess about the turnaround time is that they just have so many reviewers “on call” that even if most are unavailable within the 10-day window, at least some people will be available.

The price does seem kind of low. I wonder if the actual average price ends up being more than the list price? E.g., if drafts are above 5,000 words?

I do wonder if the price and turnaround time is too good to be true.

Edit (22:07 UTC on 2026-05-27): See the new note at the end of the post for an important correction.

Where does your doubt come from? Do you doubt that peer review in general is good quality? Or does this service seem too cheap or too fast to be any good?

There’s also the EA organization called The Unjournal, which commissions reviews of EA research from external experts. But I don’t know if this is a better option than Wiley’s service.

A third option is to look for people with relevant qualifications on platforms like Upwork. Here’s a recent freelance job posted on Upwork:

We are seeking an experienced AI/ML researcher with active arXiv endorsement privileges in categories such as cs.AI, cs.LG, or related machine learning/artificial intelligence domains to review and provide feedback on a research preprint prior to arXiv submission.

Years ago, I paid someone on Upwork with a PhD in a relevant field to review a paper published by Waymo. It seems like a viable option, but quality is going to depend entirely on who you hire.

And of course option #4 is to submit papers to peer-reviewed journals.

Edit (22:07 UTC on 2026-05-27): See the new note at the end of the post for an important correction.

So, your skepticism comes from the 10-day turnaround time? If it were 60 days or 90 days instead, you wouldn’t feel skeptical?

I wonder how/why they are able to offer such fast turnarounds and whether it’s by sacrificing quality. Do you think if you got paid, say, $150-250 per review you’d make time to do them faster? Or would it just be impossible regardless?

There are a number of other services similar to Wiley’s. I don’t know if any of them are any good.

Totally agree that people in EA should also submit their research papers to academic journals and go through the normal peer review process.