Non-EA interests include chess and TikTok (@benthamite). Formerly @ CEA, METR + a couple now-acquired startups.
Feedback always appreciated; feel free to email/DM me or use this link if you prefer to be anonymous.
What does the average scholar think about this situation? ...They think that the work falls beneath standards because it has not been through the system of review which nearly all scholars think is required for work to reliably meet minimal scholarly standards.
This is not my experience. The arxiv version of my paper has been cited 97 times; the peer-reviewed version 7 times. The only time I can remember someone saying the paper shouldn't be trusted because of not having been peer-reviewed yet was, ironically, on the EA Forum.Â
You can compare the peer review comments with those on LessWrong. Neither is a pareto improvement over the other, but what in hindsight has proven to be the strongest critique (that growth is actually superexponential, not exponential, because of e.g. post-training) was only mentioned on LW (and twitter).Â
Peer review has many things to recommend it but, as might be guessed from a post whose "Criticism of peer review" section consists solely of claims that all criticism of peer review is invalid, this post is overstating the case.Â
What's your explanation for why they attack EAs rather than, say, the AI ethics crowd?
I think the AI ethics crowd is the subject of attacks (though arguably this is because they tried to seek power and influence).
I don't disagree, it's more that this feels a bit like privileging the hypothesis? I think the modal reason I've heard from people who did capabilities work and now regret it is something like "I knew I was misaligned with leadership but I thought leaving would be even worse."Â
If, for some reason, Anthropic asked me how to prevent people from regretting working for them, I would focus much more on "have a thing for people to do once they realize their colleague is corrupt" instead of "have a more nuanced way of telling if their colleague is corrupt."
Thanks! I only know a handful of people in this category, but for what it's worth, it again feels like people who were predisposed to thinking that working on pretraining would be okay rather than them being "corrupted."Â
E.g., I recently talked to someone who told me that their main takeaway from a safety fellowship was realizing that they didn't fit in because they actually weren't worried about existential risk in the same way that the other attendees were.
People seem surprised and bewildered when AI folks defect away from AI safety towards capabilities. People trust that as AI companies grow, those gaining power and money from shares will not be adversely influenced by that power and money.
fwiw I don't actually know many examples of this, and the ones I hear cited often seem uncompelling to me. E.g.:
(Counterexamples appreciated, though!)
Thanks for writing this up! I hadn't realized that the feb protest was so big.