I don't think that's an attack on the AI ethics crowd. I think that's an attack on wokeness which maybe deals a glancing blow to AI ethics as an incidental side effect.

Like, if you look at the purpose:

One of the most pervasive and destructive of these ideologies is so-called “diversity, equity, and inclusion” (DEI).  In the AI context, DEI includes the suppression or distortion of factual information about race or sex; manipulation of racial or sexual representation in model outputs; incorporation of concepts like critical race theory, transgenderism, unconscious bias, intersectionality, and systemic racism; and discrimination on the basis of race or sex.  DEI displaces the commitment to truth in favor of preferred outcomes and, as recent history illustrates, poses an existential threat to reliable AI.

This has very little to do with what the "AI ethics" crowd wants in my experience. The topics I hear about are more like algorithmic discrimination, misinformation, the right to an explanation, child safety, job loss, copyright, accessibility, etc.

You can also skim through the papers at FAccT 2025, I think this also suggests that it's not an attack on that crowd except incidentally.

But there wasn't originally any conflict! I don't understand why everyone is presupposing conflict!

My whole point involves reasoning about what created the conflict in the first place; I'm not going to be persuaded by arguments that presume a conflict.

I am in fact claiming it is causally upstream. Idk why you think it's implausible.

The main reason I'm not persuaded by your politician analogy, is that the politician analogy bakes in the assumption that there is a zero-sum conflict going on. But the whole question here is why there is a conflict in the first place.

I don't really care whether it's "democratic" or "undemocratic" and wish I hadn't used the word in my original comment (I was mostly just mirroring the original language).

My main claim is that AI safety / EA likely created their own enemies due to an intense focus on gaining influence and power.

I am not claiming it is inherently bad to gain influence and power. I do it myself. I just think AI safety / EA is pretty naive in how it goes about it.

Tbc some of your explanations would still go against my claim if they were true. I don't think they're true, but I agree I haven't justified that here.

What's your explanation for why they attack EAs rather than, say, the AI ethics crowd?

Why was SB 1047 so controversial, while other much more onerous AI bills (esp for "little tech") were barely discussed?

If you think their goal is just to win, why attack the movement that has power and can coordinate funding to counter their actions? What exactly are they trying to win, and why would EA stop them from achieving that (if EA were not seeking power and influence)?

(I am not claiming that their target-selection rubric is calibrated to who is actually bad or good and idk why you would think that. I feel like you are committing some kind of fallacy where in any conflict there is a "good" side and a "bad" side and this is causing you to read implications into my comments that I don't intend.)

... I explicitly called them bad faith actors in my comment; I don't think I'm extending much grace to them.

In any case, whether or not they are acting in bad faith doesn't have much bearing (for me anyway) on whether EA actions have been good.

Immense, undemocratic political spending is ramping up (though spending by the enemies of AI safety is also growing)

You present the parenthetical as a meliorating factor, but I expect that these enemies exist due to previous undemocratic power-seeking actions by the AI safety community.

(This isn't based on any private information, I just think there must be some reason these enemies single out EAs in particular. Bad faith actors don't just randomly pick targets to attack. My best guess at the reason is the intense focus on gaining influence and power.)

Or is this a stronger claim that safety work is inherently a more short-time horizon thing?

It is more like this stronger claim.

I might not use "inherently" here. A core safety question is whether an AI system is behaving well because it is aligned, or because it is pursuing convergent instrumental subgoals until it can takeover. The "natural" test is to run the AI until it has enough power to easily take over, at which point you observe whether it takes over, which is extremely long-horizon. But obviously this was never an option for safety anyway, and many of the proxies that we think about are more short horizon.

Oh sorry, I missed the weights on the factors, and thought you were taking an unweighted average.

Is it because you have to run large evals or do pre-training runs? Do you think this argument applies to all areas of capabilities research?

All tasks in capabilities are ultimately trying to optimize the capability-cost frontier, which usually benefits from measuring capability.

If you have an AI that will do well at most tasks you give it that take (say) a week, then you have the problem that the naive way of evaluating the AI (run it on some difficult tasks and see how well it does) now takes a very long time to give you useful signal. So you now have two options:

1. Just run the naive evaluation and put in a lot of effort to make it faster.
2. Find some cheap proxy for capability that is easy to evaluate, and use that to drive your progress.

This doesn't apply for training / inference efficiency (since you hold the AI and thus capabilities constant, so you don't need to measure capability). And there is already a good proxy for pretraining improvements, namely perplexity. But for all the other areas, this is going to increasingly be a problem that they will need to solve.

On reflection this is probably not best captured in your "task length" criterion, but rather the "feedback quality / verifiability" criterion.

Great analysis of factors impacting automatability.

Looking at your numbers though, I feel like you didn't really need this; you could have just said "I think scheming risk is by far the most important factor in automatability of research areas, therefore capabilities will come first". EDIT Overstated, I missed the fact that scheming risk factor had lower weight than the others.

I don't agree with that conclusion for two main reasons:

1. There's clearly some significant probability that the AIs aren't scheming; I would guess that even MIRI folks don't particularly expect that human-level AIs will be scheming (I'd guess that their take would be "they might be, they might not be, who knows"). So at the very least it seems like you should have a bimodal expectation, with both significant probability that alignment will be on par with capabilities, and significant probability that capabilities will come first.
2. Task length is also a huge differentiator and will be a much bigger deal for capabilities than for alignment. Currently for capabilities you can just look at how the AI performs on tasks that take minutes, or perhaps hours. But in the future the AIs will ace tasks that take hours, but fail at tasks that take months. That's a really rough regime in which to do research. But many areas of alignment research would still be able to work in settings where the tasks take minutes / hours. (Put another way, I disagree with your numbers on task length, once we're talking about AIs much more advanced than the ones we have today, which are the ones relevant for automated research.) EDIT: Perhaps I shouldn't say "task length" specifically, see comment later for clarification.