Know when to sound alarm bells

What is the situation where people coordinate to sound alarm bells over an AI benchmark? I basically don't think people pay attention to benchmarks in the way that matters: a benchmark comes out demonstrating some new potential danger, AI safety people raise concerns about it, and the people with the actual power continue to ignore them.

Thinking of some historical examples:

• Anthropic's findings on alignment faking should have alerted people that AI is too dangerous to keep building, but it didn't.
• Anthropic/OpenAI recently finding that their latest models may cross ASL-4 / CBRN thresholds should've been an alarm bell that they can't release those models, but they released them anyway and nobody could do anything to stop them.

I think its incredibly hard to determine, but probably negative EV, as has been the history (IMO) of many (perhaps even most?) EA AI safety interventions. Well intended but for-the-worse-in-the-end. As a side note, I think the history of EA intervention in AI is a great example of just how hard it is to intentionally positively influence even the mid-term future.

It would only be positive if it actually contributed to some kind of slowdown or policy change, which seems pretty unlikely to me. I don't think an eval reslut will ever sound an alarm bell, that would only come from some kind of obvious capability shift or warning shot that affected the general public. 

More likely is that it slightly speeds up AI development basically like @MichaelDickens said.

1. Adding to the race dynamic with the clear targets and competition to be at the 'top'
2. Giving AI companies better tools to measure progress
3. Incentivising investment 

Increases the speed of ai development

This is the biggest con by far. There are at least two mechanisms by which it could happen:

• Make AI capabilities growth more apparent, incentivizing investment.
• Help AI companies learn how to build more powerful AI systems.