Specifically addressing your AI art point: In this case, you risk this fallacy being used to prop up technologies which solve problems they created. Which, I suspect, is part of the popular backlash against AI art in the first place. These justifications continue to be used by fossil fuel companies in developing ‘biofuelds’, ‘sustainable aviation fuel’, etc., and it’s not possible to falsify that some future iteration of the current harmful technology might exist; meanwhile the companies continue to pollute, often at greater and greater scales. There is a big difference between these companies developing sustainable fuels on the side, and redirecting 100% of their resources to that development. I suspect you might feel the same way about AI safety vs. general AI development.

Maybe we can amend the framing to exclude this somehow, because I really like the rest of this (the nuclear energy example felt particularly salient). To differentiate your examples, nuclear power intended to replace an existing harmful energy source, but AI art doesn’t replace… harmful manual artists? So I would perhaps frame it as occurring only when a promising new technology has potential harms today, but has some long tail of probabilities that could make it less harmful (rather than better) than the technology it replaces.

This is a really interesting take, and I agree with many elements. There is one element I want to explore more, and one I'd like to contest. 

Firstly, I find a lot of the acceleration vs deceleration debate to be mostly theoretical and academic - not unlike debating whether or not it is better to have tides or to stop them and have a still ocean. At the end of the day (four times a day in most places, if we're being pedantic) the tide is still going to do its thing. It's the same with technical progress. Could you make it harder to innovate and improve technology? Yes. But realistically speaking having a pause or freeze of status quo in anything approaching an effective manner is just not possible. It's the same issue I had with signing an open letter declaring a freeze. You can get everyone in the nation to sign an open letter saying "Don't commit crimes", but that isn't going to solve the crime problem. But that's a bit of a tangent and I don't want to hijack your post nor your comments with unrelated debate.

Secondly, I think the nuclear and AI debates are quite poor comparisons. Much of this is anecdotal, having worked in both industries in a regulation role. Firstly, the very high levels of anti-nuclear campaigning and risk aversion have resulted in nuclear energy being a very heavily (and effectively) regulated industry. If it was not for the amount of anti-nuclear sentiment, I don't think we'd have that level of security today. I think that's partly what makes it so safe. I agree when you discuss the risk tradeoffs between coal and nuclear that it's not as clear-cut as may be imagined, but I don't think it supports the core argument very well. Also, nuclear energy and AI are such different industries to undertake risk reduction in - mostly because of the leverages of control you have through licensing, resources, and capital. However, this may be because of the aforementioned lobbying resulting in very burdensome regulation and perhaps AI will be similarly easy to regulate in future.

It's also very possible that I'm misinterpreting your point, so please do let me know if that's the case.

 Ultimately I agree with your core point that this is a fallacy seen in much AI Safety reasoning, and that even stopping now would be shutting the stable door after the horse has bolted, but I think that there is a middle ground where speed of improvement and slower safeguards is a good way to lessen risk. I actually think nuclear energy is a good example of this, rather than a poor one.