Risk of AI deceleration.

While I am not an advocate of AI safety/alignment research because I don't think it is possible to align humans with future AGI, I have historically not done much of anything to "combat" it because I felt it was not doing any harm and maybe I would one day be proven wrong about my impossibility hypothesis.

Unfortunately, recent events are making it look like the situation is changing as AI safety/alignment researchers are now starting to take action rather than just spending their time trying to solve a maybe-impossible problem. I am of course referring to recent events like https://futureoflife.org/open-letter/pause-giant-ai-experiments/ and people lobbying governments for more regulatory constraints.

I have two problems with this new active role that safety researchers are taking:

People are attempting to stop others from doing the one thing that has historically brought the greatest reduction in suffering because there may be a way to prevent a catastrophe that might happen.

Technological progress has historically been the greatest reducer of human suffering. AI research is just the latest form of technological progress and one doesn't even need to be very imaginative to see how AGI could do wonders for addressing human suffering.

This needs to be weighed against the chance that we can actually align AI long term and the chance that an unaligned AGI will cause a catastrophe. It feels like much of the AI safety/alignment community are ignoring the harm caused by slowing/halting technological progress.

If I was convinced that there is a reasonable path toward AI alignment/safety, then I would likely be much more friendly to these proposed delays/regulations. At the moment AI safety/alignment researchers have not succeeded in convincing me that a path exists. Even worse, AI safety/alignment researchers have not convinced me that they even have a concrete target that is worth working toward.

The AI alignment stuff in particular seems to built on incredibly unstable ground. It seems to presuppose that there is some universal set of human values that an AI can be aligned with. However, humans are diverse and disagree on everything imaginable so at best you could align an AI with one specific set of human values and just tell the people who disagree with that set "tough luck, you should have become AI researchers and won the race to AGI". If this is the target for alignment, then I want no part in it. As much as I think my personal world view is right, I even more strongly believe that others should be allowed to have a different world view and I do not seek to force my view onto others, even if I have the opportunity to do so by inventing an AGI.

As far as the path to alignment goes, the critical problem that I see no way to solve is the one of "AI value drift". It seems quite unreasonable to believe that there is anything we can do today that would prevent AIs from changing their values in the future. At best, we could perhaps delay the time between first AGI and eventual value drift away from human values, but that doesn't really solve anything, it just kicks the can down the road. If we end up in a hard takeoff scenario (the most catastrophic), that delay may be hours, days, or weeks between hard takeoff and catastrophe, which are inconsequential amounts of time.

The proposed solutions to the immediate perceived problem won't actually slow progress, it will just change where the progress occurs and who is in control of it.

Things like halting AI research or introducing IRB-like institutions for deciding what AI research is allowed and what isn't could result in worse outcomes than if research was allowed to continue freely.

Short of putting everyone in prison and having some benevolent anti-AI dictator, you cannot actually stop AI research from happening. If you use governments to suppress research then at best you end up with the people doing research being limited in their capabilities to what can fit in a wealthy person's basement. This may slow things down, but it also may prevent some viable safety/alignment options like the invention of an exocortex.

Governments notoriously don't follow their own rules. Almost all of the solutions to slowing AI research involve getting governments to step in and forcibly stop people from doing AI research. However, what are the chances that the US, Russian, and Chinese governments will obey their own rules even if they agree to write them into law? Rules for thee, not for me. The most wealthy, and historically the most harmful, groups of people on the planet have been governments, so using regulation to stop AI research just means that the wealthiest and most harm causing organizations will be doing all of the research/advancement, and they'll do it in secret rather than in the light. This seems completely contrary to the stated goal of increasing the chances of eventual safety/alignment.

The above two points assume that regulations can actually work. While regulations do tend to be good at slowing down progress, they rarely end up the way the original proposers intended. It is likely that the final regulation will look nothing like what was intended by those who originally proposed it and instead will just serve to entrench certain actors in the ecosystem and prevent many individuals (likely the ones who care the most about alignment) from contributing.

I'm happy that people are trying to solve the problem of safety/alignment, and while I don't think it is possible, I still support/encourage the effort. Where I draw the line is when those people start negatively impacting the world's ability to reduce suffering and/or limit what knowledge is allowed to be pursued and what isn't.