On Twitter, Katja Grace wrote:
I think people should think more about trying to slow down AI progress, if they believe it's going to destroy the world soon. I know people have like eighteen reasons to dismiss this idea out of hand, but I dispute them.
The introduction to the post is below. Do read the whole thing.
Consider reading alongside:
Averting doom by not building the doom machine
If you fear that someone will build a machine that will seize control of the world and annihilate humanity, then one kind of response is to try to build further machines that will seize control of the world even earlier without destroying it, forestalling the ruinous machineâs conquest. An alternative or complementary kind of response is to try to avert such machines being built at all, at least while the degree of their apocalyptic tendencies is ambiguous.Â
The latter approach seems to me like the kind of basic and obvious thing worthy of at least consideration, and also in its favor, fits nicely in the genre âstuff that it isnât that hard to imagine happening in the real worldâ. Yet my impression is that for people worried about extinction risk from artificial intelligence, strategies under the heading âactively slow down AI progressâ have historically been dismissed and ignored (though âdonât actively speed up AI progressâ is popular).
The conversation near me over the years has felt a bit like this:Â
Some people: AI might kill everyone. We should design a godlike super-AI of perfect goodness to prevent that.
Others: wow that sounds extremely ambitious
Some people: yeah but itâs very important and also we are extremely smart so idk it could work
[Work on it for a decade and a half]
Some people: ok thatâs pretty hard, we give up
Others: oh huh shouldnât we maybe try to stop the building of this dangerous AI?Â
Some people: hmm, that would involve coordinating numerous peopleâwe may be arrogant enough to think that we might build a god-machine that can take over the world and remake it as a paradise, but we arenât delusional
This seems like an error to me. (And lately, to a bunch of other people.)Â
I donât have a strong view on whether anything in the space of âtry to slow down some AI researchâ should be done. But I think a) the naive first-pass guess should be a strong âprobablyâ, and b) a decent amount of thinking should happen before writing off everything in this large space of interventions. Whereas customarily the tentative answer seems to be, âof course notâ and then the topic seems to be avoided for further thinking. (At least in my experienceâthe AI safety community is large, and for most things I say here, different experiences are probably had in different bits of it.)
Maybe my strongest view is that one shouldnât apply such different standards of ambition to these different classes of intervention. Like: yes, there appear to be substantial difficulties in slowing down AI progress to good effect. But in technical alignment, mountainous challenges are met with enthusiasm for mountainous efforts. And it is very non-obvious that the scale of difficulty here is much larger than that involved in designing acceptably safe versions of machines capable of taking over the world before anyone else in the world designs dangerous versions.Â
Iâve been talking about this with people over the past many months, and have accumulated an abundance of reasons for not trying to slow down AI, most of which Iâd like to argue about at least a bit. My impression is that arguing in real life has coincided with people moving toward my views.
Looks like Katja posted this on the forum herself as well: https://forum.effectivealtruism.org/posts/vwK3v3Mekf6Jjpeep/let-s-think-about-slowing-down-ai-1