The Long (Self-)Correction

I propose the Long Self-Correction^[1] as an alternative name/idea/concept to AI Pause and Long Reflection.

Problem with AI Pause: Pause until when, and for what purpose? Presumably to make AI (that we'll build later) safer, but the deeper problem is that humans aren't safe, and can't safely serve as builders, overseers, or alignment targets for powerful AIs.

Problem with Long Reflection: It seems to imply that the main problem with humans is that we just haven't had enough time to think, that reflection is the main thing we need to do more of, and then we can get on with building powerful AIs or other technologies. Or that if we build aligned AIs that sincerely help us think a lot more, or do the thinking for us, then things will turn out fine.

So I think we need a catchy handle for a related but distinct idea, that humans aren't ready to build AIs or other extremely powerful technologies, because we're currently too flawed, in a variety of ways, and it will take a long process (which may or may not end up succeeding) to fix those flaws.

A summary of the flaws that I have in mind:

1. not having a workable moral framework (consequentialism, deontology, virtue ethics all having serious problems)
2. being bad at philosophy and long-horizon strategy
3. being badly calibrated about our philosophical and strategic competence, i.e., not realizing that we're incompetent, despite overwhelming evidence (see e.g. FTX and early MIRI, and many others, trying to maximize impact while assuming their own philosophical and strategic competence)
4. in practice, human morality is a kind of status game that actively disvalues careful strategy and philosophy in most places
5. positional/zero-sum values (like power and social status) being a huge part of human motivations, but almost nobody explicitly reasons/talks about this while discussing, for example, AI safety or effective altruism or how to make the long term future turn out well^[2] 
6. being easy to manipulate (or go off-rails by oneself) via things like sycophancy, plausible-sounding philosophical arguments (see e.g. Objectivism or Dialectical Materialism), limerence, hero worship, spirituality
7. not understanding the nature of philosophy so that we don't know how to go about fixing many of these issues even in principle
8. not realizing how flawed humans are, or the full scope and interlocking nature of the human and AI safety problems, and thus being prone to over-optimistically proposing or implementing partial solutions (e.g., thinking that making AIs aligned or corrigible to humans or to some favorite moral theory would be sufficient to make them safe, or that we just need the right people to win the AI race, or Tool AI, or Scientist AI)

(This list focuses on key bottlenecks that seem hard to fix even with AI assistance or intelligence enhancement, and isn't meant to be a complete list of human flaws / safety problems. It ignores e.g. that the median human is ignorant of many important issues, and that we're currently quite bad at complex large-scale coordination such as passing/implementing close-to-optimal government policies.)

My main hope for a Long Self-Correction eventually succeeding rests on the fact that humans have seemingly, mysteriously, made progress on these issues over a very long period of time, so if we preserve the environment in which we can seemingly do this, and not give anyone or anything the power to permanently derail such progress, then maybe we can continue to snowball The Correction until we reach a point when we can rightly justify reshaping the universe according to our volition.

1. ^{^}

   It will probably be shortened to "The Long Correction" at some point if it catches on, similar to how "outer space" is now often just "space".

2. ^{^}

   Why isn't there a version of EA that explicitly talks about how to leverage people's status motivations to do more good for the world? It's very possible that explicit talk about status is actually counterproductive at least in the short run, e.g. it heightens status motivations and makes people less altruistic, but then do we just march into the future while blindfolding ourselves to this aspect of human nature?