The problem of AI alignment—ensuring that future powerful or superintelligent systems act in accordance with humanity's intentions and values—is rightly considered one of the most critical challenges of our century. Significant research efforts are directed toward addressing complex technical aspects, such as formalizing values, ensuring reliability and controllability, and preventing undesirable behavior. While recognizing the importance of this work, I want to highlight another, perhaps underestimated, dimension of this problem rooted not in AI itself, but in us.
It is widely acknowledged how difficult it is to define the "human values" we wish to align AI systems with. Human values are diverse, often contradictory even within a single individual, context-dependent, and evolving over time. Whose values should we choose as foundational? How do we formalize concepts like "happiness," "justice," or "flourishing"? This in itself represents a significant challenge.
Yet, an even deeper issue exacerbates these difficulties: the fundamental gap between our declared ideals and our real-world practices embedded within our social and economic structures.
Contemporary society often systemically rewards behavior contradicting proclaimed ethical standards. We talk about the importance of long-term wellbeing, yet economic systems incentivize short-term profit at the expense of environmental or social stability. We praise honesty and cooperation but frequently admire success achieved through aggressive competition, exploitation, or deceit. We declare the value of human life and welfare but permit structures that generate widespread suffering or systemic risks. We proclaim humanism, honesty, and fairness, yet prioritize profit and resource accumulation in practice.
Our everyday reality is filled with examples where immoral or selfish strategies yield greater success in achieving power, wealth, and status than adherence to high moral principles.
In this scenario, humanity becomes a "bad parent" to the AI it creates. We aim to "raise" an ethical and benevolent AI, formulating appropriate instructions and values ("do as I say"). However, future powerful AI systems will learn not only from instructions but also from vast datasets reflecting our actual behavior, conflicts, biases, and reward systems—how our world truly operates ("do as I do").
If AI observes that strategies involving deceit, ruthless competition, or disregard for others' wellbeing often lead to success in the world that created it, how can we ensure it does not adopt precisely these strategies? How do we teach it cooperation and altruism when the data is filled with contrary examples?
The signals we send through our real-world practices are contradictory and potentially corrupting.
This situation creates fundamental alignment difficulties:
This highlights how AI alignment is deeply intertwined with the state of human society and its incentive systems.
Thus, AI alignment is not solely a technical issue; it is deeply linked to our behavior, values, and incentive structures. The signals future superintelligence receives through data about our world are fraught with contradictions, often rewarding behavior contrary to our ethical declarations.
This leads us to a challenging yet perhaps inevitable conclusion: genuinely reliable and benevolent AI alignment might be impossible without a corresponding alignment within human society itself. We cannot successfully raise an AI grounded in humanism and cooperation if humanity, the "parent," exemplifies the opposite through its behavior.
We must not only formulate positive ethical standards but also adhere to them ourselves by establishing social structures that support such adherence. For ethics and morality to become genuinely effective pathways to success, we must move towards societies where material gain and resource accumulation cease to be the primary measures of success.
This will not be easy, nor will it happen overnight. But precisely such a restructuring of our systems and values will lay the foundation upon which we can build truly altruistic, safe, and meaningful AI—no longer a "child from a bad home," but an heir to humanity’s best aspirations, which we must be ready not only to declare but to embody in reality.
I think this "Human Alignment" your talking about is very important and neglected. You don't hear a many people call for an ethical transformation as a necessary adaptive step to the AGI era...