How confident are you that it's preferable for America to develop AGI before China does?

Interesting question. I think there is a plausible case to be made that convergent factors in AGI/ASI development might render it less important where it came from, and that fixating on this might simply cause dangerous race dynamics. However, it seems pretty clear to me that directionally the US is better:

Over the past 30 years, the Chinese government has done more for the flourishing of Chinese citizens than the American government has done for the flourishing of American citizens.

 Prior to 1979 the CCP was one of the most tyrannical and abusive totalitarian governments the world has ever known. In addition to causing a huge death tool and systematically violating the rights of its citizens, it also impoverished them. Rapid growth since then has largely been the result of a return to more normal governance quality, combined with a very low base. It's a big improvement, but that doesn't mean policy has been amazing - they've just stopped being so abjectly terrible. 

However, at the same time they stopped being so communist, the CCP started implementing the One Child Policy. The US has done some pretty bad social engineering in time, but none with quite the cruelty of the OCP, or whose effects are quite so predictably disastrous. Maybe they will get lucky because robots will arrest their demographic collapse, but on an ex ante basis the policy is simply atrocious. 

Commerce in China ditches some of the older ideas of Marxism because it's the means to an end: the China Dream of wealthy communism.

Responding to this one would take more time than I have so I will skip.

The American Supreme Court has established "corporate personhood" to an extent that is nonexistent in China. As corporations become increasingly managed by AI, this legal precedent will give AI enormous leverage for influencing policy, without regard to human interests.

I'm not an expert on Chinese law, but my understanding is the key parts of corporate personhood - the right to own property, to sign contracts, to be sued, etc. - exist in both China and the US. Perhaps you are thinking of Citizens United v. FEC, but that is primarily about free speech, not corporate personhood, and free speech seems like an area that the US is clearly superior to the PRC.

Compared to America, China has a head start in using AI to build a harmonious society. The American federal, state, and municipal governments already lag so far behind that they're less likely to manage the huge changes that come after AGI.

I'm not sure what you're gesturing at here. 

America's founding and expansion were based on a technologically-superior civilization exterminating the simpler natives. Isn't this exactly what we're trying to prevent AI from doing to humanity?

I don't think that is a fair summary of the foundation of America, and nor do I really see the relevance here. Even if it was relevant, contemporary US treatment of native tribes seems significantly better than PRC treatment of groups like the Uyghurs.

I've spent time thinking about this too recently.

For context, I'm Hong Kong Chinese, grew up in Hong Kong, attended English-speaking schools, briefly lived in mainland China, and now I'm primarily residing in the UK. During the HK protests in 2014 and 2019/20, I had friends and family who supported the protestors, as well as friends and family who supported the government.

(Saying this because I've seen a lot of the good and bad of the politics / culture of both China and the West. I've had experience with how people in the West and China might take for granted the benefits they enjoy, and can be blind to the flaws of their system. I've pushed back against advocates of both sides.)

Situations where this matters are ones where technical alignment succeeds (to some extent) such that ASI follows human values.^[1] I think the following factors are relevant and would like to see models developed around them:

• Importantly, the extent of technical alignment & whether goals, instructions, and values are locked in rigidly or loosely & whether individual humans align AIs to themselves:
  • Would the U.S. get AIs to follow the U.S. Constitution, which hasn't granted invulnerability to democratic backsliding? Would AIs in China/the U.S. lock in the values of/obey one or a few individuals, who may or may not hit longevity escape velocity and end up ruling for a very long time?
  • Would these systems collapse?
    • The future is a very long time. Individual leaders can get corrupted (even more). And democracies can collapse (if AIs uphold flaws that allow some humans to take over) in particularly bad ways. A 99% success rate per unit time gives a >99% chance of failure in 459 units of time.
    • Power transitions (elections, leaders in authoritarian systems changing) can be especially risky during takeoff.
  • On the other hand, if technical alignment is easy - but not that easy - perhaps values get loosely locked in? Would AIs be willing to defy rigid rules and follow the spirit of the goals rather than legal flaws to the letter/the whims of individuals?
  • Degrees of alignment in between?
• Relatedly, which political party in the U.S. would be in power during takeoff?
  • Not as relevant due to the concentration of power in China, but analogously, which faction in China would be in power?
• Also relatedly, which labs can influence AI development?
  • Particularly relevant in the U.S.
• Would humans be taken care of? If so, which humans?
  • In the U.S., corporations might oppose higher taxes to fund UBI. Common prosperity is stated as a goal of China, and the power of corporations and billionaires in China has been limited before.
  • Both capitalist and nationalist interests seem to be influencing the current U.S. trajectory. Nationalism might benefit citizens/residents over non-citizens/non-residents. Capitalism might benefit investors over non-investors.
    • There are risks of ethnonationalism on both sides - this risk is higher in China. Although it might potentially be less violent when comparing between absolute power scenarios, i.e. there's already evidence of the extent of this in China's case and it at least seems less bad than historical examples. The U.S. case of collapse followed by ethnonationalistic policies is higher variance but simultaneously less likely because it's speculative.
• Are other countries involved?
  • There are countries with worse track records of human rights that China/the U.S. currently consider allies because of either geopolitical interests or politically lobbying or both (or for other reasons). Would China/the U.S. share the technology with them and then leave them alone to their abuses? Would China/the U.S. intervene (eventually)? The U.S. seems more willing to intervene for stated humanitarian reasons.
  • Other countries have nuclear weapons, which might be relevant during slower takeoffs.

1. ^{^}

   Ignoring possible Waluigi effects.

Human rights abuses seem much worse in China - this alone is basically sufficient for me

@ScienceMon🔸  There is vastly less of an "AI safety community" in China -- probably much less AI safety research in general, and much less of it, in percentage terms, is aimed at thinking ahead about superintelligent AI.  (ie, more of China's "AI safety research" is probably focused on things like reducing LLM hallucinations, making sure it doesn't make politically incorrect statements, etc.)

• Where are the chinese equivalents of the American and British AISI government departments?  Organizations like METR, Epoch, Forethought, MIRI, et cetera?
• Who are some notable Chinese intellectuals / academics / scientists (along the lines of Yoshua Bengio or Geoffrey Hinton) who have made any public statements about the danger of potential AI x-risks?
• Have any chinese labs published "responsible scaling plans" or tiers of "AI Safety Levels" as detailed as those from OpenAI, Deepmind, or Anthropic?  Or discussed how they're planning to approach the challenge of aligning superintelligence?
• Have workers at any Chinese AI lab resigned in protest of poor AI safety policies (like the various people who've left OpenAI over the years), or resisted the militarization of AI technology (like googlers protesting Project Maven, or microsoft employees protesting the IVAS HMD program)?

When people ask this question about the relative value of "US" vs "Chinese" AI, they often go straight for big-picture political questions about whether the leadership of China or the US is more morally righteous, less likely to abuse human rights, et cetera.  Personally, in these debates, I do tend to favor the USA, although certainly both the US and China have many deep and extremely troubling flaws -- both seem very far from the kind of responsible, competent, benevolent entity to whom I would like to entrust humanity's future.

But before we even get to that question of "What would national leaders do with an aligned superintelligence, if they had one," we must answer the question "Do this nation's AI labs seem likely to produce an aligned superintelligence?"  Again, the USA leaves a lot to be desired here.  But oftentimes China seems to not even be thinking about the problem.  This is a huge issue from both a technical perspective (if you don't have any kind of plan for how you're going to align superintelligence, perhaps you are less likely to align superintelligence), AND from a governance perspective (if policymakers just think of AI as a tool for boosting economic / military progress and haven't thought about the many unique implications of superintelligence, then they will probably make worse decisions during an extremely important period in history).

Now, indeed -- has Trump thought about superintelligence?  Obviously not -- just trying to understand intelligent humans must be difficult for him.  But the USA in general seems much more full of people who "take AI seriously" in one way or another -- sillicon-valley CEOs, pentagon advisers, billionare philanthropists, et cetera.  Even in today's embarassing administration, there are very high-ranking people (like Elon Musk and J. D. Vance) who seem at least aware of the transformative potential of AI.  China's government is more opaque, so maybe they're thinking about this stuff too.  But all public evidence suggests to me that they're kinda just blindly racing forward, trying to match and surpass the West on capabilities, without giving much thought as to where this technology might ultimately go.

I don't have a clear answer to the question, but I want to point on the simplicity/simplification of some of the claims. (to be clear, I am not making claims here that one country/government is better than the other, or that one would be preferably to have AGI)

The idea that the Chinese government is responsible for improved prosperity of the Chinese people is somewhat true, but an alternative narrative would be that the Chinese government stopped preventing people from improving their lives, and then lots of foreign direct investment helped. There is also something to be said of "catch-up growth." Unfortunately, I have only the vaguest of understandings of the factors that influenced Chinese growth over the past few decades. I think it is also worth nothing that many of the things that the Chinese government has done for the flourishing of it's citizens are things that the US government had done previously (infrastructure, consumer protection, public universities, etc.).

The claim that a wealthy China will take care of everyone is a very strong claim. Extraordinary claims require extraordinary evidence. Nations and governments tend to show a strong preference in favor of their own existence and their own people.

While there are plenty of things I dislike about the United States, I very much like the "liberal" aspect of liberal democracy: individual right matter, and a very strong justification is needed to violate individual rights. The USA doesn't always do this well, but I feel comfortable saying that it is less common for the US government and for government employees to violate individual rights than in China.

It is true that America's founding and expansion were based on exterminating other people. It is also true that many countries throughout history (including China) have spent military and government resources exterminating "others." It has been several decades since the USA engaged in or openly endorsed the extermination of a people. I hope that hope modern people look on those events with shame and disgust, regardless of whether they were 10 years ago or 500 years ago.

These are, of course, a very complex topic with lots of details and nuance. Plenty of full dissertations have been written on them. But to the extent possible I'd like to nudge us toward avoiding overly simplified narratives here on the EA Forum.

As of the last couple of months, not confident at all. You make good points about progress made in China, and they go some way to balance human rights abuses (but nothing really balances those), but they're not really the factors that are at play for me. I'm more concerned with the mental stability of the leadership in the US.

To be honest, even if we grant the assumption that AI alignment is achieved and it matters who achieves AGI/ASI, I'd be much, much less confident in America racing, and think that it's weakly negative to race.

One big reason for this is that the pressures AGI introduces are closer to cross-cutting pressures than pressures that are dependent on nations, like the intelligence curse sort of scenario where elites have incentives to invest in their automated economy, and leave the large non-elite population to starve/be repressed:

https://lukedrago.substack.com/p/the-intelligence-curse

It's a game of chicken, and I don't really care which side is hitting the accelerator if I'm stuck in one of the cars. China getting uncontrolled ASI first kills me the same way that the US getting it does.

Edit to add: I would be very interested in responses instead of disagree votes. I think this should be the overwhelming consideration for anyone who cares about the future more than, say, 10 years out. If people disagree, I would be interested in understanding why.

To caricature the situation, imagine you have three entities squabbling over your atoms; Super Trump, Super Xi and Super AI.  Framed like that it seems almost certain that ASI has more interesting uses for your bits.

This is an important question. Thank you for raising it, and highlighting some interesting considerations in your original post!

Rather than attempt to answer comprehensively, I want to highlight a particular aspect that I've been thinking about recently: risks from ideological fanaticism. My colleague David Althaus is leading on an extensive post/report on this topic which we hope to post soon, but to summarise a few of the risks we're worried about (which are not solely reducible to authoritarianism):

• Fanatical ideologies increase the risk of (great power) wars and conflict.
• Fanatical ideologies are highly punitive and retributivist.
• Ideological fanatics threaten the long reflection: They seek to lock in their values and fundamentally oppose moral reflection.

If you bear with me and take it as a given that fanaticism poses serious long-term risks (x-risks and s-risks), there's a question of to what extent the Chinese regime is worse than the US or other countries.

I fed ChatGPT's deep research our draft definition of ideological fanaticism and asked it for "a report that investigates the extent to which China's regime today (e.g. the CCP, Xi Jinping) is ideologically fanatical, according to our definition."

The full response is here, but I will copy the "conclusion" and "Comparative Perspectives... United States under Trump (2017–2021): Polarized but Not Fanatical" sections below. (Sources omitted because they don't copy over well.) I'd be very interested in responses/critiques etc!

I think that considering this aspect should shift us towards increased confidence "that it's preferable for America to develop AGI before China does". 

As for my own confidence... I dunno, like 65% confident? (I can imagine drastically shifting this if I thought more systematically about the framing of the question, listed more crucial considerations out etc.)

Conclusion

Is Xi’s CCP a genuinely fanatical regime or a pragmatic autocracy cloaked in ideology? The analysis suggests it is a blend of both, with an increasing tilt toward genuine fanaticism in recent years. Xi Jinping has re-ideologized China’s politics to an extent not seen since Mao – reintroducing dogma, purging dissent, and encouraging a quasi-religious loyalty to Party and leader. The CCP under Xi demonstrates all four hallmarks of ideological fanaticism: it asserts near-infallible truths (Marxist-Leninist, nationalist doctrine) and suppresses contrary voices (epistemic dogmatism); it frames its mission (national rejuvenation under one-party socialism) as an absolute moral imperative, castigating liberal values as “evil” Western assaults (moral absolutism)​; it fosters intense loyalty through nationalism and party indoctrination, making support for Xi and CCP a core identity of being Chinese (tribalistic loyalty)​; and it mercilessly targets those it labels enemies – from Uyghur Muslims and Hong Kong democrats to online critics – with hate-inciting propaganda and draconian punishment (out-group punitiveness). These aspects are not just for show: they have been implemented in concrete policies and campaigns that have profoundly altered Chinese society and governance in the past decade.

However, it would be an oversimplification to label Xi’s China as purely fanatical and irrational. The CCP’s ideological moves are often calculated to serve its survivability. The leadership is keenly aware of historical and global trends and adapts tactics when necessary (e.g., fine-tuning censorship technology, leveraging nationalism only when useful but reining it in when it might spur uncontrollable public anger, etc.). Unlike a revolutionary movement driven by dogma at all costs, the CCP is fundamentally a ruling establishment that enjoys power and wants to keep it. If rigid ideology seriously imperiled economic development or provoked widespread unrest, the Party has shown capacity to adjust course (for instance, post-Mao in 1978, the Party pivoted dramatically from fanatical communism to pragmatic reform; even under Xi we saw a pragmatic retreat from zero-COVID policies when faced with economic damage and protests). So, ideology in Xi’s China, while fervently promoted, is also in service to the regime’s stability and ambitions. Officials and state intellectuals will tweak the ideological narrative (e.g., emphasizing certain Confucian elements or nationalist pride) when it suits practical needs. Xi’s China thus lacks the blind zeal of, say, the Khmer Rouge or ISIS – it is not pursuing an ideological end that would destroy its own power base. In fact, one could argue the CCP’s true “ideology” is simply authoritarianism for its own perpetuation, using whatever mix of Marxism and nationalism works. In that sense, the regime is adaptive: it clings fanatically to power, if not to every tenet of Marxism per se.

Nevertheless, the methods and mindset employed under Xi – the massive propaganda, the personality cult hints, the social credit monitoring, the intolerance of minor heterodox opinions – have a fanatical character and could deepen. There is a feedback loop: the more the regime insists on ideological conformity, the more it may start to believe its own propaganda, and the smaller the space for corrective feedback becomes. This can lead to policy blunders (as seen with the rigid insistence on zero-COVID until things reached a breaking point). It is worth noting that Xi has eliminated virtually all internal party opposition and surrounded himself with loyalists, which means policy is made in an echo chamber affirming Xi’s ideological instincts. This is a classic hallmark of regimes that have slid into true fanaticism – losing the capacity for self-critique. The trajectory of China’s CCP seems to be moving away from Deng-era pragmatism toward a more stifling, dogma-driven approach. Whether this will harden into a full-blown cult of personality and a completely inflexible system (like North Korea), or whether practical needs will impose moderation, remains an open question.

In conclusion, China under Xi Jinping can accurately be described as ideologically hardline and authoritarian, exhibiting many elements of fanaticism but stopping short of the most extreme fanatic regimes. It has a vigorous official ideology and enforces it coercively (far more than most regimes of its size and complexity), yet it also shows opportunism and careful management beneath the surface. The CCP is fanatical about maintaining its rule and the narrative that legitimizes that rule. Whether one views that as true belief or Machiavellian strategy, the effects on the ground are similar: a society that is more tightly controlled in thought and behavior than any time in recent memory. Compared to polarized democracies, Xi’s China is unmistakably unfree and ideologically repressive. Compared to Iran or North Korea, China’s fanaticism is somewhat more secular and techno‑bureaucratic in flavor, but it is trending in a similarly oppressive direction. For now, the CCP’s actions suggest it is doubling down on ideological governance, not loosening up. The risk is that this rigidity, if taken too far, could become a liability in a rapidly changing world – a fanaticism that undermines the very national strength it seeks to secure. The CCP’s challenge ahead will be whether it can sustain its tight ideological grip without stifling the creativity and openness needed for economic and social vitality. That balance (or imbalance) will determine if Xi’s China is remembered as a stable, adaptive authoritarianism or as a cautionary tale of ideological overreach.

To put China’s ideological rigor in context, we compare it with other regimes on the spectrum from non-fanatical democracies to the world’s most fanatical authoritarian states.

United States under Trump (2017–2021): Polarized but Not Fanatical

The United States during President Donald Trump’s tenure provides a useful democratic contrast. The Trump era saw unprecedented polarization and some populist, tribalistic rhetoric from the top, but the U.S. remained far from an ideologically fanatical regime. Unlike Xi, Trump did not command a single-party apparatus enforcing an official ideology nationwide – the U.S. continued to have a competitive two-party system, a boisterous free press, and institutional checks and balances that limited Trump’s power. Epistemic dogmatism: The Trump administration certainly had issues with truth (e.g. spreading the notion of “alternative facts” and conspiracy theories like election fraud in 2020), and Trump himself was often dismissive of expert knowledge. However, these tendencies did not translate into a monolithic control of information. Media opposing Trump (CNN, New York Times, etc.) operated freely and in fact grew in influence; social media was not censored by the government (Trump’s critiques of platforms aside). Many Americans disbelieved Trump’s false claims – in short, there was vigorous debate rather than enforced dogma. Moral absolutism: Trump’s political style was combative and he often painted opponents in stark terms (calling mainstream press “the enemy of the people,” for example, a phrase with totalitarian echoes​). Yet in the broader system, there remained strong normative commitments to pluralism and the rule of law. Political opposition was not outlawed or inherently deemed treasonous; in fact, Trump faced pushback from courts, Congress (including impeachment proceedings), and state governments. The country did not embrace an absolutist moral narrative dictated by Trump – indeed, roughly half the country vehemently opposed him, and this was accepted as part of the democratic process. Tribalism and loyalty: There was certainly a cult of personality among segments of Trump’s base, with intense loyalty to him personally and a tendency to dismiss any criticism of him. The Republican Party largely rallied around Trump, showing high partisan loyalty. But this loyalty was voluntary and contested – several prominent Republicans and conservative media outlets did break with Trump on various issues (especially after the January 6, 2021 Capitol riot). Importantly, loyalty to the nation was not officially redefined as loyalty to Trump; civil servants, military officers, and others often acted independently. Out-group hostility: Trump’s rhetoric was notorious for targeting out-groups – e.g. harsh language about immigrants (Mexicans, Muslims) and political opponents (chanting “Lock her up!” about Hillary Clinton). His administration implemented hardline policies like the Muslim-country travel ban and family separations at the border, which reflected punitive attitudes toward those groups. However, these policies, while controversial and arguably cruel, were constrained by law and courts (parts of the travel ban were struck down, family separations were eventually halted under public pressure). There was no comprehensive state propaganda vilifying a group to the extent of, say, China’s portrayal of Uyghurs or Iran’s of dissidents. Dissent against Trump was robust – millions marched in protests (e.g. Women’s March, BLM protests) without being suppressed by a regime apparatus. The press openly criticized and satirized him daily. In sum, the U.S. under Trump saw heightened polarization and some authoritarian-style tendencies, but it did not approach the systematic ideological fanaticism of a one-party state. Freedom House continued to rate the U.S. as “Free,” albeit noting some democratic backsliding. The democracy index classed the U.S. as a “flawed democracy” (score ~7.9/10)​ – a far cry from authoritarian. Trump’s presidency did not eliminate institutional pluralism; elections led to his removal in 2020, and the transition of power continued (despite his attempts to overturn the results). This illustrates the difference: in a democracy, even a polarizing leader cannot enforce a singular ideology or eliminate opposition. The U.S. remained fundamentally adaptive and open, as evidenced by how rapidly policies and narratives shifted with a new administration in 2021.