I am not American. But compared to China, I have plenty of friends from the U.S., Australia is more culturally similar to the U.S., and I have spent more time there. So, is the fact that I support the U.S. over China in the AI race just selfish and downstream of my personal circumstances?
I would like to think not. But it is worth seriously considering what the future would be like if the CCP becomes the dominant player in AI development. Dominance in AI development could (but would not necessarily) give China a decisive strategic advantage and control over the galactic future. Here, I think through what a CCP-led future would mean for human flourishing, and avoiding risks from misalignment and war.
Present-day harms
I care about creating a Better Future – not just today's world minus poverty and disease, but a utopian future qualitatively different from today. How likely are we to achieve such a future if the CCP is in charge?
First, I think some common reasons to dislike the CCP aren't as decisive in the long term as they might seem. I take a fairly utilitarian perspective here; placing intrinsic value on diversity, democracy, or human rights regardless of welfare outcomes makes the picture look considerably worse.
Persecuting minorities: China is (in)famously not a nice place for ethnic or religious minorities (e.g. Tibetans, Uyghurs). But in a CCP-controlled future, the vast majority of people (either biological or digital) will likely be culturally Han Chinese. This is because minorities may be successfully (if brutally) assimilated, or they could simply be underrepresented in space colonization and digital minds programs. If we're scope sensitive and thinking about trillions of future beings, the persecution of minorities in the 21st century, while deeply tragic, will not feature prominently in the overall (dis)value of the future.
Repressing freedom: But it isn't just minorities that have a hard time in China – everyone's speech and action is closely policed, which is arguably incompatible with a flourishing society. However, this too might change: you only need to rule with an iron fist if you are scared of losing your grip on power. In an ASI-enabled CCP dictatorship, there could be common knowledge that overthrowing the government is impossible, so leaders might not have as much to fear from some protest action and dissident speech. For instance, minituarized drones could simply incapacitate anyone attempting serious violence, reducing the need for pre-emptive thought-policing. Of course, such a society would still be meaningfully unfree in some senses, but having a far narrower set of activities (violent rebellion) that are impermissible could be a vast improvement from today’s repression of free speech.
That said, there are strong counterarguments here. Historical authoritarian regimes have rarely relaxed control even when firmly entrenched. And free speech that cannot change anything arguably doesn’t contribute much to flourishing, given it is constrained. So while I think some loosening of day-to-day repression is possible, I'm far from confident. I tentatively think an ASI-powered CCP might allow somewhat more personal freedom than exists today.
Economic stasis: Centrally planned economies have historically not produced innovations at the same rate as liberal democracies. Restricting the free market is putatively the road to serfdom. However, ASI could for the first time allow centralised information processing to be competitive with the distributed information processing of the market. It may not be fully efficient, but an ASI-powered centralised economy is likely to avoid catastrophic blunders like the Great Leap Forward.
Based on these considerations, I tentatively expect that the average welfare of individual subjects in a CCP-led future would be fairly high—perhaps better than many pessimistic portrayals suggest. However, I think this still misses out on most possible value.
A Flourishing Future
Moral innovation: The truly best futures may require substantial moral reflection and innovation, ending up very different from today. Recent centuries have seen enormous moral progress: increasing consideration of the interests of peasants, women, ethnic minorities, animals, and future people. My impression is that most of this innovation has originated in the West and been exported later, if at all, to China and other authoritarian states. Moral philosophy research also seems far stronger in the West than in China. The ethical schools of thought I'm most aligned with—longtermism, sentientism, effective altruism, and utilitarianism—are far more prominent in the West (though still very niche).
Western countries appear more likely to expand the moral circle to include animals. If the far future contains vast numbers of animals (or especially digital minds), the ruling culture being more pro-animal might matter greatly. Of course, the U.S. has awful factory farming too, so perhaps isn’t that much better.
It is also interesting that China ranked last out of 24 major countries on charitable giving as a percentage of GDP, with 0.03%, compared to the U.S. at 1.44%. But I don’t put much weight on this, given the very different cultures and economies of the two countries.
Pluralism, liberalism, and the long reflection: Despite my tentative prediction that China might become less repressive if it controlled the future, I don't expect China to become a liberal democracy. Power will likely remain immensely concentrated in one or a few CCP leaders. And for all their faults, liberal democracies still seem far better at dynamism and taking new ideas seriously. If something like "the moral truth" exists to be discovered, it will probably look quite weird and different from any current ideology. A pluralistic, liberal society has a better chance of progressing towards the moral truth; Xi Jinping Thought surely isn’t the last word on moral truths in the universe. Even under moral anti-realism, a more pluralistic moral reflection process may produce better outcomes by most people’s lights.
It's worth noting that Taiwan, which shares Chinese cultural heritage but developed democratic institutions, scores much better on liberalism, pluralism, and moral/institutional innovation than the mainland. This suggests the issue is less about "Chinese values" and more about the governance system the CCP has imposed.
So, even if a CCP-run future delivers reasonable welfare for most beings, I expect it to miss out on the vastly greater value that could be unlocked through continued moral progress and liberal dynamism. The difference between a "pretty good" future and a truly excellent one could be astronomical in a universe-spanning civilization.
Avoiding AI catastrophe
But before we even get to designing utopia, humanity needs to safely navigate the acute risks associated with developing ASI. How would a Chinese lead in AI affect our chances of avoiding misaligned AI takeover and war?
Misalignment: Historically, most work outlining risks from misaligned AI and potential solutions has come from the West. Some safety work is emerging from China, but my impression is that there are still far fewer people who deeply grasp the risks from misaligned ASI. Part of this simply reflects that the West leads in AI research generally, not some deep cultural difference. Still, by default, I expect a Chinese lead in AI development to mean less effort from the leading AI project in preventing AI takeover.
Moreover, given that the US is currently ahead, if China has a lead, it will likely be a narrow one, with both the US and China racing recklessly to not fall behind. This would be terrible for doing deep alignment work. Conversely, it is more possible that the US will have a large lead, allowing them to slow down and invest more in safety work at the crucial moment (though whether they actually would is another question).
One countervailing consideration: conditional on a Chinese lead, China's AI developers have probably been centralized under state control, which could reduce within-country racing between projects and potentially allow for more safety work. But this effect seems relatively weak, and the centralization itself creates other problems. Overall, I expect a Chinese lead to significantly harm our chances of solving alignment in time.
War: Forecasting which AI development pathways are more likely to lead to a US-China war is extremely difficult. As I've argued previously, the commitment problems created by the possibility of decisive strategic advantage make rational war more likely than in typical geopolitical contexts.
One side (likely the US) having a large lead could reduce the chance of war, as the laggard would recognize their low chances of success (whereas in a close race the laggard would try to catch up legitimately). Conversely, the laggard might be desperate if they are far behind, or unwilling to "lose face" by accepting a lopsided bargain. Overall, the interplay between the size and direction of an AI lead and the risk of war seems murky.
Conclusion
So, I have reaffirmed the traditional conclusion that a US lead is good. What should we do about this? Probably nothing new – I think this validates the AI governance community’s focus on denying China access to AI compute, and on making the US government take AI more seriously. Still, given the possibility of a Chinese lead in AI (and, thereafter, maybe domination of space futures), an increase in people thinking about AI safety and moral innovation in China seems great.
I want to point out that the ethical schools of thought that you're (probably) most anti-aligned with (e.g., that certain behaviors and even thoughts are deserving of eternal divine punishment) are also far more prominent in the West, proportionately even more so than the ones you're aligned with.
Also the Western model of governance may not last into the post-AGI era regardless of where the transition starts. Aside from the concentration risk mentioned in the linked post, driven by post-AGI economics, I think different sub-cultures in the West breaking off into AI-powered autarkies or space colonies with vast computing power, governed by their own rules, is also a very scary possibility.
I'm pretty torn and may actually slightly prefer a CCP-dominated AI future (despite my family's past history with the CCP). But more importantly I think both possibilities are incredibly risky if the AI transition occurs in the near future.
For what it's worth, we recently ran a cross-cultural survey (n > 1,000 after extensive filtering) on endorsement of eternal extreme punishment, with questions like "If I could create a system that makes deserving people feel unbearable pain forever, I would" and "If hell didn't exist, or if it stopped existing, we should create it [...]".
~16-19% of Chinese respondents consistently endorsed such statements, compared to ~10–14% of US respondents—despite China being majority atheist/agnostic.[1]
Of course, online surveys are notoriously unreliable, especially on such abstract questions. But if these results hold up, concerns about eternal punishment would actually count against a China-dominated future, not in favor of one.
On individual questions, agreement rates were usually much higher, especially in China and other non-Western countries. The above numbers reflect a conservative conjunctive measure filtering for consistency across multiple questions.
Interesting! (And troubling - well above the lizardman constant.) It would be interesting to do some qualitative follow-up on this, maybe with having these consistently retributivist people chat with an LLM instructed to do qualitative data collection and gently nudge them towards more suffering-averse views to see how deeply held or changeable those beliefs are.
If I'm interpreting this correctly, 25% of people in China think that at least 58% of all people in the world deserve eternal unbearable pain (with similar results in 3 other countries). This is so crazy that I think there must be another explanation, e.g., results got mixed up, or a lot of people weren't paying attention and just answered randomly.
Thanks for flagging this, but you're looking at an unfiltered sample (N=2,980) which includes almost all participants regardless of data quality. All statistics in the main text use a filtered sample (N=1,084), which excludes participants who failed two attention checks, reported not answering honestly, gave invalid birth years, or strongly violated additivity (see the relevant section in the main post, including footnotes 94 and 95 for more details). The unfiltered numbers should be ignored as they clearly contain a lot of inattentive participants. (We will update the supplementary materials to label them clearly, sorry about the confusion).
Here is the table that only includes participants who passed our inclusion criteria (the 7th table in the selected stats doc.
As you can see, the numbers are much lower: 25% of Chinese respondents believe that at least 10% of people deserve unbearable pain forever.
(Note: this table shows N=1,036 rather than the N=1,084 in the main text; the small discrepancy likely reflects a stricter additivity filter. I'm confirming with my co-author Clare Harris who analyzed the survey and wrote the supplementary materials.)
Thanks for clarifying. Sill, this suggests that the Chinese participants were on average much less conscientious about answering truthfully/carefully than the US/UK ones, which implies that even the filtered samples may still be relatively more noisy.
Perplexity w/ GPT-5.2 Thinking when I asked "Are there standard methods for dealing with this in surveying/statistics?", among other ideas (sorry I don't know how good the answer actually is):
Hi, happy to speak to the methodological points here.
Thanks for sharing the link and suggestion. We agree that understanding how much we can trust the data is crucial for interpreting our results, so thank you for engaging with this critically.
We didn’t measure individual reaction times for questions so using RT modelling isn’t an option. Modelling carelessness in other ways (e.g., modelling it as a latent tendency) would be fascinating, but I don’t endorse the assumptions we’d need to model carelessness as a latent variable. (I like how Rohrer & Paulewicz 2025 point out that latent variable modelling requires making some strong assumptions.) So even though we could try to model carelessness, I currently don’t think we should do so.
The headline results were designed such that it would be very unlikely for participants selecting at random to meet our definition of “consistent and concerning” endorsers. In the main piece, the focus is on who agreed with the hell question, AND selected “Forever” for the duration question (the last of 11 options), AND selected 1% or more for the proportion question. The supplementary materials additionally report those who endorse BOTH the “endorses system” question and “would create” question, AND selected “Forever” for the duration question (the last of 11 options), AND selected 1% or more for the proportion question. Partly due to the design of these “headline” results variables, the proportion of the sample meeting our definition of “consistent and concerning” turned out to be robust to the removal of all attention checks (i.e., the inclusion of everyone who didn’t drop out before the questions of interest, with no filtering).
Regarding the other things in the list you linked to from that LLM chat, though, we did do most of those things. For example, we included unobtrusive checks and multiple different quality measures, not just attention checks - I’d be interested in your thoughts on the checks outlined in our supplementary folder. And importantly, for our headline results, we did sensitivity analyses and shared the results (including confidence intervals) in our supplementary materials folder.
(Also, just to address the point about the N varying between questions - that’s because different numbers of participants completed some questions; slightly fewer completed the duration and hell questions because we had tested different wordings for both questions early in the study, before they were replaced with new versions that were used for the rest of the study.)
Would be happy to answer follow-up questions too. Thanks!
I agree that both possibilities are very risky. Interesting re belief in hell being a key factor, I wasn't thinking about that.
Even if a future ASI would be able to very efficiently manage todays economy in a fully centralised way, possibly the future economy will be so much more complicated that it will still make sense to have some distributed information processing in the market rather than have all optimisation centrally planned? Seems unclear to me one way or the other, and I assume we won't be able to know with high confidence in advance what economic model will be most efficient post-ASI. But maybe that just reflects my economic ignorance and others are justifiedly confident.
It seems like the whole AI x-risk community has latched onto "align AI with human values/intent" as the solution, with few people thinking even a few steps ahead to "what if we succeeded"? I have a post related to this if you're interested.
I think there will be distributed information processing, but each distributed node/agent will be a copy of the central AGI (or otherwise aligned to it or shares its values), because this is what's economically most efficient, minimizes waste from misaligned incentives and so on. So there won't be the kind of value pluralism that we see today.
There's probably a lot of other surprises that we can't foresee today. I'm mostly claiming that post-AGI economics and governance probably wont look very similar to today's.