I am late to this but anyway I'd like to point out that main and huge dowside of export controls is imho that they are poisoning the atmosphere for international cooperation on AI (also on other things), for reasons I tried to explain here: alesziegler's Quick takes — EA Forum
"But I think maybe the cruxiest bits are (a) I think export controls seem great in Plan B/C worlds, which seem much likelier than Plan A worlds, and (b) I think unilaterally easing export controls is unlikely to substantially affect the likelihood of Plan A happening (all else equal). It seems like you disagree with both, or at least with (b)?"
Yep, this is pretty close to my views. I do disagree with (b), since I am afraid that controls might poison the well for future Plan A negotiations. As for (a), I don’t get how controls help with Plan C, and I don’t think Plan B/C worlds are much more likely than Plan A, as of now. But I do agree with you that controls help with Plan B, so if you see it as the main hope, I understand why you are supporting them.
Thx for your perspective; I should be upfront that my confidence in my own case is far from ironclad. Anyway.
From the ryan_greenblatt’s article you’ve linked, I think safety plan A, based on achieving international agreement, should be tried before going for plan B, based on achieving and using a secure US lead, while in “plans” C and D it doesn’t matter whether the leading company is American or Chinese, so slowing down Chinese development is useless for those “plans” (they are in fact more like scenarios than real plans).
I do agree that the US political system is in an important sense better than the Chinese political system, but my prior is that if superintelligence is developed before, let’s say, 2050 (and that is very optimistic), it is likely to go badly completely regardless of which country it’ll come from.
I take your point about slack being potentially useful. Theoretically, I can imagine the following sequence of events: a) the US AI industry crushes Chinese competition, then b) the US government, feeling secure in the US lead, imposes sensible safety regulation on companies. If it were smart about it, it would at the same time propose an international regulatory framework that the rest of the world would be prepared to sign on to, as an alternative to unrestrained US domination. In effect, this would be tantamount to getting so much leverage over China that they would drop out of the race, and then hoping that the US government would use its advantage to push for safety, instead of using it in some other way.
However, imho this is a plan that should be pursued as a first option only in circumstances where you are really confident it’ll work, since the consequences of trying and failing are likely to be dire, as per my original post. And I am not confident it will work.
A much better order of operations would be to 1) try to negotiate with China to establish an international regulatory framework (plan A), with export control and other stuff being imposed as something that is explicitly linked to China not agreeing to that framework, in the same way sanctions on Russia are imposed explicitly because its aggression against Ukraine, and 2) only if they refuse, try to crush them (plan B).
When political will in the US to try for plan A is lacking, I think waiting until circumstances make that plan realistic while preparing the groundwork for it is a better strategy than going straight ahead for plan B.
Hi, thx for engaging, but I guess we’ll have to agree to disagree.
I do agree that chip controls would make a lot of sense as a source of leverage in negotiations over an AI pause in a world where China would be trying to build superintelligence and the US would be trying to force them to do a mutual pause. But we don’t live in that world. The Biden administration had no intention to agree to a cooperative pause when it imposed chip controls, and the Trump administration has somehow even less than no intention to pause; Trump seems to be going all-in on AI development.
You might object that controls are good because they are sort of increasing the supply of leverage which in the future could be used to do a pause, and that is true, BUT to do a deal, both leverage and trust are needed. And imho even without controls, the US has a lot of leverage over China just by the sheer power of its global hegemony.
So, leverage over China is already relatively abundant, so to speak, but Chinese trust is what is relatively scarce (as is, of course, American trust in Chinese intentions). So, I am skeptical that steps which build more leverage over China but decrease Chinese trust in American intentions get these countries closer to cooperation.
And I disagree that lifting controls doesn’t recoup Chinese trust. It does not recoup it fully, as if controls never happened, but I do think that it builds trust relative to a situation of continuing controls.
To use a somewhat strained example with which most are surely familiar, when Trump put up his Liberation Day tariffs, prices of various financial assets fell off a cliff, but when he partially reversed the tariffs, prices partially recouped. How come, if Trump has shown that he is untrustworthy and might change his mind any minute? Well, many investors are evidently betting he won’t do that, and so far they’ve been correct.
Imho it is a fairly general principle of human relationships that for maintaining trust: never doing a trust-damaging action > rescinding a trust-damaging action > continuing a trust-damaging action.
Also, Chinese-American relations are themselves a vivid example that trust can be rebuilt: in the 50s, they were at rock bottom due to various trust-destroying actions, notably in the Korean War, where the Chinese army attacked UN forces advancing toward the Yalu River and pushed them back from what is now unfortunately North Korea. But, skipping ahead, by 2000, after a lot of trust building, the US granted China Permanent Normal Trade Relations. Things went downhill after that, but that could change again.
I also disagree that chips used to train/run LLMs are in the same category as F-35s or nuclear material; imho they are more like steel or oil, commodities obviously important for military purposes but with extremely broad civilian applications, and actions like an oil embargo are considered to be hostile acts, usually only deployed against pariah states, unlike an “F-35 embargo.”
Finally, imho you misunderstood what point I was trying to make (ok, that’s on me, I should’ve been clearer) about pariah states – it is not that chip controls make China into a pariah state. They don’t. Rather, by deploying measures usually only used against pariah states, these controls blur differences between “normal” members of an international community and pariah states, thus devaluing the usefulness of not doing things that put the country on the naughty list, primarily for China but secondarily for everyone else.
Since a thing that puts countries on the naughty list is waging an aggressive war, I do think this development is bad.
Is the recent partial lifting of US chip export controls on China (see e.g. here: https://thezvi.substack.com/p/selling-h200s-to-china-is-unwise) good or bad for humanity? I’ve seen many takes from people whose judgment I respect arguing that it is very bad, but their arguments, imho, just don’t make sense. What am I missing?
For transparency, I am neither Chinese nor American, nor am I a paid agent of them. I am not at all confident in this take, but imho someone should make it.
I see two possible scenarios: A) you are not sure how close humanity is to developing superintelligence in the Yudkowskian sense. This is what I believe, and what many smart opponents of the Trump administration’s move to ease chip controls believe. Or B) you are pretty sure that humanity is not going to develop superintelligence any time soon, let’s say in the next century. I admit that the case against the lifting of chip controls is stronger under B), though I am ultimately inclined to reject it in both scenarios.
Why is easing of chip controls, imho, a good idea if the timeline to superintelligence might be short?
If superintelligence is around the corner, here is what should be done: an immediate international pause of AI development until we figure out how to proceed.
Competitive pressures and resulting prisoner’s dilemmas have been identified as the factor that might push us toward NOT pausing even when it would be widely recognized that the likely outcome of continuing is dire.
There are various relevant forms of competition, but plausibly the most important is that between the US and China. In order to reduce competitive dynamics and thus prepare the ground for a cooperative pause, it is important to build trust between the parties and beware of steps that are hostile, especially in domains touching AI.
Controls make sense only if you are very confident that superintelligence developed in the US, or perhaps in liberal democracy more generally, is going to turn out well for humanity, and you are not so confident about superintelligence developed in China.
I do think that the US political system is, in an important sense, whose exposition would require more than an oversized quick take, better than the Chinese political system, but despite this, I am not at all confident that superintelligence developed in a liberal democracy would be good for humanity; frankly, I find such beliefs wholly unfounded.
Why is easing of chip controls a good idea if the timeline to superintelligence is long?
Disclaimer: I know following argument is perhaps too abstract and I am even less confident in it than in previous section, nevertheless.
It is a feature of the current international order that there is a strong taboo on military action of one country against another outside narrow and contested exceptions. I think this taboo is very good and very important. And yes, I know that this taboo is sometimes broken; that does not invalidate previous two sentences.
Another feature of the current international order is that some countries are what is sometimes called pariah states; I am vaguely aware that the word “pariah” has unpleasant connotations in the Indian cultural context, but I am going to use the term anyway, since I don’t have a suitable replacement, sorry. Anyway, good examples are, e.g., Russia or Afghanistan.
Being a pariah state means essentially that some hostile actions which are considered beyond the pale against a non-pariah state are considered ok against them. Notably, the taboo against direct military action is so strong that it mostly applies even to attacks against pariah states, but certain types of heavy restrictions on trade and cross-border investment are what essentially define pariah status. Another sometimes used action against a pariah state is external support, including by providing arms, to an internal opposition, but that is facultative. Many pariah states don’t have internal opposition capable of or interested in armed resistance.
In trade, pariah status tended to be a package deal. If a country is a pariah, it usually had heavy restrictions placed on its trade and investment; if it is a normal member of the international community, it could expect that it would not have certain kinds of restrictions placed on it.
Of course, the previous paragraph has to be immediately qualified in that there are many, many types of restrictions on international trade considered completely acceptable even among normal members of the international community, such as tariffs, bans on investment in critical infrastructure, or bans on selling some military or military-adjacent technologies. But there are other kinds of restraints that were usually deployed only against pariahs.
And pariah status is not a category enshrined in international law; really, making and unmaking a pariah is based on a decision of the government of the United States, since the US is the current hegemon of the international order. E.g., Cuba is, imho, a pariah state mainly because of random historical contingency plus quirks of the US electoral system. BUT breaking a taboo against militarily attacking another country is the sort of thing that tends to get you on the pariah list.
This all means that avoiding a pariah designation is a strong incentive for countries NOT to engage in military aggression.
Now, I do think that the chip controls imposed on China by the Biden administration are breaking this logic. These controls are restrictions on selling a critical component of consumer technology which is related to military stuff only in the sense that, like, steel or oil is related to military stuff. And restrictions on selling those is something which has not usually been done against non-pariah states; the Biden administration, however, had no plan to impose a whole package of pariah status on China. They made an awkward exception in an important area, which devalues the status of being non-pariah member of international community.
Thus, the imposition of chip controls reduces the incentive for China, and indirectly for everyone else, to avoid pariah status. Which means that it reduces their incentive to avoid military aggression. I think that that is very bad and it outweighs benefits of those controls even in no-near-term-superintelligence-threat world.
I am late to this but anyway I'd like to point out that main and huge dowside of export controls is imho that they are poisoning the atmosphere for international cooperation on AI (also on other things), for reasons I tried to explain here: alesziegler's Quick takes — EA Forum
"But I think maybe the cruxiest bits are (a) I think export controls seem great in Plan B/C worlds, which seem much likelier than Plan A worlds, and (b) I think unilaterally easing export controls is unlikely to substantially affect the likelihood of Plan A happening (all else equal). It seems like you disagree with both, or at least with (b)?"
Yep, this is pretty close to my views. I do disagree with (b), since I am afraid that controls might poison the well for future Plan A negotiations. As for (a), I don’t get how controls help with Plan C, and I don’t think Plan B/C worlds are much more likely than Plan A, as of now. But I do agree with you that controls help with Plan B, so if you see it as the main hope, I understand why you are supporting them.
(btw. how does one do a proper blockqoute here?)
Thx for your perspective; I should be upfront that my confidence in my own case is far from ironclad. Anyway.
From the ryan_greenblatt’s article you’ve linked, I think safety plan A, based on achieving international agreement, should be tried before going for plan B, based on achieving and using a secure US lead, while in “plans” C and D it doesn’t matter whether the leading company is American or Chinese, so slowing down Chinese development is useless for those “plans” (they are in fact more like scenarios than real plans).
I do agree that the US political system is in an important sense better than the Chinese political system, but my prior is that if superintelligence is developed before, let’s say, 2050 (and that is very optimistic), it is likely to go badly completely regardless of which country it’ll come from.
I take your point about slack being potentially useful. Theoretically, I can imagine the following sequence of events: a) the US AI industry crushes Chinese competition, then b) the US government, feeling secure in the US lead, imposes sensible safety regulation on companies. If it were smart about it, it would at the same time propose an international regulatory framework that the rest of the world would be prepared to sign on to, as an alternative to unrestrained US domination. In effect, this would be tantamount to getting so much leverage over China that they would drop out of the race, and then hoping that the US government would use its advantage to push for safety, instead of using it in some other way.
However, imho this is a plan that should be pursued as a first option only in circumstances where you are really confident it’ll work, since the consequences of trying and failing are likely to be dire, as per my original post. And I am not confident it will work.
A much better order of operations would be to 1) try to negotiate with China to establish an international regulatory framework (plan A), with export control and other stuff being imposed as something that is explicitly linked to China not agreeing to that framework, in the same way sanctions on Russia are imposed explicitly because its aggression against Ukraine, and 2) only if they refuse, try to crush them (plan B).
When political will in the US to try for plan A is lacking, I think waiting until circumstances make that plan realistic while preparing the groundwork for it is a better strategy than going straight ahead for plan B.
Hi, thx for engaging, but I guess we’ll have to agree to disagree.
I do agree that chip controls would make a lot of sense as a source of leverage in negotiations over an AI pause in a world where China would be trying to build superintelligence and the US would be trying to force them to do a mutual pause. But we don’t live in that world.
The Biden administration had no intention to agree to a cooperative pause when it imposed chip controls, and the Trump administration has somehow even less than no intention to pause; Trump seems to be going all-in on AI development.
You might object that controls are good because they are sort of increasing the supply of leverage which in the future could be used to do a pause, and that is true, BUT to do a deal, both leverage and trust are needed. And imho even without controls, the US has a lot of leverage over China just by the sheer power of its global hegemony.
So, leverage over China is already relatively abundant, so to speak, but Chinese trust is what is relatively scarce (as is, of course, American trust in Chinese intentions). So, I am skeptical that steps which build more leverage over China but decrease Chinese trust in American intentions get these countries closer to cooperation.
And I disagree that lifting controls doesn’t recoup Chinese trust. It does not recoup it fully, as if controls never happened, but I do think that it builds trust relative to a situation of continuing controls.
To use a somewhat strained example with which most are surely familiar, when Trump put up his Liberation Day tariffs, prices of various financial assets fell off a cliff, but when he partially reversed the tariffs, prices partially recouped. How come, if Trump has shown that he is untrustworthy and might change his mind any minute? Well, many investors are evidently betting he won’t do that, and so far they’ve been correct.
Imho it is a fairly general principle of human relationships that for maintaining trust: never doing a trust-damaging action > rescinding a trust-damaging action > continuing a trust-damaging action.
Also, Chinese-American relations are themselves a vivid example that trust can be rebuilt: in the 50s, they were at rock bottom due to various trust-destroying actions, notably in the Korean War, where the Chinese army attacked UN forces advancing toward the Yalu River and pushed them back from what is now unfortunately North Korea. But, skipping ahead, by 2000, after a lot of trust building, the US granted China Permanent Normal Trade Relations. Things went downhill after that, but that could change again.
I also disagree that chips used to train/run LLMs are in the same category as F-35s or nuclear material; imho they are more like steel or oil, commodities obviously important for military purposes but with extremely broad civilian applications, and actions like an oil embargo are considered to be hostile acts, usually only deployed against pariah states, unlike an “F-35 embargo.”
Finally, imho you misunderstood what point I was trying to make (ok, that’s on me, I should’ve been clearer) about pariah states – it is not that chip controls make China into a pariah state. They don’t. Rather, by deploying measures usually only used against pariah states, these controls blur differences between “normal” members of an international community and pariah states, thus devaluing the usefulness of not doing things that put the country on the naughty list, primarily for China but secondarily for everyone else.
Since a thing that puts countries on the naughty list is waging an aggressive war, I do think this development is bad.
Is the recent partial lifting of US chip export controls on China (see e.g. here: https://thezvi.substack.com/p/selling-h200s-to-china-is-unwise) good or bad for humanity? I’ve seen many takes from people whose judgment I respect arguing that it is very bad, but their arguments, imho, just don’t make sense. What am I missing?
For transparency, I am neither Chinese nor American, nor am I a paid agent of them. I am not at all confident in this take, but imho someone should make it.
I see two possible scenarios: A) you are not sure how close humanity is to developing superintelligence in the Yudkowskian sense. This is what I believe, and what many smart opponents of the Trump administration’s move to ease chip controls believe. Or B) you are pretty sure that humanity is not going to develop superintelligence any time soon, let’s say in the next century. I admit that the case against the lifting of chip controls is stronger under B), though I am ultimately inclined to reject it in both scenarios.
Why is easing of chip controls, imho, a good idea if the timeline to superintelligence might be short?
If superintelligence is around the corner, here is what should be done: an immediate international pause of AI development until we figure out how to proceed.
Competitive pressures and resulting prisoner’s dilemmas have been identified as the factor that might push us toward NOT pausing even when it would be widely recognized that the likely outcome of continuing is dire.
There are various relevant forms of competition, but plausibly the most important is that between the US and China. In order to reduce competitive dynamics and thus prepare the ground for a cooperative pause, it is important to build trust between the parties and beware of steps that are hostile, especially in domains touching AI.
Controls make sense only if you are very confident that superintelligence developed in the US, or perhaps in liberal democracy more generally, is going to turn out well for humanity, and you are not so confident about superintelligence developed in China.
I do think that the US political system is, in an important sense, whose exposition would require more than an oversized quick take, better than the Chinese political system, but despite this, I am not at all confident that superintelligence developed in a liberal democracy would be good for humanity; frankly, I find such beliefs wholly unfounded.
Why is easing of chip controls a good idea if the timeline to superintelligence is long?
Disclaimer: I know following argument is perhaps too abstract and I am even less confident in it than in previous section, nevertheless.
It is a feature of the current international order that there is a strong taboo on military action of one country against another outside narrow and contested exceptions. I think this taboo is very good and very important. And yes, I know that this taboo is sometimes broken; that does not invalidate previous two sentences.
Another feature of the current international order is that some countries are what is sometimes called pariah states; I am vaguely aware that the word “pariah” has unpleasant connotations in the Indian cultural context, but I am going to use the term anyway, since I don’t have a suitable replacement, sorry. Anyway, good examples are, e.g., Russia or Afghanistan.
Being a pariah state means essentially that some hostile actions which are considered beyond the pale against a non-pariah state are considered ok against them. Notably, the taboo against direct military action is so strong that it mostly applies even to attacks against pariah states, but certain types of heavy restrictions on trade and cross-border investment are what essentially define pariah status. Another sometimes used action against a pariah state is external support, including by providing arms, to an internal opposition, but that is facultative. Many pariah states don’t have internal opposition capable of or interested in armed resistance.
In trade, pariah status tended to be a package deal. If a country is a pariah, it usually had heavy restrictions placed on its trade and investment; if it is a normal member of the international community, it could expect that it would not have certain kinds of restrictions placed on it.
Of course, the previous paragraph has to be immediately qualified in that there are many, many types of restrictions on international trade considered completely acceptable even among normal members of the international community, such as tariffs, bans on investment in critical infrastructure, or bans on selling some military or military-adjacent technologies. But there are other kinds of restraints that were usually deployed only against pariahs.
And pariah status is not a category enshrined in international law; really, making and unmaking a pariah is based on a decision of the government of the United States, since the US is the current hegemon of the international order. E.g., Cuba is, imho, a pariah state mainly because of random historical contingency plus quirks of the US electoral system. BUT breaking a taboo against militarily attacking another country is the sort of thing that tends to get you on the pariah list.
This all means that avoiding a pariah designation is a strong incentive for countries NOT to engage in military aggression.
Now, I do think that the chip controls imposed on China by the Biden administration are breaking this logic. These controls are restrictions on selling a critical component of consumer technology which is related to military stuff only in the sense that, like, steel or oil is related to military stuff. And restrictions on selling those is something which has not usually been done against non-pariah states; the Biden administration, however, had no plan to impose a whole package of pariah status on China. They made an awkward exception in an important area, which devalues the status of being non-pariah member of international community.
Thus, the imposition of chip controls reduces the incentive for China, and indirectly for everyone else, to avoid pariah status. Which means that it reduces their incentive to avoid military aggression. I think that that is very bad and it outweighs benefits of those controls even in no-near-term-superintelligence-threat world.