In this post, @Jeffrey Ladish made the case that a nuclear war under current situations is unlikely to cause human extinction. I agree with his analysis that neither the destructive effect of our current nuclear arsenal nor the potential of a nuclear winter would lead to immediate human extinction (and in the long term, humanity would probably survive a civilisational collapse).
The Fourth World War
However, if a nuclear war is not destructive enough to lead to (immediate) civilisational collapse, it is quite possible that the players remain on the table, and hostilities may continue even during a nuclear winter.
Needless to say, continued global hostilities would significantly worsen our prospects of surviving a nuclear winter and rebuilding civilisation from its ruins, and I have not found any research/discussion here about how hostilities would end after an all-out nuclear exchange. I do believe there are reasons to be pessimistic, that it might be impossible to end hostilities after a nuclear exchange diplomatically.
Why
I believe it is quite possible that the major powers (China and the USA, and maybe Russia) will retain significant governmental and military capabilities after an all out nuclear exchange that exhausts the current global nuclear weapon stock, and it is then possible that massive industrialised warfare will continue during a nuclear winter.
Effect of nuclear weapon on military units
Military units deployed in the field are highly dispersed
When attacking with three regiments in a single echelon, a division zone of attack is normally 15 to 25 kilometers wide.
The 5psi damage radius of a 1 megaton warhead is ~7.5km, which just about covers the position of a division. Considering that a large number of personnel would be inside armoured vehicles or field fortifications, it is quite likely that a single 1 megaton warhead dropped on top of a division's operating zone would still leave a large proportion of its personnel alive and well.
Also considering that conventional military units are often deployed away from main nuclear targets (major cities, nuclear installations, etc), and it is quite possible that military units will be ordered to disperse when tensions are mounting (army units get deployed in the field and fleets leave port).
Hence, I believe it is likely that a large proportion of military personnel would survive an all-out nuclear exchange, and surviving governments could continue functioning using military personnel.
Destructive effect of current nuclear arsenal
Back in 1979, studies estimated that an attack on the US with ~800 warheads could kill around 40% of the US population and destroy around 60% of the US industrial capacity.
While world nuclear warhead stockpile is currently sitting at ~10000 warheads, a large number of them are in storage and not available for use without extensive preparation, hence I suspect that a large number of "stockpiled" nuclear warheads would not be available for an all out nuclear exchange, and a large number of them may be lost to counterforce strikes.
This leaves us with <2000 warheads available between Russia and China (potentially far less if Russia's nuclear maintenance is poor), and a somewhat larger number available for NATO. Hence the above "800 warheads" scenario is probably the worst possible case for the US.
China and Russia might fare worse in absolute terms, but it seems likely that at least China will retain a significant industrial capacity post-nuclear war. China has a large rural population that will remain mostly untouched by nukes, and some steel/machine tool manufacturing industry exist widely even in regional towns.
What happens next?
The surviving states might still be able to govern their population and maintain a wartime industry. Perhaps high tech facilities will be destroyed, but it is quite likely that they will be able to conscript millions of people and hand them rifles, and continue the war that got us to this place until one side is utterly destroyed.
Additional concerns are that a nuclear exchange removes a lot of moral constrains, it might become politically attractive to use other WMDs such as biological agents, and perhaps the public would be less averse (or even enthusiastically embrace) a genocidal war against a country that just dropped nuclear weapons on us.
Given that all these might happen during a nuclear winter and widespread famine, not properly ending a nuclear war might be a significant detriment to our survival.
What are all these soldiers going to eat during a nuclear winter though? Unless the food supply hasn't collapsed, in which case we are probably far enough from extinction that the continuation of a conventional group power war isn't going to get us there, or even close.
"Food supply collapse" isn't a simple binary switch, though.
It's possible that whatever food leftover will be distributed in the most militarily efficient way possible, and a large number of civilians will be left to starve so that the remenants of conventional military forces could continue their fight to the death.
Of course, I think this scenario will not lead to outright human extinction. But it does make the post-nuclear war situation a lot more difficult, despite civilisation nominally surviving the ordeal.
Well, in the food queue soldiers are likely to be the first.
Really nice point, Kinoshita!