Thanks for posting this interesting write-up! I know you said you posted only as part of the Amnesty, but I've found the information you've compiled here useful to inform other ongoing projects. 

More tractable, necessary precondition 

Similarly to Owen's comment, I also think that AI and nuclear interact in important ways (various pathways to destabilisation that do not necessarily depend on AGI). It seems that many (most?) pathways from AI risk to extinction lead via other GCRs eg pandemic, nuclear war, great power war, global infrastructure failure, catastrophic food production failure, etc. So I'd suggest quite a bit more hedging with focus on these risks, rather than putting all resources into 'solving AI' in case that fails and we need to deal with these other risks. 

Thanks for posting this. I'll comment on the bit about New Zealand's food production in nuclear winter conditions. Although the paper cited concludes there is potential for production to feed NZ's population, this depends on there being sufficient liquid fuel to run agricultural equipment and NZ imports 100% of it's refined fuels. Trade in fuel would almost certainly collapse in a major nuclear war. Without diesel, or imported fertiliser and agrichemicals, yield would be much lower. Distribution would be difficult too. See this paper: https://onlinelibrary.wiley.com/doi/abs/10.1111/risa.14297 Ideally, places like NZ would establish the potential to produce fuel locally, eg biofuels, in case of this scenario. If restricted to use in agriculture and food transport, with optimised cropping, surprisingly little biofuel would be needed. This kind of contingency planning could avert famine, and any associated disease and potential conflict. I agree that the existential risk is very low. But it is probably slightly higher when considering these factors. 

Interesting juxtaposition: 

It promotes the idea of spending considerable state resources, i.e. tax payer money, for building massive computing clusters in the US, while at the same time denying the knowledge required to build AI models from non-American people and companies.

With the following here: 

I'm Leopold Aschenbrenner. I recently founded an investment firm focused on AGI.

As you say, the whole set of writings has a propaganda (marketing) tone to it, and a somewhat naive worldview. And the temporal coincidence of the essays and the investment startup are likely not accidental. I'm surprised it got the attention it did given, as you say, the sort of red-flag writing style of which we are taught to be skeptical. Any presentation of these essays should be placed alongside the kind of systematic skepticism of eg Gary Marcus et al. for readers to draw their own conclusions. 

This all seems extremely optimistic. I don't see the words 'environment', 'externalities', 'biodiversity', or 'pollution' mentioned at all, let alone 'geopolitics', 'fragmentation', 'onshoring', 'deglobalisation' or 'supply chains'. And nothing about energy demands and cost of extraction. Based on upbeat consultancies biased models that always conclude things are good for business? I'll be extremely surprised if this 'lower bound' scenario is even the upper. 

Hopefully everyone who thinks that AI is the most pressing issue takes the time to write (or collaborate and write) their best solution in 2000 words and submit to the UN's recent consultation call: https://dig.watch/updates/invitation-for-paper-submissions-on-worldwide-ai-governance A chance to put AI in the same global governance basket as biological and nuclear weapons. And potential high leverage from a relatively small task (Deadline 30 Sept). 

Difficult to interpret a lot of this as it seems to be a debate between potentially biased pacifists, and potentially biased military blogger. As with many disagreements the truth is likely in the middle somewhere (as Rodriguez noted). Need new independent studies on this that are divorced from the existing pedigrees. That said, much of the catastrophic risk from nuclear war may be in the more than likely catastrophic trade disruptions, which alone could lead to famines, given that nearly 2/3 of countries are net food importers, and almost no one makes their own liquid fuel to run their agricultural equipment. 

Thanks for this post. Reducing risks of great power war is important, but also consider reducing risks from great power war. In particular working on how non-combatant nations can ensure their societies survive the potentially catastrophic ensuing effects on trade/food/fuel etc. Disadvantages of this approach are that it does not prevent the massive global harms in the first place, advantages are that building resilience of eg relatively self-sufficient island refuges may also reduce existential risk from other causes (bio-threats, nuclear war/winter, catastrophic solar storm, etc). 

One approach is our ongoing project the Aotearoa New Zealand Catastrophe Resilience Project.

Also, 100,000 deaths sounds about right for the current conflict in Ukraine, given that recent excess mortality analysis puts Russian deaths at about 50,000. 

100% agree regarding catastrophe risk. This is where I think advocacy resources should be focused. Governments and people care about catastrophe as you say, even 1% would be an immense tragedy. And if we spell out how exactly (one or three or ten examples) of how AI development leads to a 1% catastrophe then this can be the impetus for serious institution-building, global cooperation, regulations, research funding, public discussion of AI risk. And packaged within all that activity can be resources for x-risk work. Focusing on x-risk alienates too many people, and focusing on risks like bias and injustice leaves too much tail risk out. There's so much middle ground here. The extreme near/long term division on this debate has really surprised me. As someone noted with climate, in 1990 we could care about present day particulate pollution killing many people, AND care about 1.5C scenarios, AND care about 6C scenarios, all at once, it's not mutually exclusive. (noted that the topic of the debate was 'extinction risk' so perhaps the topic wasn't ideal for actually getting agreement on action).