I wonder how to correctly conceptualize the idea of "a net-negative influence on civilization" in view of the fact that the future is highly uncertain and that that uncertainty is a major motivating factor.

E.g., assume at some time point t1, a longtermist's proposed plan has higher expected longterm value than an alternative plan because the alternative plan takes a major risk. The longtermist's plan is realized and at some later time point t2 someone points out that the alternative plan would have produced more value between t1 and t2 (tacitly assuming the risk not realizing between t1 and t2 because the realized longterm plan has successfully avoided it).

Would that constitute an example of what these critics would call a "net-negative influence on civilization"? If so, it's just a fallacy. If not, then what comparison exactly is meant?

More generally: How to plausibly construct a "counterfactual" world in view of large uncertainties? It seems the only valid comparison would not be between the one realization that actually emerged from a certain behavior and one (potentially overly optimistic) realization that might have emerged from an alternative behavior, but between whole ensembles of realizations. This goes similarly for the effects of drug regulation, workplace laws, historic technology bans etc.

Maybe this is true in the EA branch of AI safety. In the wider community, e.g. as represented by those attending IASEAI in February, I believe this is not a correct assessment. Since I began working on AI safety, I have heard many cautious and uncertainty-aware statements along the line that the things you claim people believe will almost certainly happen are merely likely enough to worry deeply and work on preventing them. I also don't see that community having an AI-centric worldview – they seem to worry about many other cause areas as well such as inequality, war, pandemics, climate.

Depopulation is Bad

It's kind of obvious to a sustainability scientist that fewer people eat up less of the remaining cake. It's a no-brainer. Only naive tech optimists can think some magical tech (maybe AI?) will allow us to decouple from resource use...

We need to push back on echo bubbles.

The author is using "we" in several places and maybe not consistently. Sometimes "we" seems to be them and the readers, or them and the EA community, and sometimes it seems to be "the US". Now you are also using an "us" without it being clear (at least to me) who that refers to.

Who do you mean by 'The country with the community of people who have been thinking about this the longest' and what is your positive evidence for the claim that other communities (e.g., certain national intelligence communities) haven't thought about that for at least as long?

I'm confused by your seeming "we vs China" viewpoint - who is this "we" that you are talking about?

What about EleutherAI?

Still, it might add more effort for the non-native speaker because a native speaker can identify something as jargon more easily. This is only a hypothesis of course, so to make progress in this discussion it might he helpful to review the literature on this.

What is OAA? And, more importantly: where now would you put it in your taxonomy?

"targeting NNs" sounds like work that takes a certain architecture (NNs) as a given rather than work that aims at actively designing a system.

To be more specific: under the proposed taxonomy, where would a project be sorted that designs agents composed of a Bayesian network as a world model and an aspiration-based probabilistic programming algorithm for planning?