Comments
63
AI Use Note: Main body text entirely human written. Claude (Opus 4.8) helped develop models of animal life histories in the appendix.
Cross-posted from Good Structures.
Executive Summary
* Animal advocates sometimes make claims like “there are X of this animal...
What if the dot (representing where we are) was further to the left? In that case, it's not so clear that we want to speed through and incur the increased risk.
The idea is that by speeding through you increase risk initially, but the total risk is lower - i.e. smaller area under the grey curve here:
I think this probably breaks down if the peak is high enough though (here I'm thinking of AGI x-risk). Aschenbrenner gives the example of:
And argues that
I'm not sure if this applies if there is some possibility of "pulling the curve sideways" to flatten it - i.e. increase the fraction of resources spent on safety whilst keeping consumption (or growth) constant. This seems to be what those concerned with x-risk are doing for the most part (rather than trying to slow down growth).
That graph represents the comparison of increased growth (gray line) to some baseline (black).
Instead, I'm talking about the case where the dot is at the far left side of the graph: we're currently at a low level of risk, and continuing at the baseline growth rate means 'climbing the risk mountain' before we get to the other side with lower risk. If the peak is very high (relative to the dot), then it's not clear that continued (or accelerated) growth is optimal; stagnation might be better.