Comments
78
Cross-posted from my Substack.
Epistemic status: I’ve puzzled over this for a long time but I’m not sure I came to the right answers. I'm sharing this partly because it might help others work on resource allocation, but mostly because some smarter reader might tell me how to do it better.
TL;DR:
* I he...
What if the dot (representing where we are) was further to the left? In that case, it's not so clear that we want to speed through and incur the increased risk.
The idea is that by speeding through you increase risk initially, but the total risk is lower - i.e. smaller area under the grey curve here:
I think this probably breaks down if the peak is high enough though (here I'm thinking of AGI x-risk). Aschenbrenner gives the example of:
And argues that
I'm not sure if this applies if there is some possibility of "pulling the curve sideways" to flatten it - i.e. increase the fraction of resources spent on safety whilst keeping consumption (or growth) constant. This seems to be what those concerned with x-risk are doing for the most part (rather than trying to slow down growth).
That graph represents the comparison of increased growth (gray line) to some baseline (black).
Instead, I'm talking about the case where the dot is at the far left side of the graph: we're currently at a low level of risk, and continuing at the baseline growth rate means 'climbing the risk mountain' before we get to the other side with lower risk. If the peak is very high (relative to the dot), then it's not clear that continued (or accelerated) growth is optimal; stagnation might be better.