You might also find use in David Manheim's Modeling Transformative AI Risk (MTAIR) sequence. And you might also want to ask the LessWrong.com forum this same question.
I've spammed you with a few sources. But even though they do exist, they are fairly scattered and my sense is still that your impression is right that there aren't many such models.
David Manheim would know better, so I recommend you check with him.
Se also: <https://www.openphilanthropy.org/research/semi-informative-priors-over-ai-timelines/>
Se also: <https://www.openphilanthropy.org/research/semi-informative-priors-over-ai-timelines/>
You might also want to check AI impacts' work on AI timelines https://aiimpacts.org/category/ai-timelines/, particularly their surveys: https://aiimpacts.org/category/ai-timelines/predictions-of-human-level-ai-timelines/ai-timeline-surveys/
Nicole Noemi gathers some forecasts about AI risk (a) from Metaculus, Deepmind co-founders, Eliezer Yudkowsky, Paul Christiano, and Aleja Cotra's report on AI timelines.
h/t Nuño
I was hoping to write something for the Future Fund contest and - being entirely a one-trick pony - was going to look at uncertainty analysis in AI Catastrophe predictions.
I've done a review of the forums and my conclusion is that predictions around AI Catastrophe are very heavily focussed on when AI will be invented and the overall top-level probability that AI will be a catastrophe if it is invented. Other than that predictions about AI Risk are quite sparse. For example, few people seem to have offered a numerical prediction about whether they think the AI Alignment Problem is solvable in principle, few people have offered a numerical prediction about the length of time we could contain a misaligned AI and so on. The only end-to-end model of AI Risk with numerical prediction I have found is Carlsmith (2021): https://arxiv.org/abs/2206.13353.
In my review I came across the 'Database of Existential Risk Estimates' - link here: https://forum.effectivealtruism.org/posts/JQQAQrunyGGhzE23a/database-of-existential-risk-estimates. This seems to contain many estimates of exactly what I am looking for - predictions of specific events which will occur on the path to an AI catastrophe, rather than the overall risk of catastrophe itself.
Thanks so much!
This spreadsheet (found here) has estimates on the propositions in Carlsmith by (some of?) the reviewers of that paper.
This is absolutely incredible - can't believe I missed it! Thank you so much
I'm excited to see what you come up with!