(Copying my comment from Substack)

Thanks! This post was interesting and helped clarify my thoughts on some relevant issues. Still, I want to push in the other direction.

I think historical data poses its own sets of flaws and limitations -- and there are certain questions that it cannot easily answer. Therefore, I still think one should from philosophical/conceptual analysis since there's often no better approach to answering the questions. To be concrete, here are two main categories of questions that AI forecasting is looking to answer and where conceptual analysis must be used.

1. Timelines and capabilities forecasting
Here, I think the historical data about other technologies and their diffusion is pretty good to predict AI's diffusion in the economy.

But to answer questions like "will AI automate all human-level cognitive labor", I think it would be dubious to only look through the history of other transformative technology and conclude "no because electricity/TV/internet/phones did not". There is a much clearer mechanism for AI to automate these tasks than any of those past technologies, and therefore the historical data just isn't that persuasive to me.

2. What risks does TAI pose and how should we mitigate them
Here I think conceptual analysis is much more valuable. There are benefits of doing some economic modelling to see how to react to issues like job loss etc., especially for the near-term. But for questions like "how likely is human disempowerment from powerful AI", I don't see any good alternative to conceptual thinking. I really like Forethought's post about what to focus on here which encompasses non-alignment problems and basically focuses on high level strategic questions since those are easier to predict: 

Of course, there are lots of possible issues with conceptual thinking that you mention in the post. Where I'm at is just accepting that no approach is especially good and that we should keep some amount of epistemic humility in our arguments since it's very easy to be deeply confused in both directions.

Thanks for sharing! I feel the same way towards Singer's thought experiment. I personally find it much more motivating to use Joe Carlsmith's version of it.

Suppose that I am setting off to walk in the forest on a crisp fall afternoon — something I’ve been looking forward to all week. As I near the forest, I notice, far away, what looks like some sort of commotion down by the river, off of my walking route, though I can’t see very clearly. I consider going to see what’s going on, but the light is fading, so I decide to continue on my way.

I learn later that while I was walking, a man in his early forties drowned in that river. He was pinned under some sort of machinery. Five other people were there, including his wife and son, but they weren’t strong enough to lift the machinery by themselves. One extra person might have made the difference.

(Here I try to imagine vividly what it was like trying to save him — his wife desperate, weeping, pulling at him, his own eyes frantic, the fear and chaos, the eventual despair– and the pain of his absence afterwards; and the counterfactual world in which instead, another person arrived in time to help, and he lives.)

The intuition this pumps for me is: I wish I’d gone to the river. Importantly, though, at least for me, the case leaves the focus of attention on the drowned man himself, and the clear sense in which a beautiful walk would be worth trading, cancelling, disappearing, to grant him multiple decades of life, and his family multiple decades of time together. The question of whether my choice to continue walking was wrong, though not entirely absent, is less salient. That is, for me (at least as a thought experiment — who knows how I’d feel if something like this really happened), the case touches most centrally into a feeling of regret, rather than guilt. I wish I could go back, and create a world where I had one fewer beautiful walk, and he lived.