PhD Student in Statistics @UCBerkeley || AI Safety, Animal Welfare
I think the objection comes from the seeming asymmetry between over-attributing and under-attributing consciousness. It's fine to discuss our independent impressions about some topic, but when one's view is a minority position and the consequences of false beliefs are high, isn't there some obligation of epistemic humility?
Maybe the examples are ambiguous but they don't seem cherrypicked to me. Aren't these some of the topics Yudskowky is most known for discussing? It seems to me that the cherrypicking criticism would apply to opinions about, I don't know, monetary policy, not issues central to AI and cognitive science.
Hey Jack! In support of your view, I think you'd like some of Magnus Vinding's writings on the topic. Like you, he expresses some skepticism about focusing on narrower long-term interventions like AI safety research (vs. broader interventions like improved institutions).
Against your view, you could check out these two (i, ii) articles from CLR.
Feel free to message me if you'd like more resources. I'd love to chat further :)
Oh totally (and you probably know much more about this than me). I guess the key thing I'm challenging is the idea that there was something like a very fast transfer of power resulting just from upgraded computing power moving from chimp-ancestor brain -> human brain (a natural FOOM), which the discussion sometimes suggests. My understanding is that it's more like the new adaptations allowed for cumulative cultural change, which allowed for more power.
Psychology/anthropology:
The misleading human-chimp analogy: AI will stand in relation to us the same way we stand in relation to chimps. I think this analogy basically ignores how humans have actually developed knowledge and power--not by rapid individual brain changes, but by slow, cumulative cultural changes. In turn, the analogy may lead us to make incorrect predictions about AI scenarios.
Also want to shout out @Holly Elmore ⏸️ 🔸 and PauseAI's activism in getting people to call their senators. (You can commend this effort even if you disagree with an ultimate pause goal.) It could be worth following them for similar advocacy opportunities.