Next week for The 80,000 Hours Podcast I'll be interviewing Joe Carlsmith, Senior Research Analyst at Open Philanthropy.
Joe's did a BPhil in philosophy at Oxford University and is a prolific writer on topics both philosophical and practical (until recently his blog was called 'Hands and Cities' but it's all now collected on his personal site.
What should I ask him?
Some things Joe has written which we could talk about include:
- Is Power-Seeking AI an Existential Risk?
- Actually possible: thoughts on Utopia
- On infinite ethics —XIV. The death of a utilitarian dream
- Anthropics: Learning from the fact that you exist
- Against neutrality about creating happy lives
- Wholehearted choices and “morality as taxes”
- On clinging
- Can you control the past?
There seems to be two different conceptual models for AI risk.
The first is a model like in his report "Existential risk from power-seeking AI", in which he lays out a number of things, which, if they happen, will cause AI takeover.
The second is a model (which stems from Yudkowsky & Bosteom, and more recently in Michael Cohen's work https://www.lesswrong.com/posts/XtBJTFszs8oP3vXic/?commentId=yqm7fHaf2qmhCRiNA ) where we should expect takeover by malign AGI by default, unless certain things happen.
I personally think the second model is much more reasonable. Do you have any rebuttal?
See also Nate Soares arguing against Joe’s conjunctive breakdown of risk here, and me here.