I only decided to work on AI safety in the last year or so, largely because the current rate of progress convinced me there was a high enough chance that the next 1-5 years are pivotal. Thinking about what I'll be working on in 2031 assumes (1) we're alive and (2) humans have retained enough control over the world for my actions to matter. The answer entirely depends on why (1) and (2) are true.
I can think of a few scenarios:
We're alive and in control in 2031 because we rose to the challenge and created some sort of (likely international) regulatory regime and/or solved the alignment problem. What I do depends on how much maintenance the solution requires, i.e. if we appear to be in a stable equilibrium. My best guess is that we won't be and there will still be lots of work to be done to maintain the fragile peace, so to speak.
The tech fizzles; models plateau around the current frontier or slightly past it. AI would still be a huge deal in this world (the frontier is currently internal models that can solve major unsolved math problems), but catastrophe seems further off. This is the world where I'm most likely to pivot to working on something else. I'd keep an eye on things and possibly go back to working in AI safety if a new paradigm takes off and it looks like we're in crunch time again.
Funding for scaling temporarily dries up. The Anthropic IPO is a bust, Wall St gets very nervous about the AI buildout, and some hyperscalers go under. The remaining frontier labs have a harder time convincing investors that $1B+ training runs are worth it and the rate of AI progress slows dramatically. I'm probably still working on AI safety in this world because nothing has happened to show that the underlying technology doesn't pose an existential risk. This future still has a button that maybe ends the world that anyone can pay $100B to press. The fact that no one is currently willing to pay $100B (or whatever the actual number is) to press the button doesn't change the fact that the world ending button is still waiting around for someone to press it.
These predictions are all admittedly qualitative. I did have a quantitative prediction that played a big role in convincing me to pivot to AI safety - in mid-2024, I said that I would pivot if the METR long-horizon tasks trend line held through 2026. I updated a lot when that benchmark was saturated in the middle of this year. At some point I'll need to think about a comparable metric I can use to signal that it's time to pivot off AI safety. I don't feel a lot of urgency to do that though, given the way things are going.
The best I have right now is "2027 and 2028 feel significantly less crazy than 2026." That's still vibes based, but feels tangible enough. I wouldn't automatically stop working on AI safety if this happened, but it would prompt me to think a lot harder about if I should be.
My hope is that in practice it would be pretty rare for this rule to be invoked, although I think it does depend a bit on how the final rule is worded. The comment you linked arguing against human biodiversity is a tough edge case. On the one hand, I am a lot more concerned about people arguing for human biodiversity than against it, but one the other hand it doesn't seem like the end of the world if a prohibition on discussing the topic also took down comments like that.
IMO the forum rule I proposed is in my view the least important of the reforms/policies I suggested. The value comes more from signalling an opposition to racism/race science than it does from actually taking down a couple of comments here and there. Given how controversial the rule is, it would clearly be a pretty costly signal. That seems good by the lights of "making it more likely people of color engage with the EA movement."
This is a great question!
I only decided to work on AI safety in the last year or so, largely because the current rate of progress convinced me there was a high enough chance that the next 1-5 years are pivotal. Thinking about what I'll be working on in 2031 assumes (1) we're alive and (2) humans have retained enough control over the world for my actions to matter. The answer entirely depends on why (1) and (2) are true.
I can think of a few scenarios:
These predictions are all admittedly qualitative. I did have a quantitative prediction that played a big role in convincing me to pivot to AI safety - in mid-2024, I said that I would pivot if the METR long-horizon tasks trend line held through 2026. I updated a lot when that benchmark was saturated in the middle of this year. At some point I'll need to think about a comparable metric I can use to signal that it's time to pivot off AI safety. I don't feel a lot of urgency to do that though, given the way things are going.
The best I have right now is "2027 and 2028 feel significantly less crazy than 2026." That's still vibes based, but feels tangible enough. I wouldn't automatically stop working on AI safety if this happened, but it would prompt me to think a lot harder about if I should be.
My hope is that in practice it would be pretty rare for this rule to be invoked, although I think it does depend a bit on how the final rule is worded. The comment you linked arguing against human biodiversity is a tough edge case. On the one hand, I am a lot more concerned about people arguing for human biodiversity than against it, but one the other hand it doesn't seem like the end of the world if a prohibition on discussing the topic also took down comments like that.
IMO the forum rule I proposed is in my view the least important of the reforms/policies I suggested. The value comes more from signalling an opposition to racism/race science than it does from actually taking down a couple of comments here and there. Given how controversial the rule is, it would clearly be a pretty costly signal. That seems good by the lights of "making it more likely people of color engage with the EA movement."