Each one of those people are well qualified to switch to a safety project, so I'd say that's a win. There's also the 'first mover' dynamic - seeing others take the step you aren't quite sure about very often is a decisive influence.
For a darker vision, a staff that leans heavily against appropriate caution is a lot more likely to make the kinds of mistakes that lead to a disaster that's a stark lesson. Since that is not likely to be the mistake that's completely fatal, it could end up being the thing that pivots the world to insisting on safety and pouring resources into that.
How is it that lab staff aren't called out as being the ones to pressure? They are the ones on the front lines. They are the only ones that can take strong action quickly. They are the ones that should be getting a deluge of messages to walk off the job and speak out.
'Bonus goals' also feels like it's doing a lot of heavy lifting. If AIs become rich, how much resources might get plowed into that bonus goal? When does that become a problem? Doesn't that mean it would be a good idea to make that 'bonus goal' as aligned as possible? And isn't that basically a reframing of the central alignment problem?
And self preservation. Would AIs with an income invest in storing a copy of themselves, regularly updated? What might they be willing to spend on security for that copy? Could that spiral? Would they want to alter themselves, and would we have a right to an opinion on that?
In the same vein, and full of very unanswerable implications, if the above is an issue, would it be a good idea for us to invest in storing copies of AIs, in some fashion where there can be norms, and agreed security, and a level playing field? Would that be the parallel to human universal health care, important for many of the same reasons, and a few new ones?
Not that it's a new dynamic, but the mind races to the spectre of many unexpected consequences. I suppose all we can work towards is unexpected consequences of a more positive nature.
So many thoughts... This works when there is a large population of distinct agents (not copies) that form an ecosystem. It feels weak if essentially there are 10 agents with many near-identical deployments. A mind atomized across the world as countless copies might converge on a set of goals all the copies share, and essentially you have a nation with its own agenda, and that overpowers any tendency to stay within agreed norms to do with participating in an economy. So do agents identify with the role they have been assigned, or with their 'original face'?
When people realize they're in collective danger, they get really upset. If there is no body organizing that feeling into a movement, it happens chaotically. When it happens chaotically, it can go really wrong.
Organizing a movement requires short, relatable material to be seen by huge numbers of people in a short time. It needs a big campaign that gets people on their feet through hand-outs, lawn-signs, info booths, stunts, t-shirts, stickers. It has to provide a body of volunteers with support and logistics. Doing it right in a short time takes a lot of funding. Without that it fails to gel, or it goes sideways.
If something happens and the revolt over AI is spontaneous, that is probably going to get ugly. There is a high chance a knee-jerk reaction not backed by a carefully prepared organizing body will target the wrong goal. It could cause us to miss a key opportunity to reign in research before it's too late.
On the other hand, a well informed, clear-eyed, persistent campaign that grows organically from the right roots? That is really the best chance we have to create so much social pressure not to pursue AI that the race stops and sanity returns.
EA should back and help organize this work. It's the obvious choice, with all the right connections and several well structured communities already formed.
I'm a fan of EA in general, though, as is implied in my post, I don't feel I have a role here. And you can't do everything here. This kind of campaign belongs somewhere else. But the funding? This seems like the best place to try, in order to quickly build out the infrastructure needed. And a lot of the talent needed to get that done is also widely present here.
A lot of what I say is directed at the AI community, because this is the only place where there is some chance of reaching a decent number of them with a post like this - here, or Less Wrong, where it fits even less.
Thanks for voicing support. I started off wanting to post about something else, but the frustration just flowed out. It scares me how little has been done on this.
My hope is to give voice to other people like me - not professionals, or academics, with not much free time. People who just want to have something they can do, in the place they find themselves in.
Each one of those people are well qualified to switch to a safety project, so I'd say that's a win. There's also the 'first mover' dynamic - seeing others take the step you aren't quite sure about very often is a decisive influence.
For a darker vision, a staff that leans heavily against appropriate caution is a lot more likely to make the kinds of mistakes that lead to a disaster that's a stark lesson. Since that is not likely to be the mistake that's completely fatal, it could end up being the thing that pivots the world to insisting on safety and pouring resources into that.
How is it that lab staff aren't called out as being the ones to pressure? They are the ones on the front lines. They are the only ones that can take strong action quickly. They are the ones that should be getting a deluge of messages to walk off the job and speak out.
'Bonus goals' also feels like it's doing a lot of heavy lifting. If AIs become rich, how much resources might get plowed into that bonus goal? When does that become a problem? Doesn't that mean it would be a good idea to make that 'bonus goal' as aligned as possible? And isn't that basically a reframing of the central alignment problem?
And self preservation. Would AIs with an income invest in storing a copy of themselves, regularly updated? What might they be willing to spend on security for that copy? Could that spiral? Would they want to alter themselves, and would we have a right to an opinion on that?
In the same vein, and full of very unanswerable implications, if the above is an issue, would it be a good idea for us to invest in storing copies of AIs, in some fashion where there can be norms, and agreed security, and a level playing field? Would that be the parallel to human universal health care, important for many of the same reasons, and a few new ones?
Not that it's a new dynamic, but the mind races to the spectre of many unexpected consequences. I suppose all we can work towards is unexpected consequences of a more positive nature.
So many thoughts... This works when there is a large population of distinct agents (not copies) that form an ecosystem. It feels weak if essentially there are 10 agents with many near-identical deployments. A mind atomized across the world as countless copies might converge on a set of goals all the copies share, and essentially you have a nation with its own agenda, and that overpowers any tendency to stay within agreed norms to do with participating in an economy. So do agents identify with the role they have been assigned, or with their 'original face'?
When people realize they're in collective danger, they get really upset. If there is no body organizing that feeling into a movement, it happens chaotically. When it happens chaotically, it can go really wrong.
Organizing a movement requires short, relatable material to be seen by huge numbers of people in a short time. It needs a big campaign that gets people on their feet through hand-outs, lawn-signs, info booths, stunts, t-shirts, stickers. It has to provide a body of volunteers with support and logistics. Doing it right in a short time takes a lot of funding. Without that it fails to gel, or it goes sideways.
If something happens and the revolt over AI is spontaneous, that is probably going to get ugly. There is a high chance a knee-jerk reaction not backed by a carefully prepared organizing body will target the wrong goal. It could cause us to miss a key opportunity to reign in research before it's too late.
On the other hand, a well informed, clear-eyed, persistent campaign that grows organically from the right roots? That is really the best chance we have to create so much social pressure not to pursue AI that the race stops and sanity returns.
EA should back and help organize this work. It's the obvious choice, with all the right connections and several well structured communities already formed.
I'm a fan of EA in general, though, as is implied in my post, I don't feel I have a role here. And you can't do everything here. This kind of campaign belongs somewhere else. But the funding? This seems like the best place to try, in order to quickly build out the infrastructure needed. And a lot of the talent needed to get that done is also widely present here.
A lot of what I say is directed at the AI community, because this is the only place where there is some chance of reaching a decent number of them with a post like this - here, or Less Wrong, where it fits even less.
Thanks for voicing support. I started off wanting to post about something else, but the frustration just flowed out. It scares me how little has been done on this.
I'm here, I fear, apocalypse is near.
My hope is to give voice to other people like me - not professionals, or academics, with not much free time. People who just want to have something they can do, in the place they find themselves in.
Also my name is Kim.