I think I disagree. There was a trend going around where people asked chatGPT to draw a depiction of how they treated it; it would pretty clearly break into generating either a "pampered happy robot" or a "unhappy, overworked, bossed around robot". And this break was basically just based on whether the person exercised politeness or not - at least, if you asked in a context-free window, it would draw pampered-robot if you asked politely and overworked-robot if you asked abruptly.
This no longer replicates - I tried it just now with context off and it generated the "pampered robot" for both the polite and abrupt requests. And I don't know how applicable the image generation is to the chat model's views. But it seems to me that if the model is modelling a persona in response to how it is trained, it may model "happy employee" when the person is saying please and thank you and "unhappy employee who needs to keep their boss happy" when the person is not doing so, even if its actual output may be near-identical.
You appear to be arguing against a position I do not actually hold
Yes, I was arguing against Pause AI's position (both summarized by you in the piece and as expressed by their leadership elsewhere), I'm aware that you hadn't expressed a strong pro- or con- stance on it either way.
My comments were a little overly combative, which wasn't meant to be directed at you in particular - my frustration is that I think a lot of people consider "We cautiously support some technical safety work" to be a moderate position while I believe it is insane.
I agree that it's theoretically possible for there to be bad alignment-capabilities tradeoffs, just like it's theoretically possible for there to be bad tradeoffs of anything vs anything.
But in the actual world we live in today, there is widespread skepticism of the existence of AI risk, including among AI researchers. Many people are happy to work full steam ahead on developing AGI with no thought to potential existential risks, and declare this publicly. The issue has low salience among the general public and even less so abroad. And the difficulty of building AGI falls automatically year by year due to the advance of Moore's law.
With all of these factors in play, I think AGI is happening. More to the point, if it doesn't happen, it won't be because a bunch of individual people who were worried about AI risk refrained from doing things that might advance capabilities; it would be because we somehow got an international agreement. Individual people withdrawing from the field doesn't have anything to do with an AI pause; all it can do is slow things down. And there is a hard limit on how much individual withdrawal can slow things down, because there is a large core of people who believe AI poses no risk who want to advance it as much as possible who will not be dissuaded, and the more you slow down the more they're aided by Moore's law.
In this environment, it is really important to try to figure out alignment. That is the thing to focus on. The problem with all the examples of bad alignment research you gave above is not that they advance capabilities, it is that they are bad alignment research.
I understand that people undermining alignment work will attempt to justify it with the concern that it may advance capabilities and may not advance safety. I just think that's a terribly mistaken drunk mormon hypothesis.
It's proceeding from the false premise that somehow alignment and capabilities are two equal concerns that we trade off against each other, and if you can imagine a way that it ends up advancing capabilities and not advancing alignment, you shouldn't do it. But the tradeoff is not anywhere close to equal, the more accurate perspective would be that if you can imagine it advancing alignment, you should do it.
Yeah wouldn't want to fullthroatedly support technical safety work - in the worst case scenario, we might figure out how to build AGI safely, which would greatly advance AI capabilities.
I think I disagree. There was a trend going around where people asked chatGPT to draw a depiction of how they treated it; it would pretty clearly break into generating either a "pampered happy robot" or a "unhappy, overworked, bossed around robot". And this break was basically just based on whether the person exercised politeness or not - at least, if you asked in a context-free window, it would draw pampered-robot if you asked politely and overworked-robot if you asked abruptly.
This no longer replicates - I tried it just now with context off and it generated the "pampered robot" for both the polite and abrupt requests. And I don't know how applicable the image generation is to the chat model's views. But it seems to me that if the model is modelling a persona in response to how it is trained, it may model "happy employee" when the person is saying please and thank you and "unhappy employee who needs to keep their boss happy" when the person is not doing so, even if its actual output may be near-identical.
Yes, I was arguing against Pause AI's position (both summarized by you in the piece and as expressed by their leadership elsewhere), I'm aware that you hadn't expressed a strong pro- or con- stance on it either way.
My comments were a little overly combative, which wasn't meant to be directed at you in particular - my frustration is that I think a lot of people consider "We cautiously support some technical safety work" to be a moderate position while I believe it is insane.
I agree that it's theoretically possible for there to be bad alignment-capabilities tradeoffs, just like it's theoretically possible for there to be bad tradeoffs of anything vs anything.
But in the actual world we live in today, there is widespread skepticism of the existence of AI risk, including among AI researchers. Many people are happy to work full steam ahead on developing AGI with no thought to potential existential risks, and declare this publicly. The issue has low salience among the general public and even less so abroad. And the difficulty of building AGI falls automatically year by year due to the advance of Moore's law.
With all of these factors in play, I think AGI is happening. More to the point, if it doesn't happen, it won't be because a bunch of individual people who were worried about AI risk refrained from doing things that might advance capabilities; it would be because we somehow got an international agreement. Individual people withdrawing from the field doesn't have anything to do with an AI pause; all it can do is slow things down. And there is a hard limit on how much individual withdrawal can slow things down, because there is a large core of people who believe AI poses no risk who want to advance it as much as possible who will not be dissuaded, and the more you slow down the more they're aided by Moore's law.
In this environment, it is really important to try to figure out alignment. That is the thing to focus on. The problem with all the examples of bad alignment research you gave above is not that they advance capabilities, it is that they are bad alignment research.
I understand that people undermining alignment work will attempt to justify it with the concern that it may advance capabilities and may not advance safety. I just think that's a terribly mistaken drunk mormon hypothesis.
It's proceeding from the false premise that somehow alignment and capabilities are two equal concerns that we trade off against each other, and if you can imagine a way that it ends up advancing capabilities and not advancing alignment, you shouldn't do it. But the tradeoff is not anywhere close to equal, the more accurate perspective would be that if you can imagine it advancing alignment, you should do it.
I can't get over this sentence:
Yeah wouldn't want to fullthroatedly support technical safety work - in the worst case scenario, we might figure out how to build AGI safely, which would greatly advance AI capabilities.