Learning to multiply the work that you do by automating it with AI when it becomes good enough to do so it's a lot more valuable than multiplying the hours that you work per week. It's not hard to see how you could make the first one 10x your output at some point, while the second one could be 2x at most.
Not to mention the benefits for your health that you could get from having more leisure time or, if you would prefer it, the additional impact/ career capital you could get from using that time to take on more responsibilities.
Seriously, the fact that people bring present benefits is just mind-blowing to me. People are rationalizing something away or they do not actually put as much weight on x-risks/ the far future as I thought.
I’m pretty happy that we didn’t pause back in 2022 or 2023 or 2024 or 2025, since the benefits of that AI development were genuinely net good for humanity and such development gave us a lot of experience with frontier AI systems which may help us better understand how to align them in the future. However, I imagine we are now finally getting close in time to when we would need to pause and we’re going to start incurring too much risk in exchange for learning.
I genuinely cannot believe there are EAs saying this kind of stuff. Don't you think the later we pause, the closer we are to the precipice and that if someone continues to develop capabilities, they would be putting everyone at risk? Don't you think there could be algorithmic progress any moment that could make ASI attainable now???
I mean, when you consider the long-term future, we are talking about putting at risk potentially 10^50 lives!!!!!!!!
How can you say the present benefits for millions of people (that are mostly productivity and not necessarily mean better lives) can justify this?
"They are giving us alignment experience"? How wouldn't it be experience that we could eventually get once we unpause? Because compute overhang? It's not like we would immediately have a trillion dollars worth of data centers just after the pause, like we are doing now. Not to mention most AI researchers seem to think present alignment stuff and superalignment are qualitatively different and basically no progress has been made at all.
Is everyone just gone extremely shortermist or what am I missing???
I have trouble understanding the different levels of fidelity argument. It seems like a belief that would have obvious evidence in the real world, eg some humans having more neurons, more complex neurons, different readings of brain activity, and less NPC behaviour to such a degree that those types of brains/ people would be conscious while the other would not! All while having people in the other extreme of the threshold arguing that subatomic particles may be conscious.
I mean, if you care a lot about animals, knowing that some humans may be more complex than others could make you believe they are *more* conscious. Why would it move your threshold for binary consciousness?
I also haven't heard of scientifically minded people making those observations themselves (in neural stuff/NPC behaviour) and then trying to find an explanation for them. But it could be because some of these arguments imply that some people may be qualitatively better/worse than others and saying that is bad for multiple reasons.
And that could be the same reason why I don't see people arguing those differences between people just based on simulation theory, nor being interested in looking for evidence in favor of or against it (or at least sharing positive proof of it in public).
This reads to me like you interpreted me as arguing against pausing. To be clear, I am strongly in favor of pausing. I'm arguing against the mental model I had a few months ago where if we pause, then we're in the clear.
There should be more ongoing debate about whether it's good or not to call oneself an EA; I haven't seen many posts myself. We should try to think through all the pros and cons.
It probably depends on who you are and the career you want to have, but for some reason it feels like most people think it's bad in most cases.
Even if you disagree with some subgroups or like to have a small identity, it could be bad for the world that most EA-adjacents don't call themselves EAs.
Over time, it could make some of them less effective or altruistic. It could also allow crazy people to give bad first impressions and misrepresent the movement. SBF, of course, is the prime example of this; maybe it would be better if people were autistic about EA like he was. In this state, we may be a ticking bomb just waiting for more crazy EA-adjacents to join and worsen the brand even more. And it's just a matter of cases like those until it's left in an irrecoverable state.
Another reason may be that identifying with a label has to do with feelings that we cannot control and we should value honesty. But then, can we fix that from within the movement if we realize it would be better for people to identify as an EA?
It could be because it sounds cringe too, but maybe it's worth enduring it.
Idk, these are just some pros and cons I just thought about.
I think that the fact that "smart" is being applied to a "digital species" changes things because "species" implies agency & goal-directedness, and "digital" implies it will be able to access useful information and become knowledgeable.
But I can understand that maybe other species don't seem goal-directed, so maybe it feels less threatening.
slowing down a car by a few milliseconds probably radically changes the genetic make-up of posterity millions of years into the future.
What? I never read WWOTF because I thought it was like an introduction to longtermism and I'd probably not learn much from it, but maybe I should check it out.
I didn't get how it could justify a "deontological approach to norms around civility and epistemics"
Learning to multiply the work that you do by automating it with AI when it becomes good enough to do so it's a lot more valuable than multiplying the hours that you work per week. It's not hard to see how you could make the first one 10x your output at some point, while the second one could be 2x at most.
Not to mention the benefits for your health that you could get from having more leisure time or, if you would prefer it, the additional impact/ career capital you could get from using that time to take on more responsibilities.
Seriously, the fact that people bring present benefits is just mind-blowing to me. People are rationalizing something away or they do not actually put as much weight on x-risks/ the far future as I thought.
I genuinely cannot believe there are EAs saying this kind of stuff.
Don't you think the later we pause, the closer we are to the precipice and that if someone continues to develop capabilities, they would be putting everyone at risk? Don't you think there could be algorithmic progress any moment that could make ASI attainable now???
I mean, when you consider the long-term future, we are talking about putting at risk potentially 10^50 lives!!!!!!!!
How can you say the present benefits for millions of people (that are mostly productivity and not necessarily mean better lives) can justify this?
"They are giving us alignment experience"? How wouldn't it be experience that we could eventually get once we unpause? Because compute overhang? It's not like we would immediately have a trillion dollars worth of data centers just after the pause, like we are doing now. Not to mention most AI researchers seem to think present alignment stuff and superalignment are qualitatively different and basically no progress has been made at all.
Is everyone just gone extremely shortermist or what am I missing???
I have trouble understanding the different levels of fidelity argument. It seems like a belief that would have obvious evidence in the real world, eg some humans having more neurons, more complex neurons, different readings of brain activity, and less NPC behaviour to such a degree that those types of brains/ people would be conscious while the other would not! All while having people in the other extreme of the threshold arguing that subatomic particles may be conscious.
I mean, if you care a lot about animals, knowing that some humans may be more complex than others could make you believe they are *more* conscious. Why would it move your threshold for binary consciousness?
I also haven't heard of scientifically minded people making those observations themselves (in neural stuff/NPC behaviour) and then trying to find an explanation for them. But it could be because some of these arguments imply that some people may be qualitatively better/worse than others and saying that is bad for multiple reasons.
Still, I think it is evidence against it.
And that could be the same reason why I don't see people arguing those differences between people just based on simulation theory, nor being interested in looking for evidence in favor of or against it (or at least sharing positive proof of it in public).
This comment from OP may clarify some things
There should be more ongoing debate about whether it's good or not to call oneself an EA; I haven't seen many posts myself. We should try to think through all the pros and cons.
It probably depends on who you are and the career you want to have, but for some reason it feels like most people think it's bad in most cases.
Even if you disagree with some subgroups or like to have a small identity, it could be bad for the world that most EA-adjacents don't call themselves EAs.
Over time, it could make some of them less effective or altruistic. It could also allow crazy people to give bad first impressions and misrepresent the movement. SBF, of course, is the prime example of this; maybe it would be better if people were autistic about EA like he was. In this state, we may be a ticking bomb just waiting for more crazy EA-adjacents to join and worsen the brand even more. And it's just a matter of cases like those until it's left in an irrecoverable state.
Another reason may be that identifying with a label has to do with feelings that we cannot control and we should value honesty. But then, can we fix that from within the movement if we realize it would be better for people to identify as an EA?
It could be because it sounds cringe too, but maybe it's worth enduring it.
Idk, these are just some pros and cons I just thought about.
Oh, I understand what you mean now.
Yeah, but I think there is a low upper limit to how good a coversation can get.
Also, if there are too many participants and no "moderator", the game theory can benefit defecting from the norms.
Yeah, that's true
I think that the fact that "smart" is being applied to a "digital species" changes things because "species" implies agency & goal-directedness, and "digital" implies it will be able to access useful information and become knowledgeable.
But I can understand that maybe other species don't seem goal-directed, so maybe it feels less threatening.
What? I never read WWOTF because I thought it was like an introduction to longtermism and I'd probably not learn much from it, but maybe I should check it out.
I didn't get how it could justify a "deontological approach to norms around civility and epistemics"