I just want to share my appreciation for how much the EA policy community has grown in Brussels! I've organized/attended EA socials here since 2016, and the first few years a common complaint was that (nearly) no one was doing EU policy. It's extremely hard to put numbers on it, and while there's a lot of room to grow, I get the impression nowadays that the policy side competes with the non-policy side in size! Love to see it ☺️
I'll be you first exception as I love bright white light during the day and especially in the morning since it's closer to the sun and keeps me awake and energized 😅 But in the evening I do prefer warm light. But I'm just one small datapoint.
For me, it doesn't need to be hard-working or smarter people. Anyone you can cowork with who is supportive will do. But my challenge is to actually create such an environment! Online doesn't work that well for me, it needs to be in-person. It's so much more impactful than any other productivity hack.
I try to avoid it, but it's hard for me to believe it's as bad or worse than most animal products. Especially in the quantities it's usually consumed. Who eats a kg of honey per year? I do think the treatment of bees is very unclear. But I've also heard that some non-animal products involve a lot of insects, like avocados, so I'm curious how it compares.
I checked parts of the study, and the 0.12% figure is for P(AI-caused existential catastrophe by 2100) according to the "AI skeptics". This is what is written about the definition of existential catastrophe just before it:
Participants made an initial forecast on the core question they disagreed about (we’ll call this U, for “ultimate question”): by 2100, will AI cause an existential catastrophe? We defined “existential catastrophe” as an event in which at least one of the following occurs:
Humanity goes extinct
Humanity experiences “unrecoverable collapse,” which means either:
<$1 trillion global GDP annually [in 2022 dollars] for at least a million years (continuously), beginning before 2100; or
Human population remains below 1 million for at least a million years (continuously), beginning before 2100.
That sounds similar to the classic existential risk definition?
(Another thing that's important to note is that the study specifically sought forecasters skeptical of AI. So it doesn't tell us much if anything about what a group of random superforecasters would actually predict!)
I am very very surprised your 'second bucket' contains the possibility of humans potentially having nice lives! I suspect if you had asked me the definition of p(doom) before I read your initial comment, I would actually have mentioned the definition of existential risks that includes the permanent destruction of future potential. But I simply never took that second part seriously? Hence my initial confusion. I just assumed disempowerment or a loss of control would lead to literal extinction anyway, and that most people shared this assumption. In retrospect, that was probably naive of me. Now I'm genuinely curious how much of people's p(doom) estimates actually comes from actual extinction versus other scenarios...
Interesting, I thought p(doom) was about literal extinction? If it also refers to unrecoverable collapse, then I'm really surprised that takes up 15-30% of your potential scenarios! I always saw that part of the existential risk definition as negligible.
You make a fair point, but what other tool do we have than our voice? I've read Matthew's last post and skimmed through others. I see some concerning views, but I can also understand how he arrives at them. But what puzzles me often with some AI folks is the level of confidence needed to take such high-stakes actions. Why not err on the side of caution when the stakes are potentially so high?
Perhaps instead of trying to change someone's moral views, we could just encourage taking moral uncertainty seriously? I personally lean towards hedonic act utilitarianism, yet I often default to 'common sense morality' because I'm just not certain enough.
I don't have strong feelings on know how to best tackle this. I won't have good answers to any questions. I'm just voicing concern and hoping others with more expertise might consider engaging constructively.
Good point, I guess my lasting impression wasn't entirely fair to how things played out. In any case, the most important part of my message is that I hope he doesn't feels discouraged from actively participating in EA.
I feel so called out my wardrobe has to be at least 90% Uniqlo and EA merch
I just want to share my appreciation for how much the EA policy community has grown in Brussels! I've organized/attended EA socials here since 2016, and the first few years a common complaint was that (nearly) no one was doing EU policy. It's extremely hard to put numbers on it, and while there's a lot of room to grow, I get the impression nowadays that the policy side competes with the non-policy side in size! Love to see it ☺️
I'll be you first exception as I love bright white light during the day and especially in the morning since it's closer to the sun and keeps me awake and energized 😅 But in the evening I do prefer warm light. But I'm just one small datapoint.
For me, it doesn't need to be hard-working or smarter people. Anyone you can cowork with who is supportive will do. But my challenge is to actually create such an environment! Online doesn't work that well for me, it needs to be in-person. It's so much more impactful than any other productivity hack.
I try to avoid it, but it's hard for me to believe it's as bad or worse than most animal products. Especially in the quantities it's usually consumed. Who eats a kg of honey per year? I do think the treatment of bees is very unclear. But I've also heard that some non-animal products involve a lot of insects, like avocados, so I'm curious how it compares.
I checked parts of the study, and the 0.12% figure is for P(AI-caused existential catastrophe by 2100) according to the "AI skeptics". This is what is written about the definition of existential catastrophe just before it:
That sounds similar to the classic existential risk definition?
(Another thing that's important to note is that the study specifically sought forecasters skeptical of AI. So it doesn't tell us much if anything about what a group of random superforecasters would actually predict!)
I am very very surprised your 'second bucket' contains the possibility of humans potentially having nice lives! I suspect if you had asked me the definition of p(doom) before I read your initial comment, I would actually have mentioned the definition of existential risks that includes the permanent destruction of future potential. But I simply never took that second part seriously? Hence my initial confusion. I just assumed disempowerment or a loss of control would lead to literal extinction anyway, and that most people shared this assumption. In retrospect, that was probably naive of me. Now I'm genuinely curious how much of people's p(doom) estimates actually comes from actual extinction versus other scenarios...
Interesting, I thought p(doom) was about literal extinction? If it also refers to unrecoverable collapse, then I'm really surprised that takes up 15-30% of your potential scenarios! I always saw that part of the existential risk definition as negligible.
You're right that this is an important distinction to make.
You make a fair point, but what other tool do we have than our voice? I've read Matthew's last post and skimmed through others. I see some concerning views, but I can also understand how he arrives at them. But what puzzles me often with some AI folks is the level of confidence needed to take such high-stakes actions. Why not err on the side of caution when the stakes are potentially so high?
Perhaps instead of trying to change someone's moral views, we could just encourage taking moral uncertainty seriously? I personally lean towards hedonic act utilitarianism, yet I often default to 'common sense morality' because I'm just not certain enough.
I don't have strong feelings on know how to best tackle this. I won't have good answers to any questions. I'm just voicing concern and hoping others with more expertise might consider engaging constructively.
Good point, I guess my lasting impression wasn't entirely fair to how things played out. In any case, the most important part of my message is that I hope he doesn't feels discouraged from actively participating in EA.