I definitely not the "worst" PR crisis. The DOW account is a random poster for the Office of the Under Secretary of War for Research and Engineering, not Hegseth.
But I agree that it would be best for Republicans not to be polarized against AI safety. They aren't so far; thanks in part to Speaker Johnson. I suggest not letting random trolls goad us into abandoning courtesy and the presumption of good will. It is counterproductive to respond to much of this stuff, as that imbues it with more seriousness than it deserves.
If we broaden this to "value change" and maintain it as a future aspiration rather than a current priority, then I'm for it.
Some limited form of value change seems necessary for progress on wild animals and digital minds, so I don't think the topic can be avoided indefinitely.
My two cents is that the movement has yet to achieve widespread value change even in highly-educated, academic circles. I think it would be highly effective on the margin to focus on think tank staffers, elite law school students, graduate students in ecology subfields, and so on. There should be a consensus understanding on these issues - for instance, that animal suffering is a negative externality of meat eating.
Just as there came to be with the claim that climate change is occuring, and is anthropogenic.
I've been trying to go through this sequence premise by premise and failing to come up with a satisfactory rebuttal. The reason is that roughly speaking, I find it to be a correct account about a wide range of potential claims to knowledge about far-future outcomes.
Nonetheless, I expect that certain outcomes are predictable nonetheless, contingent on antecedent events. These are roughly the ones the EA community has focused on (with a few minor exceptions, e.g. the value and sign of a very near-term AI Pause). The badness of extinction. The persistence of wild animal suffering, in a range of futures. The potential for many sentient digital minds. The expected persistence and scale of factory farming. Etc...(further specificity would require a full post).
So why these ones and not others? I think the sequence has generally been rather irrationally pessimistic in regards to certain claims to knowledge, which it has obscured by staying away from object-level claims. For instance:
"Consider your favorite intervention, let’s say, advocating for digital mind welfare. And imagine you become aware of some new pessimistic hypothesis. E.g., advocating for digital mind welfare could make it easier for misaligned AIs to take over by gaining humans’ sympathy (see Fenwick). Should you update toward more pessimism about the other possibilities in the catch-all? How much more?"
This just seems like a very implausible scenario! Yes, we may be unaware of the more plausible complementary issues that make concern valid, but our inability to come up with these issues (given the effort, importantly!) should be taken as evidence against them. I realize saying "seems like a very implausible scenario" is not a great account of a credible intuitive process, but there's clearly way more mechanistic intricacy in the misaligned ASI scenario. The misalignment, the usefulness for the ASI, the weird edge-case were this was actually an instrumental factor, etc. I'm not sure how helpful it is to go through this in greater detail; I suspect it is more plausible that this type of back-lash effect is just there to serve a psychological function, the mere conception being the satisfying evidence and not the strength of the reason. This is a type of argumentation that should be itself suspect.
It's not surprising that the sequence is pessimistic in these ways: I think that it's a fairly well-evidenced sociological phenomenon now that people respond pretty doomishly to far-future scenarios. Dystopias over utopias, and disempowering cluelessness (and contrived back-lash effects) over individual agency. But we should keep in mind that some form of eutopia[1] is by considerable margins the most probable. And further, that most interventions will tend to either have small effects or do nothing. I think these are good hedges against going off-kilter in one way or another.
I agree with some of this. But let me attempt a conciliatory take: less of forecasting money and effort should go to platforms and tournaments, but more should go to identifying existing, nascent forecasts (people using the word "probably" or "unlikely" about empirical matters) and creating markets (even unsubsidized Manifold markets would be helpful on the margin). I think it would be very helpful for someone to go through popular EA forum posts and org research documents and do this systematically.
"Super dead" is a bit of an exaggeration, but there is less activity than there should be. The main issue is pretty apparent to me though: there isn't sufficient cross-posting to the forum. There's tons of good stuff on substack, but also on various org websites and blogs.
I don't think other causes are nearly as important as this.
I may be missing important context, but I think you are mistaken here on the norms at hand in this case. I do applaud you for helping your friend out; that makes you a good friend. But opportunities for people to be altruistic are completely unbounded; I could find hundreds of similar asks for help in a 5 minute google search, most of which aren't distinctively "good opportunities". If this wasn't a personal request, but instead calling for donations to a related cause you were making a case for, that would be fine. I think highlighting personal requests for help is permissible and is even virtuous interpersonal behavior between friends and family. People reach out on facebook pages like this all the time. But it just looks like spam or emotional manipulation when posted on online forums dedicated to other purposes, with colleagues or strangers. Hopefully this helps! This can definitely be a confusing discourse norm contextually.
For context: Clara is right, there is good experimental evidence that this occurs in online comment forums. This is on top of the simple mechanism that more highly upvoted content is more likely to be seen for various reasons.
I'd assume this holds true for EA forum content. I do the same thing @Toby Tremlett🔹 is describing to some extent, but I'd be surprised if my system 2 thinking outweighs my system 1 on net in this regard. I suspect I personally do this most with very low Karma posts, which I neglect to upvote because of a vague embarrassment over the possibility of promoting content with some flaw I missed.
Due to Value Lock-in, TAI poses a time constraint for farmed animal social progress.
I do not expect most issues to be resolved before this time, due to technological limitations, heightened barriers to social change relative to historic movements, and increasing developing world meat consumption.
If we open this up to wild animals rather than just farmed, net-negative outcomes are much more assured.
I do tend to favor longer AGI/TAI timelines than many for roughly these reasons. But I don't think you are exactly right about the AI data access trend. For one, whether or not me or Americans at large are "happy to give an ASI full autonomous power to gather such biomedical data", China will be.
I tentatively I expect capabilities with real-world economic importance to come to some extent in the US as well, even if the most radical and transformative stuff requires further integration into the physical world for modeling. And at that point there may simply be a iterative process of greater and greater integration, as public perception improves and dependence increases. The complication here is moral backlash of some sort, which I note you've written about before. I agree that this is plausible, I simply wouldn't call it probable. Things look more bi-modal to me; most likely we get the outcome I've described above (mild harms could still be disregarded by China), or we get a longer slow down before curing aging.
I definitely not the "worst" PR crisis. The DOW account is a random poster for the Office of the Under Secretary of War for Research and Engineering, not Hegseth.
But I agree that it would be best for Republicans not to be polarized against AI safety. They aren't so far; thanks in part to Speaker Johnson. I suggest not letting random trolls goad us into abandoning courtesy and the presumption of good will. It is counterproductive to respond to much of this stuff, as that imbues it with more seriousness than it deserves.
If we broaden this to "value change" and maintain it as a future aspiration rather than a current priority, then I'm for it.
Some limited form of value change seems necessary for progress on wild animals and digital minds, so I don't think the topic can be avoided indefinitely.
My two cents is that the movement has yet to achieve widespread value change even in highly-educated, academic circles. I think it would be highly effective on the margin to focus on think tank staffers, elite law school students, graduate students in ecology subfields, and so on. There should be a consensus understanding on these issues - for instance, that animal suffering is a negative externality of meat eating.
Just as there came to be with the claim that climate change is occuring, and is anthropogenic.
I've been trying to go through this sequence premise by premise and failing to come up with a satisfactory rebuttal. The reason is that roughly speaking, I find it to be a correct account about a wide range of potential claims to knowledge about far-future outcomes.
Nonetheless, I expect that certain outcomes are predictable nonetheless, contingent on antecedent events. These are roughly the ones the EA community has focused on (with a few minor exceptions, e.g. the value and sign of a very near-term AI Pause). The badness of extinction. The persistence of wild animal suffering, in a range of futures. The potential for many sentient digital minds. The expected persistence and scale of factory farming. Etc...(further specificity would require a full post).
So why these ones and not others? I think the sequence has generally been rather irrationally pessimistic in regards to certain claims to knowledge, which it has obscured by staying away from object-level claims. For instance:
"Consider your favorite intervention, let’s say, advocating for digital mind welfare. And imagine you become aware of some new pessimistic hypothesis. E.g., advocating for digital mind welfare could make it easier for misaligned AIs to take over by gaining humans’ sympathy (see Fenwick). Should you update toward more pessimism about the other possibilities in the catch-all? How much more?"
This just seems like a very implausible scenario! Yes, we may be unaware of the more plausible complementary issues that make concern valid, but our inability to come up with these issues (given the effort, importantly!) should be taken as evidence against them. I realize saying "seems like a very implausible scenario" is not a great account of a credible intuitive process, but there's clearly way more mechanistic intricacy in the misaligned ASI scenario. The misalignment, the usefulness for the ASI, the weird edge-case were this was actually an instrumental factor, etc. I'm not sure how helpful it is to go through this in greater detail; I suspect it is more plausible that this type of back-lash effect is just there to serve a psychological function, the mere conception being the satisfying evidence and not the strength of the reason. This is a type of argumentation that should be itself suspect.
It's not surprising that the sequence is pessimistic in these ways: I think that it's a fairly well-evidenced sociological phenomenon now that people respond pretty doomishly to far-future scenarios. Dystopias over utopias, and disempowering cluelessness (and contrived back-lash effects) over individual agency. But we should keep in mind that some form of eutopia[1] is by considerable margins the most probable. And further, that most interventions will tend to either have small effects or do nothing. I think these are good hedges against going off-kilter in one way or another.
a term I take to include pretty low-value-but-positive futures, e.g. average utilitarian outcomes
I agree with some of this. But let me attempt a conciliatory take: less of forecasting money and effort should go to platforms and tournaments, but more should go to identifying existing, nascent forecasts (people using the word "probably" or "unlikely" about empirical matters) and creating markets (even unsubsidized Manifold markets would be helpful on the margin). I think it would be very helpful for someone to go through popular EA forum posts and org research documents and do this systematically.
"Super dead" is a bit of an exaggeration, but there is less activity than there should be. The main issue is pretty apparent to me though: there isn't sufficient cross-posting to the forum. There's tons of good stuff on substack, but also on various org websites and blogs.
I don't think other causes are nearly as important as this.
I may be missing important context, but I think you are mistaken here on the norms at hand in this case. I do applaud you for helping your friend out; that makes you a good friend. But opportunities for people to be altruistic are completely unbounded; I could find hundreds of similar asks for help in a 5 minute google search, most of which aren't distinctively "good opportunities". If this wasn't a personal request, but instead calling for donations to a related cause you were making a case for, that would be fine. I think highlighting personal requests for help is permissible and is even virtuous interpersonal behavior between friends and family. People reach out on facebook pages like this all the time. But it just looks like spam or emotional manipulation when posted on online forums dedicated to other purposes, with colleagues or strangers.
Hopefully this helps! This can definitely be a confusing discourse norm contextually.
For context: Clara is right, there is good experimental evidence that this occurs in online comment forums. This is on top of the simple mechanism that more highly upvoted content is more likely to be seen for various reasons.
I'd assume this holds true for EA forum content. I do the same thing @Toby Tremlett🔹 is describing to some extent, but I'd be surprised if my system 2 thinking outweighs my system 1 on net in this regard. I suspect I personally do this most with very low Karma posts, which I neglect to upvote because of a vague embarrassment over the possibility of promoting content with some flaw I missed.
Is he going to starve if I stop reading posts?! I'm too scared to leave the forum now.
Due to Value Lock-in, TAI poses a time constraint for farmed animal social progress.
I do not expect most issues to be resolved before this time, due to technological limitations, heightened barriers to social change relative to historic movements, and increasing developing world meat consumption.
If we open this up to wild animals rather than just farmed, net-negative outcomes are much more assured.
I do tend to favor longer AGI/TAI timelines than many for roughly these reasons. But I don't think you are exactly right about the AI data access trend. For one, whether or not me or Americans at large are "happy to give an ASI full autonomous power to gather such biomedical data", China will be.
I tentatively I expect capabilities with real-world economic importance to come to some extent in the US as well, even if the most radical and transformative stuff requires further integration into the physical world for modeling. And at that point there may simply be a iterative process of greater and greater integration, as public perception improves and dependence increases. The complication here is moral backlash of some sort, which I note you've written about before. I agree that this is plausible, I simply wouldn't call it probable. Things look more bi-modal to me; most likely we get the outcome I've described above (mild harms could still be disregarded by China), or we get a longer slow down before curing aging.