Another issue is that there is no longer any org that produces career prioritization research.
I suspect people have the mistaken notion that this area of research is just a reflection of philanthropic prioritization; the places it is advisable to recommend donations are also the best to recommend for careers. But this is a major simplification, even though there's a lot of overlap. Altrusitically optimal career trajectories are waaaaay broader and more obscure due to personal fit and labour market demand considerations.
Probably good doesn't do this. 80k used to do this well, but now they are just another AI safety advocacy group (no slight intended, those are needed too).
You are indeed in the right place! I do like to see more wild animal welfare discussions happening here. Regarding the forum norms, the animal welfare and opportunities tags are most appropriate for this, but it's not a big deal. If you feel like writing a longer post outside quick takes, you may find the wild animal topic tab helpful to see what others have written.
We know that unhatched chick embryos can suffer late in development, but current evidence is mixed on mid-term development around day-13. The in-ovo sexing technology used by the NestFresh brand in the US works on day-13 (which is banned in Germany). Claude's overview is here. Given that public policy and industry tech adoption is still in early stages outside Europe, it seems to me that it is great time to fund further research on chick embryo sentience. Before day-13 becomes standard.
I think the forum team has generally shown good judgement on which posts to feature. But still, it is hard to see how this doesn't give too much authority to too few decision makers on what content does and doesn't get boosted. A major benefit of a public forum is that it can help surface new ideas by casting a wide net; but centering featured posts this much privileges established authorities.[1]
Some amount of featuring is good, particularly of old posts that can get buried (Lesswrong does this well!) or when it can concentrate the forum on one topic, as it has effectively during debate weeks. The balance is just off with the current design.
I definitely not the "worst" PR crisis. The DOW account is a random poster for the Office of the Under Secretary of War for Research and Engineering, not Hegseth.
But I agree that it would be best for Republicans not to be polarized against AI safety. They aren't so far; thanks in part to Speaker Johnson. I suggest not letting random trolls goad us into abandoning courtesy and the presumption of good will. It is counterproductive to respond to much of this stuff, as that imbues it with more seriousness than it deserves.
If we broaden this to "value change" and maintain it as a future aspiration rather than a current priority, then I'm for it.
Some limited form of value change seems necessary for progress on wild animals and digital minds, so I don't think the topic can be avoided indefinitely.
My two cents is that the movement has yet to achieve widespread value change even in highly-educated, academic circles. I think it would be highly effective on the margin to focus on think tank staffers, elite law school students, graduate students in ecology subfields, and so on. There should be a consensus understanding on these issues - for instance, that animal suffering is a negative externality of meat eating.
Just as there came to be with the claim that climate change is occuring, and is anthropogenic.
I've been trying to go through this sequence premise by premise and failing to come up with a satisfactory rebuttal. The reason is that roughly speaking, I find it to be a correct account about a wide range of potential claims to knowledge about far-future outcomes.
Nonetheless, I expect that certain outcomes are predictable nonetheless, contingent on antecedent events. These are roughly the ones the EA community has focused on (with a few minor exceptions, e.g. the value and sign of a very near-term AI Pause). The badness of extinction. The persistence of wild animal suffering, in a range of futures. The potential for many sentient digital minds. The expected persistence and scale of factory farming. Etc...(further specificity would require a full post).
So why these ones and not others? I think the sequence has generally been rather irrationally pessimistic in regards to certain claims to knowledge, which it has obscured by staying away from object-level claims. For instance:
"Consider your favorite intervention, let’s say, advocating for digital mind welfare. And imagine you become aware of some new pessimistic hypothesis. E.g., advocating for digital mind welfare could make it easier for misaligned AIs to take over by gaining humans’ sympathy (see Fenwick). Should you update toward more pessimism about the other possibilities in the catch-all? How much more?"
This just seems like a very implausible scenario! Yes, we may be unaware of the more plausible complementary issues that make concern valid, but our inability to come up with these issues (given the effort, importantly!) should be taken as evidence against them. I realize saying "seems like a very implausible scenario" is not a great account of a credible intuitive process, but there's clearly way more mechanistic intricacy in the misaligned ASI scenario. The misalignment, the usefulness for the ASI, the weird edge-case were this was actually an instrumental factor, etc. I'm not sure how helpful it is to go through this in greater detail; I suspect it is more plausible that this type of back-lash effect is just there to serve a psychological function, the mere conception being the satisfying evidence and not the strength of the reason. This is a type of argumentation that should be itself suspect.
It's not surprising that the sequence is pessimistic in these ways: I think that it's a fairly well-evidenced sociological phenomenon now that people respond pretty doomishly to far-future scenarios. Dystopias over utopias, and disempowering cluelessness (and contrived back-lash effects) over individual agency. But we should keep in mind that some form of eutopia[1] is by considerable margins the most probable. And further, that most interventions will tend to either have small effects or do nothing. I think these are good hedges against going off-kilter in one way or another.
I agree with some of this. But let me attempt a conciliatory take: less of forecasting money and effort should go to platforms and tournaments, but more should go to identifying existing, nascent forecasts (people using the word "probably" or "unlikely" about empirical matters) and creating markets (even unsubsidized Manifold markets would be helpful on the margin). I think it would be very helpful for someone to go through popular EA forum posts and org research documents and do this systematically.
"Super dead" is a bit of an exaggeration, but there is less activity than there should be. The main issue is pretty apparent to me though: there isn't sufficient cross-posting to the forum. There's tons of good stuff on substack, but also on various org websites and blogs.
I don't think other causes are nearly as important as this.
I may be missing important context, but I think you are mistaken here on the norms at hand in this case. I do applaud you for helping your friend out; that makes you a good friend. But opportunities for people to be altruistic are completely unbounded; I could find hundreds of similar asks for help in a 5 minute google search, most of which aren't distinctively "good opportunities". If this wasn't a personal request, but instead calling for donations to a related cause you were making a case for, that would be fine. I think highlighting personal requests for help is permissible and is even virtuous interpersonal behavior between friends and family. People reach out on facebook pages like this all the time. But it just looks like spam or emotional manipulation when posted on online forums dedicated to other purposes, with colleagues or strangers. Hopefully this helps! This can definitely be a confusing discourse norm contextually.
Another issue is that there is no longer any org that produces career prioritization research.
I suspect people have the mistaken notion that this area of research is just a reflection of philanthropic prioritization; the places it is advisable to recommend donations are also the best to recommend for careers. But this is a major simplification, even though there's a lot of overlap. Altrusitically optimal career trajectories are waaaaay broader and more obscure due to personal fit and labour market demand considerations.
Probably good doesn't do this. 80k used to do this well, but now they are just another AI safety advocacy group (no slight intended, those are needed too).
You are indeed in the right place! I do like to see more wild animal welfare discussions happening here. Regarding the forum norms, the animal welfare and opportunities tags are most appropriate for this, but it's not a big deal. If you feel like writing a longer post outside quick takes, you may find the wild animal topic tab helpful to see what others have written.
@Wild_Animal_Initiative may also be relevant for you: https://www.wildanimalinitiative.org/grants
Plus there's some wild animal topic conversation on the Sentient Futures slack.
We know that unhatched chick embryos can suffer late in development, but current evidence is mixed on mid-term development around day-13. The in-ovo sexing technology used by the NestFresh brand in the US works on day-13 (which is banned in Germany). Claude's overview is here.
Given that public policy and industry tech adoption is still in early stages outside Europe, it seems to me that it is great time to fund further research on chick embryo sentience. Before day-13 becomes standard.
I think the forum team has generally shown good judgement on which posts to feature. But still, it is hard to see how this doesn't give too much authority to too few decision makers on what content does and doesn't get boosted. A major benefit of a public forum is that it can help surface new ideas by casting a wide net; but centering featured posts this much privileges established authorities.[1]
Some amount of featuring is good, particularly of old posts that can get buried (Lesswrong does this well!) or when it can concentrate the forum on one topic, as it has effectively during debate weeks. The balance is just off with the current design.
It's already the case that the weighted upvote system has this effect.
I definitely not the "worst" PR crisis. The DOW account is a random poster for the Office of the Under Secretary of War for Research and Engineering, not Hegseth.
But I agree that it would be best for Republicans not to be polarized against AI safety. They aren't so far; thanks in part to Speaker Johnson. I suggest not letting random trolls goad us into abandoning courtesy and the presumption of good will. It is counterproductive to respond to much of this stuff, as that imbues it with more seriousness than it deserves.
If we broaden this to "value change" and maintain it as a future aspiration rather than a current priority, then I'm for it.
Some limited form of value change seems necessary for progress on wild animals and digital minds, so I don't think the topic can be avoided indefinitely.
My two cents is that the movement has yet to achieve widespread value change even in highly-educated, academic circles. I think it would be highly effective on the margin to focus on think tank staffers, elite law school students, graduate students in ecology subfields, and so on. There should be a consensus understanding on these issues - for instance, that animal suffering is a negative externality of meat eating.
Just as there came to be with the claim that climate change is occuring, and is anthropogenic.
I've been trying to go through this sequence premise by premise and failing to come up with a satisfactory rebuttal. The reason is that roughly speaking, I find it to be a correct account about a wide range of potential claims to knowledge about far-future outcomes.
Nonetheless, I expect that certain outcomes are predictable nonetheless, contingent on antecedent events. These are roughly the ones the EA community has focused on (with a few minor exceptions, e.g. the value and sign of a very near-term AI Pause). The badness of extinction. The persistence of wild animal suffering, in a range of futures. The potential for many sentient digital minds. The expected persistence and scale of factory farming. Etc...(further specificity would require a full post).
So why these ones and not others? I think the sequence has generally been rather irrationally pessimistic in regards to certain claims to knowledge, which it has obscured by staying away from object-level claims. For instance:
"Consider your favorite intervention, let’s say, advocating for digital mind welfare. And imagine you become aware of some new pessimistic hypothesis. E.g., advocating for digital mind welfare could make it easier for misaligned AIs to take over by gaining humans’ sympathy (see Fenwick). Should you update toward more pessimism about the other possibilities in the catch-all? How much more?"
This just seems like a very implausible scenario! Yes, we may be unaware of the more plausible complementary issues that make concern valid, but our inability to come up with these issues (given the effort, importantly!) should be taken as evidence against them. I realize saying "seems like a very implausible scenario" is not a great account of a credible intuitive process, but there's clearly way more mechanistic intricacy in the misaligned ASI scenario. The misalignment, the usefulness for the ASI, the weird edge-case were this was actually an instrumental factor, etc. I'm not sure how helpful it is to go through this in greater detail; I suspect it is more plausible that this type of back-lash effect is just there to serve a psychological function, the mere conception being the satisfying evidence and not the strength of the reason. This is a type of argumentation that should be itself suspect.
It's not surprising that the sequence is pessimistic in these ways: I think that it's a fairly well-evidenced sociological phenomenon now that people respond pretty doomishly to far-future scenarios. Dystopias over utopias, and disempowering cluelessness (and contrived back-lash effects) over individual agency. But we should keep in mind that some form of eutopia[1] is by considerable margins the most probable. And further, that most interventions will tend to either have small effects or do nothing. I think these are good hedges against going off-kilter in one way or another.
a term I take to include pretty low-value-but-positive futures, e.g. average utilitarian outcomes
I agree with some of this. But let me attempt a conciliatory take: less of forecasting money and effort should go to platforms and tournaments, but more should go to identifying existing, nascent forecasts (people using the word "probably" or "unlikely" about empirical matters) and creating markets (even unsubsidized Manifold markets would be helpful on the margin). I think it would be very helpful for someone to go through popular EA forum posts and org research documents and do this systematically.
"Super dead" is a bit of an exaggeration, but there is less activity than there should be. The main issue is pretty apparent to me though: there isn't sufficient cross-posting to the forum. There's tons of good stuff on substack, but also on various org websites and blogs.
I don't think other causes are nearly as important as this.
I may be missing important context, but I think you are mistaken here on the norms at hand in this case. I do applaud you for helping your friend out; that makes you a good friend. But opportunities for people to be altruistic are completely unbounded; I could find hundreds of similar asks for help in a 5 minute google search, most of which aren't distinctively "good opportunities". If this wasn't a personal request, but instead calling for donations to a related cause you were making a case for, that would be fine. I think highlighting personal requests for help is permissible and is even virtuous interpersonal behavior between friends and family. People reach out on facebook pages like this all the time. But it just looks like spam or emotional manipulation when posted on online forums dedicated to other purposes, with colleagues or strangers.
Hopefully this helps! This can definitely be a confusing discourse norm contextually.