I agree this moment presents an opportunity for some truth-tracking comms, and I'm sad that the administration isn't into AI safety, but I disagree with your vibe. EAs don't need to "defend" EA on social media. They can just go on doing good. Winning hearts and minds is nice but not super important; I feel unthreatened by the "trash-talk." And the truth is convergent; just doing good is a great way to improve your reputation in the long run.
In most cases, especially high-stakes cases, cost of living is small relative to prioritization/productivity/information benefits. (View not justified here.)
It seems pretty likely to me that you instead want AIs to be risk seeking for reasons discussed here. Takeover attempts that are very unlikely to succeed might speculatively actually be a great trade from the perspective of humanity and risk seeking/risk neutral AI in that they reduce overall takeover risk while being good for this AI (while deals are way less useful to humanity due to being less clear evidence). Risk avoidant AIs might do nothing or just take deals until AIs take over later (and accepting deals might not be a good strategy for them depending on their views about the chance of AI takeover and some other factors).
I also think the implicit story about how we steer these traits doesn't really hold together and assumes a type of generalization I find somewhat implausible if we condition on AIs being egregiously misaligned.
If AGI goes well for humans, it’ll probably [i.e. ≥70%] go well for animals
I think it doesn't really make sense to do a sliding-scale vote on ≥70%. If my credence on (AGI goes well for animals | AGI goes well for humans) is 60%, then I'm just a no; if my credence is 80%, then I'm just a yes.
One way people could interpret the sliding-scale is expressing their confidence/stability in their judgment about whether the probability is greater or less than 70%. But that's somewhat deranged and it's not clear how to make it precise and I think everyone will just be confused.
It would be totally reasonable to vote on a scale from 0% to 100% for P(AGI goes well for animals | AGI goes well for humans), rather than voting from fully-disagree to fully-agree for ≥70%. Obviously that requires a little recoding. But making the voting scale more flexible, rather than just from fully-disagree to fully-agree, will benefit other debates too.
My claim is that if you're worried, the correct response is to actually try to make the astronomical problem/cause go better, not to give up on it. I think if you're savvy you will probably find a way to make the astronomical thing go better—such as doing strategy/prioritization/deconfusion work, or working on robustly good intermediate desiderata, or building skills/money in case there's more clarity in the future—rather than ultimately thinking there's nothing I can do to make the thing go better.
I agree this moment presents an opportunity for some truth-tracking comms, and I'm sad that the administration isn't into AI safety, but I disagree with your vibe. EAs don't need to "defend" EA on social media. They can just go on doing good. Winning hearts and minds is nice but not super important; I feel unthreatened by the "trash-talk." And the truth is convergent; just doing good is a great way to improve your reputation in the long run.
No, funders can get even more money for effective philanthropy by investing in AI.
In most cases, especially high-stakes cases, cost of living is small relative to prioritization/productivity/information benefits. (View not justified here.)
Crossposting Ryan's comment on LW:
Prior art: this, this, and some of this.
Over on LessWrong I wrote some posts about prioritization research and donations.
Yes. Most people will directionally-agree (which is maybe a problem you were trying to solve by adding "probably") but maybe that's OK.
I think it doesn't really make sense to do a sliding-scale vote on ≥70%. If my credence on (AGI goes well for animals | AGI goes well for humans) is 60%, then I'm just a no; if my credence is 80%, then I'm just a yes.
One way people could interpret the sliding-scale is expressing their confidence/stability in their judgment about whether the probability is greater or less than 70%. But that's somewhat deranged and it's not clear how to make it precise and I think everyone will just be confused.
It would be totally reasonable to vote on a scale from 0% to 100% for P(AGI goes well for animals | AGI goes well for humans), rather than voting from fully-disagree to fully-agree for ≥70%. Obviously that requires a little recoding. But making the voting scale more flexible, rather than just from fully-disagree to fully-agree, will benefit other debates too.
I haven't engaged with your posts and so don't know the arguments.
I respect that you and a few others legitimately feel deeply clueless. Alice and Bob are just whining about how not everything is clear-cut.
My claim is that if you're worried, the correct response is to actually try to make the astronomical problem/cause go better, not to give up on it. I think if you're savvy you will probably find a way to make the astronomical thing go better—such as doing strategy/prioritization/deconfusion work, or working on robustly good intermediate desiderata, or building skills/money in case there's more clarity in the future—rather than ultimately thinking there's nothing I can do to make the thing go better.