Let's imprecisely interpret judgment calls as "hard-to-explain intuitions" as you wrote, for simplicity. I think that's enough, here.

For the US 2024 presidential election, there are definitely such judgment calls involved. If one tries to make an evolutionary argument undermining our ability to predict US 2024 presidential election, P1 holds. P2 visibly doesn't however, at least for some good predictors. There is empirical evidence against P2. And presumably, the reason why P2 doesn't hold is that people who have decent hard-to-explain intuitions vis-a-vis "where the wind blows" in such socio-political contexts survived better. The same can't be said (at least, not obviously) for forecasting whether making altruistic people more longtermists does more good than harm, considering all the consequences on everything from now until the end of time.

> But I don't see why we would need to end up at 50%

Say you say 53% and Alice says 45%. The two of you can give me all the arguments you want. At the end of the day, you both undeniably made judgment calls when weighing the reasons to believe making altruistic people more longtermists does more good than harm, all things considered, and reasons to believe the opposite (including reasons, in both cases, that have to do with aliens, acausal reasoning, and how to deal with crucial unknown unknowns). I don't see why I should trust any of your two different judgment-cally "best guesses" any more than the other.

In fact, if I can't find a good objection to P2, I have no good reason to trust any of your best guesses any more than a dart-throwing chimp. If I had an opinion on the (dis)value of making altruistic people more longtermists without having a good reason to reject P2, I'd be blatantly inconsistent. [1]

Do you agree now that we've hopefully clarified what is a judgment call and what isn't, here? (I think P2 is definitely the crux for whether we should be clueless. Defending that we can identify positive longtermist causes without resorting to any sort of hard-to-explain intuitions seems really untenable. And I think there may be better objections to P2 than the ones I address in the post.)


[1] Btw, a bit tangential but a key popular assumption/finding in the literature on decision-making under deep uncertainty is that "not having an opinion" or "suspending judgment" =/= 50% credence -- see this post from DiGiovanni for a nice overview).

Nice, thanks for bringing this up! Let's take this example of yours

Find the most altruistic person you know, and direct their attention towards crucial considerations about moral patienthood, population ethics, decision theory etc. The effect size is probably pretty small, but having altruistic people learn more about longtermism seems good in expectation.

Say Alice (someone with similar intellectual capacities to yours and equivalent GPR knowledge) comes and tells you she thinks that "having altruistic people learn more about longtermism seems (slightly) bad in expectation" (and she gives plausible reasons -- I'll intentionally not give examples to avoid anchoring us on a particular one). The two of you talk and agree on all the relevant factors to consider here but reach different verdicts when weighing the reasons to believe preaching longtermism to altruists is good and those to believe the opposite. How would you explain your disagreement if not by the fact that the two of you have different intuitions and make different judgment calls?

Like maybe your beliefs don't need to track the truth better than random to be warranted? Fair. I was also implicitly assuming not that.

could anything other than evolutionary pressures (direct or indirect) work "against individuals unable to make the correct judgment calls regarding what actions do more good than harm (in expectation) considering how these impact the far future"?

Fair! One could say it's not evolution but God or something that gave us such ability (or the ability to know we have such ability although for unknown reasons).

Another: does making correct judgement calls enough to have warranted beliefs (in humans) about something require any (past) pressure against incorrect judgement calls (about those things in particular, or in domains from which there is generalization)?

I don't understand how this differs from your first example. Can you think of a way one could argue for the negative on this? That'd probably help me spot the difference.

I've looked into this and here's (this specific section of the post) a case against this cancelation postulate that builds upon the literature on the topic. :) For something that addresses Owen's above point in particular, see the second half of the section discussing Tomasik's argument (which is basically the same afaiu).

(Sharing this here on the off-chance this is also of interest to someone else finding themself lost in this comment thread someday).

Sorry, that wasn't super clear. I'm saying that if you believe that there is more total suffering in a human-controlled future than in a future not controlled by humans, X-risk reduction would be problematic from the point of view you defend in your post.

So if you endorse this point of view, you should either believe x-risk reduction is bad or that there isn't more total suffering in a human-controlled future. Believing either of those would be unusual (although this doesn't mean you're wrong) which is why I was curious.

Mogensen and Wiblin discuss this problem in this podcast episode, fwiw. That's all I know, sorry.

Btw, if you really endorse your solution (and ignore potential aliens colonizing our corner of the universe someday, maybe), I think you should find deeply problematic GCP's take (and the take of most people on this Forum) on the value of reducing X-risks. Do you agree or do you believe the future of our light cone with humanity around doing things will not contain any suffering (or anything that would be worse than the suffering of one Jones in the “Transmitter Room Problem”)? You got me curious.

I would even take this further and argue that your chain of reasoning could be applied to most causes (perhaps even all?), which seems valid.

Would you disagree with this?

I mean, I didn't actually give any argument for why I don't believe AI safety is good overall (assuming pure longtermism, i.e., taking into account everything from now until the end of time). I just said that I would believe it if there was evidence robust to unknown unknowns. (I haven't argued that there wasn't such evidence already; although the burden of the proof is very much on the opposite claim tbf). But I think this criterion applies to all causes where unknown unknowns are substantial, and I believe this is all of them as long as we're evaluating them from a pure longtermist perspective, yes. And whether there is any cause that meets this criterion depends on one's values I think. From a classical utilitarian perspective (and assuming the trade-offs between suffering and pleasure that most longtermists endorse), for example, I think there's very plausibly none that does meet this criterion.

If you do have a determinate credence above 50% for AI safety work, how do you arrive at this conclusion?

It happens that I do not. But I would if I believed there was evidence robust to unknown unknowns in favor of assuming "AI Safety work" is good, factoring in all the possible consequences from now until the end of time. This would require robust reasons to believe that current AI safety work actually increases rather than decreases safety overall AND that increased safety is actually good all things considered (e.g., that human disempowerment is actually bad overall). (See Guillaume's comment on the distinction). I won't elaborate on what would count as "evidence robust to unknown unknowns" in such a context but this is a topic for a future post/paper, hopefully.

Next, I want to push back on your claim that if ii) is correct, everything collapses. I agree that this would lead to the conclusion that we are probably entirely clueless about longtermist causes, probably the vast majority of causes in the world. However, it would make me lean toward near-term areas with much shorter causal chains, where there is a smaller margin of error—for example, caring for your family or local animals, which carry a low risk of backfiring.

Sorry, I didn't mean to argue against that. I just meant that work you are clueless about (e.g. maybe AI safety work in your case?) shouldn't be given any weight in your diversified portfolio. I didn't mean to make any claim about what I personnally think we should or shouldn't be clueless about. The "everything falls apart" was unclear and probably unwarranted.

Do you think that AI safety is i) at least a bit good in expectation (but like with a determinate credence barely higher than 50% because high risk/uncertainty) or ii) you don't have determinate credences and feel clueless/agnostic about this? I feel like your post implicitly keeps jumping back and forth between these two positions, and only (i) could support your conclusions. If we assume (ii), everything falls apart. There's no reason to support a cause X (or the exact opposite of X) to any degree if one is totally clueless about whether it is good.

Thanks for writing this :)