AIM just announced that they opened applications to the Charity Entrepreneurship Incubation Program. For those interested, I recently wrote this piece outlining what I did to get accepted to the program. I hope it can be a useful resource for you.
Unless you have a good reason to believe otherwise, whatever you are doing now is probably not the most impactful thing that you will ever do. Over time, you will have more career capital, which will open more opportunities for impact. If that's true, a large portion of the value of your current project is building career capital for future projects.
As a result, all else equal, we should perhaps be slightly skewed towards tractability in the typical ITN framework. A successful project looks better for career capital than a failed one, even if the two projects were equal from a pure EV comparison.
With that said, most people skew risk-averse, so maybe this is already built in?
EDIT: I agree with @Charlie_Guthmann 's comment that while this may be true, most people already (unintentonally) take this consideration into account due to natural biases, like risk-aversion, MORE than they should.
Without getting into the specifics, this donation opportunity only made sense above a certain level.
I did donate, and I'm glad I did, but I think it's valuable to note the inconsistencies of human psychology that appear even in communities that pride themselves on rationality.
Thinking back to my first big (well, big for me) donation and an unusual series of thoughts I had. Sharing in case anyone has experienced the same. --
I've practiced frugality to a significant extent, largely because I want to donate/salary sacrifice so much of my income to effective charities. As a result, whenever I'm spending a significant amount of money on anything, alarm bells go off.
I was surprised that alarm bells went off when donating. I had thought through where I donated extensively, and charity was the reason why I wanted to save money in the first place. But I still felt stressed because I was "spending money."
This is such a clear example of missing the forest for the trees. I think we need to be careful about instrumental values/rules and making sure they don't become absolute limitations that decrease overall impact.
I DO believe that preventing extinction is valuable and there are some projects promising enough to prioritize over work in other cause areas.
I DON'T believe that longtermism dominates all other cause areas in expectation, or that the argument you have made in favor of longtermism is convincing.
I agree that rejecting both A and B would not make sense, if you are informed of both. I think the author is wrong to treat A and B as separate decisions, when the agent knows about both in advance.
Knowing that you have the option to take bet B later fundamentally changes the considerations for bet A. As a result, we are not making 2 independent decisions (A: yes or no, and B: yes or no). We are making 4 (A, B, BOTH, NEITHER).
When considering that list, we can see that BOTH is strictly greater than NEITHER in all worlds and rule out NEITHER. We are left with A, B, and BOTH to choose from, all of which might make sense depending on the agent's choices.
At no point did I need to employ NARROW, PLAN, or SEQUENCE. I didn't even consider the probability of H, let alone whether that probability is sharp. I just considered the available options differently.
EDIT: I think this is close in effect to SEQUENCE. As a result, there might be the objection, "What if, of the 4 options, you choose B? Could you change your mind after rejecting A and then reject B as well?" To this I would say that a rational actor does not change their mind without new information. They would only choose B if they believe B > BOTH > NEITHER. Any rational actor who believes B > NEITHER would end up betting B. They would never bet NEITHER.
What might have muddied the waters:
I separately considered how I might deal with these probabilities separately, WITHOUT knowledge that one will follow the other. This is a distinct problem from the original dilemma. However, I think it's the only situation where a rational actor who follows UNSHARP might behave differently.
Without knowledge beforehand, if you hold UNSHARP, the following can happen:
You receive A, evaluate it, conclude it's optional due to UNSHARP probabilities, and reject it. Then, you are offered B, evaluate it, conclude it's optional, and reject it. You look back and think "I wish I would have known beforehand. I would have taken advantage of the arbitrage. Oh well. I guess rational actors with less information make worse decisions."
I think it is rational for an actor to hold unsharp probabilities for some hypotheses.[1] I think it's rational to not engage in sports gambling when no arbitrage exists. My initial example was designed to connect the two.
I haven't made my mind up on whether it's necessary to hold unsharp probabilities in theory but I'm much more confident in practice.
When you see a new opportunity that you know very little about that might be massively valuable, using your minimally informed baseline model to direct action seems irresponsible. Upon further investigation, everything regresses to the mean.
In the sports gambling example I gave, you should reject unless you see arbitrage because ~all available information is priced in. In the case of impact, new opportunities look more exciting than reality due to (e.g.) selection effects and stable equilibria.
This discussion of whether or not we should have unsharp probabilities is beside the point. My argument is about whether we can have unsharp probabilities without sacrificing rationality. I believe we can.
This seems to me to be another instance of the 1% fallacy (or the 0.1% fallacy, or the 10^-18 fallacy).
In your post, you talk about being skeptical of arguments where infinities cancel. I would argue that uncanceled infinities are generally a sign of a model being applied beyond its range of efficacy.
If you start off with a rough model and extrapolate it out to get a big enough number, all you have to do is come up with a set of conditions under which that number could plausibly be achieved, no matter how improbable. Then, say "Zero isn't a probability, and you don't have enough evidence to show that the probability cancels my big number." from the start and suddenly you have a large expected value.
However, when you refine the model (considering higher order effects, more thoroughly treating counterfactuals, turning exponential curves into more accurate s-curves), the big number drops out.
The base rate for the infinite is 0. As a result, I am much more skeptical of models where infinities don't cancel.
Could you please provide any concrete grounding for the probability of counterfactually shifting from extinction to a vast future (not delaying extinction temporarily) that is not based on a very small subjectively "conservative" probability?
I agree that accepting both bets is consistent with a sharp probability at 50%, though I'm just trying to give an example of a case where I would have an unsharp probability range where I would reject both bets in isolation but take them when they arrive together.
I don't employ any of the 3 strategies. My argument is that you don't need a fancy strategy because, in the example, you know that bet B is coming when you're asked about bet A. I think it's reasonable for a rational actor to reject bet A and reject bet B if the two are presented separately but accept them both if they are presented together. My example is intended to demonstrate that. A rational actor doesn't need NARROW, PLAN, or SEQUENCE. They need to consider the future: "Bet B is coming, so there's an arbitrage opportunity regardless of the probability." The article seems to disagree, treating every action in isolation and requiring that we make the right decision without global thinking.
My recommendation for portfolios is not an argument for, but an implication of, unsharp probabilities. A lot of cause prioritization is about the core philosophical positions you hold underpinning it. If you have a sharp probability, you might be comfortable investing all in one cause. If you have an unsharp one, you might not be convinced that investing in any one cause is net positive. However, you might find a combination of causes that seems robustly better than no action.
For example, you might be concerned about climate policy's constraints on growth as well as growth's effect on the climate. If you believe that the second order effects of investing in growth on the climate are smaller than the direct benefits of donating to climate policy (and vice versa), it is strictly better to donate to both in some combination than to do nothing. Someone with a sharp probability might be comfortable donating to just one in a way someone with unsharp probabilities would not.
As a result, portfolios are better (i.e. are more often optimal) in a world where UNSHARP is true.
I see and agree with your point about marginal returns. Depending on how strong that effect is, portfolios are also good in a world with sharp probabilities only.
AIM just announced that they opened applications to the Charity Entrepreneurship Incubation Program. For those interested, I recently wrote this piece outlining what I did to get accepted to the program. I hope it can be a useful resource for you.
Most importantly, my first tip: APPLY!
I agree, and this is a (more thorough and better written) version of the point I was trying to make with the question I asked at the bottom.
Unless you have a good reason to believe otherwise, whatever you are doing now is probably not the most impactful thing that you will ever do. Over time, you will have more career capital, which will open more opportunities for impact. If that's true, a large portion of the value of your current project is building career capital for future projects.
As a result, all else equal, we should perhaps be slightly skewed towards tractability in the typical ITN framework. A successful project looks better for career capital than a failed one, even if the two projects were equal from a pure EV comparison.
With that said, most people skew risk-averse, so maybe this is already built in?
EDIT: I agree with @Charlie_Guthmann 's comment that while this may be true, most people already (unintentonally) take this consideration into account due to natural biases, like risk-aversion, MORE than they should.
Without getting into the specifics, this donation opportunity only made sense above a certain level.
I did donate, and I'm glad I did, but I think it's valuable to note the inconsistencies of human psychology that appear even in communities that pride themselves on rationality.
Thinking back to my first big (well, big for me) donation and an unusual series of thoughts I had. Sharing in case anyone has experienced the same.
--
I've practiced frugality to a significant extent, largely because I want to donate/salary sacrifice so much of my income to effective charities. As a result, whenever I'm spending a significant amount of money on anything, alarm bells go off.
I was surprised that alarm bells went off when donating. I had thought through where I donated extensively, and charity was the reason why I wanted to save money in the first place. But I still felt stressed because I was "spending money."
This is such a clear example of missing the forest for the trees. I think we need to be careful about instrumental values/rules and making sure they don't become absolute limitations that decrease overall impact.
At least on mobile, my prior feed customization options to reduce certain topics has been removed, and I have been unable to reinstate my preferences.
I figured this was important to keep separate:
I DO believe that preventing extinction is valuable and there are some projects promising enough to prioritize over work in other cause areas.
I DON'T believe that longtermism dominates all other cause areas in expectation, or that the argument you have made in favor of longtermism is convincing.
I agree that rejecting both A and B would not make sense, if you are informed of both. I think the author is wrong to treat A and B as separate decisions, when the agent knows about both in advance.
Knowing that you have the option to take bet B later fundamentally changes the considerations for bet A. As a result, we are not making 2 independent decisions (A: yes or no, and B: yes or no). We are making 4 (A, B, BOTH, NEITHER).
When considering that list, we can see that BOTH is strictly greater than NEITHER in all worlds and rule out NEITHER. We are left with A, B, and BOTH to choose from, all of which might make sense depending on the agent's choices.
At no point did I need to employ NARROW, PLAN, or SEQUENCE. I didn't even consider the probability of H, let alone whether that probability is sharp. I just considered the available options differently.
EDIT: I think this is close in effect to SEQUENCE. As a result, there might be the objection, "What if, of the 4 options, you choose B? Could you change your mind after rejecting A and then reject B as well?" To this I would say that a rational actor does not change their mind without new information. They would only choose B if they believe B > BOTH > NEITHER. Any rational actor who believes B > NEITHER would end up betting B. They would never bet NEITHER.
What might have muddied the waters:
I separately considered how I might deal with these probabilities separately, WITHOUT knowledge that one will follow the other. This is a distinct problem from the original dilemma. However, I think it's the only situation where a rational actor who follows UNSHARP might behave differently.
Without knowledge beforehand, if you hold UNSHARP, the following can happen:
You receive A, evaluate it, conclude it's optional due to UNSHARP probabilities, and reject it. Then, you are offered B, evaluate it, conclude it's optional, and reject it. You look back and think "I wish I would have known beforehand. I would have taken advantage of the arbitrage. Oh well. I guess rational actors with less information make worse decisions."
I think it is rational for an actor to hold unsharp probabilities for some hypotheses.[1] I think it's rational to not engage in sports gambling when no arbitrage exists. My initial example was designed to connect the two.
I haven't made my mind up on whether it's necessary to hold unsharp probabilities in theory but I'm much more confident in practice.
When you see a new opportunity that you know very little about that might be massively valuable, using your minimally informed baseline model to direct action seems irresponsible. Upon further investigation, everything regresses to the mean.
In the sports gambling example I gave, you should reject unless you see arbitrage because ~all available information is priced in. In the case of impact, new opportunities look more exciting than reality due to (e.g.) selection effects and stable equilibria.
This discussion of whether or not we should have unsharp probabilities is beside the point. My argument is about whether we can have unsharp probabilities without sacrificing rationality. I believe we can.
This seems to me to be another instance of the 1% fallacy (or the 0.1% fallacy, or the 10^-18 fallacy).
In your post, you talk about being skeptical of arguments where infinities cancel. I would argue that uncanceled infinities are generally a sign of a model being applied beyond its range of efficacy.
If you start off with a rough model and extrapolate it out to get a big enough number, all you have to do is come up with a set of conditions under which that number could plausibly be achieved, no matter how improbable. Then, say "Zero isn't a probability, and you don't have enough evidence to show that the probability cancels my big number." from the start and suddenly you have a large expected value.
However, when you refine the model (considering higher order effects, more thoroughly treating counterfactuals, turning exponential curves into more accurate s-curves), the big number drops out.
The base rate for the infinite is 0. As a result, I am much more skeptical of models where infinities don't cancel.
Could you please provide any concrete grounding for the probability of counterfactually shifting from extinction to a vast future (not delaying extinction temporarily) that is not based on a very small subjectively "conservative" probability?
I agree that accepting both bets is consistent with a sharp probability at 50%, though I'm just trying to give an example of a case where I would have an unsharp probability range where I would reject both bets in isolation but take them when they arrive together.
I don't employ any of the 3 strategies. My argument is that you don't need a fancy strategy because, in the example, you know that bet B is coming when you're asked about bet A. I think it's reasonable for a rational actor to reject bet A and reject bet B if the two are presented separately but accept them both if they are presented together. My example is intended to demonstrate that. A rational actor doesn't need NARROW, PLAN, or SEQUENCE. They need to consider the future: "Bet B is coming, so there's an arbitrage opportunity regardless of the probability." The article seems to disagree, treating every action in isolation and requiring that we make the right decision without global thinking.
My recommendation for portfolios is not an argument for, but an implication of, unsharp probabilities. A lot of cause prioritization is about the core philosophical positions you hold underpinning it. If you have a sharp probability, you might be comfortable investing all in one cause. If you have an unsharp one, you might not be convinced that investing in any one cause is net positive. However, you might find a combination of causes that seems robustly better than no action.
For example, you might be concerned about climate policy's constraints on growth as well as growth's effect on the climate. If you believe that the second order effects of investing in growth on the climate are smaller than the direct benefits of donating to climate policy (and vice versa), it is strictly better to donate to both in some combination than to do nothing. Someone with a sharp probability might be comfortable donating to just one in a way someone with unsharp probabilities would not.
As a result, portfolios are better (i.e. are more often optimal) in a world where UNSHARP is true.
I see and agree with your point about marginal returns. Depending on how strong that effect is, portfolios are also good in a world with sharp probabilities only.