When do you think it's best to use WFM vs expected value calculation?
Asking as I tend to use EV for charity choice. As I see it: pro of EV is it captures variables like scale really nicely, pro of WFM is it's more robust to imperfect inputs.
Agree with Alex. The clincher for me is the Climeworks assessment. If you have a range of candidate technologies which will remove Co2 from the atmosphere, then cost must be a factor. If a candidate is expensive and has no path to becoming cheap, it doesn’t matter that it’s amazing by all other metrics. This seems basic.
Giving Green talk about the near certainty of Climeworks. This tells you that the expected cost per tonne of Co2 right now is close to £1,150 (https://climeworks.com/subscriptions), perhaps plus or minus £5, making the “true” cost £1,145 - £1,155.
On the other hand, other charities might have a broader range. For example I believe CfRN has a “true” cost between $5 - $30 to reach the same reduction in Co2. This is a broader range ($25 vs £10). Yet CfRN is clearly better (by a factor of 38 - 230!) We can disagree about the exact numbers: the point is that cost effectiveness shouldn’t be left out of charity assessment.
The near certainty of Climeworks may give peace of mind for some. However it’s an adjunct to cost, no more.
On future value. Climeworks themselves aim to get to $100 per tonne by 2025, so one might give in expectation of helping to bring about this price. Alex mentions Metaculus’ forecasts, which can be used to calculate how probable this theory of change might be. Cost (current or future) should always be considered.
(On analyst jobs specifically) I think the impact various hugely by position.
Being in the right place at the right time is a huge deal: this means being closer to decision makers. If you're an analyst such opportunities are going to vary a lot depending on the nature of your work, team and colleagues, norms of sharing information, and how the teams are structured.
Holding all these constant, I think you can still increase your likelihood of finding yourself in the right place at the right time. How?
The more skills you have and the better connected you are in the organisation (lots of weak ties) the more likely you are to spot a useful thing that would've otherwise fallen by the wayside.
Some skills that I think are likely enough to pay dividends (vaguely ordered by perceived usefulness and how quick they are to learn):
Statistics 101 (confidence intervals, hypothesis testing, probability distributions), judgemental forecasting (Metaculus is a great place to start), programming in 1+ data science language, systems thinking 101 (much of the useful stuff is in Thinking in Systems - Meadows), non-deep learning ML, behavioural economics, basically all operational research, social research and economic techniques (if you are labelled as one of these professions I see no reason not to try and get good at all of them!). Plus generic things like writing well, presenting, running meetings well, ability to teach others, consuming long documents quickly, presenting well, management skills (this is bumped up if you're officially managing people), etc etc.
Being an expert in everything is unnecessary (if laudable). Even having a sense of what's possible technically can open doors, and you can learn more later if need be. If you can spot something and implement it, that's good counterfactual impact.
The civil service will often put on training, however MOOCs can be better. They can also be done in your own time if you're a keen bean (I am).
If you're working as an analyst, hopefully you find some of these interesting enough that learning them isn't a chore!
A few random thoughts
You can also link people to useful things, e.g.: prediction markets if a policy choice hinges on a few predictions. Or linking them to useful people (this is where being clued in on the EA network can be helpful).
One (possibly) high impact a action could be go to an area where analytics or other objectively beneficial things tend to fall by the wayside (and other analysts might have given up or been discouraged), and "sell" your services. You could spot and offer to do something beyond the official scope of your job. This might not be fun though!
Some analyst jobs have lots of room for coming up with clever and nifty ideas, whereas others are dependent on knowing lots of facts. I'd expect the latter type to benefit more from staying power of the employees, as it takes time to accumulate specialist knowledge, so just staying put in the job for a long time could add impact. Obvious reasons why you might not want to do this.
Thanks for writing this up :)
When do you think it's best to use WFM vs expected value calculation?
Asking as I tend to use EV for charity choice. As I see it: pro of EV is it captures variables like scale really nicely, pro of WFM is it's more robust to imperfect inputs.
Agree with Alex. The clincher for me is the Climeworks assessment. If you have a range of candidate technologies which will remove Co2 from the atmosphere, then cost must be a factor. If a candidate is expensive and has no path to becoming cheap, it doesn’t matter that it’s amazing by all other metrics. This seems basic.
Giving Green talk about the near certainty of Climeworks. This tells you that the expected cost per tonne of Co2 right now is close to £1,150 (https://climeworks.com/subscriptions), perhaps plus or minus £5, making the “true” cost £1,145 - £1,155.
On the other hand, other charities might have a broader range. For example I believe CfRN has a “true” cost between $5 - $30 to reach the same reduction in Co2. This is a broader range ($25 vs £10). Yet CfRN is clearly better (by a factor of 38 - 230!) We can disagree about the exact numbers: the point is that cost effectiveness shouldn’t be left out of charity assessment.
The near certainty of Climeworks may give peace of mind for some. However it’s an adjunct to cost, no more.
On future value. Climeworks themselves aim to get to $100 per tonne by 2025, so one might give in expectation of helping to bring about this price. Alex mentions Metaculus’ forecasts, which can be used to calculate how probable this theory of change might be. Cost (current or future) should always be considered.
(On analyst jobs specifically) I think the impact various hugely by position.
Being in the right place at the right time is a huge deal: this means being closer to decision makers. If you're an analyst such opportunities are going to vary a lot depending on the nature of your work, team and colleagues, norms of sharing information, and how the teams are structured.
Holding all these constant, I think you can still increase your likelihood of finding yourself in the right place at the right time. How?
The more skills you have and the better connected you are in the organisation (lots of weak ties) the more likely you are to spot a useful thing that would've otherwise fallen by the wayside.
Some skills that I think are likely enough to pay dividends (vaguely ordered by perceived usefulness and how quick they are to learn):
Statistics 101 (confidence intervals, hypothesis testing, probability distributions), judgemental forecasting (Metaculus is a great place to start), programming in 1+ data science language, systems thinking 101 (much of the useful stuff is in Thinking in Systems - Meadows), non-deep learning ML, behavioural economics, basically all operational research, social research and economic techniques (if you are labelled as one of these professions I see no reason not to try and get good at all of them!). Plus generic things like writing well, presenting, running meetings well, ability to teach others, consuming long documents quickly, presenting well, management skills (this is bumped up if you're officially managing people), etc etc.
Being an expert in everything is unnecessary (if laudable). Even having a sense of what's possible technically can open doors, and you can learn more later if need be. If you can spot something and implement it, that's good counterfactual impact.
The civil service will often put on training, however MOOCs can be better. They can also be done in your own time if you're a keen bean (I am).
If you're working as an analyst, hopefully you find some of these interesting enough that learning them isn't a chore!
A few random thoughts
You can also link people to useful things, e.g.: prediction markets if a policy choice hinges on a few predictions. Or linking them to useful people (this is where being clued in on the EA network can be helpful).
One (possibly) high impact a action could be go to an area where analytics or other objectively beneficial things tend to fall by the wayside (and other analysts might have given up or been discouraged), and "sell" your services. You could spot and offer to do something beyond the official scope of your job. This might not be fun though!
Some analyst jobs have lots of room for coming up with clever and nifty ideas, whereas others are dependent on knowing lots of facts. I'd expect the latter type to benefit more from staying power of the employees, as it takes time to accumulate specialist knowledge, so just staying put in the job for a long time could add impact. Obvious reasons why you might not want to do this.