Why Forecasting Fails Decision Makers

Well written and pragmatic, thanks. Although forecasting does sounds great, I have no idea how animal nonprofits (to take one example) could use it to make better work.

Jo_🔸

I can actually think of one example in animal welfare: the EA Animal Welfare Fund forecasts grant outcomes.

CB🔸

Great, good to know !

2mo*

I think this is a clear sign the community hasn’t been able to communicate its use case well at all. This is one reason I often use “predictive reasoning” as a more general concept when talking to people, interestingly especially if they are already aware of forecasting (as they’ve been conditioned to think it means prediction markets, tournaments, and resolution criteria).

Take your example of animal welfare, I don’t know the exact use case best aligned to you, but fundamentally i’m confident 95%+ of the decisions an animal nonprofit will make are based on two predictions:

1) What will the world be like in the future (insert timeline)?

2) What interventions will most impact/change that future closer to what you’d like it to be?

Forecasting, or more specifically, the processes that underpins the science of forecasting, can be used to increase the accuracy and efficiency of those two predictions.

Once you do that, a better estimation of the future world + a better estimation of the efficacy of your actions, will occur.

CB🔸

Yes, makes sense, shorter or longer AI timelines do change the game a lot, for instance.

david_reinstein

2mo*

For example, forecast on the potential viability of cultured meat could have several actionable implications.

.m It might be something worth directly funding, particularly if we think that certain types of investment will have a strong impact on it’s like it would be successful. It also might be something to start investing in planning for in legal messaging communications et cetera.

Also, if we anticipate a massive global shift towards culture meat and thus away from factory farming it makes lobbying for factory farming reforms and corporate welfare of pledges somewhat less valuable.

This is much of the theory of impact/theory of change behind The Unjournal’s cultured meat pivotal questions, project and our upcoming workshop. We are working in some belief elicitation and some forecasting elements. However, it’s not clear whether these are situations in which the broad Wisdom of the Crowd adds a lot of value or whether it’s dependent on small groups of experts.

Also see Support Metaculus' First Animal-Focused Forecasting Tournament which is something we are trying to support and we try to make sure that each of the forecasting questions has actionable value of information.

david_reinstein

Trying to connect the “forecasting to decisions” or “evidence to decisions” pipeline with the “Pivotal questions”: an Unjournal trial initiative and our workshops https://uj-pq-workshops.netlify.app/

But the challenges you mention are real and this tracks my experience. Upvotes.

Oleksandr Bondar

Point 2 is the one I'd push further. Once a probability has done its real job, exposing that two people in the room are 40pp apart, there's a second lever most skip: checking whether the reasoning each side exposed was actually licensed by the evidence, or just confidently asserted. The interesting part of a 40pp gap usually isn't the number. It's that one person treated as established what the evidence only suggested.

That also reframes the accountability worry. Brier scores need a tournament's worth of resolved questions to mean much, which slow or one-shot decisions never supply. But you can benchmark a different way: name the load-bearing reasons up front, seal them before the outcome, then later check whether the ones you flagged as weak were the ones that actually drove the result. That works on a single decision, and it survives the clearance filter you describe, because what reaches the minister isn't a percentage, it's "here is why this was a reasonable call on what we knew at the time."

I work on exactly this in pharma R&D, where go/no-go calls have dated readouts a year or two out, so the reasoning can be scored against reality without waiting for a tournament's worth of questions. The decision-makers there want the defensible record, not the probability, which matches your experience.

Jared T Peterson

A lot of political and organisation decision making is not based on how accurately you’ve predicted the world. Sad, but true.

I do not consider this sad, but just a fact about how decision-making works in the real world. Economists model humans as-if they use probabilities to make choices between well-defined options. If you model humans as making decisions in this way, then forecasting makes sense.

But the real world is not a multiple choice test offering pre-defined options which need probabilistic estimates. In real world decision-making, problem formulation and problem solving are the same cognitive process. There are no problems out there waiting for you to find them, rather, you have to define what is even a problem in the first place and how to think about it. Figuring out how to solve a problem is a process of sensemaking until you feel like you have a grip on the situation; like you know what is relevant, how it is relevant, what the moving parts are, how they interact, what your leverage point is, and what your values are. A probabilistic estimate can never offer that. A probabilistic estimate abstracts away so much that it actually leaves you feeling like you don't know the space at all.

Dominic Cummings has mentioned this

During the pandemic, Dominic Cummings said some of the most useful stuff that he received and circulated in the British government was not forecasting, it was qualitative information explaining the general model of what’s going on, which enabled decision-makers to think more clearly about their options for action and the likely consequences. If you’re worried about a new disease outbreak, you don’t just want a percentage probability estimate about future case numbers, you want an explanation of how the virus is likely to spread, what you can do about it, how you can prevent it. Not the best estimate for how many COVID cases there will be in a month, but why forecasters believe there will be X COVID cases in a month.
https://www.samstack.io/p/five-questions-for-michael-story

Forecasting is based on an as-if (read: wrong) model of decision-making. You wouldn't decide when to make a left-hand-turn at a busy intersection based on some narrow probabilistic estimate (there is a 99% chance you won't get t-boned if you turn now) because you want to understand the decision yourself, and that probabilistic estimate is missing so much (Will I T-bone someone? Are there people in the cross walk? Are the cars slowing down because the light already changed colors? Or are they speeding up because it already changed colors?)

In real world decision-making, framing/modeling is everything. How you take large problems and turn them into something tractable that the human mind can comprehend and reason about. But superforecasting assumes that problem away. It assumes away the most important aspect of decision-making; understanding.

I think this misunderstands forecasting or mischaracterises decision making.

Your brain absolutely does decide when to make a left-hand-turn at a busy intersection based on a probabilistic estimate. Your brain is just pretty reliable at making said judgments that you assume some binary choice was made. Subconciously you are weighing up the likelihood of various risks based on your senses observing them and concluding that there is a very low (or an appropriately low) probability of it going wrong. The easy way to test this is go to a zip line, one with a drunk operator and one with a sober operator. The zip line may look identically safe but your brain will (hopefully) consider the former more risky - not because the impact of something going wrong is different between the zip lines (both falls would hurt, if not kill you), but because the likelihood of something going wrong is different (as you'd calculate that a drunk operator may not be as cognitively aware of what they are doing). That's a probabilistic assessment.

The difference is, most decisions we deal with in the world don't need us to sit down and do a formal prediction process to ensure good decision making. However, most organisational, and especially those on policy and governance etc., do require it. They are concretely two predictions you are making:

1. What will the world be? (through the lens of what you care about, e.g. will there be an oil crisis?)

2. What influence on the world will my policy have? (e.g. if I send Trump a friendly email will he avoid attacking another petrostate?)

Viewing forecasting as just the probabilistic estimate is missing the entire benefit. Forecasting is the entire process. Stating your view probabilistic just allows you to understand your own and others uncertainty (and to provide a purity of accountability and incentive alignment). It only adds decision fatigue if you have not put in the right processes to interpret the result (such as upfront thresholds for action). If you are following an optimal predictive decision making process, you should be making your assumptions and your weighing up of information explicit. This is how you determine what is relevant, what isn't relevant etc.

Dominic Cummings statement misses that everything he said there IS forecasting. What he's actually saying is "the point estimate from forecasters was not useful - it was the explicit reasoning about the causal chain etc. from forecasters that was useful".

However, benchmarking against accuracy ensures the incentives are correct and that decisions aren't manipulated by elements that decrease the end outcomes efficacy.

If you conclude that it'd be better if decision makers had: 1) the most accurate view of the world; and 2) the most accurate view of how their actions may influence that world towards their objectives. Then I would stand by the statement that it's sad.

Mahi

Thank you for posting this!

“we’ve found people don’t know what issues they should be focusing on, and rather than a probability estimate, they want help to identify the most prescient questions”.

Interesting point. Curious whether this alludes to a broader bottleneck within forecasting around question generation itself, rather than question answering/calibration. My vague impression (though I don’t know anything about the practical workflows of forecasting organisations) is that a lot of attention/status accrues to producing forecasts, whereas identifying and specifying the right questions may be at least as important (and potentially harder?)
I'd be interested to hear how much effort (as a rough %) forecasting orgs currently devote to question discovery/specification versus answering already-defined questions.

Also wondering at what “level” this problem mainly exists. Is the issue:

identifying the broad strategic questions institutions should care about in the first place (e.g., cause prioritisation, emerging risks, etc), or
the more granular forecasting-design questions like framing, timelines, resolution criteria, etc?
(or perhaps both of these are linked too tightly in practice to separate?)

So my experience is that identifying/specifying/generating the right questions is at least 50% of the benefit, if not higher. There are lots of reasons organisations struggle with this, from: organisational incentives; incoherent steers and views from senior managers; lack of accountability and ownership; to simply not recognising that they are trying to do a prediction.

This is why forecasting funding that has focused on improving forecasting accuracy is flawed, because it doesn't matter how accurate you are if your question isn't of use to the decision making process.

The problem exists at both those "levels", but the most important one to solve for an organisation is the first one. Issues with resolution criteria etc. decrease accuracy but as long as people's rationale's are explicit you can bridge that gap (i.e. I know why one person was higher and another was lower, it was because they both took the resolution to mean something slightly differently). But if the question/problem you are trying to solve in the first instance is wrong, then the whole thing is a waste of time and energy.

Vasco Grilo🔸

Thanks for the post, James. It made sense to me.

I worked at HM Treasury

This is "the Government of the United Kingdom’s economic and finance ministry".

The real value of forecasting is in the moment you realise two people in the same room have forecasts 40% apart.

You mean 40 pp apart?

Counting animals: Stable population size is not equivalent to priority level

Yeah 40pp, though 40% difference may also be informative depending on the question and distribution.

Regardless, “40” was just a random number. Basically the interesting thing are the areas of greatest difference, not the probability itself.

Comments

Curated and popular this week

abrahamrowe, mal_graham🔸·4d ago·Curated 1d ago·16m read

AI Use Note: Main body text entirely human written. Claude (Opus 4.8) helped develop models of animal life histories in the appendix. Cross-posted from Good Structures. Executive Summary * Animal advocates sometimes make claims like “there are X of this animal...

150

Let's taboo the V-word

lincolnq·5d ago·8m read

“How long have you been v*g*n?” This is one of the most common icebreakers at animal protection events. It’s a baseline assumption, and it mostly holds true: if you’re out advocating for animals not to be tortured or abused, realistically these days you are v**n, or close. And it makes for good conversation. It seems fairly safe to assume when you meet strangers. But this assumption is hurting the movement in a way which we don’t always notice: someone new comes into the sp...

Spiro: an update 2.5 years on and a fundraising ask for expansion

Habiba Banu·2d ago·6m read

Summary Back in November 2023 I posted here to launch Spiro and raise our first $198k. Two and a half years later this is an update and a fundraiser for the next step. The short version: we've now reached over-5,900 people with TB preventive medicine, including over 3,000 children under five years old. Our early results have held up well an...