Summary
EA and rationalists got enamoured with forecasting and prediction markets and made them part of the culture, but this hasn’t proven very useful, yet it continues to receive substantial EA funding. We should cut it off.
My Experience with Forecasting
For a while, I was the number one forecaster on Manifold. This lasted for about a year until I stopped just over 2 years ago. To this day, despite quitting, I’m still #8 on the platform. Additionally, I have done well on real-money prediction markets (Polymarket), earning mid-5 figures and winning a few AI bets. I say this to suggest that I would gain status from forecasting being seen as useful, but I think, to the contrary, that the EA community should stop funding it.
I’ve written a few comments throughout the years that I didn’t think forecasting was worth funding. You can see some of these here and here. Finally, I have gotten around to making this full post.
Solution Seeking a Problem
When talking about forecasting, people often ask questions like “How can we leverage forecasting into better decisions?” This is the wrong way to go about solving problems. You solve problems by starting with the problem, and then you see which tools are useful for solving it.
The way people talk about forecasting is very similar to how people talk about cryptocurrency/blockchain. People have a tool they want to use, whether that be cryptocurrency or forecasting, and then try to solve problems with it because they really believe in the solution, but I think this is misguided. You have to start with the problem you are trying to solve, not the solution you want to apply. A lot of work has been put into building up forecasting, making platforms, hosting tournaments, etc., on the assumption that it was instrumentally useful, but this is pretty dangerous to continue without concrete gains.
We’ve Funded Enough Forecasting that We Should See Tangible Gains
It’s not the case that forecasting/prediction markets are merely in their infancy. A lot of money has gone into forecasting. On the EA side of things, it’s near $100M. If I convince you later on in this post that forecasting hasn’t given any fruitful results, it should be noted that this isn’t for lack of trying/spending.
The Forecasting Research Institute received grants in the 10s of millions of dollars. Metaculus continues to receive millions of dollars per year to maintain a forecasting platform and conduct some forecasting tournaments. The Good Judgment Project and the Swift Centre have received millions of dollars for doing research and studies on forecasting and teaching others about forecasting. Sage has received millions of dollars to develop forecasting tools. Many others, like Manifold, have also been given millions by the EA community in grants/investments at high valuations, diverting money away from other EA causes. We have grants for organizations that develop tooling, even entire programming languages like Squiggle, for forecasting.
On the for-profit side of things, the money gets even bigger. Kalshi and Polymarket have each raised billions of dollars, and other forecasting platforms have also raised 10s of millions of dollars.
Prediction markets have also taken off. Kalshi and Polymarket are both showing ATH/growth in month-over-month volume. Both of them have monthly volumes in the 10s of billions of dollars. Total prediction market volume is something like $500B/year, but it just isn’t very useful. We get to know the odds on every basketball game player prop, and if BTC is going to go up or down in the next 5 minutes. While some people suggest that these trivial markets help sharpen skills or identify good forecasters, I don’t think there is any evidence of this, and it is more wishful thinking.
If forecasting were really working well and was very useful, you would see the bulk of the money spent not on forecasting platforms but directly on forecasting teams or subsidizing markets on important questions. We have seen very little of this, and instead, we have seen the money go to platforms, tooling, and the like. We already had a few forecasting platforms, the market was going to fund them itself, and yet we continue to create them.
There has also been an incredible amount of (wasted) time by the EA/rationality community that has been spent on forecasting. Lots of people have been employed full-time doing forecasting or adjacent work, but perhaps even larger is the amount of part-time hours that have gone into forecasting on Manifold, among other things. I would estimate that thousands of person-years have gone into this activity.
Hits-based Giving Means Stopping the Bets that Don’t Pay Off
You may be tempted to justify forecasting on the grounds of hits-based giving. That is to say, it made sense to try a few grants into forecasting because the payoff could have been massive. But if it was based on hits-based giving, then that implies we should be looking for big payoffs, and that we have to stop funding it if it doesn’t.
I want to propose my leading theory for why forecasting continues to receive 10s of millions per year in funding. That is, it has become a feature of EA/rationalist culture. Similar to how EAs seem to live in group houses or be polyamorous, forecasting on prediction markets has become a part of the culture that doesn’t have much to do with impact. This is separate from parts of EA culture that we do for impact/value alignment reasons, like being vegan, donating 10%+ of income, writing on forums, or going to conferences. I submit that forecasting is in the former category.
At this point, if forecasting were useful, you would expect to see tangible results. I can point to you hundreds of millions of chickens that lay eggs that are out of cages, and I can point to you observable families that are no longer living in poverty. I can show you pieces of legislation that have passed or almost passed on AI. I can show you AMF successes with about 200k lives saved and far lower levels of malaria, not to mention higher incomes and longer life expectancies, and people living longer lives that otherwise wouldn’t be because of our actions. I can go at the individual level, and I can, more importantly, go at the broad statistical level. I don’t think there is very much in the way of “this forecasting happened, and now we have made demonstrably better decisions regarding this terminal goal that we care about”. Despite no tangible results, people continue to have the dream that forecasting will inform better decision-making or lead to better policies. I just don’t see any proof of this happening.
Feels Useful When It Isn’t
Forecasting is a very insidious trap because it makes you think you are being productive when you aren’t. I like to play bughouse and a bunch of different board games. But when I play these games, I don’t claim to do so for impact reasons, on effective altruist grounds. If I spend time learning strategy for these board games, I don’t pretend that this is somehow making the world better off. Forecasting is a dangerous activity, particularly because it is a fun, game-like activity that is nearly perfectly designed to be very attractive to EA/rationalist types because you get to be right when others are wrong, bet on your beliefs, and partake in the cultural practice. It is almost engineered to be a time waster for these groups because it provides the illusion that you are improving the world’s epistemics when, in reality, it’s mainly just a game, and it’s fun. You get to feel that you are improving the world’s epistemics and that therefore there must be some flow-through effects and thus you can justify the time spent by correcting a market from 57% to 53% on some AI forecasting question or some question about if the market you are trading on will have an even/odd number of traders or if someone will get a girlfriend by the end of the year.
Conclusion
A lot of people still like the idea of doing forecasting. If it becomes an optional, benign activity of the EA community, then it can continue to exist, but it should not continue to be a major target for philanthropic dollars. We are always in triage, and forecasting just isn’t making the cut. I’m worried that we will continue to pour community resources into forecasting, and it will continue to be thought of in vague terms as improving or informing decisions, when I’m skeptical that this is the case.
I don’t disagree with some of the fundamentals of this post. Before diving into that, I want to correct a factual error:
“the Swift Centre have received millions of dollars for doing research and studies on forecasting and teaching others about forecasting”
The Swift Centre for Applied Forecasting has not received millions in funding. The majority of our earnings have been through direct projects with organisations who want to use forecasting to inform their decisions.
On your wider argument. I think forecasting has probably received too much funding and the vast majority of that has misallocated on platforms and research. I believe some funding (hundreds of thousands) to maintain core platforms like Metaculus as a public good of information. Though, services like Polymarket can probably fill most of this need in the future (but many useful, informative markets would never reach the necessary volume to be reliable).
Where I think we disagree most is in the application of forecasting and some of the achievements. We’ve worked with frontier AI labs to inform their decisions, are currently advising a U.K. Minister’s team on a central piece of their policy, and are about to start a secondment where I will be advising one of the most influential decision making committees in the country to help improve their scenario analysis and forecasting. Forecasting, and specifically, the science of decision making that it is built on, has the ability to structurally improve decisions in institutions. Significantly better than asking two or three of your smartest friends. That was just never funded, so instead we conclude forecasting is not useful.
"We’ve worked with frontier AI labs to inform their decisions"
This feels likely net negative to me? But don't have enough information to know.
We could “forecast” the likelihood of that haha.
I can’t get into specifics. But if you believe activities like evaluations of models to test for dangerous behaviour etc. is net negative, then that may give credence to your assumption. As an extra data point of whether we’d do work we thought was net negative, I was Head of Policy at ControlAI and co-authored narrowpath.co, and our forecasters have done numerous AI safety focused projects (with and outside of the Swift Centre, including AI 2027).
Personally I think any working with AI labs (except perhaps anthropic) supports dangerous acceleration, but I think the opposing view is almost as strong.
That other stuff sounds way better than working with the labs too ;)
Appreciate the correction. I simply did totals I saw from Open Phil/CG spreadsheets. Ill correct the post.
Hi Marcus. Thanks for the post. I broadly agree.
Coefficient Giving's (CG's) Forecasting Fund has recently been closed.
I think this is more likely to make forecasting grants useful. They will presumably be assessed with the criteria used to evaluate the non-forecasting grants of the respective fund.
@NunoSempere wrote about the end of CG's Forecasting Fund in the last edition of the Forecasting Newsletter. Only paid subscribers can check the relevant section.
Right.
I'm not a paid sub to Nuno so I can't see.
I had this post in my drafts for 3 years. I was happy to see the Forecasting Fund close down, I don't expect we will see less than $5M of forecasting grants done by CG in 2026 or 2027 though.
I don't have an opinion on if I would rather the forecasting grants be made within or separate from the forecasting fund so long as the grants are still being made. I see pros and cons.
I think Holly's post you linked is awesome and is in my Mount Rushmore of posts (top 4 all time).
What are the other 3 on your Mount Rushmore?
Me neither.
CG's Forecasting Fund granted 15.9 M$ in 2025.
I tend to agree with the OP, but think there are a couple of other points about subsidising prediction markets which could have had more emphasis
I guess without a platform cut/spread you get marginally more precision, but how many forecasts actually need that precision and are sufficiently liquid to get it?
actually worse than zero sum on a for-profit exchange, obviously...
many forms of traditional gambling relies heavily on "whales" with a mixture of non trivial amounts of money to lose and impulse control problems for much of their volume and profit; some of them ruin their lives doing so, even more so the people with the same impulse control problems and less starting money. This may not apply to niche prediction markets, but I'm sure people can become addicted to the idea of winning their money back even if they know casino "betting systems" are -EV and don't like sports or machines with flashing lights
For Kalshi specifically, it seems to have essentially become a backdoor to deregulate sports gambling in every US state. The mass deregulation of gambling in the US this decade feels harmful and like something we’ll probably really regret (legalisation seems fine but not like this).
It doesn’t seem popular to criticise the gambling aspects of prediction markets here, but it does seem strange to me that EAs seem to care a lot about reducing harms from tobacco and alcohol, but seem indifferent to gambling.
I suspect that the main use of forecasting is if you need a probability for something and you don't really have time to look into it yourself or you wouldn't trust your judgement even if you did.
I think this is great and makes sense, but this isn't where 90 percent of the money is going.
Sort of, but that also doesn’t capture the significant accuracy and efficiency benefits the process of structured reasoning and communication that forecasting enables. There’s substantial risks and issues of “just looking into an issue yourself” - especially when you are more confident in your judgement (because that’s a clear risk of confirmation bias/overconfidence).
The main use of forecasting is in utilising the core scientific benefits it can bring as above into, to help real world decision makers. But fundamentally, that hasn’t been funded - instead we’ve funded tournaments and research.
I have at least three reasons to be hopeful:
That's not to say that every project previously funded around forecasting was a good use of money. I would probably agree with you regarding most of the projects you have in mind, while disagreeing with the title and framing which is way too broad.
Related comment I made 2 years ago and ensuing discussion: https://forum.effectivealtruism.org/posts/ziSEnEg4j8nFvhcni/new-open-philanthropy-grantmaking-program-forecasting?commentId=7cDWRrv57kivL5sCQ
Off topic, but one additional thing I noticed about this list:
Is the glaring lack of tangible advances in technical AI safety. It's a different case from your post, as it's about a problem rather than a tool; but I think it still shows something about whether we understand the dangers of AI and the systems they stem from enough to do anything about it.
Prediction markets seem to be a great business (mostly gambling with all the problems associated with it) so “funding” in the sense of investing in them could be sensible while “funding” in the donation sense not. (And then later donation to AMF or similar).
In general, I’m hesitant to donate to stuff that’s plausibly just a really good business in its own right.
I agree with some of this. But let me attempt a conciliatory take: less of forecasting money and effort should go to platforms and tournaments, but more should go to identifying existing, nascent forecasts (people using the word "probably" or "unlikely" about empirical matters) and creating markets (even unsubsidized Manifold markets would be helpful on the margin). I think it would be very helpful for someone to go through popular EA forum posts and org research documents and do this systematically.
I like bets involving donations, and investments as alternatives to forecasting without money on the line.
That's still a sort of game/cultural thing rather than a means for more positive impact, though. I've seen that around EA basically forever, but I don't think people who bet on their beliefs have been "more right" than those who don't.
Hi Guy. The bets would be directly beneficial if people who are more accurate donate to more cost-effective interventions? In addition, I wonder whether the discussions of bets involving donations, and investments could have higher quality than ones of forecasting questions without money on the line. The prospects of winning or losing money usually leads to people investigating their views more.
That seems to be a general cultural view in EA, but what I'm saying is that I've yet to see any evidence these bets actually help. I think the notion is unfounded.
This isn't something I've thought a ton about but I think forecasting should plausibly still receive funding in a specific way:
Funders should either pay forecasters to make predictions on important questions, or subsidize prediction markets on those questions.
I don't think forecasting is a "solution seeking a problem." There are tons of important but hard-to-predict questions that I'd like better forecasts on! The problem is that the ecosystem hasn't done a great job of turning dollars into good forecasts.
For example, most of my Metaculus questions are things I wanted answers to, but I tended not to update on the results because the questions usually don't receive a lot of forecasts. If someone wanted to pay money to get more predictions on questions, I'd learn something useful!
I'm not sure how valuable this is compared to other uses of money (I wouldn't pay for it myself) but at least it's better than more general-purpose research on forecasting.
The problem is that prediction markets on useful questions, many years out, suffer from problems due to capital lockup/interest rates, among other things.
Also, I just don't think the wisdom of the crowds emerges as much as you'd want. I think you can just ask 3 smart people what they think, and this will elicit more useful info.
The wisdom of crowds effect kicks in with very few forecasts. In the working paper I cite elsewhere in the comments, even 5 forecasts gets you pretty far along into the WoC effect, and 10 even more so. This is for asking people what they think, not prediction markets - the latter should, theoretically, require more forecasts, since seeing the implicit beliefs of others through the market price could lead to herding etc. But the wisdom of crowds effect kicking in for very small N is well established in the literature.
I have a different takeaway as you, though, that we only know about this effect - or about the biases people have and how to adjust their forecasts - because of work on forecasting. I don't know how we'd know this stylized fact without work on it. For the wisdom of the crowds effect specifically, perhaps you could stop funding early since that one is well known, but it's sufficiently surprising to most people that there could be value in showing it for more domains, and it is really just one example of what we learn more generally from research on forecasting - and these other results on how to optimally weigh forecasts can shrink error much more even after taking the wisdom of the crowds effect into consideration. (In our work, WoC gets you a ~60% reduction in the MSE, but other small adjustments lead to an improvement of an additional ~60% reduction in error compared to the WoC estimate, and those aren't even all the improvements we can make.)
Today I would never run an experiment without using forecasts to help with power calculations. And there is very recent work I'd use to adjust those forecasts, and we're collectively not near the optimum in terms of learning what we can learn to make more accurate forecasts or integrating them into workflows. As I said elsewhere in the comments, the claims in the OP are far too strong. Even your asking a few experts - that's something that could be improved on and integrated into workflows and is part of the titular "forecasting". (It reads to me kind of like: don't do forecasting, do this other thing which is itself forecasting and is informed by and improved upon by... forecasting.)
A more defensible claim imo would be that there are some projects that are self-supporting and those should not be funded, or that in some but not all cases if the market doesn't pay for it then it's not valuable (abstracting from coordination failures and other market failures, or the externalities of basic research).
I definitely have my own gripes about EA/rationalist attitudes towards forecasting (see here), but maybe your objection is a level confusion:
(My gripes discussed in the linked post above, FWIW: Re: (1), the typical EA operationalization of "well-calibrated", and judgments about how to defer to people based on their calibration on some reference class of past questions, are based on very questionable epistemological assumptions. See also this great post.)
I think one should distinguish between several things here:
This post really belabours the first and second bullet point, perhaps because that is where a lot of money has gone to, but there can be a lot of value in the third.
"I don’t think there is very much in the way of “this forecasting happened, and now we have made demonstrably better decisions regarding this terminal goal that we care about”."
I assume some people disagree with this strong claim. One example I've heard was AGI timelines and their influence on AI safety field priorities - though I guess one could answer that certain reports or expert opinions where disproportionately more useful than prediction markets.
On a different point, I appreciated Eli Lifland's past comment on many intellectual activities (such as grantmaking) being forms of forecasting.