The latest episode of Hard Fork has a 20 min section dedicated to prediction markets! Kevin Roose and Casey Newton go into much more depth in this podcast than in the earlier NYT article, covering the history of prediction markets, the rationalist movement, Google’s internal markets, insider trading, etc. They even talk about embedding prediction markets in the New York Times!

You can listen to the full podcast here (the segment runs from 29:30 to 48:15); a transcript is available there too. Some highlights:

Excerpts

Manifest, LK99 and Manifold

Casey Newton

Kevin, you did some great reporting this week about prediction markets, and it all started at something called Manifest, which — I read that and I thought, well, is this just a festival for men? Tell us about what manifest is.

Kevin Roose

[LAUGHS]: So this is a field trip that I had been planning for a while. This was a very fun and interesting reporting trip to a conference for what they call forecasting nerds, so people who like to predict the future and bet on the future. And this was actually something that came out of an episode that we did several months ago about LK-99. Do you remember this episode?

Casey Newton

Yes, of course.

Kevin Roose

So this was the room-temperature superconductor that a group of scientists in South Korea had claimed to have come up with. And there was this period of maybe a week or two where people were hotly debating whether this was real or not. And we mentioned on the show the existence of something called Manifold Markets, which is a prediction-markets platform where people can go and wager fake money on real-world events. And one of the most popular markets was about LK-99. And it was a way to track, like, what the smart-money people thought was going to happen and whether this prediction of a room-temperature superconductor would pan out. Now, it did not, right? LK-99 did not turn out to be a room-temperature superconductor. But I heard from one of the founders of Manifold Markets who said, if you’re interested in prediction markets, we’re actually having a big conference in a few weeks in Berkeley called Manifest, and you should come report on it. And I thought, well, that sounds like a fun trip.

Casey Newton

Yeah, I’d actually predicted that you were going to go to that, so that was interesting. [KEVIN LAUGHS] So you get there. And sort of describe the scene. Because what you’ve described — I’ll say it — sounds a little bit dull, but then I read your story, and it actually seemed like it might be a good time.

Kevin Roose

Yeah, it was a very strange event. And I say that — I had a good time and I learned a lot, but it was definitely not what I was expecting. I was expecting, like, a sort of statistics conference where people in dress shirts and Dockers would be going around, like, comparing their predictive models of the world. But it was more like a party than I thought. I described it in the article as sort of a cross between a math Olympiad and Burning Man. Like, there was actually an orgy at this conference. And I know that because there was also a prediction market asking whether or not there was going to be an orgy. I think when I got there, it was, like, 28 percent possibility, and by the time I left, someone had had an orgy and closed out the market.

History of prediction markets

Kevin Roose

When we talked about it on the podcast in the context of LK-99, I believe we made some snarky comments about, oh, these are just gamblers who like to bet on everything.

Casey Newton

Yeah.

Kevin Roose

But I would say, after going to this Manifest conference, there’s also a real movement that I think is worth paying attention to here. Prediction markets — this is not a new idea, right? People have been betting on things like elections for centuries. Actually, in the 19th and early-20th centuries, it was common to open up the newspaper and see a sort of betting-odds breakdown of who people thought was going to win the next election.

Casey Newton

And I feel like my entire life, I’ve been hearing about the wisdom of crowds.

Kevin Roose

Yes. That was a very popular idea. This idea of prediction markets was sort of revived in the 1990s by a group of economists who thought, well, markets collect information. You can bet on the price of a company’s stock, or you can bet on corn futures — what will the price of corn be a year from now? You can also bet on sports games. Why can’t you bet on other things? Why can’t you bet about scientific discoveries? Why can’t you bet about policy implementation? Why can’t you bet about silly things like whether there’s going to be an orgy at a statistics conference?

On Rationalists

So there’s been sort of a real resurgence in the last few years, led by this group of people called the Rationalists. Do much about the Rationalists?

Casey Newton

A lot of it from reading your reporting, but yeah, tell us a little bit more about the Rationalists.

Kevin Roose

So Rationalists are a sort of loose collective of people who are sort of committed to examining their own beliefs. They want to get closer to the truth. Big figures in the movement are people like the guy who runs this blog, “Astral Codex Ten,” which used to be known as “Slate Star Codex.” Eliezer Yudkowsky is sort of an AI safety researcher and a prominent Rationalist blogger who started a website called “LessWrong.” So there’s a crew of people, largely based in the Bay Area but also spread out throughout the world, who are sort of doing what they would describe as rigorous empirical testing of everything that they believe and do. They love attaching probabilities to things. So I want to sketch out the vision for what they believe prediction markets could do. Because they’re not just saying like this could be a way to make money by betting right on things. They’re saying, if you have everyone betting on everything, then you end up with a system where people are incentivized to understand the truth.

Google’s internal prediction markets

But interestingly, I was told that this is not a new idea to the world of tech, that actually, Google has run its own prediction markets internally for employees. So if you worked at Google, you could bet —

Casey Newton

On whether it would become a search monopoly.

Kevin Roose

[LAUGHS]: Yeah, and everyone who made that bet got paid. No, it was — they used a fake currency called googles, and you could bet on things like, will this project launch in time, or will Gmail get to this many users by this date. And the company’s leaders use this as a way to gauge employee sentiment. And sort of, when people could bet anonymously, they could actually get people’s true opinions.

Casey Newton

Wait, that’s fascinating. And do they not do this anymore?

Kevin Roose

Well, they were playing around with this as recently as 2021. And there were people I talked to at Manifest who believed that this is ultimately how all companies should run. Like, you get a job at a company and a prediction market opens up that says, in a year, will Casey Newton be more or less successful than we expect him to? And everyone in the company would bet on whether they think you will succeed at your job or not. And over time, you would essentially see who is the best at forecasting people’s performance. And you could put them in charge of your hiring process.

Casey Newton

That sounds so unbelievably stressful. [KEVIN LAUGHS] By the way, if you are really good at the Google predictions market, and so you had more googles than anyone else, and then they shut down the market and you were stuck with all these googles you couldn’t use, we want to hear from you. I think that would make for a good story.

Insider trading

Well, the Google story is really interesting. But it brings up something else I want to ask you about, which is the potential for people to manipulate these markets, right? You set up a market, and then you either have some insider information, or you just sort of try really hard to make the thing happen.

Kevin Roose

Yes, I actually saw insider trading happen. Because I was interviewing someone at this conference, and he pulled out his phone, and he showed me a prediction market that had been placed on Manifold about whether “The New York Times” would cover Manifold in an article in the year 2023. And as I was talking to him for this article, he was placing a large bet on yes on that market with his insider information, which is that a reporter from “The New York Times” was, in fact, interviewing him for an article.

Casey Newton

And how does the platform view that? Is it just sort of, well, all’s fair in love and prediction markets?

Kevin Roose

So they actually think that insider information and insider trading is good. Because people with inside information have the best information, and they can bring it to a market. They have some sort of elaborate theoretical underpinning for not believing that insider trading actually should be illegal. Right now, all of this is play money, right? Because of our gambling laws in the US, there are a couple sort of small real-money prediction markets that are very limited, and it’s not worth going into why. But most forms of real-money prediction markets are not legal in the US.

Casey Newton

Right. And at Manifold, they use something called mana as the virtual currency, which is also the same currency that you use to cast spells and Magic the Gathering and Diablo, so you’re going to want to manage your resources wisely.

Kevin Roose

Exactly. So this is this play currency called mana that you can use on the platform. They have leaderboards for who’s got the most mana. You can also convert it into charity donations. They are not allowed to pay out real money for people who are right on these gambles. And the people at this conference were upset about that. They think this should be legal. I have some concerns about that. I just don’t know what it would look like for a society to be gambling on everything all the time. But they are of the mind that the benefits of legalizing this kind of prediction market would outweigh the costs.

Skepticism of “skin in the game”

Casey Newton

You know, I got to say, Kevin, I’m of really mixed mind about this. Because on one hand, the idea of people, like, betting play money to guess what might happen seems totally innocuous. Have a good time. It seems like you had a great time at this conference, seemed like all the other people who were there did, too. But I start to hear things like, well, these folks think that insider trading should be legal, and I just start to think, keep them away from the real economy. And this whole idea that we make better decisions when we have skin in the game, I just think, has been really challenged over the past few years, right? Like, this was one of the big arguments for crypto. And crypto is the place where we used to hear all the time, you got to have skin in the game. I use — we were told for years, you can make a better social network if you have skin in the game. Right? You can develop a better relationship between musicians and fans if the fans have skin in the game. The whole idea of the Bored Ape Yacht Club was, give people skin in the game and they’ll be able to make movies. And it all just kind of came to nothing. And one of the reasons was that when you give people skin in the game and everything just has this, like, gross economic incentive tied to it, it just changes behavior. And people start, kind of, behaving in antisocial ways. So what do you think about the value of people having skin in the game? And is it possible that they’re overstating the benefits here?

Prediction markets in the New York Times?

Kevin Roose

One application of this that I actually think is kind of interesting would be, in our industry, in media — I had a conversation with the guy who runs Astral Codex Ten, and one thing that he was saying is like, if “The New York Times” put little prediction-market things at the bottom of articles, for example, that might give readers a better sense of what the probabilities behind the news events that they’re reading about are. So you could have an article about who will be the next speaker of the House, and then at the bottom of the article, you could have a little widget that sort of gave you the prediction market for someone specific or sort of an indicator of where the betting odds were on various people. And that might actually help you come to a better conclusion than just reading the article alone. Do you think that makes any sense?

Casey Newton

Yeah, I think you subscribe to “The New York Times,” and you’re given a certain amount of New York toobles, and then you sort of bet your toobles on who will be the next House speaker and — I mean, what that makes me think of is the way that polls would be gamed on Twitter in the heyday, right? People would say, like, hey, do you think this thing is going to happen? And then, it would get gamed as the most zealous partisans would stuff the ballot box until the poll was over. And I wonder what mechanisms might be put in place to prevent something like that from happening here.

But on balance, I’m persuaded that this is an interesting technology. And one thing that I have just observed in moving through Silicon Valley is that you do just constantly meet people who are into prediction markets. You know, it’s like, along with poker, these are the two preferred forms of gambling and, increasingly, ways of socializing here in our strange little corner.

Calling pundits to account

Kevin Roose

Totally. I mean, the first place I saw this take off was among AI researchers, who love to bet on, for example, what year we will get AGI or, like, when the first AI-generated screenplay will win an Oscar and things like that. And so they really are sort of running prediction models in their heads at all times. There’s this sort of cohort of people who are very into what they call Bayesian analysis or, like, attaching probabilities to things and living their lives that way. Do I think that is the way most people live their lives? Absolutely not. But it is sort of an interesting idea. And as someone who makes predictions sometimes as part of my job, it’s interesting to contemplate a world in which your position as a pundit or a columnist or a newsletter writer would be quantifiable in some way. Like, readers of “Platformer” could go in and say, OK, Casey’s predictions were 75 percent right last year, so I’m going to trust him more. But if his predictions fall to only 50 percent right next year, maybe I’ll cancel my subscription.

Casey Newton

And, like, by the way, how amazing would it be to have the pundit score that sort of said this person predicted these 50 concrete things in the past year and 12 of them happened? That feels like — that feels like the sort of information that the right person would sue to get taken off of Google.