Nuanced Models for the Influence of Information

A common fallacy I see regarding information sources is the assumption that they either have zero impact or completely determine critical decisions.

• "Our World In Data isn't directly cited by the US President, therefore it's worthless."
• "Prediction markets haven't been adopted by top elected officials, so they contribute zero value."
• "Prediction markets are already more competent than many intellectuals, and people don't listen to them. Therefore, there's no value in them being more competent. It's a bottleneck issue, and all that matters now is marketing."
• "LLMs can't match the reasoning of elites, therefore they offer nothing."
• "The moment LLM forecasters outperform human forecasting, in certain experiments, LLMs will completely take over."

This binary thinking misrepresents how information actually propagates through systems. In reality, valuable insights rarely travel directly to ultimate decision-makers. The Hollywood model  (i.e. the highly simplified model) of direct access to power oversimplifies real-world influence dynamics.

Instead, impact diffuses gradually through networks. An article might influence Sally, who shares the concept with Mark, who develops it further before passing it to Will. Information sources create ripple effects with varying degrees of influence throughout human networks - strongest at the source but still propagating outward.

This doesn't mean "systems are complex, therefore all information is equally valuable." We can still confidently predict that certain content will gain no traction. I'm very supportive of work to better model the impacts of information, but suspect that it would take some nuance.

Human influence (i.e. Power) operates similarly. Individuals exert varying degrees of impact on different nodes within large systems. Their immediate sphere of influence matters, as does the proximity of those connections to other significant actors.

Lastly, I'll recommend the metaphor of "a voice in the room." Adding another thinker to a decision-making group creates a limited but non-zero contribution. They'll speak on topics where they add value and remain silent elsewhere, producing neither zero nor total influence.

For example, I suspect that LLMs are clearly "a voice in the room" of limited impact in many key situations now, and I expect that it will gradually become a louder and louder voice in the room.

I'm unsure how best ways to visually represent "a voice in the room." One idea is something like the diagram below. ^[1] 

Appendix A: Other Plots

During this project, I experimented with ChatGPT and Claude to produce related diagrams. Below are two from Claude that I found particularly interesting. I'm eager to see similar analytical approaches applied to information relevant to our community. For instance, a tool that maps which specific journalists, authorities, and intellectuals are influenced by Metaculus forecasts, and subsequently, who those people influence. LLM agents could potentially make useful attempts at such network analyses.

To clarify, I'm not claiming this type of diagram is novel. Similar analytical approaches have a well-established history in academic literature.

1. ^{^}

    The key decision process occupies the center, with various parties positioned according to their relative influence. Additional dimensions of interest can be represented through visual attributes such as dot size, dot color, and line type. For example, I can easily picture a situation where someone wants to indicate different participants' stance or orientation on a particular topic of interest.