Various people are sceptical of epistemic interventions to reduce power concentration.

Here is a half-hour note sketching out what I think the bull case here is:^[1]

• Base rates suggest this will matter. Historically, control of the information environment has played a significant role in backsliding^[2] and coups^[3]. So the starting expectation should be that it’s going to be significant

• There are reasons to update the base rates upwards:       
  • AI will probably significantly raise the ceiling on how good epistemics can get, which should make this a more important lever than historically
  • I think it’s likely that good and bad epistemics will be self-reinforcing, more so with AI than they have been historically. Which should also make this more important than previously

• ‘Everyone correctly understands their own interests’ effectively blocks several important ways to backslide^[4]

  • Makes it hard to get a plebiscitary majority or a legislative majority (unless power concentration is actually in the interests of the majority)

    • Still possible for the executive to be so disproportionately powerful that it can just present things as a fait accompli in spite of unpopularity - but then there’s a question of why people have failed to anticipate this and coordinate against it

    • Still possible for the executive to collude with powerful private actors against the public interest

• ‘Everyone correctly understands their own interests’ empowers other branches of government to resist power concentration
  • The courts and the legislature are responsive to public opinion
  • E.g. if everyone could see that e.g. a better funded Congress was in their interests, then it’s much more likely that Congress would use its legislative powers to vote itself more funding
  • E.g. if everyone could see that some executive action was power-seeking, then it’s much more likely that the courts stand up to that
• ‘Everyone correctly understands their own interests’ probably isn’t actually necessary
  • Some kinds of epistemic uplift you only get the benefits if everyone is uplifted. E.g.
    • Lots of the societal benefits of provenance tracing, rhetoric highlighting, reliability tracking come from widespread trust in and use of those tools
    • Some of the paths to impact from coordination tech involve most of civil society coordinating using AI tools
  • But other important kinds you only need some important people to be uplifted. E.g.
    • Automated OSINT for journalists
    • If only academics/lawyers/economists were using things like provenance tracing, rhetoric highlighting, reliability tracking, that could still enable them to use their political and advisory capital to much greater effect
    • Similarly, automated superforecasting and scenario planning could help if only adopted by influential elites (though there are also dual use concerns here)
    • Angels-on-the-shoulder type tools like reflection scaffolding and guardian angels could be extremely consequential if adopted by a handful of key decision-makers
• Epistemic interventions are unusually tractable
  • You don’t need legislation, you can just go and build stuff
  • Adoption is hard, but as above, there are important cases where narrow adoption is sufficient
• Epistemic interventions are approximately uncorrelated with other intervention classes, so they make the overall portfolio more robust
  • Lots of interventions route through a) government or b) lab policy. These are super important, but also all or a)/b) might fail if the environment for that kind of policy remains/becomes unfavourable

Thanks to Ben Stewart and John Bridge for scepticism that prompted me to write this and useful comments; and to Owen Cotton-Barratt, Abra Ganz and Oly Sourbut for comments. I haven't edited the text in light of comments, but sharing as is because maybe it will prompt more useful discussion.

1. ^{^}

    NB I’m leaning into the bull case in the expectation that others will represent the bear case, rather than trying to reach an all-things-considered take in this doc.

2. ^{^}

    Boese et al find that freedom of expression and civil society freedoms are usually the first things to go (though other scholars point to different ordering:

   • Levitsky and Ziblatt, How Democracies Die (2018): capturing the referees (like courts and law enforcement), then hampering the opposition (through bribery, corruption and lawfare), then changing the rules (the constitution and the electoral system)
   • Sato et al: undermining horizontal accountability (separation of powers), then diagonal accountability (media freedom and freedom of speech), then vertical accountability (elections)
3. ^{^}

    Singh argues that coups are basically coordination games, and control of the information environment is key.

^{^}
 Ways to backslide, from Riedl et al:
• Legislative capture, when there’s a strong party. Examples: India, Turkey, Hungary
• Plebiscitary overrides, when there’s a populist president. Examples: Venezuela
• Executive powergrabs, when opposition parties and institutions are weak. Examples: Tunisia, Brazil
• Elite collusion, when parties and civil society are weak and state capacity is low. Examples: Indonesia, Guatemala
Show all footnotes