Take the 2025 EA Forum Survey to help inform our strategy and prioritiesTake the survey

It seems that everyone who looks into information hazards eventually develops their own typology. This isn't so surprising; Bostrom's original paper lists about thirty different categories of information hazards and isn't even completely exhaustive, and those who wish to work with a simpler system have many ways of dividing up that space.

In general, I've tended to use a typology adapted from Anders Sandberg, with some tweaks to fit it better into my own brain. This typology divides the majority of infohazards into three broad types[1]:

  1. Capability hazards: Information that gives other actors new or improved abilities to harm you or your values[2], either by acting in ways you wish they would not, or by threatening to do so to extort actions from you. Examples: Instructions for building nuclear weapons; incriminating information suitable for blackmail; most biosecurity infohazards; that master key hack.

  2. Direct hazards[3]: Information that directly harms the possessor, either through infliction of suffering[4] or by otherwise reducing their ability to achieve their goals. Examples: News that a loved one has died unpleasantly; political news you can do nothing about but find very distracting; the examples from this post; a certain well-known example from the rationality community.

  3. Mindset hazards: Information that, while true, interacts with actors' other beliefs or biases in a way that motivates them to act badly – either by acting rationally based on beliefs or values you hold to be false, or by acting irrationally according to their own beliefs and values. Examples: Alleged examples of this class of hazard are common but tend to be very controversial[5]. Some frequently-claimed examples include the heritability of desirable cognitive traits; the generally low efficiency of HIV transmission; the fact that such-and-such public figure you support once said such-and-such controversial thing.

Of course, these three categories all bleed into each other at the edges. The boundary between a direct hazard (that hurts the knower) and a mindset hazard (that hurts others via the actions of the knower) is especially fuzzy; information that makes someone act irrationally is likely to be both. Some direct and mindset hazards are also capability hazards, in that an enemy or careless actor in possession of the information gains the capability to hurt you by sharing it with you or your allies (or threatening to do so). And the distinction between "this information might enable someone to do harm" (capability hazard) and "this information might motivate someone to do harm" (mindset hazard) isn't always terribly crisp.

In general, though, most real-world cases I've seen seem to fall fairly naturally into one of these three categories, with the other two making relatively minor contributions. That is to say, in most cases where someone is worried about some piece of true information being hazardous, they seem to mostly be worried about one of these three kinds of harm.

I like this typology because it's very simple and memorable, while focusing attention on the central question of how a given piece of information might potentially do harm[6]. Hopefully it will be of some use to some of you, as well.


  1. Note that, as with all infohazards, these categories only apply to true information. The harms of spreading false information are generally not controversial, or especially interesting. ↩︎

  2. Note that this definition is not limited to malicious actors. Information that provides careless or incompetent actors with new capacities to cause accidental harm also falls under this category. ↩︎

  3. Also known as memetic hazards or cognitohazards, though both of these sound more esoteric than I'd ideally like. ↩︎

  4. If there morally-(dis)valuable mental states other than happiness/suffering, then information that damages the good states or induces the bad ones would also count as a direct information hazard. ↩︎

  5. If you claim a piece of information is a mindset hazard, you are implicitly claiming that (a) the new information is true, and (b) the actions prompted by communication of the information are bad. Naturally, the actor themselves will tend to strongly contest (b), while those who agree with (b) will often strongly contest (a). ↩︎

  6. Having established this typology, I think there's quite a lot you can say about how these three categories of infohazards tend to differ from one another: in breadth, in severity, in available strategies for mitigation, in exploitability by bad actors, et cetera. More on that in a future post. ↩︎

Comments3


Sorted by Click to highlight new comments since:

You credit Anders Sandberg here and elsewhere, but you don't provide a reference. Where did Sandberg propose the typology that inspired yours? A search for 'direct information hazard' (the expression you attribute to Sandberg) only results in this post and your LW comment.

A mixture of conversations and shared Google Docs. Nothing publicly citable as far as I know.

It seems that Sandberg is discussing something like this typology in https://www.youtube.com/watch?v=Wn2vgQGNI_c

Edit: Sandberg starts talking about three categories of hazards at ~12:00

Curated and popular this week
 ·  · 6m read
 · 
I am writing this to reflect on my experience interning with the Fish Welfare Initiative, and to provide my thoughts on why more students looking to build EA experience should do something similar.  Back in October, I cold-emailed the Fish Welfare Initiative (FWI) with my resume and a short cover letter expressing interest in an unpaid in-person internship in the summer of 2025. I figured I had a better chance of getting an internship by building my own door than competing with hundreds of others to squeeze through an existing door, and the opportunity to travel to India carried strong appeal. Haven, the Executive Director of FWI, set up a call with me that mostly consisted of him listing all the challenges of living in rural India — 110° F temperatures, electricity outages, lack of entertainment… When I didn’t seem deterred, he offered me an internship.  I stayed with FWI for one month. By rotating through the different teams, I completed a wide range of tasks:  * Made ~20 visits to fish farms * Wrote a recommendation on next steps for FWI’s stunning project * Conducted data analysis in Python on the efficacy of the Alliance for Responsible Aquaculture’s corrective actions * Received training in water quality testing methods * Created charts in Tableau for a webinar presentation * Brainstormed and implemented office improvements  I wasn’t able to drive myself around in India, so I rode on the back of a coworker’s motorbike to commute. FWI provided me with my own bedroom in a company-owned flat. Sometimes Haven and I would cook together at the residence, talking for hours over a chopping board and our metal plates about war, family, or effective altruism. Other times I would eat at restaurants or street food booths with my Indian coworkers. Excluding flights, I spent less than $100 USD in total. I covered all costs, including international transportation, through the Summer in South Asia Fellowship, which provides funding for University of Michigan under
 ·  · 1m read
 · 
This is a personal essay about my failed attempt to convince effective altruists to become socialists. I started as a convinced socialist who thought EA ignored the 'root causes' of poverty by focusing on charity instead of structural change. After studying sociology and economics to build a rigorous case for socialism, the project completely backfired as I realized my political beliefs were largely psychological coping mechanisms. Here are the key points: * Understanding the "root cause" of a problem doesn't necessarily lead to better solutions - Even if capitalism causes poverty, understanding "dynamics of capitalism" won't necessarily help you solve it * Abstract sociological theories are mostly obscurantist bullshit - Academic sociology suffers from either unrealistic mathematical models or vague, unfalsifiable claims that don't help you understand or change the world * The world is better understood as misaligned incentives rather than coordinated oppression - Most social problems stem from coordination failures and competing interests, not a capitalist class conspiring against everyone else * Individual variation undermines class-based politics - People within the same "class" have wildly different cognitive traits, interests, and beliefs, making collective action nearly impossible * Political beliefs serve important psychological functions - They help us cope with personal limitations and maintain self-esteem, often at the expense of accuracy * Evolution shaped us for competition, not truth - Our brains prioritize survival, status, and reproduction over understanding reality or being happy * Marx's insights, properly applied, undermine the Marxist political project - His theory of ideological formation aligns with evolutionary psychology, but when applied to individuals rather than classes, it explains why the working class will not overthrow capitalism. In terms of ideas, I don’t think there’s anything too groundbreaking in this essay. A lot of the
 ·  · 1m read
 · 
I’m a long-time GiveWell donor and an ethical vegan. In a recent GiveWell podcast on livelihoods programs, providing animals as “productive assets” was mentioned as a possible program type. After reaching out to GiveWell directly to voice my objection, I was informed that because GiveWell’s moral weights currently don’t include nonhuman animals, animal-based aid is not categorically off the table if it surpasses their cost-effectiveness bar. Older posts on the GiveWell website similarly do not rule out animal donations from an ethical lens. In response to some of the rationale GiveWell shared with me, I also want to proactively address a core ethical distinction: * Animal-aid programs involve certain, programmatic harm to animals (breeding, confinement, separation of families, slaughter). * Human-health programs like malaria prevention have, at most, indirect and uncertain effects on animal consumption (by saving human lives), which can change over time (e.g., cultural shifts, plant-based/cultivated options). Constructive ask to GiveWell: Until you have publicly considered how to incorporate animal welfare into your moral weights, please avoid funding programs that use animals as aid. I share this with respect for GiveWell’s impact and to help animal rights-aligned donors make informed choices. If I’ve misunderstood anything, I’m happy to be corrected.