Hide table of contents

A huge thanks to @Agustín Covarrubias 🔸  for his feedback and support on the following article:


Shapley values are an extremely popular tool in both economics and explainable AI.

In this article, we use the concept of “synergy” to build intuition for why Shapley values are fair. There are four unique properties to Shapley values, and all of them can be justified visually. Let’s dive in!

A figure from Bloch et al., 2021 using the Python package SHAP
A figure from Bloch et al., 2021 using the Python package SHAP

The Game

On a sunny summer day, you and your two best friends decide to run a lemonade stand. Everyone contributes something special: Emma shares her family’s secret recipe, Liam finds premium-quality sugar, and you draw colorful posters.

The stand is a big hit! The group ends up making $280. But how best to split the profits? Each person contributed in a different way, and the success was clearly due to teamwork…

Luckily, Emma has a time machine. She goes back in time — redoing the day with different combinations of team members and recording the profits. This is how each simulation went:

Who was involved?Total profits
Only Emma$20
Only you$30
Only Liam$10
Emma and you$90
You and Liam$100
Liam and Emma$30
Emma, Liam, and you$280

Individually, Emma makes 20 dollars running the lemonade stand, and you make 30 dollars. But working together, the team makes 90 dollars.

The sum of individual profits is 20 + 30 = 50 dollars, which is clearly less than 90 dollars. That extra 90 - 50 = 40 dollars can be attributed to team dynamics. In game theory, this bonus is called the “synergy” of you and Emma. Let’s visualize our scenario as a Venn diagram.

Note: Synergies can also be negative (e.g., if Liam and Emma fought it could hurt profits).

The synergy bonuses in the Venn diagram are “unlocked” when the intersecting people are part of the team. To calculate total profit, we add up all areas relevant to that team.

For example, when the team consists of just you and Liam, three portions of the Venn diagram are unlocked: the area exclusive to you (30 dollars), the area exclusive to Liam (10 dollars), and the area exclusively shared by you and Liam (60 dollars). Adding these areas together, the total profit for team “You and Liam” comes out to 30 + 10 + 60 = 100 dollars.

Referring to our Venn diagram, the same formula holds true for every other team:

Team membersSum of synergiesTotal profits
Emma$20$20
You$30$30
Liam$10$10
Emma, You$20 + $30 + $40$90
You, Liam$30 + $10 + $60$100
Liam, Emma$10 + $20 + $0$30
Emma, You, Liam$20 + $30 + $10 + $40 + $60 + $0 + $120$280

Emma and Liam are impatient and want their fair share of money. They turn to you, the quick-witted leader for help. While staring at the Venn diagram, an idea strikes!

Take a moment to look over the visual. How would you slice up the Venn diagram fairly? Pause here, and continue when ready.


You decide to take each “synergy bonus” and cut it evenly among those involved.

Doing the math, each person’s share comes out to:

Emma and Liam agree the splits are fair. The money is handed out, and everyone skips happily home to dinner.

In this story, the final payouts are the Shapley values of each team member. This intuition is all you need to understand Shapley values. For the adventurous reader, we now tie things back to formal game theory.


The Formalities

Shapley values are a concept from cooperative game theory. You, Liam, and Emma are all considered “players” in a “coalition game”. Every possible “coalition” (or team) has a certain “payoff” (or profit). The mapping between coalition and payoff (a.k.a. which just corresponds to our first table of profits) is called the “characteristic function” (as it defines the nature, or *character*, of the game).

We define a set of players  (which, in this case, is You, Emma, and Liam), and a characteristic function , where :

We can see how this is the same mapping we had in our table of profits by players:

Who was involved?Total profits
Only Emma$20
Only you$30
Only Liam$10
Emma and you$90
You and Liam$100
Liam and Emma$30
Emma, Liam, and you$280

We also define a synergy function labeled  where :

Similarly, the synergy function just corresponds to areas of the Venn diagram:

https://miro.medium.com/v2/resize:fit:4800/format:webp/1*QilJDt6fx0PcgpSnLks_fw.png

Thus, for a given player , the Shapley value is written as:

Which, in more compact notation, becomes:

The last is exactly the formula described on Wikipedia.


Concluding Notes

Shapley values are the ONLY way to guarantee:

  1. Efficiency — The sum of Shapley values adds up to the total payoff for the full group (in our case, $280).
  2. Symmetry — If two players interact identically with the rest of the group, their Shapley values are equal.
  3. Linearity — If the group runs a lemonade stand on two different days (with different team dynamics on each day), a player’s Shapley value is the sum of their payouts from each day.
  4. Null player — If a player contributes nothing on their own and never affects group dynamics, their Shapley value is 0.

Take a moment to justify these properties visually.

No matter what game you play and who you play with, Shapley values always preserve these natural properties of “fairness”.

Hopefully, you have gained some intuition for why Shapley values are “fair” and why they account for interactions among players. Proofs and more rigorous definitions can be found on Wikipedia.

Thanks for reading! :)

Comments4


Sorted by Click to highlight new comments since:

Thanks for posting this.

Do we have an intuition for how to apply shapely values in typical EA scenarios, for example: • How much credit goes to donors, vs charity evaluators, vs object level charities? • How much credit goes to charity founders/executives, vs other employees/contractors? • How much credit goes to meta vs object organizations?

Seconding this question, and wanted to ask more broadly: 

A big component/assumption of the example given is that we can "re-run" simulations of the world in which different combinations of actors were present to contribute, but this seems hard in practice. Do you know of any examples where Shapley values have been used in the "real world" and how they've tackled this question of how to evaluate counterfactual worlds?

(Also, great post! I've been meaning to learn about Shapley values for a while, and this intuitive example has proven very helpful!)

Thanks for cross posting, this got Shapley values to "click" for me!

Executive summary: Shapley values, which determine fair allocation of rewards in cooperative games, can be intuitively understood using Venn diagrams to visualize how individual contributions and synergies between players combine to create total value.

Key points:

  1. Shapley values calculate fair payouts by dividing both individual contributions and "synergy bonuses" (additional value from cooperation) among relevant team members
  2. Using a lemonade stand example, the method shows how to split $280 total profit among three contributors based on their individual and combined performances
  3. Venn diagrams provide an intuitive visualization where overlapping regions represent synergies between specific team combinations
  4. The approach guarantees four key properties: efficiency (payouts sum to total), symmetry (equal contributors get equal shares), linearity (additive across games), and null player (no contribution means no payout)
  5. This framework has practical applications in both economics and explainable AI for understanding and attributing value fairly

 

 

This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, and contact us if you have feedback.

Curated and popular this week
 ·  · 16m read
 · 
This is a crosspost for The Case for Insect Consciousness by Bob Fischer, which was originally published on Asterisk in January 2025. [Subtitle.] The evidence that insects feel pain is mounting, however we approach the issue. For years, I was on the fence about the possibility of insects feeling pain — sometimes, I defended the hypothesis;[1] more often, I argued against it.[2] Then, in 2021, I started working on the puzzle of how to compare pain intensity across species. If a human and a pig are suffering as much as each one can, are they suffering the same amount? Or is the human’s pain worse? When my colleagues and I looked at several species, investigating both the probability of pain and its relative intensity,[3] we found something unexpected: on both scores, insects aren’t that different from many other animals.  Around the same time, I started working with an entomologist with a background in neuroscience. She helped me appreciate the weaknesses of the arguments against insect pain. (For instance, people make a big deal of stories about praying mantises mating while being eaten; they ignore how often male mantises fight fiercely to avoid being devoured.) The more I studied the science of sentience, the less confident I became about any theory that would let us rule insect sentience out.  I’m a philosopher, and philosophers pride themselves on following arguments wherever they lead. But we all have our limits, and I worry, quite sincerely, that I’ve been too willing to give insects the benefit of the doubt. I’ve been troubled by what we do to farmed animals for my entire adult life, whereas it’s hard to feel much for flies. Still, I find the argument for insect pain persuasive enough to devote a lot of my time to insect welfare research. In brief, the apparent evidence for the capacity of insects to feel pain is uncomfortably strong.[4] We could dismiss it if we had a consensus-commanding theory of sentience that explained why the apparent evidence is ir
 ·  · 40m read
 · 
I am Jason Green-Lowe, the executive director of the Center for AI Policy (CAIP). Our mission is to directly convince Congress to pass strong AI safety legislation. As I explain in some detail in this post, I think our organization has been doing extremely important work, and that we’ve been doing well at it. Unfortunately, we have been unable to get funding from traditional donors to continue our operations. If we don’t get more funding in the next 30 days, we will have to shut down, which will damage our relationships with Congress and make it harder for future advocates to get traction on AI governance. In this post, I explain what we’ve been doing, why I think it’s valuable, and how your donations could help.  This is the first post in what I expect will be a 3-part series. The first post focuses on CAIP’s particular need for funding. The second post will lay out a more general case for why effective altruists and others who worry about AI safety should spend more money on advocacy and less money on research – even if you don’t think my organization in particular deserves any more funding, you might be convinced that it’s a priority to make sure other advocates get more funding. The third post will take a look at some institutional problems that might be part of why our movement has been systematically underfunding advocacy and offer suggestions about how to correct those problems. OUR MISSION AND STRATEGY The Center for AI Policy’s mission is to directly and openly urge the US Congress to pass strong AI safety legislation. By “strong AI safety legislation,” we mean laws that will significantly change AI developers’ incentives and make them less likely to develop or deploy extremely dangerous AI models. The particular dangers we are most worried about are (a) bioweapons, (b) intelligence explosions, and (c) gradual disempowerment. Most AI models do not significantly increase these risks, and so we advocate for narrowly-targeted laws that would focus their att
 ·  · 1m read
 · 
Disclaimer: Post written in a personal capacity. These are personal opinions and do not in any way represent my employer's views TL;DR: * I do not think we will produce high reliability methods to evaluate or monitor the safety of superintelligent systems via current research paradigms, with interpretability or otherwise. * Interpretability still seems a valuable tool and remains worth investing in, as it will hopefully increase the reliability we can achieve. * However, interpretability should be viewed as part of an overall portfolio of defences: a layer in a defence-in-depth strategy * It is not the one thing that will save us, and it still won’t be enough for high reliability. EDIT: This post was originally motivated by refuting the claim "interpretability is the only reliable path forward for detecting deception in advanced AI", but on closer reading this is a stronger claim than Dario's post explicitly makes. I stand by the actual contents of the post, but have edited the framing a bit, and also emphasised that I used to hold the position I am now critiquing, apologies for the mistake Introduction There’s a common argument made in AI safety discussions: it is important to work on interpretability research because it is a realistic path to high reliability safeguards on powerful systems - e.g. as argued in Dario Amodei’s recent “The Urgency of Interpretability”.[1] Sometimes an even stronger argument is made, that interpretability is the only realistic path to highly reliable safeguards - I used to believe both of these arguments myself. I now disagree with these arguments. The conceptual reasoning is simple and compelling: a sufficiently sophisticated deceptive AI can say whatever we want to hear, perfectly mimicking aligned behavior externally. But faking its internal cognitive processes – its "thoughts" – seems much harder. Therefore, goes the argument, we must rely on interpretability to truly know if an AI is aligned. I am concerned this line of