We need more discussion and clarity on how university groups create value

Intro

This red team has been written by four community builders^[1] who are keen to investigate their own skepticisms and uncertainties pertaining to community building itself, with a particular focus on university programming. This post was written as part of the Red Team Challenge by Training For Good.

For the sake of simplicity, we assume the article: A huge opportunity for impact: movement building at top universities offers the strongest arguments for our chosen red teaming topic. More specifically, we narrow our attention to the claim/conclusion: If you’re able to help students at top universities to devote their career to the world’s most pressing problems, this is one of the most impactful things you could do. Our aim here is not to argue against movement building at universities per se, but rather to sustain and add to the stress testing that this idea has seen recently.

Lastly, the key takeaway that we hope to communicate is that EA might be overestimating and oversimplifying the extent to which university groups can change career plans.

Note: an objection could be made at this point that: given CEA is discontinuing its focus university programming, passing funding to Open Philanthropy, there is little left to add to this discussion at the organizational level. To this, we would say that we have tried to steer this conversation towards the wider community level. Additionally, the assumptions and convictions that shaped this discontinued program could still be around and shape future projects.

Difference between HEA and high impact EA 

An important thing to note early on in this discussion is that there is a common conflation between the concepts of “a highly-engaged EA (an HEA)” and “a highly-impactful person”.

In the context of this post, we imagine a highly-impactful person as someone who is (planning on) pursuing a career where they work on problems which have a high EV, such as working on AI alignment research, biosecurity, global health and wellbeing, animal welfare, or other recognised EA cause areas. It is possible that what it takes to produce an HEA is different to what it takes to generate a highly-impactful person. 

The reason this distinction is important to note is because we think it is crucially unclear whether university groups (and university group organizers) think they should be “producing” HEAs or high-impact individuals - the point being that we think it is likely true that they are mutually exclusive categories. If they are mutually exclusive, it is possible that creating an HEA has orders of magnitude difference in impact than creating a highly-impactful person (or vise versa^[2]), and that we would therefore be making orders of magnitude worse funding decisions if one was true over the other if we allocated equal funding in both scenarios. 

The value of a high-impact EA

The first reason we believe that we might be overestimating and oversimplifying the extent to which university groups can change career plans is because putting a dollar value on such a change in career plans is far from clear.

There have been a number of attempts to estimate the expected value of someone working in a high-impact, EA field over their career. Such a number is difficult to pin down due to general uncertainties, as well as the variation that exists between people regarding time, efficiency, role, cause area and various other context specific details such as educational experience and social capital. A place to start would be the general rate for a standard CEA contractor, which is currently at around $38/hour. This covers roles like events organizing and logistics, and it’s reasonable to assume that a more specialized EA working in a higher impact role would be paid significantly more, but it is difficult to pin down an exact figure.

While looking at salary indicates that EA values engaged EAs significantly higher than they value an average role (over 5 times the federal minimum wage in America), there are of course reservations about using salary as a gauge for the impact of a role. Salaries may not be an accurate representation of the impact being generated through a specific role, as pay is limited by the amount of funding that EA has access to as well as individual factors like propensity to negotiate and country of residence. Additionally, salaries will likely not be commensurate with impact generated as they are generally a compensation for one’s labor and expertise more so than one’s impact.

It seems salaries are not the most useful starting point for estimating the value of a HEA, so let’s leave compensation out of the picture and instead look at monetary estimates that try to quantify impact. Here, the range increases by several orders of magnitude. We cannot always be certain how the authors of such estimates intended them to be interpreted. For the purposes of this post we use the most neutrally phrased definition we could find, inspired by the top comment of the a dollar of value post: "$X of value" means we value something as much as EA as a whole having access to $X of extra funding.

This effectively means we should be willing to spend up to $X to generate the desired outcome. Along this definition, an estimate from 80,000 Hours states that someone working in a high impact role can generate millions of dollars per year, and CEA Executive Director Max Dalton pins it at tens of millions of dollars, though a timescale is not specified.

There are a few concerns that arise when looking at this method of analyzing the impact of EAs, which is not formalized and has not been widely corroborated or well researched. The most obvious concern would be that we are currently spending a lot of money (~tens of millions of dollars per year according to OpenPhil reports) and hedging a lot of bets on the idea that spending very heavily on community building will eventually yield millions or tens of millions of dollars worth of impact over the course of one’s career without much publicly available and peer reviewed data which backs this claim.

The value of a role in community building at universities

The value of a role in community building is another crux in the argument for movement building at top universities. The value of a community builder is dependent on not just the value of highly-engaged EAs as discussed above, but also on how many of these are counterfactually created.

Although they are put forward with epistemic humility, current estimates involve huge numbers, such as 80,000 Hours estimating that doubling the current spending of around $10-100 million on community building would take us to around 10% of the way toward seeing the full potential of EA realized, which they estimate at saving 100-1,000 million QALYS annually or reducing x-risk by 1-10%. CEA argues university groups are among the best ways to achieve this.

The main line of reasoning that is used to quantify the impact of a community builder at a (top) university seems to boil down to the multiplier argument - if you can convince others to dedicate their careers or other resources to doing good, you’ve multiplied your impact! CEA expected campus specialists to create roughly 10 people annually that go on to have high impact careers which, if you take the millions of dollars per career estimate of 80,000 Hours seriously, brings the annual value of a campus specialist at a top university into the tens of millions of dollars range.

Additionally, as the post itself emphasizes the estimates are not adjusted for counterfactuals, which could reduce them 'several-fold'. In this regard the high estimates might be a double-edged sword. If the counterfactual to someone's community building efforts would have been that the efforts of different individuals or organizations would have done a better job, the sign flips and the estimated value could quickly fall well below zero, some examples of which were recently discussed in the Bad Omens in Current Community Building post.

Finally, it is still unclear to what extent this value can be attributed to the university group. From mentors to internships or the general EA infrastructure at large, there are plenty of possible alternative explanations to counterfactually attribute the value of creating an EA with a high impact career to. For these reasons, the use of astronomical guesstimates to inform or justify how to approach or what to prioritize in movement building at universities should be much more rigorously scrutinized and justified by those encouraging or funding it.

To be clear, we are not here to rule out the possibility of either an impactful career or a role in community building at top universities being valued at tens of millions of dollars, or to say that it is not worth spending generously on univeristy community building. The 2020 surveys from Open Philanthropy and Rethink Priorities showed that EAs self-report university groups to play a significant role in their path to impact. On top of that, even if the value of university community building spending is even a fraction of the various EV estimates in the millions, it would still be worth pursuing. Rather, we are here to claim that the epistemic standards applied to estimating the value of a highly impactful career and roles in community building at (top) universities are not rigorous enough. Not to mention, even if the careers creating tens of millions of dollars are out there, it is plausible that it could come across as jarring and off-putting for university groups to try and target these individuals. Therefore, encouraging an approach for university groups that focuses on identifying specific, high-potential students without solid reasoning transparency might be overly simplistic or lacking the rigorous evidence we tend to expect.

Risks and reframings

Having looked at the empirical assumptions lying behind impactful careers and movement building, this third and final section takes a practical turn to look at the very means of identifying talent and whether the abilities of university groups are a good fit to the task. Put differently, the aim is to reframe how we think about talent search. 

First, to paint a backdrop, it is useful to point out where a lot of the motivation for writing this section is coming from.

Consider the context that effective altruism is becoming and will continue to become a whole lot more famous. What would the media come and see if they decided to research how the movement viewed its own practices for increasing publicity and finding new members? What issues might they have with how members of the movement view the practice of building the movement - the value they put on the impact of the practice and the goals of the practice itself?

It wouldn’t take a competent journalist very long to do some research, take quotes out of context and say something like “This movement thinks recruiting new members could be worth millions to the world”.

Hopefully the previous two sections of this post show that there is a lot of nuance in these numbers and that even though they could still have merit, we should be aware of the possible risks associated with publicly putting these values on certain members of our community. 

As a quick aside, it is worth noting the slight tension here in case we sound like we are contradicting our earlier point that argued for greater transparency. On one hand we hope this article highlights a need for more reasoning transparency around valuing career changes to roles with more impact and the community building that influences them. On the other hand, even with transparency and well-reasoned estimates for these values, an outsider looking in might quickly come to the conclusion that our movement is self-aggrandizing given the size of the estimated value of individuals is often into the hundreds of thousands and millions. Yet these risks are not necessarily unforeseen, as a self-aggrandizing objection could possibly be made on the same grounds as elitism or cold, calculating, and impersonal. 

But putting these numbers aside, the real question, again, is to what extent this value can be counterfactually created by the actions of university groups. When it comes to the idea that university groups are well-positioned for talent search, we think this could be false or misguided. Namely, if university groups have the goal of finding highly-engaged EAs, we do not think that it can then also be inferred that university groups can get someone’s career off to an impactful start. Put more clearly, we are not saying that university groups are ineffective at achieving the goal of finding highly engaged EAs. We are saying that university groups are unlikely to be able to help highly engaged EAs enter high impact careers once they have been identified.

And the reason we say this is because we believe finding highly engaged EAs requires selecting for value-alignment whereas helping highly engaged EAs enter high impact careers requires selecting for talent and competency. Now, we are not the first to be making this point, so consider the recent forum post Bad Omens in Current Community Building and the following two excerpts:

• If you’re high-status within EA, think carefully about how you react to community builders who seem to create lots of HEAs. Don’t just praise them, but also coach them and monitor them. The more HEAs created, the more you should be suspicious of Goodharting (despite the best of intentions), so work together to avoid it. (see Part 4)
• One issue I've always had with the "highly engaged EA" metric is that it's only a measure for alignment, but the people who are most impactful within EA have both high alignment and high competence. If your recruitment selects only on alignment this suggests we're at best neutral to competence and at worst (as this post describes) actively selecting against competence. (see comments)

It is for much the same reasons that we argue the journey of someone successfully taking on new EA/EA-adjacent values and career plans while at university is a journey that is quite different to finding the right role with the highest degree of personal fit. And this is in part due to the task of creating “highly-engaged EAs” seeming much more tractable than “recruiting” specific, high-potential students. Selecting for alignment and competence at the same time is hard because the criteria are rarely the same. 

Alignment entails things like: an interest in the ideas, a commitment to attending events, reading, listening to, and watching relevant content. Whereas competence entails skill bottlenecks - insights, leadership, coordination - which university groups might have difficulty in helping to address because they differ from one organization to the next. Trying to select for both is not impossible, but we do think it means university groups should be careful about how confident they are in selecting for both successfully.

Rounding off this point of engagement seeming much more tractable than impact in the context of movement building at university, consider the following hypothetical example. Charlotte is a student at university and over the course of the last 18 months of her degree, she gradually becomes more engaged with effective altruism - she completes a fellowship, attends events, and self-starts a part-time research project into alternative proteins. She says the university group helped her a lot in making the most of these opportunities. Charlotte plans to take a gap year to work and travel once she finishes university and after that hopes to join a graduate program outside of EA to build career capital and test out her fit for different types of work. The timelines here for Charlotte’s engagement and impact differ greatly, which makes it harder to track her university group’s influence and determine if it is affecting her impact to an extent similar to her engagement. 

To avoid overlapping too much with what has already been said on the forum, allow us to put forward some concrete suggestions:

• At a community level, we should temper the strength of our language when it comes to describing the opportunity for impact in the domain of movement building in general, and at top universities in particular.
  • As an example, instead of saying “movement building is one of the impactful things you could do”, consider instead “movement building is a great opportunity to ...
    • build an aptitude for identifying talent
    • experiment with your own ideas for community building
    • upskill in areas that are currently talent-constrained within EA
      • figuring out what matters most and set the right priorities
      • entrepreneurship
• Let’s do what Daniel Gross calls “jumping into the Dunning-Kruger abyss” of talent search. Instead of assuming that we can legibly identify talent, let’s instead assume that we are, at best, neutral, and that we could be so much better through investing more resources into the very question: how can we identify hidden talent?
• Such investments and resources could come from having university groups collaborate more extensively with EA-adjacent institutions like Emergent Ventures and Schmidt Futures who are tackling talent bottlenecks in a way that tries to best select for competence.
• Better defining and measuring the impact of the work done by university groups
  • With a similarly well-intentioned call for reasoning transparency as in this post, we would encourage funders of university groups to provide some quantitative explanation for their decisions to allocate large amounts of money to university groups. One of the things we hope to have brought to attention in this post is the lack of real public writing on these decisions, which is inconsistent with most of the grantmaking made by EA funders.
    • This might include some deep investigation into the differences in EV for HEAs vs highly-impactful people, and therefore which one of them university groups should be aiming at “generating”.
  • The Open Phil EA/LT Survey 2020 made a good start on allowing respondents to self-report how much a group’s influence affected them, but the impact point metric could be improved.
    • One way would be to anchor the different numerical values with examples
      • “Helped me a lot” = “found me a new job”
      • “Slightly helped me” = “introduced me to a potential employer”

Conclusion

As pointed out at the beginning of this article, this has been written by four community builders keen to investigate their own skepticisms and uncertainties. To that end, we want to stress that the kernel of our red team has more to do with trying to challenge and improve the theory of change used for community building at universities. We are not trying to get any extant activities and projects put on by organizations in the community building space to be discontinued - we are in fact very excited and supportive of many things happening in EA university groups. But we do hope to see new approaches for estimating and communicating values of high impact careers and community building roles be developed and improved. 

Finally, we want to acknowledge the numerous reviewers whose feedback really helped us to specify and find the core thrusts of this post - thank you for your comments.

1. ^{^}

   Kaleem: Founded and ran a university group at Northeastern University, and then worked full-time on community building in Boston on an EAIF after graduating. He has been a UGAP mentor to two groups, and an unofficial mentor to around a dozen more. He was also an organizer of EAGx Boston.

   Jelle: Co-founded the university group in Utrecht in early 2022 and currently founding groups at the universities of Eindhoven and Tilburg in the Netherlands. 

   Jennifer: organizes a university group too.

   Oscar: volunteers with Giving What We Can and High Impact Professionals, as well as facilitating for the Introductory EA Program.

2. ^{^}

    Although this seems unlikely, given it would mean that being a highly-engaged EA makes you, on expectation, even more highly impactful than a highly-impactful person.