Comments
Funding for talent-focused organisations (TFOs) in EA remains highly concentrated, with more than 80% coming from Open Philanthropy and EA Funds. To understand whether clearer impact reporting could unlock new funding for the space, this pre-study interviewed seven funders.
While historical impact data isn't the main decision criterion for most funders, an improved impact reporting infrastructure could potentially open up more funding to the talent space. In addition, a shared reporting framework could support funders in making more informed decisions, and provide clearer funder expectations for the TFOs.
We suggest three next steps: (1) develop standardised indicators with input from both funders and TFOs, (2) gather TFO perspectives to complement this funder-focused study, and (3) explore the value of an independent evaluation agency for the talent space.
We define a TFO as any organisation whose explicit goal is to help people increase the impact they have through their careers. Some examples are meta EA groups like EA Sweden or EA Netherlands, and coaching or training organisations like 80k, SuccessIf, and BlueDot Impact.
The 2024 EA Meta Funding Landscape Report found that 80% of funding in the talent field is coming from Open Philanthropy and EA Funds, with the share being even higher when only talent-focused organisations (TFOs) are included. In a memo for the Meta Coordination Forum 2024, Patrick Gruban argues that the lack of legibility in TFO's impact reporting might be a hurdle for new funders to enter the field, and that some impactful TFOs are underfunded due to this.
In the fall of 2025, Emil Wasteson Wallén led a pre-study to investigate this assumption and whether a shared impact reporting framework for TFOs could increase the total funding to the talent space.
Below, we summarise the findings and outline three potential interventions for taking this project forward.
Seven "funders" - individual donors, grantmakers, and philanthropic advisors - were interviewed to understand the funders' perspective. Two were established funders within the talent space. Three others had made, or seriously considered making, grants in the talent space at least once. The remaining two had no direct experience in the talent space but were active funders in other parts of the EA meta ecosystem (primarily effective giving), making them particularly interesting as potential future funders to the talent space.
The interviews aimed to explore:
Three funders reported using historical impact data as a key criterion for decision-making. But even for them, it wasn't the main decision criterion. A pattern was that larger and more established funders placed greater weight on historical data.
The criteria that most funders highlighted as most decisive in their decisions were:
Naturally, the younger a project or organisation is, the less relevant historical impact data becomes relative to these other factors.
Finally, four funders mentioned that Open Philanthropy’s funding recommendations and decisions significantly influenced their own.
Of the five funders who were not already established in the talent space, one noted that uncertainty about an organisation’s impact had delayed their decision by nearly ten months, and that clearer impact reporting could have accelerated the process.
The remaining four did not view the lack of robust historical impact data as a major gap in their decision-making. However, three of them said they would likely take such data into account if it were easily accessible and commonly used by other funders in the field.
This suggests that a shared impact reporting framework could potentially increase the total funding for the talent space.
In addition to potentially unlocking new funding for the talent space, the funders highlighted three other ways in which a shared impact reporting framework could add value:
Assuming a shared impact reporting framework is developed, two aspects were highlighted as especially important to include for it to be useful. The first one is focused on the framework's content, and the other on its adoption.
Based on the findings, we remain uncertain whether developing a full shared impact reporting framework would be valuable. We, however, believe there is evidence to support continued work. Below, we outline three promising future interventions, presented in increasing order of complexity.
As a first low-effort intervention, we recommend establishing a few small sets of indicators for evaluating TFOs. This would help existing funders in the space to compare TFOs more accurately, provide new funders with a better understanding of the impact TFOs create, and enable TFOs to have a clearer understanding of how they are evaluated.
We believe it’s essential that both funders and TFOs are actively involved in developing these indicators, to ensure diverse perspectives are represented and to build buy-in on both sides. More research and conversations are needed to determine the most useful indicators, but some examples mentioned by the funders include:
This pre-study focused on the perspectives of funders. We think there is substantial value in complementing this with insights from TFOs themselves. They may, for example, identify bottlenecks, considerations, or practical challenges that funders may have overlooked.
These findings could also be beneficial to improve coordination and collaboration between the TFOs, in addition to a funder-facing framework. This might take the form of a shared Monitoring, Evaluation & Learning (MEL) playbook or a resource bank with best practices and templates.
Just as GiveWell is a trusted evaluator for global health interventions, and Animal Charity Evaluators in the animal welfare space, an independent evaluator in the talent space (and potentially in the EA meta space at large) might be desirable. The recommendations from such an agency would not only guide how the more than USD 100 million spent annually in the space is allocated, but also strengthen the credibility of the talent ecosystem.
That said, it's not obvious that such an agency would be desirable. One argument would be that historical data isn't the most important decision criterion. Another that TFO's theories of change might vary too much in their approaches, making it hard to make fair comparisons. A third that it wouldn't affect the funding allocation enough to justify its own costs.
Still, we believe this idea warrants deeper investigation to assess its potential in more depth.
We hope these findings spark further discussion among funders, TFOs, and other actors in the ecosystem. If you would be interested in contributing to the next phase of this work, please reach out to Emil or Patrick. While neither of us will have the capacity to continue leading the project, we’d be happy to share insights, material, and connections to help others take it forward.
A big thanks to David Moss for the help with the interview outline, and to Devon Fritz and Marieke de Visscher for the support with introductions to funders.
Thanks for doing this!
Could you define TFOs? Based on your backgrounds, I'm guessing you mean community building organisations like EA Sweden, EA Netherlands, etc., and coaching/training/advising organisations like Successif, 80k, Talos, AIM, Tarbell, etc.?
While both of these sets of organisations are ultimately about helping talent make a difference, I think they have quite different theories of change, and therefore require different M&E systems.
See my proposal below for how I think community building organisations should do things.
An M&E Framework for EA Community Building Based on Community Capital
The proximate objective: Community capital
EA community building organisations are ultimately aiming for impact. But measuring final impact directly is nearly impossible for community builders. How do you attribute a career transition to AI safety, or a crucial research insight, or a new organisation being founded, to your intro fellowship or community event?
Instead, the proximate objective should be increasing community capital, defined as:
Community Capital = (Sum of individual career capital) × (Coordination ability)
This formula captures something important about how EA communities actually create value. Career capital - the skills, knowledge, credentials, and connections that enable someone to have impact - matters enormously. But a collection of capable individuals who don't coordinate is far less valuable than a community that can pool knowledge, collaborate on projects, and leverage each other's expertise. The multiplication relationship reflects that coordination acts as a multiplier: high coordination ability means individual career capital gets leveraged far more effectively.
For an EA national group like EA Netherlands, this means success looks like: growing the number of people with relevant career capital, increasing the average career capital per person, and strengthening the community's ability to coordinate effectively. Do this well, and impact should follow (via EA's broader theory that career capital directed at priority problems matters).
Measuring community capital: The annual survey approach
I think this can be best measured with an annual community survey that collects data on both components simultaneously.
Individual career capital can be measured through self-assessment questions:
Coordination ability is best measured through network questions borrowed from social capital research:
These network questions serve multiple purposes. First, they give you objective data about who's connected to whom, rather than just subjective feelings about connectedness. You can map the actual network structure, identify clusters, and measure density. Second, they differentiate between mere acquaintance and genuine collaboration-readiness - knowing someone versus being able to work with them effectively.
Estimating community size
Apparently, we could then estimate total community size using capture-recapture methods. If survey respondents collectively name 150 unique people, but only 60 of those actually took the survey, the overlap pattern tells you what proportion of the community you're reaching. This lets you estimate:
Combined with career capital measures, you now have all three components of the formula:
Programme-level indicators
The annual survey tells you whether you're winning overall, but you need more frequent feedback on whether specific programmes are working. Programme-level indicators provide this:
These don't measure community capital directly, but they're leading indicators that tell you if you're on track between annual surveys.
Attributing changes to your programmes
Measuring community capital is one thing; showing that your programmes actually contribute to it is another. Three complementary approaches:
1. Cohort tracking: Survey fellowship participants before and after the programme. Survey attendees of major events like EAGx before and after. Track how their career capital and network connections change. This gives you programme-specific deltas, though you can't fully prove causation without control groups.
2. Attribution questions in the annual survey: Simply ask people which EAN programmes most increased their career capital or helped them build connections. This relies on self-reported attribution, which isn't perfect, but people generally have decent intuitions about what helped them.
3. Qualitative contribution analysis: Interview a sample of community members annually and ask them to tell the story of how they became more connected or capable. Code their responses for whether EAN programmes feature in their causal narratives. This captures unexpected pathways and avoids leading them toward giving you credit. We're experimenting with the QUIP methodology at the moment.
Realistically, you'd use all three: cohort tracking for major programmes, attribution questions in the annual survey, and some qualitative interviews.
Connecting community capital to impact
This is the hardest link in the chain. You can measure community capital and show your programmes contribute to it, but does community capital actually produce impact?
The honest answer: you can't measure final impact (lives saved, existential risk reduced) directly. You're relying on a theory of change with two key assumptions:
What you can do is track intermediate outcomes that validate this theory:
Then correlate these with community capital levels: do people with higher career capital and better networks achieve these outcomes more frequently? Do collaborative projects require the coordination infrastructure you've built?
What this system gives you
This M&E approach offers several advantages:
Practical: One annual survey gives you the core metrics, supplemented by programme-level data you're probably already collecting.
Actionable: The formula highlights where to invest. If career capital is low but coordination is high, focus on upskilling and recruitment. If career capital is high but coordination is low, invest in events and infrastructure.
Honest about limitations: It doesn't pretend you can measure final impact. Instead, it measures the proximate objective you actually control, while acknowledging the remaining uncertainty.
Theory-driven: It's based on an explicit model of how communities create value, not just a collection of metrics. This makes it easier to explain to funders and board members why you're measuring what you're measuring.
Hot take: right now I think most regions have high coordination but low career capital but unfortunately are spending waaaaaay more on coordination
Thanks for your comment James. I would define a TFO as any organisation whose explicit goal is to help people increase the impact they have through their careers. So yes, both meta EA groups and coaching and training organisations are included. I’ve now clarified this in the post too.
While I agree the theories of change between different interventions and organisations likely differ substantially, I think a set of standardised outcome-related indicators is still both relevant and necessary. Just as different organisation's interventions in global health differ significantly, but their effects still can be estimated in comparable units like QALYs.
With that said, some organisations (e.g., national EA groups) will have additional outcome indicators that aren’t directly tied to talent pipelines or career transitions, just as your M&E framework illustrates.
Thanks for clarifying!
But at what level should that standardised set of outcome-related indicators operate?
As you mention, we already have indicators for ultimate impact (QALYs, etc). And the indicators at the opposite end of the spectrum are pretty simple (completion rates, NPS, etc.).
It feels like you're looking for indicators that occupy the space in between? Something like 80k's old DIPY metric or AAC's ICAP?
I thiiiiink both organisations tried these metrics and then discontinued them because they weren't so useful?
Yes, as you sugget I think the biggest gap is indicators that say something meaningful about an organisation's impact, rather than just outputs like completion rates, while still being feasible to track continuously (it seems unrealistic for TFOs to regularly estimate their ultimate impact, e.g. QALYs).
These can be on different levels, including simpler ones like number of placements/transitions and their cost-effectiveness. Slightly more advanced ones would take counterfactual and attribution into account.
And more complex ones could assign a quantitive value of a transition. This could be something similar to DIPY or ICAP. If establishing a new standardised quantified indicator for career changes, I think there could be a lot of learning to understand what worked well and less well with these. (Not sure that AAC has stopped using ICAPs internally.)
One possible structure for a quantified indicator could include:
If cause area per placement is reported, funders could adjust their comparison between different orgs depending on their cause prioritisation.