Thank you, Benjamin, for writing this in-depth profile and to the whole 80,000 hours team for your work!

Since grantmaking is one of the highlighted careers, I'm going to allow myself to shamelessly plug two opportunities at the EA Animal Welfare Fund that we posted today: full-time and part-time Fund Manager role (deadline is 29th of December) and our expression of interest form for the Fund Development Officer/Manager/Director position. 

Thank you for these thorough reports and the project as a whole! As Chair of the EA Animal Welfare Fund, I'm very grateful for GWWC's continued work evaluating evaluators and grantmakers in the animal welfare space and personally grateful for their work across all cause areas. I think this sort of meta-evaluation is incredibly valuable. This year, I'm particularly excited to see ACE's Movement Grants join the recommended list this year - their improvements are great developments for the field. Last year's evaluation of AWF was also very helpful for our team, and we're looking forward to the re-evaluation next year. It's encouraging to see the evaluation and funding ecosystem becoming increasingly robust. Thank you for your work! 

Thank you, Angelina! I'm very excited about it, too!

Thank you for raising this question, Emre! We value transparency and recognize how outcome data helps potential donors make informed decisions. We would like to move more toward that direction, but there are some limitations to that.
First, we could only present the result in aggregation potentially with individual data for successful grants/work. Publishing individual grant outcomes, particularly unsuccessful ones, could discourage grantee candor and lead others to draw overly broad conclusions about interventions or grantee capabilities. That's why we lean toward aggregate reporting - for example, sharing overall success rates or highlighting particularly impactful grants that make up the bulk of the impact.
The second limitation is our capacity. Even after I joined the fund in greater capacity, we still only have 1.3 FTE, most of which goes toward grant sourcing, evaluation, decisions, and internal impact tracking. While we're planning to expand our team soon, we need to carefully balance any new initiatives with other strategic priorities. Therefore, even though we would be excited to increase the amount of public grant outcome reporting, we're still assessing the extent to which we can implement it while balancing other goals.
We hope that the steps we are taking right now, like regularly publishing payout reports, and annual reports like the one above, will already help supporters assess our work until we could do more on that front.

Thank you for the update about your program!

In general, the examples we listed in the post are not exhaustive, and there are opportunities that haven't been explicitly mentioned, so I encourage reaching out to me if one is interested in making a contribution and would like to learn more about the opportunities we identified. 

Thanks! We are grateful for all the work that our grantees do.

From our experience, in general, work in Europe tends to be more tractable than in North America, especially on the margin. This is especially true for policy opportunities that show higher expected cost-effectiveness in the European context than in the US. When I look at the grants we funded in Europe over the last year, many are indeed focused on policy advocacy. What also plays a role is that we simply receive more applications from European organizations, which naturally affects our grant distribution. 

Hey Vasco! I agree that AWF should be more transparent, and since I started working on it full-time, we have more capacity for that, and we are planning to communicate about our work more proactively.

In light of that, we just published a post summarizing how 2024 went, what changes we recently introduced, and what we are planning. We touched on updates to our evaluation process as well. Here is the relevant section from that post: 

"Grant investigations:
Updated grant evaluation framework: We've updated our systematic review process, enabling us to evaluate every application using standardized templates that vary based on the required depth of investigation. This framework ensures a thorough assessment of key factors while maintaining flexibility for grant-specific considerations. For example, for the deep evaluations, (which are the vast majority of all evaluations), key evaluation areas include assessment of the project’s Theory of Change, scale of counterfactual impact, likelihood of success, back-of-the-envelope cost-effectiveness and benchmarking, and the expected value of receiving funding. It also includes forecasting grant outcomes. You can read more about our process in the FAQ.
Introduced new decision procedures for marginal grants: We introduced an additional step in our evaluation that enables us to make better decisions about grants that are just below or just above our funding bar. Since AWF gives grants on a rolling basis rather than in rounds, it is important to have a process for this to ensure decisions are consistent."

We also slightly updated our website and added a new question to the FAQ - I'm copying that below: 

"How Does the EA Animal Welfare Fund Make Grant Decisions?

Our grantmaking process consists of the following stages:

Stage 1: Application Processing. When we receive an application, it's entered into our project management system along with the complete application details, history of previous applications from the applicant, evaluation rubrics, investigator assignments, and other relevant documentation.

Stage 2: Initial Screening. We conduct a quick scope check to ensure applications align with our fund's mission and show potential for high impact. About 30% of applications are filtered out at this stage, typically because they fall outside our scope or don't demonstrate sufficient impact potential.

Stage 3: Selecting Primary Grant Investigator and Depth of the Evaluation. For applications that pass the initial screening, we assign investigators who are most suitable for a given evaluation. Based on various heuristics, such as the size of the grant, uncertainty, and potential risk, the Fund’s Chair also determines the depth of the evaluation.

Stage 4: In-Depth Evaluation. Every grant application undergoes a systematic review. For each level of depth of investigation required, AWF has an evaluation template that fund managers follow. The framework balances ensuring that all key factors have been considered and that evaluations are consistent, while leaving space for additional, grant-specific crucial considerations. For the deep evaluations, (which are the vast majority of all evaluations), the primary investigator typically examines:
 

• Theory of Change (ToC) - examining how activities translate into improvements for animals and whether the evidence supports its merits
• Scale of counterfactual impact - assessing the problem's scale, neglectedness, and strategic importance
• Likelihood of success - evaluating track record, team competence, and concrete plans
• Cost-effectiveness and benchmarking- conducting calculations to estimate impact per dollar and compare it to relevant benchmarks
• Value of funding - analyzing counterfactuals and long-term sustainability
• Forecasting - forecasting the probability that the project will succeed or fail and due to what reasons (validity of the ToC or performance in achieving planned outcomes )
• In the case of evaluations that require the maximum level of depth, a secondary investigator critically reviews the completed write-up, raises additional questions and concerns, and provides alternative perspectives or recommendations.

Stage 5: Collective Review and Voting. After the evaluation, each application undergoes a thorough collective assessment. The Fund Chair and at least two Fund Managers review the analysis. All Fund Managers without conflicts of interest can contribute additional insights and discuss key questions through dedicated channels. Finally, each Fund Manager assigns a score, which helps us systematically compare the most promising grants.

Stage 6: Final Recommendation Looking at the average score, the Fund Chair approves grants that are clearly above our funding bar and rejects those clearly below it. For grants near our funding threshold, we conduct another step where all found managers compare those marginal grants against each other to select the strongest proposals.

Once decisions are finalized, approved grants move to our grants team for contracting and reporting setup.

Throughout this process, we maintain detailed documentation and apply consistent standards to ensure we select the most promising opportunities to help animals most effectively."

Thanks Ozzie! 

> “I'm not sure if this was done very intentionally, or that's more a representation of who applied, but overall, I'm more net-optimistic about investments in larger projects. 
[...]
Now, especially with the recent changes at OP, it seems like some significant animal cause areas (invertebrate welfare) will likely be overlooked by other funders. I'd expect that going forward, there should be significant opportunities for other funders to be active here, and I'd expect much of the gain would come from funding larger projects. “

This represents who applied at the time, how developed some of the projects are, and how uncertain their outcomes are. We would often fund “an experimental, new project” for 6 months for a pilot, then for 1 year, and if it is proven, we would provide a larger-scale grant. Sometimes a project of this type also “graduates” to a larger funder like Open Phil and that’s why you do not see them here. EA AWF’s comparative advantage is often in funding small and medium-scale projects and I think it makes sense to serve this role in the project development pipeline. 
That being said, there are some grantees that have a strong track record in areas where EA AWF has a comparative advantage and we provide larger grants ($150-$400k). Those typically include projects in wild animals, invertebrate-related work and research on neglected species, although not exclusively. We plan to continue and hopefully scale our grantmaking in those areas given the Good Ventiured update. 

Additionally, there were also instances where we would like to provide a larger amount to top applicants, but thought that the value of the marginal grant was higher than more funding for top applicants. If we had more funding, we would have provided both and in the past, have communicated that EA AWF has significant RFMF. This is still the case. 

Hey Vasco,

Yes, it's right that we don’t conduct CEAs in all of our evaluations, but they are part of our analysis for some of our grant investigations. GWWC only looked at 10 grant evaluations, so it's possible they didn't come across those where we did model BOTEC CEA. With the upcoming increase in the capacity of the fund, we plan to invest more in creating BOTECs for more evaluation. We are hoping to be reevaluated by GWWC so the evaluation reflects the changes we have made and are planning to make in the future.

In the past, we tended to do CEAs more often if: a) The project is relatively well-suited to a back-of-the-envelope calculation b) A back-of-the-envelope calculation seems decision-relevant. At that time, a) and b) seem true in a minority of cases, maybe ~10%-20% of applications depending on the round, to give some rough sense. However, note that there tends to be some difference between projects in areas or by groups we have already evaluated versus projects/groups/areas that are newer to us. I'd say newer projects/groups/areas are more likely to receive a back-of-the-envelope style estimate.

Even in evaluations where we didn’t explicitly model CEA, we tended to look more at factors that help us judge marginal cost-effectiveness, such as the scale of the problem and potential number of animals affected, whether the work is happening in a country with high production of target species, how neglected it is (to get at the counterfactual impact), the goals of the grant and whether we think the applicant is likely to achieve them given their track record or strength of the plan. We also use and reference more in-depth independent CEAs, like the one on cage-free corporate outreach, shrimp stunning,  ballot initiatives or fish stunning while noting that they have limitations and we do not take them at face value. 

However, since then, we've started conducting BOTEC CEA more frequently and using benchmarking in more of our grant evaluations. For example, we sometimes use this BOTEC template and compare the outcomes to cage-free corporate campaigns (modified for our purposes from a BOTEC that accompanied RP's Welfare Range Estimates).

For harder-to-quantify grants like movement or capacity building, we would also occasionally model expected outcomes in numerical terms and ask whether this outcome is something we would pay x amount (the expected cost per unit). 

We also have a score calibration guide we use when we score grants to make them comparable across grants.

We do not put that much weight in applicant’s CEA as they are impossible to compare to CEAs that use different methodologies and are very sensitive to assumptions that we often cannot verify. 

I hope that helps to understand our methodology. Let me know if you have any questions.