Cross-posted from my Substack.

Epistemic status: I’ve puzzled over this for a long time but I’m not sure I came to the right answers. I'm sharing this partly because it might help others work on resource allocation, but mostly because some smarter reader might tell me how to do it better.

TL;DR:

• I helped set up a large foundation and we wanted a consistent way to allocate grants
• We planned to allocate substantially more than our field was able to absorb right now, giving us the opportunity to actively shape our field
• We outlined key principles and considerations
• We quickly learned we could not just copy the for-profit world: so what could we do?
• We could try and optimize perfectly, mathematically, but it gets absurd and impractical
• Instead, we tried out a bunch of different approaches to finding a consistent grant allocation formula, settling on the one that most “felt” right given our starting principles and constraints
• I quite liked this process of setting principles and constraints, and then vibing your way to an allocation formula that feels right vs being super mathematical
• I wanted to share this process as a real example of trying to work out how to allocate funding. It might be useful to people in the same position, or people might have ideas for how we could have done better 

x = height of the opposing player

When I was about 10, I started to learn algebra in school. Clearly a future EA, I had a brainwave on my way home one day: I can use this new math to become really good at Goldeneye, an N64 shooting game. I got increasingly excited as I walked downhill home, imagining myself briefly pausing the game, whipping out my notebook, firing off some calculations, and then, BAM, quick console controller movements would finish off my nemeses with the precision only numbers could provide. 

So I got home, opened my notebook, and wrote “x = height of opposing player”.

I paused.

I paused longer.

"Eugh"

I closed my notebook and gave up. 

I hadn’t been entirely wrong to try. In principle, there are optimal moves at any given moment that can maximize your chances of winning any game, and those can indeed be identified mathematically. But my 10-year-old brain realized the absurdity of computing all of my actions on paper, while constantly updating the calculations as the game progressed. 

But this post isn’t about Nintendo, it’s about doing as much good as possible with limited resources. 

Searching for the perfect grant-allocation formula

A few years ago I helped set up a private foundation aiming to spend a few hundred million dollars over the coming years on global development. There is a staggering responsibility to spend that money effectively. As we planned our first year (a few $10ms in grants), I thought a lot about how to allocate funding to very different organizations under quite a bit of uncertainty on grantee potential. I didn’t solve it, but I ended up getting to a place that was good enough to begin. Maybe it’s useful, especially if you are working out how to spend sums that your field is not currently ready to absorb.

Our core metrics

1. Cost-effectiveness based on historic performance and discounted for evidence quality (i.e. we preferred large scale RCTs to pre/post studies of small pilots)
2. Chance of success defined by the % chance they would exceed our minimum cost-effectiveness bar. Obviously the higher our cost-effectiveness estimate, the higher their chance of success, but this allowed us to factor in other things like “expanding to a new country”, “impressive new CEO” and “need to hire a new, unproven team” that would not be included in the cost-effectiveness estimate, which was based on historic impact assessments

These metrics allowed for a neat division of labor. The cost-effectiveness estimate would be fully owned by the research team, while the % success estimates would be owned by the programs team.

Setting metrics was the easy part, the hard part was working out how to turn those numbers into a neat “Give Org A $3m this year” output. 

We couldn’t just copy from other grantmakers or the for-profit world

Hold on, you might say, can’t you just copy the funding allocation model of [one of the famous big EA grant makers]. Surprisingly, at least a few years ago, no org I spoke to had a formula that says how much to allocate to a given opportunity. More common is to see if the opportunity passes your funding bar and fund it if it does, but there is no formula that says “we ideally would give this org $xM”, with the grant sizing generally left to the grantee to request. 

Our case was also unique in that we had more money to spend than grantees in our field could currently absorb, and so we were in the position of building our field rather than just allocating to the opportunities that existed in front of us. We had the ability to support the most promising grantees to grow to absorb more, meaning we didn't need to simply "rank everyone by cost-effectiveness and give them as much money as they can take, starting at the top". We wanted to answer "how much would this grantee ideally take?".

So I turned to the VC/hedge fund world for inspiration. But I gradually realized that there are some fundamental differences between for-profit investment and grant-making that prevented us from simply copy-pasting from a hedge fund.

Non-profit funders are not penalized for missing amazing start-ups: A for-profit investor will be penalized for missing a great start-up by paying much more for the same share later on e.g. an investor buying shares in Uber in 2020 will really wish she had invested in Uber in 2010. 

Grant-making doesn’t work this way: if I'm faced with a high-risk/reward non-profit, I can give them a small grant, learn if they do a good job with it, and then donate more, later, once my confidence is higher. All else equal, I don't take a hit on impact from waiting, but I do reduce my risk. 

Non-profits don’t have to optimize for keeping their pot of money: A failed for-profit investment is more damaging than the immediate loss: if I lose a $100k investment, not only have I lost $100k, I’ve lost the compounding profits that that $100k could have made if I hadn’t lost it and been able to invest in future opportunities. What's more, a total failure will lead to going bankrupt and entirely prevent future investments. This dynamic will lead to rational for-profit investors being somewhat risk averse: you don’t want to risk losing your money and not longer being able to play the game, which typically results in investments being spread around opportunities of varying risk/reward rather than betting on just one amazing opportunity. 

But a successful $100k grant costs a grant-maker just as much as an unsuccessful $100k: it costs $100k. All things equal, this makes it easier to take risks with grants that you might not want to take with for-profit investments

Note that the 2 dynamics above are somewhat at odds with each other. The former points to being less aggressive with high risk/reward opportunities since you can just wait until you have better information. The latter points to being more risk tolerant since you aren’t trying to maintain your pot of funding. The later sections will talk more about how we tried to balance these dynamics.^[1] 

Turning our principles into a utility function + optimizing

We wanted to generally maximize impact, but with some risk aversion and spreading of bets, both because we could adjust our funding over time as our confidence levels changed but also because this was our first year of grant-making, and if we totally failed to generate impact, our donors might just drop us. 

If we wanted to present that mathematically, we could model the implied utility function (roughly, "How excited are we by a particular grant") as a linear combination of:

• Maximize utility: Utility is linear with funding*cost-effectiveness
• Better to spread bets: Utility grows at, say, the logarithm of funding invested into each opportunity. So we want to put more into good opportunities but the more we’ve already put in, the less attractive those opportunities get
• Delay funding for risky bets: An aggressive function like 1-f - f^2, where f is the probability of failure, would penalize high risk. This looks pretty linear at smaller risk levels, but rapidly drops to zero/negative utility if the risk goes above a threshold, incenvitizing us to wait for the risk to drop before spending more

We could then bring this all together into a single utility function, like AX + B ln(X + C) -D f- Df^2, where X is the funding allocated, f is risk, and A is cost-effectiveness, and B, C, and D would be constants that we could all align on through internal conversations and talking to our funders, based on how we valued risk. 

Except… this all felt a lot like 10-year-old me writing “x= height of opposing player”. What numbers could we possibly substitute for B, C, and D that had any meaningful bearing on reality? That wouldn’t change as soon as we made our first grants and started learning about just how wide our error bars had always been?

So, and I will not pretend to have had a grand strategy behind this, we simply tried out a bunch of different methods for allocating funding using our key metrics, to see if we could tell if any “felt” more right than the others.  

“Vibing” our way to a utility function 

Here are toy examples based on our real grantees. If you want to play along, I suggest taking a look at these first and thinking through how you might allocate $30m:

Below, I’ll walk through the different allocation strategies we considered for our first $30m in grants, along with the gut reactions from colleagues and internal discussions.

The "Obvious" Option: Allocate by Expected Cost-Effectiveness

Straightforward enough. Multiply the cost-effectiveness by probability of success, get an expected return, allocate proportionally. Here’s how that would look for some example grantees:

“Impact” here is just put into generic units, you could imagine something like “lives saved”, “DALYs averted”, etc

This approach felt wrong. It pushed us to spend large sums on orgs that felt quite high risk, when we knew that with a smaller grant we could learn more about their chances of success and choose to go bigger next year– it actually wasn’t optimal to go big immediately if we could gain more confidence in our higher risk investments first. 

The "Risk-Buckets" Approach

The Risk Buckets idea was a reaction to the above. With Risk Buckets allocate funds into different buckets based on risk levels, then optimize within each bucket for cost-effectiveness. This would enable us to avoid going big on higher-risk grants by learning first with a smaller grant. 

We tried something like this:

… which would lead to funding allocation like this:

We didn’t love it. We were left with the opposite problem from our first approach: we were allocating a lot of money to things we didn't think would deliver lots of impact, just because they were safe. 

The Kelly Criterion

Originating in finance, Kelly tells you the optimal fraction of your funds to bet on opportunities based on risk and reward. I first encountered it through Michael Lewis's book on Sam Bankman-Fried, who notably rejected Kelly as too conservative.

Kelly’s formula: optimal allocation (%) = success rate (%) - [(failure rate %) / (return per dollar)]. On paper, it looked ideal, directly optimizing risk vs. reward trade-offs. Also it came from finance, and finance is cool.

Here’s what a Kelly-optimized grant portfolio would look like for us:

But Kelly has a crucial assumption: you’re growing an investment portfolio where losses directly reduce your available capital, your ability to keep playing the game. In grant-making, if a grant fails or succeeds, you lose the grant money either way, as discussed earlier. This causes Kelly to be overly cautious for grantmaking, pushing too much toward low-risk, low-return bets. For example, something like Org F (high success chance (75%) but modest cost-effectiveness) still received outsized allocations ($3m). Kelly optimized for "not going broke," a constraint that people giving money away for free don’t have. 

Sharpe ratio

The Sharpe ratio is another concept from the finance world. It allows for a risk-adjusted estimate of the value of an opportunity and is calculated by taking the expected return, minus a risk-free return (like government bonds), all divided by the standard deviation of the portfolio's excess returns (vs risk-free). The idea of meaningfully estimating the standard deviation of excess returns of a bunch of nonprofits with varying data quality seemed pretty unrealistic. We did not pursue the Sharpe ratio further.

Edge

Finally, through Nate Silver's book "On the Edge," I encountered "edge," a poker player's heuristic defined as the return to success divided by odds of failure. This is very similar to Kelly, but because it uses odds rather than percentages, it subtly nudges towards higher-risk, higher-reward opportunities.

Here’s how paying grants in proportion to edge looked for us:

The initial consensus of colleagues? This feels right! By funding in proportion to edge, we had a straightforward way of being cautious with high-risk investments but ambitious with promising investments. Importantly, none of the recommended grants felt very off, though some recommendations came off as useful challenges (“Huh, the edge is high and I guess we could go bigger with those guys”).

Here’s how all of the strategies look together: 

What We Actually Did

We really liked the use of edge to generate an initial grant recommendation, subject to occasional adjustment based on “gut”, but adjustment wasn’t common. 

Surprisingly, the biggest benefit was psychological. The concept of edge generated a useful tool for our program officers as they engaged (potential) grantees, focusing conversations around estimating and increasing our "edge". Seeking opportunities to boost and balance cost-effectiveness and probability of success. 

I left the org (Prevail Fund) a few years ago and from catching up with colleagues, the concept of edge ended up most useful for our early years of grants and less relevant to our longer term state, which was more of a "repeatable game" with clearer data on each grantee. 

If we settled on edge, doesn’t that mean there was a calculable utility function in the background the whole time?

A quick, skippable, willdy-under-explained detour into some of the math, for the curious.

You could reasonably point out that the approach we took of trying out different methods until we found one that “felt” right was really just a trial-and-error way of identifying our underlying utility function, which is fundamentally still a mathematical object. And you’d be right! In fact, you can write out the “edge” formula and integrate it to identify the implied utility function that it maximes. I gave this a go and graphed it, it looks like this:

The y-axis here is utility, the x-axis is the grant allocated, shown for a fixed org risk. The units on each axis are arbitrary, I just threw in numbers to generate the shape of the graph. One way to interpret the graph is "how excited am I by this opportunity as I spend more money on it?": you'll see that the more we spend on a grantee at first, the better, but the gains diminish with further funding and eventually peak, where it gets worse to spend more. I 100% acknowledge I'm not properly justifying this section but decide to just "ship the damn post".

The utility function behaves exactly as we wanted it to, somewhat linear at low allocations (i.e. spending $200k on an org is about twice as promising as spending $100k), getting more log-ish (risk averse) as more funding goes to the opportunity, peaking at a point (in fact, the point where the grant allocated is in proportion to the org's edge), and rapidly falling off after that point (since we can just wait a year before giving much more). 

Remember my “in principle, optimal” allocation formula of  Ax + B ln(X + C) -D f- Df^2? It is actually possible to fit a combination of those functions to this graph, playing with different values of B, C and D, producing a good fit up until just about where the “edge” point is reached:

The green line above shows a version of the Ax + B ln(X + C) -D f- Df^2 formula fit to overlap with the edge graph: they more or less do overlap, up until the point where the org's funding is proportional to its edge

So our final selection of “edge”, chosen based on what “felt” right based on various constraints, did indeed look close to the theoretical optimal, but the simpler formula of edge didn't require our programs team to do a load of calculus on every field visit. 

What I learned in all of this

The Goldeneye story is cute but it wouldn't be fair to say the moral of the grantmaking story is “don’t use math”. We did use math, lots! But we found our way to our final fund allocation approach by setting initial principles and then trial and error. Much smarter people than me have noted that we should be wary of trusting individual numbers when we are dealing with high uncertainty.

Although I can’t claim that it was all a masterplan, I feel quite good about the approach we ended up taking:

• Identify principles and constraints, and the implications for strategy, but don’t hardcore quantify it all
• Try out a bunch of options in a way that you can see the actual outputs 
  (e.g. in our case, the actual $ recommendations)
• Be willing to “vibe” your way to what feels right
• Double check your final result: does it accurately get across your initial principles?
• Use the results to guide you, but don’t tie your hands

We got half of the way through math, half of the way through our guts, and I’m pretty happy where we ended up. 

But if we missed something… I’d love to hear suggestions and takes (on philanthropy or Goldeneye, I'm still figuring out both). 

1. ^{^}

   I’m not detailing specifics of what the grants were for or what the impact metric actually was, partly because it doesn’t really matter to the conclusions and also they might not want people trying to game their system. That said, “the secret is to be really cost-effective and reliable” is definitely a wholesome way to game a system…