Hide table of contents

May 2024 marked the last month of the Center for Effective Aid Policy. This post serves as the public post-mortem. I have strived for it to be interesting to the average forum reader, who may not know much about the cause area.

For professionals in development, we have a few internal private write ups which we may be more interesting, such as an overview of development asks we tried[1], their strengths and weaknesses, and our experience advocating for them.

Our mission

Our mission was to improve the cost-effectiveness of development assistance through policy advocacy. Governments spend billions on projects to help the world’s poorest, few of them cost-effective.

For example, one could propose the use of cash-benchmarking to the ministry or push through a political motion to increase the proportion of spending going to the Least Developed Countries.

If one could make even a small part of this very large budget more cost-effective, it would be massively impactful. In October 2022 we were incubated through AIM’s Charity Entrepreneurship programme and came out with $160,000 to get started.

How far did we get?

The first months

Barely a month after receiving funding, we noticed Sweden’s new government was likely to cut the aid budget. The cut would hinge on one party breaking its campaign promise not to cut, perhaps we could campaign for the party to hold their promise.

Over two hectic weeks we put together a write-in campaign for dissatisfied voters. Our execution was not good enough (too little, too late), and we were not able to get voters to write in. Sweden cut its aid spending, and we moved on.

Figuring out where to focus from there was difficult. We tried many things across different geographies, but nothing we did seemed to get much of a response from civil servants and decision makers. Writing credible reports was difficult. We were still learning the development world’s many acronyms, and were struggling to find partners whose trustworthiness we could lean on.

Things pick up

Week by week our network and knowledge expanded. With it came opportunities to get our points across. Through monumental luck we got to present on cost-effective development aid for His Majesty’s Treasury in the United Kingdom. In Denmark we moderated our first public debate between MPs on improving the cost-effectiveness of development.

We eventually fell into a groove of spending the majority of our time writing briefs, taking meetings, and networking.

Between events and meetings, we spent extensive time researching and preparing. Before our first meeting with one Dutch MP, we for example did message testing on 400 voters, broke the answers down by political affiliation, and were able to show with data what voters thought of our ideas. (cash-benchmarking was popular, cash-transfers less so!)

In our record month we had meetings in three countries' parliaments (though it certainly was an outlier!). Our record event had almost 300 attendees and a keynote speech from the Dutch foreign ministry’s chief of science.

A little over a year in we got our first intermediary success. The election programmes of two Dutch political parties now stated their intention to increase the proportion of ODA going to the Least Developed Countries.

The decision to shut down

Our execution eventually became good enough that we got to sit in front of the busy people at the very top, whom we needed to persuade. Speaking to these people we became pessimistic of our odds. Decision makers just weren’t buying what we were selling. You can lead a horse to water, but you can’t make it drink.

Many were skeptical that the RCT-driven approach we recommended would lead to the best outcomes. Those who were on board with our ideas, faced several constraints which made concrete action difficult.

We were failing to move from initial meetings to tangible next steps. Any plans we made with decision makers kept fizzling out, when the next steps were not taken. Several consecutive misses made us believe there were more underlying reasons than just bad luck.

Through our work we got to know several highly effectiveness-minded civil servants with influential roles inside development ministries in multiple countries. We can’t be certain that pushing for another year wouldn’t have yielded a motion or pilot, but seeing how difficult it was even for people at the top to affect change, we became skeptical that persistent outside advocacy from our organization would be cost-effective.

Why changing aid is difficult

Development is a competitive market

The core thesis of our charity fell prey to the 1% fallacy. Within any country, much of the development budget is fixed and difficult to move. For example, most countries will have made binding commitments spanning several years to fund various projects and institutions. Another large chunk is going to be spent on political priorities (funding Ukraine, taking in refugees, etc.) which is also difficult for an outsider to influence.

What is left is fought over by hundreds, if not thousands of NGOs all looking for funding. I can’t think of any other government budget with as many entities fighting over as small a budget. The NGOs which survive in this space, are those which were best at getting grants. Like other industries dependent on government subsidies, they fight tooth and nail to ensure those subsidies stay put.

Development aid can be seen as an efficient market where NGOs are heavily optimized towards receiving funding by whatever means necessary, including lobbying.

When a startup wants to break into a competitive market, the right question to ask is: “what’s your edge?”. This is also the right question to ask any organization trying to move aid spending. Why should the decision maker listen to you over the WFP director who is telling them to do the opposite?

CEAP did not have a good answer to this question.

Political champions are few and far between

We weren’t able to find allies in parliaments. Alone getting to meet the MPs who were development rapporteurs proved challenging.

Of those we met, we found MPs on the left-wing tended to be skeptical of our language around cost-effectiveness. Not everything that matters can be measured and such. Left-wing voters, however, responded positively to our ask for a larger proportion of aid to be spent on the least developed countries.

MPs from right-wing parties were more willing to meet. That too much of aid was ineffective was not a hard sell. It was more difficult, but typically possible, to convince them that some things really were quite effective. I usually came out of these meetings feeling optimistic.

But all things considered, we found that development assistance just isn’t that high a priority for right wing parties. If an MP agreed with us, they'd have to sell it to their party. As one MP put it: “I have to show my party how this will benefit our country”. Spending aid money to prevent migration is a vote winner, vaccinating needy kids in Africa is not.

Among those we got to agree, we at best got lukewarm excitement. We found ourselves unable to move from initial meetings to the next steps such as drafting a motion. Lukewarm excitement doesn't move policy, at least not in the short term. 

Decreasing budgets limit maneuverability

While budget cuts present an excellent opportunity to fight them, once passed, they severely restrict the maneuverability of development ministries.

We believed budget cuts presented an opportunity for us. Cuts shake things up, and by pushing for ministries to keep the most cost-effective programming we hoped to have a large counterfactual impact. As we came to better understand the challenges a development ministry faces when facing a significant budget cut, we became less certain in this hypothesis.

For a ministry which has to get by on half its previous budget, reducing spending without breaking existing agreements and harming diplomatic relations is already a very difficult challenge for a director to face.

Going to a ministry during their most stressful time and asking them to ALSO consider all these additional factors on the cost-effectiveness of programming, is a steep ask.

Was our execution sufficient?

I’d love to tell a narrative where the founders made all the right decisions, and failed because the task was just insurmountable, but that would not be the whole truth.

From its very inception CEAP was blessed with talented volunteers and early employees who did brilliant work. The founders were not quite as brilliant.

Looking back, I am proud of the networks we built and meetings we managed to get. On this, I think we performed above expectations. But what we did present during those crucial meetings was underdeveloped and not sufficiently thought through.

We needed far better material and far more persuasive arguments. To be persuasive we needed to dig far deeper into the government’s aid budget and programming, get input from many more experts, and really think through the policies we wanted to push for.

Our briefs, reports and letters needed to be more detailed and our recommendations much more refined. If there is any single clear failure of execution to point to, this is it. Our materials were far from good enough and we were not sufficiently trustworthy as a result.

Insufficient founder-fit

We lacked development experience

As directors of a policy charity, you have to present as an expert in the topic you’re speaking about.

Neither founder had experience from the development field, and spent the first many months learning the field and its many acronyms. Unlike a policy idea, such as Tobacco Control or Road Traffic Safety, which requires the founders to become experts on a single specific topic, being a credible voice on prioritization within the development sector requires a good understanding of the many facades of development, from educational interventions to democracy building.

Becoming a credible voice on *all of development* is far from easy!

Personally, wanting to present as an expert and a fear of looking stupid made me learn much slower than I needed to. For example, had I pushed to ask more stupid questions about cash-benchmarking early, I would have arrived at the conclusion that it has enormous implementation difficulties much faster.

We came to struggle with morale

I don’t remember exactly when I started noticing my motivation had taken a dip. Early on, I remember leaving the AIDEX conference feeling very demotivated. This was a sign of what was to come.

Seemingly every talk had brought up how their NGO was adopting a new holistic approach to aid, each featuring six new buzzwords and a curious lack of measurement.

Initially the field’s questionable priorities encouraged us to think our mission was all the more important, but eventually it came to feel completely crushing. Every time a report introduced some awful acronym only to use it twice thereafter, my sanity took a hit.

I believe many issues relating to execution were downstream of founder motivation. Why couldn’t we produce higher quality material? Part of it was that it required reading reports that I really did not want to read. Why did we not meet with more civil servants in the Netherlands? Because I procrastinated on writing those emails! And so forth.

Around a year into the charity’s life, my co-founder had his first child. After returning from paternity leave, child rearing and co-running a charity proved too much, and he made the decision to step away.

This did not make things any easier. The intervention still looked impactful in expectation, so I soldiered on, but eventually I lost conviction that we had any realistic shot of succeeding.

Should others try this intervention?

Perhaps!

In some sense, ‘improving the cost-effectiveness of aid’ is not really an intervention any more than ‘improving public health in Africa’ is.

Starting a charity with the mission of “improving public health in Africa” without any specific idea as to how is rarely a good idea. With hindsight, this is a bit how I view our venture into aid advocacy.

But like the mission of improving public health in Africa, there absolutely is room for many charities working on this! Similarly there’s room for lots of great projects in the aid space, but they will benefit from a more specific remit.

Throughout CEAPs life, we tried many instantiations of the broader mission. Perhaps we could fight cuts in Sweden. Perhaps we could push for commitments to increasing spending for the Least Developed Countries in political party programmes. All of these had different reasons for not working out, and we weren’t able to find a slam dunk.

Broad lobbying for improved cost-effectiveness is very difficult unless you have a clear competitive edge. The space is too crowded. The counterpoint to my view, is that our political advisor for our work in the Netherlands believes more time and better execution could have gotten language on cost-effectiveness into the coalition agreement and have had the ministry ready to execute.

My best guess is that a pair of highly talented and motivated founders will run into many of the same issues as we did, but very well could succeed with a pivot or a niche that we missed. I think a founding pair with at least one person with seniority in the development field, would significantly increase their odds of success.

It’s also difficult to know for certain that it wasn’t just a matter of folding too quickly. With sustained effort for 5-8 years things might eventually have come to look different. But as the director, I did not want to take people’s hard earned money to continue a project I no longer believed in.

My loss of conviction is not necessarily an indicator of any objective facts having changed. The odds were always against us, we knew that going in. The glass might just have gone from looking half full to half empty.

I don’t want to say the idea can’t work. I see a million things I could have done differently, hours I could have worked harder, and a thousand ways we could have succeeded. At the end of the day execution is everything. The Center for Global Development proves the core concept can work.

Ultimately aid remains as important as ever. $200B is spent every year. It is imperative to ensure this money does as much good as possible.

I remain optimistic for projects in the space which:

  1. Presents a clear problem (eg. India’s poorest states lack capacity to spend the water sanitation budget they have been allotted)
  2. Presents a clear solution (The Dutch Government should send sanitization experts to advise on how to spend the budget)
  3. Makes a credible case for stakeholders want the problem solved (The Dutch government wants to carry out WASH projects, local Indian NGOs will benefit from new contracts to bid on, etc.)

But if someone wanted to start a project similar to ours because the EV looks really good on paper, I am less optimistic about their chances.

Thank you to everyone who helped

I want to thank everyone who helped along the way.

From start to finish The Effective Altruism community had our back and without it we wouldn’t have gotten half as far. The caliber of volunteers and advisors we got to work with was a privilege I won’t soon forget.

Especially the Dutch EA community went above and beyond. The number of people who reached out offering to volunteer or introduce us to someone in their network was astounding. Whatever went well for us in the Netherlands, did so because of the help we received from the Dutch EA community. You people seriously rock.

Lastly, thank you to the donors who placed their trust in us and donated their hard earned money knowing the risks involved. Publicly admitting defeat is never easy. I’ve had my fair share of failures in life, but none as hard to swallow as this one.

On failure, Peter Thiel states:

"Most businesses fail for more than one reason. So when a business fails, you often don't learn anything at all because the failure was overdetermined. You will think it failed for reason 1, but it failed for reasons 2 through 5."

This was the case for CEAP. I’m hesitant to point to a single reason for why we failed or to claim with confidence whether it lied with the intervention or the execution. It was anything from all execution to a good deal of both.

  1. ^

     Such as pushing for more spending on LDCs or on global health

511

1
0
46
12

Reactions

1
0
46
12

More posts like this

Comments35
Sorted by Click to highlight new comments since:

This post is an impressive feat of honesty and humility

It's a great idea, glad you tried. Here's a few thoughts from a 60 year old with many years overseas and 3 years in Washington, DC.  Maybe they can help (given from a USA perspective). 

The feeling many activists who really care get after spending any time adjacent to the DC development-industrial-complex is that so much of it is a big political scam...DC is not trying to help the world, they're deploying taxpayers money to increase USA power and influence to the ends of benefiting USA (same for all wealthy countries). That's the DC side, then living overseas for almost a decade of my life I saw the actual aid workers in the field and it's like the newbies are actually there for good idealistic reasons, then as they age and go up the ladder you see them just realizing it's all a scam but it's my job so I just play along or they are just expat types who love living overseas and with a USA salary in a poor country you can live like a king. 

All of that is going on, while your holding meetings with individual politicians who may care, but they don't even see the whole big picture themselves because for example State Department priorities and strategies may be opaque to congress members.

The other issue that occurs to me. You are applying to aid what EA applied to charity so successfully...the idea of scientific method for evidence based impact...but the audience who responded so powerfully to EA's charity changing idea was donors deciding what to do with money they control directly...individuals and foundations. But your audience was politicians who are seeking to get votes by doing what voters want, and by needing to follow their leadership. So even though they may be moved by "effective impact" as an idea, the audience they serve - their leaders and voters - are far removed from your efforts and don't hear your ideas, ie. your energies spent at the wrong spot...the solution would be to not only influence politicians but influence the voting public by creative media campaigns in film, tv, youtube and substack and change the zeitgeist, and get the voters to demand a change in how our governments do aid, and start doing it this smarter evidence based way. Now your energies are hitting directly.

But of course, this is not a methodology EA really understands since they don't invite creative media people to the table. They should and more of us are on the peripheries stating the importance of creative media to everything EA does and pointing out EA's blindspot on this. Nobody in lobbying or advocacy would ever imagine effecting a change without creative media to influence the public as a significant part of it...you gotta hit 'em on both sides...inside lobbying, outside changing of minds/votes. 

Perhaps another attempt with a combination of more experienced development lobbyists paired with a creative media team also would have a better chance...but of course it would require a much bigger budget.

EA needs to learn from Tech...Tech is a bunch of STEM people like EA...but Tech has a backroom of STEM talent making stuff, and a frontroom of arts & humanities creatives communicating that stuff to the world. EA on the other hand only has a backroom of STEM people and no frontroom at all, leaving massive value on a non-table that never gets communicated to the world it was for. Philanthropic funding artificially enables EA to perpetuate this misunderstanding of how the world works, Tech lives in the real world.

Perhaps another attempt with a combination of more experienced development lobbyists paired with a creative media team also would have a better chance...but of course it would require a much bigger budget.

FYI, there's a US-focused organization with a similar mission to CEAP's called Unlock Aid that seems to be doing really good work. Open Philanthropy is also doing a lot of engagement with governments directly through its Global Aid Policy program.

Yeah those are both really good initiatives...now they need more creative media to win the broader public to their vision. 

I feel like effective aid policy is at a similar stage to what animal well-being was at a few decades ago. People would agree that animal well-being is good, but they wouldn't feel it's important.

Maybe we need an org that does targeted public campaigns on how a certain aid organization is wasting money, combining that with pushing them to a commitment to more effectiveness. This approach has worked with some meat-intensive companies, and it might also work for non-profits if it can threaten their donor base.

A on the other hand only has a backroom of STEM people and no frontroom at all, leaving massive value on a non-table that never gets communicated to the world it was for.

I don't understand this sentance. The value on the table is good ideas that don't get realised because they're poorly communicated?

Sorry to hear about the lack of success.

I'd strongly caution against overlearning that 'this sort of thing cannot work'. The big thing that stands out to me is the issue of founder-fit:

Neither founder had experience from the development field, and spent the first many months learning the field and its many acronyms.

I have the impression that there are a lot of aligned people with this background and expertise looking to fill roles like this. Another attempt should be able to improve on this.

Yep, I think credibility and credentials in policy advocacy are very important, especially when you need to build networks sort of from scratch. Perhaps AIM can pay attention to this aspect going forward when founding more policy oriented charities?

There's truth here, but I'm not sure how much it would make a difference in terms of moving the needle and changing minds. I'm very uncertain, but I feel like more credentials might get you more conversations and into more rooms but I'm not sure it would make those conversations more likely to lead to change.

Perhaps, but I wouldn't want to overgeneralise against non-experts in policy either. As the post said, a broad effort to improve aid policy requires a much broader expertise across many areas than specific efforts to target specific policies. AIM has incubated several more targeted policy orgs and I think they're much easier for non-experts to build knowledge and credibility in.

Mathias, I hope you feel proud of what you did at CEAP. I have a lot of respect for the thought and effort you put into this project, and I thought your decisionmaking was well-reasoned throughout. A number of people in the field and at CE have told me they feel the same way.

Running an organization with such a broad remit and weak feedback loops is a challenge. Doing so by yourself with waning morale is an uphill battle. You were in a tough position and I thought you handled it well. And as Jason wrote, you may have had more success developing Dutch, Danish, and British champions than you know.

I’m looking forward to following what you do next :)

This is a wonderful post, basically everything you've said here makes sense and lines up with my (limited) experience of those in control of the aid dollars. 

This sent shivers down my spine a little. I too feel there is a new wave of "Holistic" aid vibes coming through at the moment, but I'm surprised its as prevalent as you saw at the conference.
"Seemingly every talk had brought up how their NGO was adopting a new holistic approach to aid, each featuring six new buzzwords and a curious lack of measurement."  

Here's three gems of wisdom I especially appreciated.

"What is left is fought over by hundreds, if not thousands of NGOs all looking for funding. I can’t think of any other government budget with as many entities fighting over as small a budget.

"In some sense, ‘improving the cost-effectiveness of aid’ is not really an intervention any more than ‘improving public health in Africa’ is.

"But if someone wanted to start a project similar to ours because the EV looks really good on paper, I am less optimistic about their chances."
 

I have one nagging question here (no pressure to answer) - do you think 18 months was long enough to really give a good go at a project like this? So much of policy/lobbying work is about relationships, and I would doubt that anyone really could build the strength of relationships that might be needed to move the needle. And you did have some success in that short time as well. I wasn't completely clear how much of the decision to shutdown was about results/funding vs. your energy levels and optimism.

"Holistic" has to be the word that I feel is most antithetical to EA

Because of the down votes, I'll explain why.

I think "holistic" is often a way of simply "doing whatever I feel like in the first place without looking at data" or doing it on "vibes". I think it is the opposite of rigor and evidence.

That's right, but in the original meaning of the word, it's actually not against EA at all. Us too would prefer sustainable interventions that lead to a better system and that do not have hidden costs. And I think rigorous RCTs that measure general markers are a good tool to find such interventions.

One of EAs anti-examples, Play Pumps, failed because it turned out not to be holistic at all.

This was a really courageous post. It's not easy to make yourself vulnerable even among EA "friends". Your org's shutdown and that of other recently incubated charities makes me wonder if the whole startup-type approach of co-founding an organization with 2 people and ~$100K and a few months of research isn't just setting people up for failure. The work you were trying to do takes years of effort and deep expertise but you were clearly under the gun from funders to get results quickly. I can't see how anyone else, no matter how brilliant and motivated, wouldn't end up in the same place you did without substantially more resources and time. I think you are right that a much narrower focus or limiting efforts to a specific political initiative/motion would have worked better but it still seems like an uphill battle.

How realistic do you think your initial expectations of the work and policymaker reception to it were? That's a vague question, but I'm wondering if an expectations-reality mismatch made it harder to maintain motivation.

On my (mostly uninformed) reading of your postmortem, you achieved significantly better than I would have predicted on some important domains (e.g., gaining access to important decisionmakers). Unfortunately, it seems like the theory of change required strong showings in a number of different domains (e.g., political strategy/savvy), and a middling showing on any of those domains would be fatal to the project. My guess is that it would be very rare for a two-person team to have strengths in all the needed domains.

Finally, although you speak of having "failed," the potential downstream effects of the conversations you had are unknown and unknowable. That measurable outcomes were in short supply doesn't mean that your advocacy didn't have a counterfactual effect on any of the policymakers. That you concluded that didn't make sense to continue is might be as much about your awareness of the strong alternative uses of your donors' money as it is about your execution and results.

Thank you for trying this, even if it didn't pan out as you had hoped.

This is a courageous post, and I commend you for it. I aspire to the level of open reflection and dispassionate analysis you've displayed here.

I'm fairly new on the forum, and am trying to decide if I should go all-in on EA or remain on the outside looking in. This vignette resonated with me:

As one MP put it: “I have to show my party how this will benefit our country”.

I'm a (small-r) republican and an ardent believer in my country's social contract. I find myself agreeing with the MP--there's usually an expectation that good stewardship of taxpayer money means using it in a way that primarily benefits citizens and/or the national interest. Despite my EA interest, I'm a tad uncomfortable advocating for public funds to be diverted in a way that provides no tangible benefits to the society providing it (unless there's an explicit democratic mandate to do so).

That's why I love the idea of earn to give and other forms of private EA money, even though I'm aware that 1% of a Western country's budget earmarked for foreign aid is likely substantially more money than all but the richest donors can chip in.

I'm curious about what your next steps are, and I wish you all the best.

My view on this: I think governments do foreign aid for a variety of reasons, some are selfish (like providing contracts for domestic companies), some are more altruistic (though a more developed Africa benefits us all through better market access and more efficient use of resources, as well as fewer global health crises and political crises to deal with).

And a different way of looking at it: Some amount of foreign aid is going to be directed towards altruistic goals. Anyone should want that money to be used as effectively as possible.

A great post-mortem, and thanks for your kind words about the Dutch EA community - it was really inspirational to see you in action! 

Your line, "Spending aid money to prevent migration is a vote winner, vaccinating needy kids in Africa is not" made me think of the Make Poverty History campaign, timed to coincide with the UK's hosting of the G8 in 2005.

According to Wikipedia, these were the main health and development agreements from that summit:

  • US$50 billion pledged (some of it previously announced) in aid to developing countries by 2010, of which US$25 billion for Africa, on top of the ministerial-level agreement to forgive debt to Highly Indebted Poor Countries
  • Universal access to anti-HIV drugs in Africa by 2010
  • G8 members from the European Union committed to a collective foreign aid target of 0.56% of GDP by 2010, and 0.7% by 2015
  • Stated commitment to reduce subsidies and tariffs that inhibit trade

Also, from memory, I think most UK political parties committed to the target of spending 0.7% of national income on aid and, although this was rolled back for a bit under Johnson, the next Labour government have pledged to reinstate it. 

I think there's an argument to be made that these agreements were heavily influenced by the campaign. 

Blair, chair of the summit, notes the following in his biography: 

“On Africa, I knew that without real figures it was going to be another ‘poor Africa, we care so much about you’ load of old rubbish in a communiqué that wasn’t going to fool anyone. Bob, Bono and the NGO alliance had mounted an effective campaign, essentially going to each main nation in turn and trying to frighten the pants off the leadership by demonstrating the breadth of public support for action on Africa. It was done cleverly, with them always giving enough praise to the leaders to encourage them. With Bob and Bono at the helm, there would be a sensible debate. If we delivered, they’d say we’d delivered. If not, they would condemn us. Fair enough.”

Of course, this was a huge campaign: a global audience of approximately 3 billion for Live 8, millions of people wearing the campaign’s white band, a quarter of a million people marching on Edinburgh, and a brand recognition that leapt from zero to 90% in just six months. It would be a huge undertaking.[1] And one could argue that, 20 years on, the effects have died out. 

But then again, maybe not. I was 13 years old when I attended the march in Edinburgh, but it's one of my more vivid memories from that time. This early exposure likely contributed to my ongoing support for GHD through my donations and my work today, and I suspect it's the same for many other people.

  1. ^

    I asked Claude to produce a BOTEC for the cost and it arrived at $120 million

Thank you for writing this and for your kind words about the Dutch EA community!


I am curious to know whether you feel like an organisation that doubles down on a single country could be more effective? At least in the political realm, it should be possible to build good relationships with the relevant political actors, though obviously you would trade away a lot of expertise that comes from having a more international perspective. 

As a side note, one thing I find amusing is just how much it sucks to announce your org's shut down after Maternal Health Initiative set the bar so ridiculously high.

Even at shutting down they have us beat!

Unfortunately, your shutdown announcement didn't meet our bar. However, it was in the top 5% of rejected shutdowns, and we strongly encourage you to announce a shutdown in the next iteration.

Thanks for writing this. I imagine it was painful, and I’m grateful.

Would you mind saying a little more about this bit?

had I pushed to ask more stupid questions about cash-benchmarking early, I would have arrived at the conclusion that it has enormous implementation difficulties much faster.

Hi Ian,

Thanks for the question! I've been meaning to write down my thoughts on this for a while, so here is a longer perspective:

 

In 2015 USAID teamed up with Givewell to cash-benchmark one of its programs. The evidence came back showing that cash-transfers outperformed the program on every metric. What gets brought up less often is that the programme got its funding renewed shortly after anyways! The cash-benchmark alone was not sufficient, you also need some policy to require programs worse than cash should be wound down.

This is a sentiment I'm fully behind. But what exactly that policy should look like is where it gets tricky.

How should the ministry cash benchmark a music festival in Mali?[1] What is the cash-benchmark for a programme to monitor the Senegalese election to ensure a fair election? If the cash-benchmark should only be for certain types of programming amenable to cash comparisons, such as global health, how will that shift funding?

I worry that instituting a selective high bar will move funding from away broadly cost-effective areas which can be benchmarked against cash, to broadly ineffective areas which can't be easily benchmarked against cash.

But even within areas amenable to cash-benchmarking, it's unclear what the policy should look like. How should the ministry cash-benchmark its funding to a large multilateral which will go to fund a thousand programmes across the world?

The answer to this, which many arrive at is: "Cleary we need to move from demanding literal cash-arms to just making estimates of how impactful programmes and organizations are compared to cash-transfers. That way we still get the nice hurdle-rate that programmes must be compared against, which is what we were really after anyways"

But that development ministries should systematically estimate and compare the impact of projects is what development economists have been shouting for decades!

To an extent, the ministry's lack of systematic measurement and comparison is a feature not a bug. Almost any instantiation of cash-benchmarking removes wriggle room to fund projects which are valuable for reasons you didn't want to state out loud. From a ministers perspective, cash-benchmarking doesn't solve any problems, it creates one!

  1. ^

    This is not a facetious example, but a real project funded by the Norwegian government.

This is a little horrifying to hear put this starkly, but makes perfect sense.

BTW I have no problem at all believing that music festivals are funded, GIZ for example fund all kinds of strange "events" here in Uganda without much of a theory of change. 

I agree that having a more experienced founder could quite possibly make a difference.

Beyond that, I wonder whether it would make sense for people to consider interventions that are further upstream, ie. some kind of fellowship course for people interested in going into policy in this kind of area.

Thanks for trying, and for sharing this write up!

I found your description of applying effort to a really difficult task, and eventually making the hard decision to cut your losses, inspiring and moving. Thank you to CEAP’s founders, funders, and other supporters.

Thanks for your work on this, Mathias!

Sorry it didn't work out, but congrats on getting all those meetings and thank you for trying! :) 

I appreciate the frankness and reasoning transparency of this post.

Executive summary: The Center for Effective Aid Policy (CEAP) shut down after 18 months due to difficulties in influencing aid policy and improving cost-effectiveness of development assistance, despite some initial progress.

Key points:

  1. CEAP's mission was to improve cost-effectiveness of development aid through policy advocacy.
  2. Initial challenges included lack of expertise and difficulty gaining traction with decision-makers.
  3. Some progress was made, including presentations to government officials and influencing party platforms.
  4. Ultimately, CEAP struggled to overcome skepticism from decision-makers and competition from established NGOs.
  5. Founders faced motivation issues and lacked sufficient development experience.
  6. Future attempts at similar interventions may benefit from more specific focus and founder expertise in development.

 

 

This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, and contact us if you have feedback.

Very helpful to learn from your perspective here - thanks so much for taking the time to write it up. Best of luck in what comes next!

Thank you for sharing this. Like many others, I admire your courage and your honesty.

Entrepreneurship is a brutal business. Most startups fail, whether they are in the private or public sector. There is no shame in shutting down. The lessons you have learned (and shared so eloquently) will help future charity entrepreneurs be more effective and perhaps give them the knowledge to succeed.

I hope you will continue to look for opportunities and use the lessons in your next venture. You may not have had the impact you hoped for, but it is clear from the comments that you did make an impact among your fellow EAs. Don’t stop now — you are just getting started!

Curated and popular this week
Relevant opportunities