I was the main person at Open Philanthropy working on our recent funding evaluation of Wytham Abbey. I’d like to share some key points on why I recommended that we cease future funding for the Wytham project and that the Abbey be sold. I can’t speak for Effective Ventures, which made the final decision to sell the property, or other people at Open Phil who, as per our standard process, approved my recommendation; but it’s probably fair to say that my work was among the most important inputs into the decisions that led to the outcome announced here.

Contrary to speculation in another comment, my recommendation was primarily driven by a cost-effectiveness analysis. In doing so, I considered the following benefits Wytham was providing:

1.  Cost savings for non-counterfactual events
   1. Wytham provides a “free” venue to events that would otherwise have had to pay for some other venue (but would have happened, just elsewhere)
2. Counterfactual events
   1. The availability of Wytham causes some events that wouldn’t have happened otherwise, e.g., because the event organizers couldn’t have gotten funding for a venue in time or didn’t have the staff capacity for things like venue scoping, organizing food, and some other ops/logistics things that Wytham takes care of. There’s probably also some effect of increasing the salience that running retreat-like events is something one can relatively easily do thanks to Wytham.
3. Improving the quality/impact of events
   1. You could imagine that due to factors like its more “secluded” location, closeness to nature, the layout and decoration of rooms, etc., Wytham enables better or deeper conversations than would have happened elsewhere; and in fact several people reported that Wytham seemed like a good venue for events to them in terms of its vibe. In addition, probably some non-counterfactual events would otherwise have happened as a shorter version, or not as residential events where participants stay overnight, thus allowing for less conversation/bonding time; and some events would plausibly have benefitted in various ways from the accumulated event hosting experience of the Wytham team. 

The bottom line was that historically, i.e. based on looking at past Wytham events, Wytham’s operating expenses (for now ignoring the opportunity cost of capital being bound up in the property) clearly exceeded Open Phil’s willingness to pay for these benefits given our current funding bar. (Under our pre-Nov 2022 funding bar, which was the one we were using at the time of the original grant to buy and operate Wytham Abbey, it would have comfortably been the other way around.) 

I then considered how likely this seemed to change in the future. While there were several reasons to expect both falling costs and increasing benefits, I didn’t end up being sufficiently optimistic, though that seemed like a much closer call. PR-related questions played a role at this stage, but mainly via the relatively specific channel of making the venue less attractive for some organizations that may have wanted to host future events there if not for worries that they’d then be associated with the venue (though in some of these cases, the desire to avoid being associated with the EA community more generally was more important for that than the implications of Wytham-specific media coverage). These considerations were not decisive on their own. Depending on how you individuate the reasons considered in my analysis, I’d say that PR considerations were among the top 10, but not among the top 3 most important considerations, ranking behind e.g. Wytham’s track record, some more mundane fundamentals of the venue (e.g., the number of bedrooms and bathrooms) constraining the kind of events and audiences Wytham seemed like a good fit for, and strategic background assumptions about the value of different kinds of events.
 

The above analysis established that continuing to fund Wytham’s operating expenses wouldn’t have met our funding bar. However, this doesn’t conclusively answer the question whether, to maximize impartial impact, we should recommend that EV sells the property (e.g., what if they could find other funding for some or all of Wytham’s operating cost?). That also depends on the opportunity cost of capital being bound up in Wytham – capital that could otherwise be used for other high-impact projects or invested into other assets to maximize its financial return (with those returns being directed to high-impact projects in the future). I spent more time looking into this question, and again PR considerations were relatively minor; my analysis was primarily based on theoretical arguments on the financial returns of different asset classes (e.g., real estate vs. stocks), data on the price appreciation of real estate in various relevant reference classes, estimates of foregone rental income while EV is holding the property, the appropriate amount of risk aversion for financial returns, the willingness to pay vs. cost estimates from the previous analysis, and whether the conclusion could plausibly change depending on different assumptions on how much the existing portfolio of capital intended for global catastrophic risk reduction might be underinvested in Wytham-like assets. On net, it seemed to me that EV holding on to Wytham would amount to a sufficiently large cost that hypothetical third-party funders or other realistic changes would be unable to compensate, such that from an impartial perspective it seemed better to recommend that EV sells the property.

A commenter asked what would have happened if my cost-effectiveness analysis had come back more positive. There are two ways to understand this question. One is: How would it have affected my recommendation if my impression of Wytham’s impact potential (and/or potential cost savings) had been stronger? It’s hard to make confident claims about such counterfactuals, but I suspect that I would have spent more time considering the implications of the broader PR picture, and what that might mean for Wytham’s future impact and any potential externalities on other projects (e.g., effects on the perception of Effective Ventures or EA more broadly), since that seems like a relatively ‘high-variance’ consideration that could potentially flip an initially positive-seeming picture. My guess is that I would’ve concluded that these broader PR issues were on net a moderately strong reason against funding Wytham, though the issue is sufficiently complex that my error bars are fairly wide here. I would then have considered this together with all other relevant considerations, including the amount by which (in this hypothetical) Wytham seemed above our funding bar based on quantitative estimates that ignore these broader PR ramifications; depending on my impression of the strength of the various considerations at play, I might or might not have recommended to renew our funding for Wytham. To be clear, this would have been an entirely normal process: For reasons that will be very familiar to many readers (see, e.g., discussions here and here, though note I don’t necessarily agree with every point), our funding recommendations are typically based on a holistic analysis of several quantitative and qualitative inputs, rather than being mechanistically determined by the output of a single-number quantitative cost-effectiveness model; and when using the term “cost-effectiveness analysis” in this comment, I was referring to just such a holistic analysis. For this reason, I would summarize this scenario as “if some aspects considered in the cost-effectiveness analysis had suggested a higher expected impact for Wytham, I would have spent more time on the analysis, which may or may not have changed my ultimate recommendation” – not as “if the cost-effectiveness analysis had come back more positive, I might still have recommended to not fund Wytham based on some highly unusual and qualitatively different additional step”.

The second way to understand the question is: If I had recommended to fund Wytham, would the final Open Phil decision-makers have agreed? The short answer is that I don’t know, and can’t speak for other people. On one hand, the base rate of grant recommendations not being approved is low. On the other hand, this was clearly an unusual case, given that our previous funding for Wytham had been an outlier regarding the amount of externalities on other projects. I would therefore have expected decision-makers such as Open Phil leadership or possibly Cari and Dustin (who, as described in our annual review blog post, have recently been more engaged with our work) to review a hypothetical recommendation to renew our funding for Wytham’s operations in more depth than usual, and would have anticipated that the risk of non-approval was higher than baseline. That expectation likely also had some effects on how I went about my evaluation, e.g., how much time I was willing to spend to proactively search for long-shot attempts to change the bottom line such as by proposing changes to the project, thinking about potential different uses of the property, etc., (though these would generally be relatively uncommon uses of time for a grant investigator since we can typically achieve a higher grant volume – and thereby have more impact – by evaluating more shovel-ready proposals rather than by trying to bolster projects that at first blush don’t meet our bar).

So I don’t want to claim that PR or adjacent considerations had zero influence here: Specific ways in which they might limit Wytham’s future impact were among the top 10 (though not top 3) most important considerations behind my recommendation not to renew funding; and while I don’t have any reason to believe that they would have been decisive in all or even most possible worlds, there are some counterfactual worlds in which broader PR considerations regarding e.g. the impact on other projects would have tipped the scales against renewing our funding. But I want to be very clear that, from my perspective, we fundamentally conducted a standard grant investigation that aimed to holistically weigh an opportunity's expected impact against its cost, and concluded that it did not meet our bar.
 

How to best handle the property until it’s sold was outside the scope of my investigation, so I don’t think it’d be appropriate for me to comment on it. We haven’t advised EV what to do with Wytham Abbey until it’s sold, nor have we recommended a specific date by which to conclude the sale (though I gave some input on how they might want to trade off timing and price).

I don’t think I’ll have the bandwidth to engage in further discussion, and I’m sorry that this comment likely won’t answer all the questions people might have. But it seemed better to provide some high-level information on the work I did than to let readers reach an inaccurate understanding of what happened.

As a closing note, my personal view is that this has basically been a tragic story: We funded a project, the project team overall did a good job running it from an operational perspective (based on the impression I got during my evaluation), and then events beyond the control of anyone involved meant they were no longer above our funding bar.

(See also this comment by Carl Shulman which is already mentioned in the post.)

The link to Carl's comment doesn't work for me, but this one does. 

This link from the main text to the same comment also doesn't work for me:

as Carl Shulman observed many years ago

(personal views only) In brief, yes, I still basically believe both of these things; and no, I don't think I know of any other type or action that I'd consider 'robustly positive', at least from a strictly consequentialist perspective.

​To be clear, my belief regarding (i) and (ii) is closer to "there exist actions of these types that are robustly positive", as opposed to "any action that purports to be of one these types is robustly positive". E.g., it's certainly possible to try to reduce the risk of human extinction but for that attempt to be ineffective or even counterproductive (i.e., to on net increase the risk of extinction, or to otherwise cause significant harms such that I'd consider the action impermissible), it's possible for resources that were acquired for impartial welfarist purposes to eventually be misused, etc.,

I made some nuanced updates about "acquiring resources for longtermist goals", but they are mostly things like me having become more or less excited about particular examples/substrategies, me having somewhat richer views on some pitfalls of that strategy (though I don't think I became aware of qualitatively 'new' pitfalls), etc., as opposed to sweeping updates about that whole class of actions and whether they can be robustly positive.

I don't remember I'm afraid. I don't recall having seen the article you link to, so I doubt it was that. Maybe it was this one.

Thanks, appreciate it!

Do you have any data you can share on how the population responding to the FTX section/survey differs from the full EAS survey population? E.g. along dimensions like EA engagement, demographics, ... – Or anything else that could shed light on the potential selection effect at this stage? (Sorry if you say this somewhere and I missed it.) Thanks for all your work on this!

This isn't quite what you're looking for because it's more a partial analogy of the phenomenon you point to rather than a realistic depiction, but FWIW I found this old short story by Eliezer Yudkowsky quite memorable.

In the short-term (the next ten years), WAW interventions we could pursue to help wild animals now seem less cost-effective than farmed animal interventions.

Out of curiosity: When making claims like this, are you referring to the cost-effectiveness of farmed animal interventions when only considering the impacts on farmed animals? Or do you think this claim still holds if you also consider the indirect effects of farmed animal interventions on wild animals?

(Sorry if you say this somewhere and I missed it.)

I like this idea! Quick question: Have you considered whether, for a version of this that uses past data/conjectures, one could use existing data compiled by AI Impacts rather than the Wikipedia article from 2015 (as you suggest)?

(Though I guess if you go back in time sufficiently far, it arguably becomes less clear whether Laplace's rule is a plausible model. E.g., did mathematicians in any sense 'try' to square the circle in every year between Antiquity and 1882?)

Tail-effects in education: Since interventions have to scale, they end up being mediocre to "what could be possible."

Related: Bloom's two-sigma problem:

Bloom found that the average student tutored one-to-one using mastery learning techniques performed two standard deviations better than students educated in a classroom environment with one teacher to 30 students

(haven't vetted the Wikipedia article or underlying research at all)