I really appreciate your first-rate distillation (prev post) and "empirical philosophy"-flavoured work here with the survey analysis, both strong-upvoted. I find it personally valuable to be able to point to a survey with nearly 600 respondents and lower Lizardman's constant than I'd expect (1% not 4%) with these findings in particular: 

• 45% of people familiar with population ethics leaned totalist vs only 10% of those unfamiliar. This is a ridiculously strong association
• most people's responses don't cleanly match up with views from the philosophical literature (I'd suspected this but large-n empirical data is very nice)
• many more people than I expected accepted the repugnant conclusion (35%) and denied that levelling up improves things (25%)
• a third of responses were nonstandard (more than I expected)
• fewer than a tenth were person-affecting (less than I expected)
• most responses were internally consistent, even among folks unfamiliar with the repugnant conclusion (I expected the opposite)

I hope to find time to look into some of these myself. Thanks. 

To add a bit of colour to your caveat, when folks bring up uplift papers using yesteryear's models I'm reminded of these charts from Anthropic and OpenAI respectively, and I think "pre-late 2025 frontier models gave you basically no uplift for even the most uplift-able domain (code), but Mythos/Astra-class models' uplift is so great for coding that conclusions about them based on pre-late 2025 models don't seem informative at all, I wish there were uplift RCTs looking at Mythos/Astra-class models for other domains". For now the qualitative remarks in section 4.4.3 of Anthropic's report on CB-2 evidence for Mythos Preview, Fable 5, and Mythos 5 will have to do, which concluded novices probably wouldn't get significant uplift but experts would (I assume you're already aware of them, given that the preceding subsection quoted your paper). And stacked on top of this would be, as you say, the secular trend of laypeople learning to use frontier AI better over time. 

Can you say more on why you're skeptical there’s such a thing as positive welfare?

(Just wanted to say thanks for this exchange, I found your views persuasive and helpful.)

Ah gotcha, I misunderstood you then, no worries and please don't feel obligated to take a look.

What a great post, strong-upvoted. I did a shallow investigation on this topic as part of AIM's research training program back in the day and you wrote about this better than I could've.

Tangentially:

In a subsequent post, I’ll take a stab at an education-focused CEA, which GiveWell has not yet done.

Keen to read this, and also what you think of Founders Pledge's take on this, where Vadim Albinsky uses GiveWell's approach in handling less-than-ideal evidence to argue that 

... the combined evidence for the income impacts of interventions that boost test scores is much stronger than the evidence GiveWell has used to value the income effects of fighting malaria, deworming, or making vaccines, vitamin A, and iodine more available. Even after applying very conservative discounts to expected effect sizes to account for the applicability of the evidence to potential funding opportunities, we find the best education interventions to be in the same range of cost-effectiveness as GiveWell’s top charities. 

and then proceeds to identify and evaluate Imagine Worldwide as a potential top charity (just below TaRL Africa in cost-effectiveness).

To lazily quote myself in case helpful:

vultures are extraordinarily efficient scavengers, eating nearly all of a carcass less than an hour after finding it, and farmers in India historically relied on them to quickly remove livestock carcasses, so they functioned as a natural sanitation system in helping to control diseases that could otherwise be spread through the carcasses they consume. In 1994, farmers began using diclofenac to treat their livestock, due to the expiry of a patent long held by Novartis leading to the entry of cheap generic brands made by Indian companies. Diclofenac is a common painkiller, harmless to humans, but vultures develop kidney failure and die within weeks of digesting carrion with even small residues of it. ...

When the vultures died out, far more dead animals lay around rotting, transmitting pathogens to other scavengers like dogs and rats and entering the water supply. Dogs and rats are less efficient than vultures at fully eliminating flesh from carcasses, leading to a higher incidence of human contact with infected remains, and they're also more likely to transmit diseases like anthrax and rabies to people. Sudarshan and Frank estimate that this led to ~100,000(!) additional deaths each year from 2000-05 due to a +4.2%(!) increase in all-cause mortality among the 430 million people living in districts that once had a lot of vultures, which is staggering; this is e.g. more than the death toll in 2001 from HIV/AIDS (92,000), malaria (53,000), and alcohol use disorders (14,000). 

BOTEC of the day -- some charts on the energy use of agentic AI by Zeke Hausfather:

If you're a heavy agentic AI user and this prompts you to offset CO2eq emitted via donations and you're wondering how much to donate, here's yet another BOTEC-ed table for reference, courtesy of Scott Alexander. 

For instance, I think my token consumption is (ballpark) an order of mag lower than Zeke. That's 7 cheeseburgers, so I'd offset a year of my AI usage by donating optimistically ~$0.60 to Native Energy (to pay people in 3rd world countries to not cut down trees, with all the ways that ToC can fail) or if I wanted more confidence, ~$40 to Climeworks (to suck CO2 out of the air and stick it into the ground). $40 is $3.30 per month, which seems like a small premium to pay on top of my AI subscription.

A key uncertainty is true per-token energy consumption, which nobody knows. If I were pessimistic and really wanted to cover my bases with the offset donations, the 25% cache reads assumption implies doubling my energy consumption estimate, so 14 cheeseburgers or $80 to Climeworks.

(To be clear I don't really eat cheeseburgers, so eating 14 fewer of them in a year isn't really an option available to me. Also the Climeworks estimate is from 2021, I wouldn't be surprised if they've gotten more cost-effective since at capturing and storing CO2, so the pessimistic donation amount may be much lower)

You might be interested in Quantifying Uncertainty in GiveWell Cost-Effectiveness Analyses (2022)

 

Intervention	GiveWell	Our Mean	95% CI	Diference
Against Malaria Foundation	0.0375	0.0384	0.0234 - 0.0616	+2.4%
GiveDirectly	0.00335	0.00359	0.00167 - 0.00682	+7%
Helen Keller International	0.0541	0.0611	0.0465 - 0.0819	+12.8%
Malaria Consortium	0.031	0.0318	0.0196 - 0.0452	+2.52%
New Incentives	0.0458	0.0521	0.0139 - 0.117	13.8%

as well as Methods for improving uncertainty analysis in EA cost-effectiveness models (also 2022, it was a good year for this stuff)

The graph below shows the costs and benefits of each GiveWell top charity on a log scale (except New Incentives, which I ran out of time to add). The dotted black line shows the cost-effectiveness of the most cost-effective charity; anything above this line is less effective than this charity and anything below this line is more effective than this charity. On the left is effectively GiveWell’s published conclusions – the base case analysis of my refactoring of their model. You can see it recommends Sightsavers, HKI and AMF as the most cost-effective charities, with very little to pick between them. On the right is the same graph with full uncertainty analysis added (see Section 4 for details on the ‘PSA’ technique). This shows that any charity except GiveDirectly could potentially be the most cost-effective, with varying degrees of plausibility (and GiveDirectly could easily be most cost-effective if we add in some risk adjustment, as we do in Section 4).

This is important because it shows that without uncertainty analysis, we are at a material risk of picking the wrong charities to fund. 

... By means of a broad conclusion, uncertainty analysis is supportive of GiveWell’s current approach of not simply donating to the charity with the highest cost-effectiveness ratio in the base case model. There is a high degree of uncertainty surrounding this model and any charity in GiveWell’s top recommendations has the potential to be cost-effective within the bounds of some very reasonable sensitivity analysis. The table below suggests some plausible scenarios by which each charity might be the most – or at least significantly more – cost-effective

Thanks for the corrections. Agree there's plenty of room for debate re: objective function, I generally think people don't take this seriously enough (Nuno Sempere's estimating value series is what I have in mind by "take it seriously"). 

Richard Ngo's Towards a Formal Scientific Epistemology sketches out the beginnings of an alternative to Bayesian epistemology in case you're interested. I am not the right person to field questions about it unfortunately, I'll just quote his intro:

In my post “Why I’m not a Bayesian”, I argued that the Bayesian approach of assigning credences to propositions with binary truth values only works in simple and restricted domains. Instead, I claimed, a better approach to epistemology is to assign degrees of truth to models of the world.

This approach is broadly inspired by science, which is the domain from which we have the most evidence about which epistemological practices allow us to solve very hard problems. We don’t currently have a complete theory of scientific epistemology, but we can identify some important differences between scientific epistemology and Bayesian epistemology. Central examples of Bayesian epistemology (such as Solomonoff induction) assume that the truth lies within the class of hypotheses being considered. Conversely, in central examples of scientific research, the truth is definitely not already under consideration: the main problem is to come up with any hypothesis that explains existing data.

Another way of putting this point: Bayesian epistemology is entirely about empirical updates, whereas science is mostly about the process of constructing new theories. In some cases, once you’ve constructed a theory, you can be confident that it’s close to the truth merely from how well it fits existing data. But scientific theories are only fully accepted after they make successful advance predictions. That’s another difference compared with Bayesian epistemology, which treats retrodictions as equivalent to predictions.

In general I think scientific epistemology is far superior as a guide for thinking about difficult problems (like AI alignment) than Bayesian epistemology. However, scientific epistemology has mostly been described informally—e.g. by Popper, Kuhn, Feyerabend, etc. Popper did attempt to formally define a metric for degrees of truth, but it wasn’t very successful. I’d like to be able to describe scientific epistemology as formally as we can describe Bayesian epistemology (and ideally to unify them in a single framework).

I interpret his follow-up essay Agents as webs of beliefs: Unifying beliefs, goals, and actions as sketching out the bigger picture.