Karthik Tadepalli's Quick takes

During the animal welfare vs global health debate week, I was very reluctant to make a post or argument in favor of global health, the cause I work in and that animates me. Here are some reflections on why, that may or may not apply to other people:

1. Moral weights are tiresome to debate. If you (like me) do not have a good grasp of philosophy, it's an uphill struggle to grasp what RP's moral weights project means exactly, and where I would or would not buy into its assumptions.
2. I don't choose my donations/actions based on impartial cause prioritization. I think impartially within GHD (e.g. I don't prioritize interventions in India just because I'm from there, I treat health vs income moral weights much more analytically than species moral weights) but not for cross-cause comparison. I am okay with this. But it doesn't make for a persuasive case to other people.
3. It doesn't feel good to post something that you know will provoke a large volume of (friendly!) disagreement. I think of myself as a pretty disagreeable person, but I am still very averse to posting things that go against what almost everyone around me is saying, at least when I don't feel 100% confident in my thesis. I have found previous arguments about global health vs animal welfare to be especially exhausting and they did not lead to any convergence, so I don't see the upside that justifies the downside.
4. I don't fundamentally disagree with the narrow thesis that marginal money can do more good in animal welfare. I just feel disillusioned with the larger implications that global health is overfunded and not really worth the money we spend on it.

I'm deliberately focusing on emotional/psychological inhibitions as opposed to analytical doubts I have about animal welfare. I do have some analytical doubts, but I think of them as secondary to the personal relationship I have with GHD.

Often people post cost-effectiveness analyses of potential interventions, which invariably conclude that the intervention could rival GiveWell's top charities. (I'm guilty of this too!) But this happens with such frequency, and I am basically never convinced that the intervention is actually competitive with GWTC. The reason is that they are comparing ex-ante cost-effectiveness (where you make a bunch of assumptions about costs, program delivery mechanisms, etc) with GiveWell's calculated ex-post cost-effectiveness (where the intervention is already delivered, so there are much fewer assumptions).

Usually, people acknowledge that ex-ante cost-effectiveness is less reliable than ex-post cost-effectiveness. But I haven't seen any acknowledgement that this systematically overestimates cost-effectiveness, because people who are motivated to try and pursue an intervention are going to be optimistic about unknown factors. Also, many costs are "unknown unknowns" that you might only discover after implementing the project, so leaving them out underestimates costs. (Also, the planning fallacy in general.) And I haven't seen any discussion of how large the gap between these estimates could be. I think it could be orders of magnitude, just because costs are in the denominator of a benefit-cost ratio, so uncertainty in costs can have huge effects on cost-effectiveness.

One straightforward way to estimate this gap is to redo a GiveWell CEA, but assuming that you were setting up a charity to deliver that intervention for the first time. If GiveWell's ex-post estimate is X and your ex-ante estimate is K*X for the same intervention, then we would conclude that ex-ante cost-effectiveness is K times too optimistic, and deflate ex-ante estimates by a factor of K.

I might try to do this myself, but I don't have any experience with CEAs, and would welcome someone else doing it.

I often return to this bit of 80000 Hours' anonymous career advice, about how when you're great at your job, no one's advice is that useful.

I think in general if you want to accomplish something, or you want to be good at something, or you want something to happen — you want to throw yourself into it, and get really deep into it. Focus on it. Obsess over it. If you don’t, you probably won’t get the result you want. And if you do — you’re going to find that advice from other people often isn’t that useful. By “advice” here I mean talking to people who aren’t paying much attention to what you’re doing and don’t know you very well and are just reacting to your description of the situation and your questions. And I’m assuming this isn’t a case where you’re asking the world’s expert on X to answer a narrowly defined question that they’re clearly the best person to answer because it’s 100% about X.

I think EAs tend to ask for a lot of advice, as defined above. And they should, I think asking for advice is good. But I think it’s good to understand what that advice is and isn’t. And usually when you’re doing your job really well, no one’s advice is going to be that useful. They’re never going to know what you should do. They’re never going to know as much as you do, and if they do, you’re not doing it right.

I like it a lot. It reminds me of Agnes Callard's observation about a young writer asking Margaret Atwood for advice and getting only the trite advice to "write every day":

The young person is not approaching Atwood for instructions on how to operate Microsoft Word, nor is she making the unreasonable demand that Atwood become her writing coach. She wants the kind of value she would get from the second, but she wants it given to her in the manner of the first. But there is no there there. Hence the advice-giver is reduced to repeating reasonable-sounding things she has heard others say—thoughts that are watered down so far that there’s really no thought left, just water.

How do we find new interventions that can beat the best? The main approach has been to seek leverage; interventions where you pay a dollar to move a larger amount of money/resources to a cause (e.g. advocacy interventions. effective giving campaigns). But people don't often recognize market-based leverage; interventions where you pay a dollar to enable a market transaction whose value is much larger than that dollar. Two examples:

1. High-school educated workers in urban South Africa struggle to find their first job, because school performance is a very weak signal of ability, and businesses are reluctant to take chances because firing workers is costly. Harambee assesses job-seekers' skills and gives them certification that they can show to employers as a credible signal. Getting the certification increases their chances of finding a job. Thus, Harambee increased their income without giving them any money, just by fixing an information problem in the labor market.
2. Rug manufacturers in Egypt would like to export their rugs for big profit. An unnamed German retailer would like to buy these rugs, since they are cheaper than alternatives. However, the global market is huge, and these two parties have no way of getting in touch with each other, or knowing that they are mutually interested in a sale. Aid to Artisans helps the local intermediary for the rug manufacturers get in touch with the German retailer, helps them travel to meet with the retailer to provide samples of their products, and arranges an initial large order. The rug manufacturers gain a reputation, and over the next four years, they get orders totalling $150,000. Thus, ATA increased the incomes of all the artisans without giving them any money, just by connecting them with buyers who wanted to give them money.

These interventions get their leverage from fixing market failures. Both of these examples are failures of information, which are both ubiquitous and cheap to fix. I suspect that it shouldn't be too hard to find one where spending $1 generates more than $10 in income, which is roughly the bar for a GiveWell top charity.

People often justify a fast takeoff of AI by pointing to how fast AI could improve beyond some point. But The Great Data Integration Schlep is an excellent LW post about the absolute sludge of trying to do data work inside corporate bureaucracy. The key point is that even when companies seemingly benefit from having much more insights into their work, a whole slew of incentive problems and managerial foibles prevent this from being realized. She applies this to be skeptical of AI takeoff:

If you’re imagining an “AI R&D researcher” inventing lots of new technologies, for instance, that means integrating it into corporate R&D, which primarily means big manufacturing firms with heavy investment into science/engineering innovation (semiconductors, pharmaceuticals, medical devices and scientific instruments, petrochemicals, automotive, aerospace, etc). You’d need to get enough access to private R&D data to train the AI, and build enough credibility through pilot programs to gradually convince companies to give the AI free rein, and you’d need to start virtually from scratch with each new client. This takes time, trial-and-error, gradual demonstration of capabilities, and lots and lots of high-paid labor, and it is barely being done yet at all.

This is also the story of computers, and the story of electricity. A transformative new technology was created, but it took decades for its potential to be realized because of all the existing infrastructure that had to be upended to maximize its impact.

In general, even if AI is technologically unprecedented, the social infrastructure through which AI will be deployed is much more precedented, and we should consider those barriers as actually slowing down AI impacts.

The True Believer by Eric Hoffer is a book about the psychology of mass movements. I think there are important cautions for EAs thinking about their own relationship to the movement.

There is a fundamental difference between the appeal of a mass movement and the appeal of a practical organization. The practical organization offers opportunities for self-advancement, and its appeal is mainly to self-interest. On the other hand, a mass movement, particularly in its active, revivalist phase, appeals not to those intent on bolstering and advancing a cherished self, but to those who crave to be rid of an unwanted self. A mass movement attracts and holds a following not because it can satisfy the desire for self-advancement, but because it can satisfy the passion for self-renunciation.

I wanted to write a draft amnesty post about this, but I couldn't write anything better than this Lou Keep essay about the book, so I'll just recommend you read that.

Shooting from the hip here - if the future of AI progress is inference-time scaling, that seems inherently "safer"/less prone to power-seeking. Expensive inference means that a model is harder to reproduce (e.g. can't just upload itself somewhere else, because without heavy compute its new version is relatively impotent) and harder for rogue actors to exploit (since they will also need to secure compute for every action they make it do).

If this is true, it suggests that AI safety could be advanced by capabilities research into AI architecture that can be more powerful yet also more constrained in individual computations. So is it true?

One essay that I thought about a lot growing up is The Quality of Life. It argues that the metrics of wellbeing that we optimize for are basically meaningless, the dreams of bureaucrats rather than of people.

Why isn’t the most important financial threshold in the inner lives of many, rich or poor, the subjective notion of fuck-you money, the first thing to study? Why isn’t there a major UN study tracking what people consider fuck-you money? Why aren’t Nobel-winning behavioral economists designing clever experiments to tease out how we think about this quantity? It is, after all, our main subjective measure of how not-free we perceive ourselves to be.

Nobody, other than bureaucrats who fund research and economists, asks the question “how much income is needed to be happy?” We already know that talking about happiness without talking about what trade-offs we are making to pursue it is meaningless. The rest of us real people ask the question “how much wealth is required to be free of scripts that dictate what trade-offs you are allowed to make?”

It does not really matter if you generalize beyond income to various in-kind quality-of-life elements like a clean environment or access to healthcare. If you are not measuring prevailing levels of freedom you are measuring nothing relevant. Until people start answering $0 to the fuck-you-money question across the planet, you can be sure that they do not perceive themselves to be free enough to properly pursue quality of life.

This essay definitely had an impact on me, highlighting the agency benefits of cash transfers and pushing me towards a large recurring GiveDirectly donation. Re-reading it today, it feels terribly confusing and difficult to grok. But it feels difficult to grok in a way that suggests it's not a confused or incoherent or obviously wrong take. It's just pointing out the water we swim in, which is a valuable thing for more EAs to keep in mind.

I find it encouraging that EAs have quickly pivoted to viewing AI companies as adversaries, after a long period of uneasily viewing them as necessary allies (c.f. Why Not Slow AI Progress?). Previously, I worried that social/professional entanglements and image concerns would lead EAs to align with AI companies even after receiving clear signals that AI companies are not interested in safety. I'm glad to have been wrong about that.

Caveat: we've only seen this kind of scrutiny applied to OpenAI and it remains to be seen whether Anthropic and DeepMind will get the same scrutiny.