On focusing resources more on particular fields vs. EA per se - considerations and takes

Arden Koehler

On focusing resources more on particular fields vs. EA per se - considerations and takes

Arden Koehler

10 min read · Jun 24, 2023

138

Comments 23

Sorted by

New & upvoted

jackva

Thanks so much for this, I think it's really valuable to have this on the forum and start a structured conversation on this.

To me, the first reason mentioned against separate field building -- declining critical thinking and questioning within subfields -- seems really key.

I think a subfield-oriented loose movement structure would probably lead to decline of epistemic standards and an underplaying of cause prioritization, differences in expected impact, etc as a result of groupthink and appealing to non-EA parts of these fields and I think to some degree this is already happening.

Deogin

Where would you say this is already happening?

jackva

I don't want to turn this into an argument about specific orgs because it is more of a general observation and more about incentive structures.

I have the general impression that single-cause organizations do not face the right incentives for truth seeking and cause prioritization, both in terms of epistemics (everyone working there sharing some beliefs) and organizational strategy and goals (the bar for an org saying that what they worked on has become less important seems v high, indeed fundraising incentives will always point in the other direction).

To focus more on the positive case which I find easier to talk about publicly: Being in a team (research team at FP) where researchers are working on different causes, with different beliefs and different methodologies, brings a lot of benefits in all directions -- e.g. encouraging GHD research to be more risk-neutral (not combining topical engagement with risk aversion, as GiveWell does), to work on different GCRs being more comparative, to work on climate being benchmarked against other risks, etc.

This is not an all-things-considered view, all I am really saying is that I think this argument against sub-field building seems quite important.

William_MacAskill

Strong upvote on this - it’s an issue that a lot of people have been discussing, and I found the post very clear!

There’s lots more to say, and I only had time to write something quickly but one consideration is about division of effort with respect to timelines to transformative AI. The longer AI timelines are, the more plausible principles-led EA movement-building looks.

Though I’ve updated a lot in the last couple of years on transformative-AI-in-the-next-decade, I think we should still put significant probability mass on “long” timelines (e.g. more than 30 years). For example, though Metaculus’s timelines have shortened dramatically — and now suggest a best-guess of superintelligence within a decade — the community forecast still puts a 10% chance that even “weakly general” AI is only developed after 2050, and it puts about a 20% chance that the time from weakly general AI to superintelligence is more than 30 years. AI progress could slow down a lot, perhaps because of regulation; there could also be some bottleneck we haven’t properly modelled that means that explosive growth never happens. (Relatedly, some things could happen that could make near-term effort on AI less important: AI alignment could end up being easy, or there could be sufficient effort on it, such that the most important questions are all about what happens post-AGI.)

In very short (e.g. <10yr) timelines worlds, then an AI-safety specific movement looks more compelling.

In long timelines worlds (e.g. >30 years), EA looks comparatively more promising for a few reasons:

EA is more adaptive over time. If our state of knowledge changes, or if the environment changes, such that X thing is no longer the best thing to do, then the EA recommendation changes, and (ideally) someone who follows EA reasoning will switch to the better thing.
- This is much more likely to be relevant in long timelines worlds: there are decades for things to change (e.g. geopolitical changes, development of new technologies), for there to be further learning about issues relevant to cause-prioritisation, and for the world as a whole to invest dramatically more into AI safety.
Relatedly, EA is accessible to more people. A greater diversity of skills is more useful in long timelines worlds than they are in short timelines worlds, because of greater uncertainty about what will be of most value in long timelines worlds.
Principles-focused EA might have greater long-term compounding benefits (as a result of EAs doing outreach to recruit new EAs, some of whom do outreach to recruit new EAs, in a way similar to the compounding of financial investment over time) than cause-specific movements. I started trying to write this out quantitatively but ran out of time and there are a lot of subtle issues, so I’m not certain how exactly it shakes out; it’s something I’ll have to come back to. (Even better, someone better at maths than me could do this instead!)

This suggests that we might want to focus in particular on both short and fairly long timelines worlds (and pay less attention to in-the-middle timelines worlds). In short timelines worlds, we have outsized impact because the world hasn’t yet caught up to the importance of safely managing advanced AI and the issue is highly neglected. In fairly long timelines worlds, we get particular benefits from the long-run compounding of EA, and its diversity and adaptability.

This could look like some people focusing on AI for the next ten years, and then potentially switching after that time for the rest of their careers. Or, and in addition, it could look like some % of EA-minded people focusing wholly on AI, and some % focusing wholly on principles-first EA.

A final consideration concerns student groups in particular. For someone aged 18 — unless they are truly exceptional — it’ll probably be at least 5 years before they are able to meaningfully contribute to AI safety. If you think that the next 10 years are the particularly-likely time to get an intelligence explosion (because of an expected but unsustainable huge increase in investment into AI, as discussed by Carl Shulman here), then half of that opportunity can’t be capitalised upon by the 18-year-old. This gives an additional boost to the value of EA movement-building vs AI safety specific movement-building when it comes to campus outreach.

Overall, I agree that given the rapid progress in AI there should be some significant reallocation from EA movement-building to AI in particular, and I think we’re already seeing this happening. I’m currently unsure on whether I expect EA as a whole to ultimately under-correct or over-correct on recent AI developments.

Ben_West🔸

Thanks for this comment; I found it helpful and agree with a lot of it. I expect the "university groups are disproportionately useful in long timelines worlds" point to be useful to a lot of people.

On this bit:

EA is more adaptive over time... This is much more likely to be relevant in long timelines worlds

This isn't obvious to me. I would expect that short timeline worlds are just weirder and changing more rapidly in general, so being adaptive is more valuable.

Caricature example: in a short timeline world we have one year from the first sentient LLM to when we achieve value lock in, and in a long timeline world we have 100 years. In the former case EA seems more useful, because we can marshal a bunch of people to drop what they are doing and focus on digital sentience. In the latter case digital sentience probably just becomes an established field, without any need for EA.

There are counterbalancing factors (e.g. more of our resources should probably go towards AI safety, in short timelines worlds) but it seems pretty plausible to me that this nets out in favor of EA being more useful in shorter timelines.^[1]

^{^}
JP has made a related point to me, and deserves credit for a lot of this idea

JP Addison🔸

I like this comment a whole bunch.

This suggests that we might want to focus in particular on both short and fairly long timelines worlds [...]

I've recently started thinking of this as a playing to your outs strategy, though without the small probabilities that that implies. One other factor in favor of believing that long timelines might happen, and those worlds might be good worlds to focus on, would be that starting very recently it's begun to look possible to actually slow down AI. In those worlds, it's presumably easier to pay an alignment tax, which makes those world more likely to survive.

Chris Leong

For someone aged 18 — unless they are truly exceptional — it’ll probably be at least 5 years before they are able to meaningfully contribute to AI safety.

This might be true for technical. Less true for things like trying to organise a petition or drum up support for a protest.

Jaime Sevilla

I'm pretty into the idea of putting more effort into concrete areas.

I think the biggest reason for is one which is not in your list: it is too easy to bikeshed EA as an abstract concept and fool yourself into thinking that you are doing something good.

Working on object level issues helps build expertise and makes you more cognizant of reality. Tight feedback loops are important to not delude yourself.

JP Addison🔸

I'm curating this post. I think the arguments presented here are not written up anywhere else publicly, and are well-put. I would like to see more discussion of them, and more exploration.

freedomandutility

I’d like to add that from my perspective:

global health and development will almost permanently be a pressing cause area
it’s very likely that within our lifetimes, we’ll see enough progress such that biosecurity and farmed animal welfare no longer seem as pressing as global health and development
it’s feasible that AI safety will also no longer seem as pressing as global health and development
new causes may emerge that seem more pressing than global health, biosecurity, AI and farmed animal welfare
growing EA is very important to help more people with “optimiser’s mindsets” switch between cause areas in response to changes in how pressing they are in the future

(but I still think there’s a case for a modest reallocation towards growing cause areas independently)

Linch

global health and development will almost permanently be a pressing cause area

I don't find this likely over the medium-long term, unless we have large-scale stagnation or civilizational collapse.

As a sanity check, 4% economic growth (roughly the rate in Africa in the last 20 years; many developing countries have higher growth rates) over 100 years translates to 50x growth. So under such conditions, the equivalent (positionally) of someone living on $1.50 today for someone in 2123 is someone living on $75/day or $27k/year in 2023 dollars, well into the standards of developed countries today. ^[1]

Will there still be large economic disparities and/or significant health problems sans intervention? Well, yeah, probably. But in a world that's fabulously wealthy, a dire need case for drastic EA interventions doesn't just rely on market failure but also state and civil society failure as well. So you should expect many problems that today look like global health and development to be solved then via more traditional public health and civil society means.

I also think it's reasonably plausible that progress in public health will outstrip GDP improvements, such that people living on (say) $5k/year in the future will not have many of the public health problems associated with people living on $5k/year today. Both because of specific technological changes and because catchup growth on health might be faster than catchup growth on development (similar to how people in countries with median incomes of $5k/year today do not have to worry about smallpox or polio, or (in most cases) even malaria).^[2]

And of course crazy AI stuff might make all of this close to irrelevant.

^{^}
This might be too optimistic. But even at 3% GDP growth (roughly global average), we're looking at a 20x increase, or about $10k/year for someone positionally equivalent to living on $1.50/day today.
^{^}
Eyeballing some graphs, US real GDP per capita was between 6k and 9k in the 1930s (~9k at the beginning and the end, middle was lower. The Great Depression was rough). There was significant malaria in the US during that time, and of course it'd be multiple decades before smallpox and polio were eradicated in the US.

JKM

global health and development will almost permanently be a pressing cause area

Really? You don't think there will come a time perhaps in the next few centuries where pretty much everyone lives above an absolute poverty line and has access to basic health resources? Our World in Data shows the progress that has been made in reducing extreme poverty, and whilst there is more work to do, saying that global health and development will "permanently" be a pressing cause areas seems an exaggeration to me.

Also, if you're a longtermist now, you'll probably be a longtermist later even if we do reduce existential risks from AI and bio etc. There are other longtermist interventions that we are aware of such as improving values, investing for the future, economic growth.

freedomandutility

I think when we reach the point where everyone lives above a certain poverty line and has access to basic health resources, the global distribution of wealth will still be very inefficient for maximising welfare, and redistributing resources from the globally richest to the globally poorest / increasing consumption of the poorest will still be one of the best available options to improve aggregate welfare.

Sidenote - but I think it's better for debate to view this as disagreement, not exaggeration. I also don't entirely agree with total utilitarianism or longtermism, if that makes my point of view easier to understand.

JKM

I think when we reach the point where everyone lives above a certain poverty line and has access to basic health resources, the global distribution of wealth will still be very inefficient for maximising welfare

Agreed.

redistributing resources from the globally richest to the globally poorest / increasing consumption of the poorest will still be one of the best available options to improve aggregate welfare.

I highly doubt this. Longtermism aside, I find it very hard to accept that redistribution of resources in a world where everyone lives above a certain poverty line would be anywhere near as pressing as reducing animal suffering.

I think it's better for debate to view this as disagreement, not exaggeration.

Fair enough. "Permanently" is a very strong word though so I guess I disagree with its usage.

Jason

The quote was "almost permanently," which I took to mean something like: of sufficient permanence that for purposes of the topic of the post -- focusing on cause areas vs. focusing on EA (in the medium run is implied to me) -- we can assume that global health and development will remain a pressing cause area (although possibly relatively less pressing than a new cause area -- point four).

I don't think that's inconsistent with a view that GHD probably won't be a pressing cause area in, say, 250 years. Knowing whether it will or won't doesn't materially affect my answer to the question posed in the original post. (My ability to predict 250 years into the future is poor, so I have low confidence about the importance of GHD at that time. Or just about anything else, for that matter.)

Anyway, I am wondering if part of the disagreement is that we're partially talking about somewhat different things.

JKM

I might be wrong but I think "almost" was an addition and not there originally. It still reads weirdly to me.

From the follow-on comments I think freedomandutility expects GHD to be a top cause area beyond 250 years from now. I doubt this and even now I put GHD behind reducing animal suffering and longtermist areas so there does seem to be some disagreement here (which is fine!).

EDIT: actually I am wrong because I quoted the word "almost" in my original comment. Still reads weird to me.

Jason

I also think that most future worlds in which humanity has its act together enough to have solved global economic and health security are worlds in which it has also managed to solve a bunch of other problems, which makes cause prioritization in this hypothetical future difficult.

Jason

I wouldn't view things like "basic health resources" in absolute terms. As global prosperity and technology increases, the bar for what I'd consider basic health services increases as well. That's true, to a lesser extent, of poverty more generally.

JKM

Sure, but there is likely to be diminishing marginal utility of health resources. At a certain point accessibility to health resources will be such that it will be very hard to argue that boosting it further would be a more pressing priority than say reducing animal suffering (and I happen to think the latter is more pressing now anyway).

Jason

One could say much the same thing about almost any cause, though -- such as "investing for the future, economic growth" at the end of your comment. The diminishing marginal returns that likely apply to global health in a world with 10x as many resources will generally be applicable there too.

JKM

Different causes have different rates at which marginal utility diminishes. Some are huge problems so we are unlikely to have even nearly solved them in a 10x richer world (e.g. wild animal suffering?) and others can just absorb loads of money.

Investing for the future is one such example - we can invest loads of money with the hope that one day it can be used to do a very large amount of good.

Also, in a world where we are 10x richer I'd imagine reducing total existential risk to permanently minuscule levels (existential security) will still be a priority. This will likely take loads of effort and I'd imagine there is likely to always be more we can do (even better institutions, better safeguards etc.). Furthermore, in a world where we are all rich, ensuring safety becomes even more important because we would be preserving a really good world.

Ben_West🔸

Thanks for posting this – I feel like these discussions have been happening in the background, and I would like to see more of it in public.

Minor: your "explain some reasons" link is broken

Chris Leong

If EA wanted to invest more effort into specific cause areas, it might want to consider running intro courses in those areas as part of EA virtual programs.

Comments

On focusing resources more on particular fields vs. EA per se - considerations and takes

On focusing resources more on particular fields vs. EA per se - considerations and takes

Overall take

Reasons I think people might have for a shift away from EA per se, and my quick takes on them

Some reasons for continuing to devote considerable resources to EA per se community building: