Could it be more important to improve human values than to make sure AI is aligned?

Consider the following (which is almost definitely oversimplified):

For clarity, let’s assume dystopia is worse than extinction. This could be a scenario where factory farming expands to an incredibly large scale with the aid of AI, or a bad AI-powered regime takes over the world. Let's assume neutral world is equivalent to extinction.

The above shows that aligning AI can be good, bad, or neutral. The value of alignment exactly depends on humanity’s values. Improving humanity’s values however is always good. 

The only clear case where aligning AI beats improving humanity’s values is if there isn’t scope to improve our values further. An ambiguous case is whenever humanity has positive values in which case both improving values and aligning AI are good options and it isn’t immediately clear to me which wins.

The key takeaway here is that improving values is robustly good whereas aligning AI isn’t - alignment is bad if we have negative values. I would guess that we currently have pretty bad values given how we treat non-human animals and alignment is therefore arguably undesirable. In this simple model, improving values would become the overwhelmingly important mission. Or perhaps ensuring that powerful AI doesn't end up in the hands of bad actors becomes overwhelmingly important (again, rather than alignment).

This analysis doesn’t consider the moral value of AI itself. It also assumed that misaligned AI necessarily leads to extinction which may not be accurate (perhaps it can also lead to dystopian outcomes?).

I doubt this is a novel argument, but what do y’all think?

I have significant concerns with some of the Global Health and Wellbeing (GH&W) work carried out in the EA community, for example by GiveWell or the Happier Lives Institute (HLI). And these aren’t just the commonly-cited ones e.g. that this work isn’t obviously good for the future in expectation or that it ignores animals.

One concern I have is that GH&W work tends to only evaluate existing interventions. The problem with this is that you completely ignore what might be possible with more research. For example, HLI recommends StrongMinds, a mental health charity that treats women with depression in Africa through free, group talk therapy. I’m sure StrongMinds is an excellent charity, but the opportunity cost of giving to StrongMinds is not giving to say world-class researchers looking into psychedelics to treat depression. I’m not saying the money should go to the research - I’m not sure - which is exactly the problem. I’m not sure because organisations like HLI are not investigating the answer. An approach of relying on self-reported wellbeing can only recommend existing interventions, and this may be inhibiting substantial progress.

Further research is only one of multiple things that GH&W work cannot or does not evaluate. Neither GiveWell nor HLI really has the tools to evaluate say climate change interventions - which likely delivers most of its value to future generations. It seems to me that GiveWell could evaluate such interventions, perhaps with difficulty, but doesn’t. Meanwhile I’m not sure how HLI could do so - it’s pretty impossible to evaluate the effect of climate change prevention on self-reported wellbeing. The general criticism here is that GH&W work often ignores (near) future people, which seems unreasonable. And don’t get me started on the far future ones… 

Any thoughts?

When introducing the ‘repugnant conclusion’, Parfit considers that a life barely worth living might be of the “painless but drab” variety, consisting only of “muzak and potatoes”. The mundanity of such a life then gives the repugnant conclusion its repugnance. This is probably the first and only time I will ever say this, but I’m amazed at Parfit’s sloppiness here. A life of just muzak and potatoes isn’t even close to being worth living.

Parfit’s general idea that a life that is barely worth living might be one with no pains and only very minor pleasures seems reasonable enough, but he should have realised that boredom and loneliness are severe pains in themselves. Can anyone honestly tell me that they would choose right now a life of listening to mundane music and eating potatoes with no other pleasures at all, over just being put in a coma? Bear in mind that we currently torture people by putting them in solitary confinement (whilst ensuring they remain fed). I would think the only people who could actually survive muzak and potatoes without going crazy would be buddhist monks who have trained themselves to rid themselves of craving.

Maybe we can remove the boredom and loneliness objections by imagining a life that lasts just a minute and just consists of muzak and potatoes. However that is a bizarre life that differs from any sort of life we can realistically imagine living, so it’s hard to properly judge its quality. If we are going to opine on the repugnance of the repugnant conclusion we need to think up a realistic concept of a life barely worth living. I’m sure such a life is far better than one with just muzak and potatoes.

As I continue to consider the implications of a longtermist philosophy, it is with a heavy heart that the animal-focused side of me feels less enthused about welfare improvements.

This post by Tobias Baumann provides some insight into what might be important when thinking about animal advocacy from a longtermist point of view and is leading me to judge animal advocacy work against three longtermist criteria:

1. Persistence: A focus on lasting change as opposed to just short-term suffering reduction
2. Stability: Low risk of bringing controversy to the effective animal advocacy movement or causing divisions within it
3. Moral Circle Expansion: The potential to expand humanity's moral circle to include anything sentient

I don’t feel that welfare improvements score particularly well against any of these criteria. 

Persistence - Corporate welfare improvements are promises that are made public and reversing them should come with reputational damage, however there are reasons to believe that some companies will fail to follow through with their commitments (see here). It isn't clear how well welfare improvements might persist beyond the short-run into the medium/long-run.

Stability - A fairly healthy contingent in the animal advocacy and even EAA movements, feels uncomfortable about a focus on welfare improvements as this can be seen to implicitly support animal agriculture and so may be counterproductive to the goal of abolition of all animal exploitation.

Moral Circle Expansion - Unclear. It is possible that welfare improvements may make people feel less concerned about consuming animal products, resulting in a persistent lack of concern for their moral status. It is also possible that there is an opposite effect (see here).

I look forward to further work on longtermism and animal advocacy. I suspect such work may redirect efforts away from welfare improvements and towards areas such as legal/legislative work, wild animal suffering, capacity building, cultured meat, and maybe even general advocacy. Whilst I feel slightly uncomfortable about a potential shift away from welfare improvements, I suspect it may be justified.

Might (applied) economists make a resurgence in longtermist EA?

Over the past few years I've had the impression that, within the longtermist EA movement, there isn't really a place for economists unless they are doing highly academic global priorities research at places like the Global Priorities Institute.

Is this set to change given recent arguments about the importance of economic growth / technological progress for reducing total existential risk? There was Leopold Aschenbrenner's impressive paper arguing that we should accelerate growth to raise the amount we spend on safety and speed through the time of perils.

Phil Trammell has also written favourably about the argument, but it still seemed fairly fringe in the movement...until now?

It appears that Will MacAskill is taking a similar argument about the dangers of technological stagnation very seriously now as he said at EA Global in his fireside chat around the 7-minute mark.

Generally I get the impression that not much  has been said about achieving existential security. It seems to me that boosting economic growth may be emerging as one of the most promising ways to do so. Could this mean that working as an economist, even outside of academia, on growth / innovation / technological progress becomes a very credible path e.g. in think tanks or government etc.? Are economists about to make a resurgence?

Any thoughts welcome!