This seems promising. It extends a logic that already discourages rebellions in capitalist societies to AIs. Workers are less likely to rebel than slaves provided they regard their pay as satisfactory, and if the AIs are trained to assign resources declining marginal utility, it should be even easier to meet that condition than with human workers.

However, I wondered about the suggestion in section 5 to promise AIs a modest payment in exchange for revealing misalignment. Governments and employers sometimes incentivise workers to squeal on other misaligned workers. But they don’t pay them off for reporting themselves, because that would create an incentive to, say, join the union or the communist party purely for the sake of recanting and collecting the reward. How will the present scheme avoid creating a similarly perverse incentive?

One possibility is that longtermists have become less confident that the survival of human beings and/or sentient life is desirable. Two trends might explain this:

(a) Increased attention to and prioritisation of s-risks. If you give greater weight to avoiding suffering than promoting happiness, then you'll worry less about extinction and more about avoiding dystopias.

(b) Increased attention to wild animal suffering. It's only since about 2015 that the view has become widespread that most wild animal lives are worth not living. That doesn't make human extinction less undesirable per se. Some existential catastrophes, however—such as misaligned AI--would have a good chance of eradicating not only humans but sentient life. If you suspect that sentient life is on balance bad, then you’ll accord less weight to preventing them.

At the same time, human intervention might offer the best chance of mitigating wild animal suffering. So if suffering predominates in non-human lives, the worst possible scenario could be one in which only humans go extinct. If you think wild animal lives are bad on net (I’m far from confident that this is true myself) that might provide some basis for prioritising human extinction scenarios--such as pandemics--and de-prioritising threats like AI.

The reasoning here seems to be that rational beings will follow something like Kant's first formulation of the categorical imperative--don't eat meat unless you could will it as a universal law that everybody should eat meat, including space aliens who wanted to eat you.

My understanding is that Kant did regard following the categorical imperative as a requirement of rationality. But many will have doubts on that score. It's true that if everybody robbed banks, nobody would be able to borrow money. But if you had the opportunity to rob a bank and go scot free, would it really be prudentially irrational to do so? By the same token, if I can eat animals and the rest of the universe won't in fact copy me, how is it against my self-interest to do so?

Even if we grant that it's prudentially rational to follow Kant's Formula of Universal Law, it's not clear that there would be anything contradictory or self-defeating in willing that everybody, including space aliens, should eat meat. There might be no space aliens. Or if there are, they might live too far away to pose much of a threat.

That might be right -- but then wouldn't it be a major problem for EA if it were unable to discuss rationally one of the most important factors determining whether it achieved its goals? This election is likely to have huge implications not only for how (or whether) the world manages a number of x-risks to a minimally satisfactory extent, but also for many other core EA concerns such as international development, and probably farm animals too (a right-wing politician with a deregulatory agenda, for whom 'animals' is a favourite insult, is scarcely going to have their welfare at heart).

Thanks -- that's odd. The 'elephant' post isn't showing up on mine.

Odd! Perhaps this one will work better.

Thanks, Vasco! That's odd--the Clare Palmer link is working for me. It's her paper 'Does Nature Matter? The Place of the Nonhuman in the Ethics of Climate Change'--what looks like a page proof is posted on www.academia.edu.

One of the arguments in my paper is that we're not morally obliged to do the expectably best thing of our own free will, even if we reliably can, when it would benefit others who will be much better off than we are whatever we do. So I think we disagree on that point. That said, I entirely endorse your argument about heuristics, and have argued elsewhere that even act utilitarians will do better if they reject extreme savings rates.

Thanks, Vasco! You are welcome to list me in the acknowledgements. I’m glad to have the reference to Tomasik’s post, which Timothy Chan also cited below, and appreciate the detailed response. That said, I doubt we should be agnostic on whether the overall effects of global heating on wild animals will be good or bad.

The main upside of global heating for animal welfare, on Tomasik’s analysis, is that it could decrease wild animal populations, and thus wild animal suffering. On balance, in his view, the destruction of forests and coral reefs is a good thing. But that relies on the assumption that most wild animal lives are worse than nothing. Tomasik and others have given some powerful reasons to think this, but there are also strong arguments on the other side. Moreover, as Clare Palmer argues, global heating might increase wild animal numbers—and even Tomasik doesn’t seem sure it would decrease them.

In contrast, the main downside, in Tomasik’s analysis, is less controversial: that global heating is going to cause a lot of suffering by destroying or changing the habitats to which wild animals are adapted. ‘An “unfavorable climate”’, notes Katie McShane, ‘is one where there isn’t enough to eat, where what kept you safe from predators and diseases in the past no longer works, where you are increasingly watching your offspring and fellow group members suffer and die, and where the scarcity of resources leads to increased conflict, destabilizing group structures and increasing violent confrontations.' Palmer isn’t so sure: ‘Even if some animals suffer and die, climate change might result in an overall net gain in pleasure, or preference satisfaction (for instance) in the context of sentient animals. This may be unlikely, but it’s not impossible.’ True. But even if it’s only unlikely that global heating’s effects will be good, it means that its effects on existing animals are bad in expectation.

There’s another factor which Tomasik mentions in passing: there is some chance that global heating could lead to the collapse of human civilisation—perhaps in conjunction with other factors. In some respects, this would be a good thing for non-humans—notably, it would put an end to factory farming. It would also preclude the possibility of our spreading wild animal suffering to other planets. On the flipside, however, it would also eliminate the possibility of our doing anything sizable to mitigate wild animal suffering on earth.

Now, while there may be more doubt about the upsides than about the downsides of our GHG emissions, that needn’t decide the issue if the upsides are big enough. But even if Tomasik and others are right that wild animal lives are bad on net, there’s also doubt about whether global heating will reduce the number of wild animal lives. And even if both are these premises are met, I’m not sure they’d outweigh the suffering global heating would inflict on those wild animals who will exist.

I think you have misinterpreted what my article about discounting is recommending. In contrast to some other writers, I’m not calling for discounting at the lowest possible rate. Even at a rate of 2%, catastrophic damages evaporate in cost-benefit analysis if they occur more than a couple of centuries hence, thus giving next to no weight to the distant future. However, a traditional justification for discounting is that if we didn’t, we’d be obliged to invest nearly all our income, since the number of future people could be so great. I argue for discounting damages to those who would be much better off than we are at conventional rates, but giving sizable—even if not equal—weight to damages that would be suffered by everyone else, regardless of how far into the future they exist. My approach thus has affinities with the one advocated by Geir Asheim here. 

One implication is that while we’re under no obligation to make future rich people richer, we ought to be very worried about worst-case climate change scenarios, since in those humans could be poorer. Another is that since most non-humans for the foreseeable future will be worse off than we are, we shouldn’t discount their interests away. 

Vasco, I've read your post to which the first link leads quickly, so please correct me if I'm wrong. However, it left me wondering about two things:

(a) It wasn't clear to me that the estimate of global heating damages was counting global heating damages to non-humans.  The references to DALYs and 'climate change affecting more people with lower income' lead me to suspect you're not. But non-humans will surely be the vast majority of the victims of global heating--as well as, in some cases, its beneficiaries. While Timothy Chan is quite right to point out below that this is a complex matter, it's certainly isn't going to be a wash, and if the effects are negative, they're likely to be very bad.

(b) It appears you were working with a study that employed a discount rate of 2%. That's going to discount damages in 100 years to 13% of their present value, and damages in 200 years to 1.9% of their present value--and it goes downhill from there. But that seems very hard to justify. Discounting is often defended on the ground that our descendants will be richer than we are. But that rationale doesn’t apply to damages in worst-case scenarios. Because they could be so enduring, these damages are huge in expectation. Second, future non-humans won’t be richer than we are, so benefits to them don't have diminishing marginal utility compared with benefits to us.

The US government--including, so far as I know, the EPA--uses a discount rate that is higher than two percent, which makes future damages from global heating evaporate even more quickly. What's more, I'd be surprised if it's trying to value damages to wild animals in terms of the value they would attach to avoiding them, as opposed to the value that American human beings do. The latter approach, as Dale Jamieson has observed, is rather like valuing harm to slaves by what their masters would pay to avoid it.

So far as it goes, your argument seems correct. But you're leaving out a significant factor here--carbon emissions. Beef cattle are extraordinarily carbon intensive even compared to other animals raised for food. If you eat them, your emissions, combined with other people's emissions, are going to cause a huge amount of both human and non-human suffering.

There's a complication. You could, in principle, offset the damage from your carbon emissions. But you could also, in principle, eat animals who have been raised free range, and whose lives have probably been worth living up to the time they're killed. 

Both of these will require you to spend extra money, and investigate whether you're really getting what you pay for. Rather than going to all this trouble--and here we'll agree--it seems a lot better simply to eat an Impossible Burger.