I think the key thing to test is when plant based substitutes are significantly cheaper than animal products. The two examples I know of now are margarine and egg replacer. The former got big market penetration (and I think some actual substitution) and the latter didn't. As I've argued elsewhere, we could get plant based burgers to cost parity or better if we built a bigger factory. Then I think there would be some substitution for price sensitive consumers.

Evidence of bumblebees liking and disliking foods depending on context.

What do you think about AI 2040's idea of giving all people an equal slice of the lightcone and allow them to sell theirs? The endowment would extremely roughly give everyone a cut of the Dyson swarm the size of the earth, a star in the Milky Way, and a galaxy. But since the star would typically take thousands of years to reach, it might be better to allow people to sell their galaxy, star, and slice of the Dyson swarm separately. Then the people who value the long-term future could buy most of the stars and galaxies.

Has anyone tried to estimate the social cost of carbon taking into account AI? I've seen several well-calibrated people estimating that there is a 20% probability that AI fizzles in the next few decades. If you believe that that would be permanent and AGI/ASI would solve the climate problem or result in catastrophe, to a first approximation, that would mean the social cost of carbon would be about 1/5 as much as people typically calculate. In reality, AGI/ASI may come further in the future, and there could be some benefits of reduced carbon even with AGI/ASI.

Or we can just leave crop residues in place. As it is, half of earth’s soil has a lifespan of 1,000 years or greater. 

Or we could reclaim eroded soil from rivers (e.g. where they are depositing it into the oceans).

Nitrogen is extracted from the air using natural gas, but can also be extracted using solar power with efficiency losses of 1%.

Thanks for citing me! To clarify, it's only about 1% of the land area taken up by the solar energy to make the nitrogen fertilizer - the losses in going from solar energy to nitrogen fertilizer are much more than 1%.

There are moderately more expensive alternatives and we would switch to those once we started to experience scarcity. 

Since doing that podcast, I have found that if we had to meet the current demand for gold and silver from common rock, it might take more than our entire economy to do it. Now as you say, we could substitute, e.g. another metal as a store of value, jewelry substitution, etc. But gold is used in computers because of its resistance to corrosion, which may be hard to substitute for. So it could be a drag on the economy (though I agree, not an existential risk).

They don’t translate into a threat to civilization much the way earthquakes aren’t a threat to civilization or an existential risk factor. 

That's generally true, though it has been pointed out that an earthquake could cause a lab leak of an engineered pathogen.

Zvi was not so charitable:

If you believe there is going to be an intelligence explosion or singularity soon, then you should absolutely spend down your philanthropic money as quickly as possible to try and make that event go well. 

...

The contrary position, taken by Will McAskill, is so bonkers I cannot wrap my mind around how he got this wrong. Yes, you can perhaps make a lot of money, but that money will be vastly less impactful, because either you will already be dead or we will have collectively already lost, or we will be winning and funding will be abundant and the points of highest leverage will be gone.

That does not mean you cannot hedge your bets to cover different possible futures, and it does not mean you cannot or shouldn’t invest or save for your own personal use, that is fine. But investing in AI (and thus if anything accelerating the problem) in order to later give more money away is a really bad bet. Don’t do that.

It does seem like @William_MacAskill  hardly talks about risk in that podcast.

Thanks for linking to this - it is important.

They do mention charitable things that have long lead times, but some others include growing plants in simulated nuclear winter conditions and building factory pilots. I think it’s important to build resilience significantly before AGI because the risk close to AGI of nuclear war and pandemics gets a lot higher.

And this general desire to "save" species doesn't always hold up. Consider:

• A common willingness to eradicate disease-carrying mosquitoes
• Strains of lab rats bred for specific research purposes, then killed off
• Little concern about purposefully breeding new species, or reclassifying existing ones

If we valued the sheer diversity or number of species, then the above wouldn't happen. So what makes a group of animals "special" enough to warrant concern, over and above the individuals within it?

Some conservationists do bite the bullet of opposing the eradication of disease-carrying mosquitoes. I think those rat strains would typically not be considered a separate species. There is interest in de-extinction, though I don't know if that qualifies as breeding new species. Some people classify domestic dogs as a separate species from wolves, and I think it's interesting to think about how humans could create new species to somewhat compensate for past extinctions. 

Thanks for doing an estimate of the long-term impact of ALLFED!

A marginal donation to a resilient-food organization produces about one-third as much risk reduction per dollar as Buhl's modeled pilot program. Buhl spent about 10 hours on the initial analysis and another 5-10 hours revising it after feedback from ALLFED. The model covered a hypothetical $10-$100 million project rather than a current marginal donation. Additional hazards, research, and advocacy provide some omitted upside. This assumption raises the estimated cost per microprobability by 3x as one all-things-considered adjustment.

Since ALLFED has not gotten that much money, I would argue that the marginal cost effectiveness should be higher than Buhl's estimate, not 3x lower. Indeed, we are focusing now on pilots that cost a lot less money, but still have similar impact. One example is growing plants in simulated nuclear winter conditions.

Strong work across this broader category produces half as much risk reduction per dollar as the resilient-food organization-level anchor. Resilient food is the only intervention in this category with a quantitative independent estimate of this pathway. Cross-hazard planning, infrastructure continuity, and recovery work have less direct evidence, although some interventions could be better. This assumption raises the estimated cost per microprobability by 2x.

For the impact on this century, the AI argued that policy was evaluated, but other things like resilient food pilots have not been evaluated, so it assigned an overall lower cost effectiveness. Now for the long-term future impact, the AI is arguing that resilient food pilots have been quantified, and other things like policy have not, so it is assigning a lower cost effectiveness. I think it's more defensible to assign about the same cost effectiveness across the board (we do try hard at ALLFED to equate the marginal cost effectiveness of different projects we can work on), though I agree it would be best to do a separate cost effectiveness analysis on the biosecurity work.

But the most important thing is that there is a long-term future impact of ALLFED quantified at all, so I appreciate your effort.

Thanks!