Very provocative post. A few minor comments, not criticisms!
Maybe some of your reference cases are more complex than you imagine. Temperature, for example, is not trivial. Depending how you measure it (ideal gas vs electrical conductivity), you get two aifferent seemingly linear scales that agree only at the two points where you force agreement (e.g. Oc and 100C). But also even using Kelvin, it's not at all obvious that this is the correct linear scale to use. Maybe the true scale is exponential or logarithmic relative to this: (e.g. one could argue that the differencebetween 10 K and 100 K is the same as that between 100 K and 1000 K ... or you could argue (if you're a chemist, that in the correct scale 300 k is 2x 290 K which is 2x 280 K ...). Today, after mant years, we have a settled system, but it doesn't mean anything - anybody working with quantum computers will know that one degree close to 0K is a lot more "difficult" than going from 285 to 284 K, while maybe cosmologists use phrases like "10's of millions of degrees" where Celsius and Kelvin are basically the same, and only worry if the temperature changes by a million degrees ...
All this to point out that the question of measurement is rarely trivial once you get past counting things!
What people do in science and engineering is to make two lists:
what are the tangible outcomes we need? (e.g. what is the cost per kJ of energy delivered to a home? ...)
what are the physical, tangible measures we can make? (e.g. how many kg of fuel did we use? what temperature was reached? what was the % completeness of the reaction? ...)
And then, they try to build a model in which the available data will predict the desired output. Sometimes it's possible, but usually it requires a lot of assumptions, which need to be tested and validated continuously.
They always also keep in mind that all models are wrong, but some are useful, and constantly re-check if their model is useful and remind themselves of the ways it is wrong, especially if applied in and area it wasn't intended for.
In this context, IMHO asking how capable an AI is is just far too difficult and too broad. To use your football analogy, Messi might be better than anyone in the world at scoring with his left foot, but there are guys on my amateur team who would be better at defending a corner than he would.
What's most realistic is to ask more granular questions (e.g. likelihood to enable a bioterrorist to produce a new virus, likelihood to successfully complete a problem that takes a skilled human 1 week, ... ), and then look at more tangible ways to test exactly that. Which of course is what AI Safety labs do. Just as IQ is meaningful but flawed, any measure of general capability will be flawed.
All this was a long-winded way of saying that yours is a great analysis and a valuable contribution, but that with the right mentality, people have found ways to measure and predict in complex situations before. It feels that AI is another level of complexity, but the principles that give us the most insight do not change.
So hedge-fund managers can become absurdly rich despite adding very little value to society. IMHO if a fund manager had to set these out as numbers rather than percentages, clients would be more likely to complain that (for large investors) the management fees are ridiculous. ("How can you justify earning $30 m in one year for managing my money?"). But when they see "2%" maybe it feels cheap to objest to such a "small" number.
Great post. I'm not sure most of it relates to Bayesian thinking, but just bad logic. I especially appreciate your point about low-probability events - but then the entire hedge-fund industry is based on the fact that people think 2% is a very small number, so we're not alone.
I think there is a lot of value in discounting estimates' influence on actions based on uncertainty.
You actually wrote "I agree that we should embrace non-vegans in the animal rights movement." You actually wrote that !!
As in, in your eyes, the true animal-rights movement is limited to the 1-2% of people who are vegans, but you deign to accept the help of some of the 98% of people who are not.
Vegans do not own the animal rights movement. People who care about animals and make commitments to help them own the animal rights movement.
If I, as an individual, decide that I want to do as much as possible to help end factory-farming, then becoming vegan would probably not be in the top 5 more impactful things I could do. On the other hand, it might well be among the most difficult things I could do.
Imagine if all the people who work on artificial protein decided that they owned the animal rights movement (because many people would argue that they are working on a much more viable long-term solution than veganism), but they were not so tolerant and decided that they didn't want vegans in their movement unless they also dedicated their careers to artificial protein.
I do agree that it's not about making veganism taboo. Vegans play a huge role in animal-rights, especially if they are clear that they do this because they oppose the killing of animals for food. (those of you who live in the kind of EA-bubble may not realise this, but most people do not immediately make this association - veganism is often seen as a kind of hippy / mindful / health-driven kind of thing rather than a robust position held by strong-willed people who care about animals.)
But when it's used to exclude (as in, you wouldn't be happy to have non-vegans leading the animal-rights movement), it is absolutely not helping.
Unless you see veganism as a total solution in itself. In that case, it's not nearly enough to be vegan. You need to convince everyone to become vegan. Not just people who care about animal rights. Everyone. There is absolutely space for a movement to convince more people to become vegans, but (maybe I'm out of the loop) I haven't seen any evidence of this. This is not about asking individual vegans to become advocates - they are already doing their part by role-modelling the behaviour - but about a movement that aggressively communicates how horrific it is that we kill animals and eat them, and then proposes a solution.
(Ironically, even this doesn't necessarily have to be led by vegans - it might even work better if the protagonists were non-vegans who were trying to become vegans and encountering the same challenges that everyone else faces, rather than people who are already past them).
Vegans role-model a behaviour that can be a powerful signal and something that makes others pause. Not because it's easy, but because it's not easy and they do it anyway. in that context - where veganism is seen as an "extreme" form of commitment, it can be very powerful and can motivate others to fight for animal rights.
But once veganism becomes a minimum criterion, the problems highlighted in this post come to the fore, and it does more harm than good. Outside the EA-bubble, most people do not meet many vegans in their daily lives, people hosting a small dinner party do not plan vegan options, and very few men are (openly) vegans.
On the other hand, if you talk about factory farming and suffering animals in tiny cages, most people will immediate react with disgust and agree (in principle at least) that this needs to stop, that it is a stain on our society. We absolutely need to channel this energy without filtering it though purity-criteria like veganism.
There are many ways that people can contribute to animal rights. Working on artificial protein. Advocating for better legislation and for tax-incentives that promote better treatment of animals. Donating to organisations like FarmKind which fight factory farming. Protesting at factory farms. Opposing the Big Food protests in Brussels and elsewhere that try to pass themselves off as standing up for poor farmers' rights. Being a vegan and role-modelling your veganism in a positive, inclusive way.
Animals need all of these right now. They do not want anyone excluded because they are not pure enough.
Being located in Brussels, it always feels that there must be ways to influence the people within the EU who have so much power. Not just the politicians (for whom there are thousands of professional and well-paid full-time lobbyists), but also the eurocrats who manage a lot of the tangible work the EU does.
It always feels like the bad guys (the oil-companies and tobacco companies and people looking for reduced regulation (often using the euphemism "simplification" to describe what is really reducing consumer-protection, reducing environmental protection, slowing AI legislation, etc.) have all the money, power and influence. I would love to see more EA's aggressively and effectively working in this area. There are many NGO's of course who do this, full of people who do great work. But it just feels like they are often out-manned and struggling to be heard.
The example that always comes to mind is the few people lobbying for better legislation on issues like factory farming or alternative protein trying to be heard when there are regular many-thousand strong violent demonstrations by "farmers" with tractors opposing anything that they don't like. It's annoying because I strongly believe that the farmers involved in these protests represent the huge industrial factory farms, and not the many smaller farmers who do have genuine concerns and challenges. The legislation serves neither the animals nor the true farmers, just the factory farms who have the money to lobby and protest.
And just so people don't get the wrong idea about this being just for people straight out of college, I got accepted after many years working as an R&D engineer with no direct connection to EA or charity. If you're experienced and motivated, chances are you may already have done some of these exercises, which makes following this advice even easier!
Just under 800 people applied. I split the pile with a colleague, which gave me 400 resumes and 1200 short-answer questions to read.1
To get this done within the week, while also doing the rest of my job, I had to spend less than five minutes per person. And we only had time to grade a few dozen work tests, so we had to cut over 90% of the applicants. These numbers aren’t unusual for an open hiring round.
I just cannot believe that this is right. IMHO we delude ourselves that we're being fair and effective, while in reality we're putting a lot of energy, hard-work and resources into applying for jobs (800 people did this for one job !!! - think about how many hours that is - and they all had to answer 3 short-answer questions with enough care and attention. For many of them, that application process was probably at least half a day of work. So, 1-2 years of "EA work" went into just applying for this one role. Surely there were more valuable ways the EA movement could have used all this energy and creativity ???
There is something wonderful about this conversation, and about what it says about effective altruism and our aim to do the most good possible.
Cash-transfers as done by Give Directly are a phenomenal initiative. So obvious in retrospect, but only when you remove your preconceived bias about poor people being partly responsible for their own predicament.
So it would be great, and laudable, for any organisation to look for ways to scale and expand programs like this. (And by supporting Give Directly, we can all help to enable this).
But the effective altruists (in this case, GiveWell) go a step further. How can we make this even better?
[I've spend many years working in one of the best corporate R&D organisations in the world, and this was a big part of what made us successful - to be always asking "how could we make this even better?"]
Once you start asking this, the ideas arrive: who are the most effective people to give the money to? when is the most effective time to give money? how to coordinate cash-transfers with other initiatives to maximise impact? Etc.
Of course there's still a lot of research needed to answer these. But the value of asking these questions is enormous, and unfortunately I get the impression that this is where many charities fail - they find a model that's good enough, and stick to it - doing a lot of good, but also not helping as many people as they might, as much as they might.
I realise this entire comment falls under what Basil Fawlty would describe as "the bleedin' obvious" for this EA audience - but to me it is a beautiful example we can use to explain how and why EA's can always create more impact. I will write about this in the Effective Giving Ireland journal and maybe convince one or two extra donors to support effective charities ...
Fair, It seems you know more about this than I do! Thanks!
Very provocative post. A few minor comments, not criticisms!
All this was a long-winded way of saying that yours is a great analysis and a valuable contribution, but that with the right mentality, people have found ways to measure and predict in complex situations before. It feels that AI is another level of complexity, but the principles that give us the most insight do not change.
Sorry, I wasn't clear. I was referring to the fee structure, where many funds charge 2% of the investment as annual management fees. Two and Twenty: Explanation of the Hedge Fund Fee Structure.
So hedge-fund managers can become absurdly rich despite adding very little value to society. IMHO if a fund manager had to set these out as numbers rather than percentages, clients would be more likely to complain that (for large investors) the management fees are ridiculous. ("How can you justify earning $30 m in one year for managing my money?"). But when they see "2%" maybe it feels cheap to objest to such a "small" number.
Great post. I'm not sure most of it relates to Bayesian thinking, but just bad logic. I especially appreciate your point about low-probability events - but then the entire hedge-fund industry is based on the fact that people think 2% is a very small number, so we're not alone.
I think there is a lot of value in discounting estimates' influence on actions based on uncertainty.
You actually wrote "I agree that we should embrace non-vegans in the animal rights movement." You actually wrote that !!
As in, in your eyes, the true animal-rights movement is limited to the 1-2% of people who are vegans, but you deign to accept the help of some of the 98% of people who are not.
Vegans do not own the animal rights movement. People who care about animals and make commitments to help them own the animal rights movement.
If I, as an individual, decide that I want to do as much as possible to help end factory-farming, then becoming vegan would probably not be in the top 5 more impactful things I could do. On the other hand, it might well be among the most difficult things I could do.
Imagine if all the people who work on artificial protein decided that they owned the animal rights movement (because many people would argue that they are working on a much more viable long-term solution than veganism), but they were not so tolerant and decided that they didn't want vegans in their movement unless they also dedicated their careers to artificial protein.
I do agree that it's not about making veganism taboo. Vegans play a huge role in animal-rights, especially if they are clear that they do this because they oppose the killing of animals for food. (those of you who live in the kind of EA-bubble may not realise this, but most people do not immediately make this association - veganism is often seen as a kind of hippy / mindful / health-driven kind of thing rather than a robust position held by strong-willed people who care about animals.)
But when it's used to exclude (as in, you wouldn't be happy to have non-vegans leading the animal-rights movement), it is absolutely not helping.
Unless you see veganism as a total solution in itself. In that case, it's not nearly enough to be vegan. You need to convince everyone to become vegan. Not just people who care about animal rights. Everyone. There is absolutely space for a movement to convince more people to become vegans, but (maybe I'm out of the loop) I haven't seen any evidence of this. This is not about asking individual vegans to become advocates - they are already doing their part by role-modelling the behaviour - but about a movement that aggressively communicates how horrific it is that we kill animals and eat them, and then proposes a solution.
(Ironically, even this doesn't necessarily have to be led by vegans - it might even work better if the protagonists were non-vegans who were trying to become vegans and encountering the same challenges that everyone else faces, rather than people who are already past them).
This is an absolutely wonderful post.
Vegans role-model a behaviour that can be a powerful signal and something that makes others pause. Not because it's easy, but because it's not easy and they do it anyway. in that context - where veganism is seen as an "extreme" form of commitment, it can be very powerful and can motivate others to fight for animal rights.
But once veganism becomes a minimum criterion, the problems highlighted in this post come to the fore, and it does more harm than good. Outside the EA-bubble, most people do not meet many vegans in their daily lives, people hosting a small dinner party do not plan vegan options, and very few men are (openly) vegans.
On the other hand, if you talk about factory farming and suffering animals in tiny cages, most people will immediate react with disgust and agree (in principle at least) that this needs to stop, that it is a stain on our society. We absolutely need to channel this energy without filtering it though purity-criteria like veganism.
There are many ways that people can contribute to animal rights. Working on artificial protein. Advocating for better legislation and for tax-incentives that promote better treatment of animals. Donating to organisations like FarmKind which fight factory farming. Protesting at factory farms. Opposing the Big Food protests in Brussels and elsewhere that try to pass themselves off as standing up for poor farmers' rights. Being a vegan and role-modelling your veganism in a positive, inclusive way.
Animals need all of these right now. They do not want anyone excluded because they are not pure enough.
Thanks for this Michelle (and Pause AI).
Being located in Brussels, it always feels that there must be ways to influence the people within the EU who have so much power. Not just the politicians (for whom there are thousands of professional and well-paid full-time lobbyists), but also the eurocrats who manage a lot of the tangible work the EU does.
It always feels like the bad guys (the oil-companies and tobacco companies and people looking for reduced regulation (often using the euphemism "simplification" to describe what is really reducing consumer-protection, reducing environmental protection, slowing AI legislation, etc.) have all the money, power and influence. I would love to see more EA's aggressively and effectively working in this area. There are many NGO's of course who do this, full of people who do great work. But it just feels like they are often out-manned and struggling to be heard.
The example that always comes to mind is the few people lobbying for better legislation on issues like factory farming or alternative protein trying to be heard when there are regular many-thousand strong violent demonstrations by "farmers" with tractors opposing anything that they don't like. It's annoying because I strongly believe that the farmers involved in these protests represent the huge industrial factory farms, and not the many smaller farmers who do have genuine concerns and challenges. The legislation serves neither the animals nor the true farmers, just the factory farms who have the money to lobby and protest.
Great post!
And just so people don't get the wrong idea about this being just for people straight out of college, I got accepted after many years working as an R&D engineer with no direct connection to EA or charity. If you're experienced and motivated, chances are you may already have done some of these exercises, which makes following this advice even easier!
I was just reading this post, How to become a better applicant in one week, from 80,000 Hours, and was struck by this paragraph:
Here’s what my previous hiring round was like:
Just under 800 people applied. I split the pile with a colleague, which gave me 400 resumes and 1200 short-answer questions to read.1
To get this done within the week, while also doing the rest of my job, I had to spend less than five minutes per person. And we only had time to grade a few dozen work tests, so we had to cut over 90% of the applicants. These numbers aren’t unusual for an open hiring round.
I just cannot believe that this is right. IMHO we delude ourselves that we're being fair and effective, while in reality we're putting a lot of energy, hard-work and resources into applying for jobs (800 people did this for one job !!! - think about how many hours that is - and they all had to answer 3 short-answer questions with enough care and attention. For many of them, that application process was probably at least half a day of work. So, 1-2 years of "EA work" went into just applying for this one role. Surely there were more valuable ways the EA movement could have used all this energy and creativity ???
There is something wonderful about this conversation, and about what it says about effective altruism and our aim to do the most good possible.
Cash-transfers as done by Give Directly are a phenomenal initiative. So obvious in retrospect, but only when you remove your preconceived bias about poor people being partly responsible for their own predicament.
So it would be great, and laudable, for any organisation to look for ways to scale and expand programs like this. (And by supporting Give Directly, we can all help to enable this).
But the effective altruists (in this case, GiveWell) go a step further. How can we make this even better?
[I've spend many years working in one of the best corporate R&D organisations in the world, and this was a big part of what made us successful - to be always asking "how could we make this even better?"]
Once you start asking this, the ideas arrive: who are the most effective people to give the money to? when is the most effective time to give money? how to coordinate cash-transfers with other initiatives to maximise impact? Etc.
Of course there's still a lot of research needed to answer these. But the value of asking these questions is enormous, and unfortunately I get the impression that this is where many charities fail - they find a model that's good enough, and stick to it - doing a lot of good, but also not helping as many people as they might, as much as they might.
I realise this entire comment falls under what Basil Fawlty would describe as "the bleedin' obvious" for this EA audience - but to me it is a beautiful example we can use to explain how and why EA's can always create more impact. I will write about this in the Effective Giving Ireland journal and maybe convince one or two extra donors to support effective charities ...