To my understanding, Animal Policy International is a great org trying to advocate for policies to not import lower welfare animal products, within their resource constraints. This is effectively fighting the international competition to local farmers. They could certainly use more help and/or resources to implement these policies worldwide, but I did want to point out part of the animalist lobby is focusing on this.
In case it's helpful, I figured I'd share a few reasons why these quantifiable metrics are so tricky to find.
It's not obvious how to calculate a given metric (such as DALYs). DALYs are calculated as the sum of YLLs YLDs. Implicitly, this assumes YLLs averted would be a good thing. However, if you believe broiler chickens living in battery cages are living a net-negative life (not an unpopular opinion in EA), then this is pretty obviously not the case. Therefore, YLLs would actually have a positive effect on welfare because YLLs imply the suffering has ended? Of course this rests on the idea the chicken is not going to be replaced by another chicken, which it almost certainly will be. This would imply YLLs don't matter at all for at least some farm animals. However, maybe the killing of the animal matters too? Again, not a crazy opinion to have (at least in the animal movement more broadly). Then YLLs would be affected by this, but probably not in a way aligning with what you intuitively think matters.
There is not common agreement about which metrics matter. In the GHD space, some people care more about QALYs, others DALYs, and others lives saved. However, in reality, everyone cares about all three and all the metrics are pretty closely related to each other. This is not the case in the animal space. Many in the animal space care about the abolition of the factory farming system while others put more value on reducing suffering in the short term. These interventions CAN be somewhat different (and there's plenty of debate about how symbiotic these frameworks are or can be). Others will care about whether animals have rights. This is an entirely different framing which may not overlap with other frameworks at all for some interventions. In GHD, it's pretty generally agreed upon that women and people in poverty should have rights and also that having rights is a component of their quality of life. With animals, many people disagree on some of these basic concepts.
Interaction effects between different interventions play such a large role in the effectiveness of an intervention. When one organization gets a company to sign a cage-free commitment, most reasonable metrics would give them credit for the welfare benefits of the commitment. However, sometimes a more radical animal rights group has been protesting their activity for years and this made it much easier for the more moderate organization to succeed. However, how would we attribute this contribution to the more radical group? It's not obvious at all. My guess is you won't reach reasonable agreement on this no matter how you quantify it. This type of thinking about change in the animal space is the reason why plurality in the movement is so valued my many (including myself) and, again, virtually any metric of cost-effectiveness of a given charity would miss this critical component of impact/social change.
The funding in the animal space is so much lower than in the GHD space. The GHD space has many billions of dollars to allocate to support a few billion people. The farmed animal space has about $200 million to allocate to support trillions of animals. This difference implies there is much less benefit to developing precise cost-effectiveness analyses for every intervention in every context. Rather, we put more resources into implementation and try to get a reasonable idea of the effectiveness of different interventions.
There are far fewer (human) stakeholders in some areas of the animal space and even fewer who are friendly. Working in MEL in the animal space, I've been able to adapt a lot of the standard practices in the GHD space. However, the biggest difference I see (other than the funding environment) is how GHD charities generally have many thousands of people in their target group and will target a few thousand each year while many animal charities have a few hundred people in their target group and they will likely target a few dozen per year (some target far fewer). This is particularly true when it comes to corporate pressure campaigns. Additionally, when you target corporations with a pressure campaign, your target group generally hates you. These differences mean methods pretty common in the GHD space, such as widescale surveying, oftentimes do not work at all in the animal space. For pressure campaigns, you're lucky if you can conduct some in-depth interviews with members of your target group, but this is often not possible because of the hostile nature of your relationship with them.
All of these point together generally support a much more qualitative type of evaluation in the animal space. While qualitative information may be much less precise, it has the possibility of being much more accurate. Meanwhile quantitative metrics are likely to be more precise, but are also very prone to being inaccurate given the budget and sample size constraints.
I speculate the welfare per time of a hen in organic egg production as a fraction of the welfare range of chickens equals the welfare of a random human as a fraction of the welfare range of humans. Consequently, for the welfare range just above, I get a welfare per time of a hen in organic egg production of 0.332 times that of a random human.
I know you say this is speculative, so I'm not holding you to actually thinking this is correct. However, I know this would probably be a hotly-contested claim by many. From a rights perspective, there's an obvious disagreement, but even from a welfare/pain perspective, organic hens are still susceptible to keel bone fractures (though possibly to a slightly lesser extent). Because keel bone fractures play such a large role in the suffering of layer hens, I think it's worth reconsidering this claim.
This could alter your conclusion pretty dramatically. For instance, if we say we assume the suffering of organic hens is only half the suffering of barn hens (I want to be clear I picked one half just to be easy and not because I think it's correct - haven't gone through the calculations here), then I calculate corporate campaigns are 2,247 times as cost-effective as buying organic eggs keeping everything else in this model the same.
This would imply even more variation in cost-effectiveness in animal welfare interventions. In case it wasn't clear above, I am also not certain (or anywhere near so) of the difference in variation. I just wanted to point out this one particular area of high uncertainty is likely a major driver of this analysis.
I certainly value my first job out of university for teaching me all sorts of lessons about productivity, communication, writing emails, management, and just work in general. I didn't appreciate it at first, but the value of these many lessons became apparent after about 8 months. I also just had to "lean in" to mastering the little things about the job, no matter how pointless they seemed.
I have recently been speaking with some people who are about to retire from their normal job, are looking for stuff to do with their time, and wish they had more meaning in their work. Additionally, they aren't really worried about a paycheck - they have their retirement plan. This has me thinking about how soon-to-be retirees with a chritable/altruistic bent may be a great unexplored (or maybe just lesser explored) source of talent for the EA movement and EA orgs.
Unfortunately, I feel a lot of the intro EA materials aren't targeted to this demographic. Therefore, I find it challenging to provide the people I've spoken to with appropriate materials to explore further. Basically, if someone has been thoroughly trained by non-EA orgs, how do we best bring them into EA? Maybe I'm totally off-base with this idea, but doesn't seem too fanatical at the moment.
Consequently, the cost per additional plant-based meal in 2023 was 0.0600 $ (= 248*10^3/(4.13*10^6)). Sarah expects this to decrease in the future.
Therefore, I just assumed the "this" referred to the "cost per additional plant-based meal" and not the effectiveness per dollar. This is a factor which can change the effectiveness of School Plates pretty radically, so I'd be careful. Obviously, from a comparison point of view, this alone probably won't make up the difference, but claiming 186 times as cost-effective is very different from saying 93 times as cost-effective or even 47 times as cost-effective.
I would almost insist this is not a situation where the law of diminishing marginal returns is applicable. From my work at The Mission Motor, I've found most organizations are not able to identify where their most-effective activities may be. This isn't to say anything bad about the organizations themselves, just that these activities are not at all obvious to identify. I spend a fair bit of my time trying to brainstorm reasonable target groups with organizations, but there's so little information to work from and contexts vary so widely that it just becomes a semi-educated guess for most organizations.
To clarify this point a bit, it's not too tricky to identify the biggest targets, but it's really challenging to estimate the resources needed to obtain a commitment and enforce it before you start a campaign. However, once you start a campaign, you can't really stop because that hurts the success of all future campaigns. Disclaimer: Some organizations are more equipped with MEL or data support than others and ProVeg does have an MEL team who can assist with some of this work. However, ProVeg has two people working on MEL and there are many dozens of interventions being carried out in many different countries; as great as they are, I doubt they can identify the most cost-effective targets very reliably for all their interventions given doing this for one intervention is challenging enough.
Additionally, it appears to me you calculated the average cost per meal rather than the marginal cost per meal in this calculation "0.0600 $ (= 248*10^3/(4.13*10^6))". I imagine the $248,000 is the total budget for the program serving 4,130,000 meals. At early phases of interventions, it is very common for the marginal cost to be below the average cost and it is also not uncommon for the derivative of the marginal cost function to be negative (it's kind of expected at the very beginning of a venture). So, I would argue the marginal cost per meal is possibly far less than $0.06 instead of the $0.12 you estimated.
Overall for point 1, over time organizations generally get better at identifying the best targets, staff are upskill and improve their tactics, and many of the supporting materials and tools for an intervention can be reused once created. All of these components would suggest the marginal cost for an intervention would decrease over time rather than increase. Additionally, as an organization grows, specialization and other aspects of economies of scale could continue to decrease costs. We're also dealing with a social movement, so there may just be less pushback over time as well. I would probably use a marginal cost of $0.03 instead of $0.12 if I had to pick a point estimate here. This changes the comparison to corporate campaigns being 47 times as cost-effective as School Plates - still a large margin, but feels a bit different.
On point 2, many of your points are well taken - namely the linearity of your model. I'm not a huge fan of sheets and would have written then model in Python where it would be relatively easy to turn the model into a MCS, so I would have just done that first instead of thinking through everything you wrote in your comment (different work styles and it seems yours is much more efficient here). Additionally, I may want a lot of this modeling sitting in Python anyway for comparing other interventions or tactics so building the MCS has other benefits for me (not to mention the fact that having a computer program spit out some examples is a nice communication tool for people without a background in probability).
I started playing around in your sheet to get a better sense of why this result seems so counterintuitive to me (nothing wrong with counterintuitive, but if I can understand why, I can learn how to update my ideas in this space). While I am a bit skeptical of the 8.2 chicken lives affected per dollar, I'm not going to jump into all these calculations at the moment, so I'll just have to accept it for the point of conversation.
However, it does appear to me there is an additional point of major uncertainty for corporate campaigns not present in the School Plates model - the improvement of conditions from conventional systems to cage-free systems. You get to an estimate that cage-free systems generate 22.3% as much suffering as conventional systems. But this is a point estimate on many very uncertain variables. While your point estimate is probably reasonable as a point estimate, I know people who would try to argue this number should be more like 95%. I'm not saying they're correct or endorsing these estimates in any way, but I feel the need to keep that uncertainty. With your particular ethical slant (particularly the EXPECTED component of your utilitarianism), this probably isn't very relevant to you personally. Additionally, even using the 95% estimate AND the $0.03 marginal cost estimate would not be enough to make School Plates more effective than corporate campaigns, but the estimate changes to 3 times as effective, which is considerably different.
I think there are other factors such as how much these interventions can shape society in the long run and whatnot which could make the School Plates intervention more effective than corporate campaigns. However, a lot of things would need to go right.
On a higher level, while I am a Bayesian, I still believe there is a "true value" as I think most Bayesians do, even if they don't talk about it much because Frequentists are so obsessed with this theoretical "true value". Because there is so much uncertainty in many of these calculations, and corporate campaigns will inevitably never lead to a world without animal exploitation (I know this may not be perfectly utilitarian but I'm not certain of this either) without other complementary interventions, I think abolitionist interventions have their place in the movement - even if just to lay the groundwork for the future. Additionally, I have heard numerous accounts of corporate campaigners sharing how much easier the more extreme abolitionists make their job. After corporations work with an extreme abolitionist, working with THL is so much more attractive.
Overall, I think these interventions do and must work together towards the world we want to create for animals, even if there may be some disagreement about what that ultimate world looks like exactly. This leads me to prefer a pluralistic movement and err on the side of endorsing less effective interventions (at least in the short run) if they are of a different "flavor". By different "flavors", I basically mean the tactics and the theories of change are not very related and may even be complementary.
Thanks for going through the analysis here! I was a bit confused about a few things.
You cite Sarah Morton as thinking the costs will decrease over time and then you you speculate the marginal cost-effectiveness to be 0.5 (seemingly implying costs will increase over time). You cite diminishing returns as your support for this decision, but Sarah's comment implying this intervention is in a period of increasing returns (yes this is a thing). Additionally, this may be less of an issue of increasing returns and this intervention could just benefit from economies of scale which would also decrease costs and increase cost-effectiveness. If Sarah is right (and thus the marginal cost-effectiveness is >= 1), School Plates is at least twice as effective as you've estimated here. However, very happy to hear I'm misunderstanding something on my end.
It seems like the sole unit of impact measurement is time and intensity of suffering. However, it's not obvious to me this is all that matters. Suppose we care about the killing of an animal differently than the suffering when they're alive (I don't think this is entirely unreasonable, though maybe not the most mathematically convenient model). Then School Plates would get a boost in effectiveness. We could also develop models where the number of animals suffering is a component of our overall utility function (there's a disutility associated with a factory-farmed animal existing at all), in which case School Plates would also get a boost in effectiveness. My point here is I'm not entirely sure such a narrow definition of impact is appropriate here. I know it makes the math easier and this is just meant to be an estimate, but when the definition of impact favors one intervention over the other in basically every component, the difference in the estimates is going to be exaggerated. Then again, I'm not here to insist your ethics are wrong (I think that's a personal choice), but I think this adds a lot of nuance to your conclusions.
I hope this isn't coming off as overly critical. I enjoyed reading this post and think it's a great starting point for further, highly-relevant work. I'm thinking of potentially building out some Monte Carlo Simulations of your model and Saulius' model (with Open Phil's comments) to see how accounting for variance impacts these estimates (my hunch is there will be so much uncertainty it will be hard to decide between the interventions). One additional benefit of the Monte Carlo Simulations is how they point to where collecting more evidence and decreasing our uncertainty would improve our estimates most. Thanks again for posting this!
Thank you Kevin and thank you Hive.
To my understanding, Animal Policy International is a great org trying to advocate for policies to not import lower welfare animal products, within their resource constraints. This is effectively fighting the international competition to local farmers. They could certainly use more help and/or resources to implement these policies worldwide, but I did want to point out part of the animalist lobby is focusing on this.
In case it's helpful, I figured I'd share a few reasons why these quantifiable metrics are so tricky to find.
All of these point together generally support a much more qualitative type of evaluation in the animal space. While qualitative information may be much less precise, it has the possibility of being much more accurate. Meanwhile quantitative metrics are likely to be more precise, but are also very prone to being inaccurate given the budget and sample size constraints.
I know you say this is speculative, so I'm not holding you to actually thinking this is correct. However, I know this would probably be a hotly-contested claim by many. From a rights perspective, there's an obvious disagreement, but even from a welfare/pain perspective, organic hens are still susceptible to keel bone fractures (though possibly to a slightly lesser extent). Because keel bone fractures play such a large role in the suffering of layer hens, I think it's worth reconsidering this claim.
This could alter your conclusion pretty dramatically. For instance, if we say we assume the suffering of organic hens is only half the suffering of barn hens (I want to be clear I picked one half just to be easy and not because I think it's correct - haven't gone through the calculations here), then I calculate corporate campaigns are 2,247 times as cost-effective as buying organic eggs keeping everything else in this model the same.
This would imply even more variation in cost-effectiveness in animal welfare interventions. In case it wasn't clear above, I am also not certain (or anywhere near so) of the difference in variation. I just wanted to point out this one particular area of high uncertainty is likely a major driver of this analysis.
I certainly value my first job out of university for teaching me all sorts of lessons about productivity, communication, writing emails, management, and just work in general. I didn't appreciate it at first, but the value of these many lessons became apparent after about 8 months. I also just had to "lean in" to mastering the little things about the job, no matter how pointless they seemed.
I have recently been speaking with some people who are about to retire from their normal job, are looking for stuff to do with their time, and wish they had more meaning in their work. Additionally, they aren't really worried about a paycheck - they have their retirement plan. This has me thinking about how soon-to-be retirees with a chritable/altruistic bent may be a great unexplored (or maybe just lesser explored) source of talent for the EA movement and EA orgs.
Unfortunately, I feel a lot of the intro EA materials aren't targeted to this demographic. Therefore, I find it challenging to provide the people I've spoken to with appropriate materials to explore further. Basically, if someone has been thoroughly trained by non-EA orgs, how do we best bring them into EA? Maybe I'm totally off-base with this idea, but doesn't seem too fanatical at the moment.
Thanks for the welcome!
For 1, you wrote
Therefore, I just assumed the "this" referred to the "cost per additional plant-based meal" and not the effectiveness per dollar. This is a factor which can change the effectiveness of School Plates pretty radically, so I'd be careful. Obviously, from a comparison point of view, this alone probably won't make up the difference, but claiming 186 times as cost-effective is very different from saying 93 times as cost-effective or even 47 times as cost-effective.
I would almost insist this is not a situation where the law of diminishing marginal returns is applicable. From my work at The Mission Motor, I've found most organizations are not able to identify where their most-effective activities may be. This isn't to say anything bad about the organizations themselves, just that these activities are not at all obvious to identify. I spend a fair bit of my time trying to brainstorm reasonable target groups with organizations, but there's so little information to work from and contexts vary so widely that it just becomes a semi-educated guess for most organizations.
To clarify this point a bit, it's not too tricky to identify the biggest targets, but it's really challenging to estimate the resources needed to obtain a commitment and enforce it before you start a campaign. However, once you start a campaign, you can't really stop because that hurts the success of all future campaigns. Disclaimer: Some organizations are more equipped with MEL or data support than others and ProVeg does have an MEL team who can assist with some of this work. However, ProVeg has two people working on MEL and there are many dozens of interventions being carried out in many different countries; as great as they are, I doubt they can identify the most cost-effective targets very reliably for all their interventions given doing this for one intervention is challenging enough.
Additionally, it appears to me you calculated the average cost per meal rather than the marginal cost per meal in this calculation "0.0600 $ (= 248*10^3/(4.13*10^6))". I imagine the $248,000 is the total budget for the program serving 4,130,000 meals. At early phases of interventions, it is very common for the marginal cost to be below the average cost and it is also not uncommon for the derivative of the marginal cost function to be negative (it's kind of expected at the very beginning of a venture). So, I would argue the marginal cost per meal is possibly far less than $0.06 instead of the $0.12 you estimated.
Overall for point 1, over time organizations generally get better at identifying the best targets, staff are upskill and improve their tactics, and many of the supporting materials and tools for an intervention can be reused once created. All of these components would suggest the marginal cost for an intervention would decrease over time rather than increase. Additionally, as an organization grows, specialization and other aspects of economies of scale could continue to decrease costs. We're also dealing with a social movement, so there may just be less pushback over time as well. I would probably use a marginal cost of $0.03 instead of $0.12 if I had to pick a point estimate here. This changes the comparison to corporate campaigns being 47 times as cost-effective as School Plates - still a large margin, but feels a bit different.
On point 2, many of your points are well taken - namely the linearity of your model. I'm not a huge fan of sheets and would have written then model in Python where it would be relatively easy to turn the model into a MCS, so I would have just done that first instead of thinking through everything you wrote in your comment (different work styles and it seems yours is much more efficient here). Additionally, I may want a lot of this modeling sitting in Python anyway for comparing other interventions or tactics so building the MCS has other benefits for me (not to mention the fact that having a computer program spit out some examples is a nice communication tool for people without a background in probability).
I started playing around in your sheet to get a better sense of why this result seems so counterintuitive to me (nothing wrong with counterintuitive, but if I can understand why, I can learn how to update my ideas in this space). While I am a bit skeptical of the 8.2 chicken lives affected per dollar, I'm not going to jump into all these calculations at the moment, so I'll just have to accept it for the point of conversation.
However, it does appear to me there is an additional point of major uncertainty for corporate campaigns not present in the School Plates model - the improvement of conditions from conventional systems to cage-free systems. You get to an estimate that cage-free systems generate 22.3% as much suffering as conventional systems. But this is a point estimate on many very uncertain variables. While your point estimate is probably reasonable as a point estimate, I know people who would try to argue this number should be more like 95%. I'm not saying they're correct or endorsing these estimates in any way, but I feel the need to keep that uncertainty. With your particular ethical slant (particularly the EXPECTED component of your utilitarianism), this probably isn't very relevant to you personally. Additionally, even using the 95% estimate AND the $0.03 marginal cost estimate would not be enough to make School Plates more effective than corporate campaigns, but the estimate changes to 3 times as effective, which is considerably different.
I think there are other factors such as how much these interventions can shape society in the long run and whatnot which could make the School Plates intervention more effective than corporate campaigns. However, a lot of things would need to go right.
On a higher level, while I am a Bayesian, I still believe there is a "true value" as I think most Bayesians do, even if they don't talk about it much because Frequentists are so obsessed with this theoretical "true value". Because there is so much uncertainty in many of these calculations, and corporate campaigns will inevitably never lead to a world without animal exploitation (I know this may not be perfectly utilitarian but I'm not certain of this either) without other complementary interventions, I think abolitionist interventions have their place in the movement - even if just to lay the groundwork for the future. Additionally, I have heard numerous accounts of corporate campaigners sharing how much easier the more extreme abolitionists make their job. After corporations work with an extreme abolitionist, working with THL is so much more attractive.
Overall, I think these interventions do and must work together towards the world we want to create for animals, even if there may be some disagreement about what that ultimate world looks like exactly. This leads me to prefer a pluralistic movement and err on the side of endorsing less effective interventions (at least in the short run) if they are of a different "flavor". By different "flavors", I basically mean the tactics and the theories of change are not very related and may even be complementary.
Thanks for going through the analysis here! I was a bit confused about a few things.
I hope this isn't coming off as overly critical. I enjoyed reading this post and think it's a great starting point for further, highly-relevant work. I'm thinking of potentially building out some Monte Carlo Simulations of your model and Saulius' model (with Open Phil's comments) to see how accounting for variance impacts these estimates (my hunch is there will be so much uncertainty it will be hard to decide between the interventions). One additional benefit of the Monte Carlo Simulations is how they point to where collecting more evidence and decreasing our uncertainty would improve our estimates most. Thanks again for posting this!