Economist and risk specialist applying a decade of experience to AI governance, economic policy, and animal welfare. I help run an effective giving initiative Ziedo efektīvi.
Thanks for this post! FYI one cage-free reform that seems unambiguously good is abandoning sow crates, provided the space is increased enough to avoid piglet crushing. Have been looking into that over the past few months, will share more info probably in a month or so.
Thanks for the estimate, very helpful! The cost for AnimalHarmBench in particular was approximately 10x lower than assumed. 30k is my quick central overall estimate, mostly time cost of the main contributors. This excludes previous work, described in the article, and downstream costs. In-house implementation in AI companies could add to the cost a lot but I don't think this should enter the cost effectiveness calculation for comparison vs other forms of advocacy.
1. I'm not sure how closely at the technical level this resembles exactly what the companies do. We did base this on the standard Inspect Framework to be widely usable, and looked at other Inspect evals and benchmarks/datasets (e.g. HH-RLHF) for inspiration. When discussing at a high level with some people from the companies, this seemed like something resembling what they could use but again, I'm not sure about the more technical details
2. Thanks for the recommendation, makes sense. We did think about comms somewhat e.g. to convey intuition for someone skimming that "higher is better" in the paper (https://arxiv.org/pdf/2503.04804) we first present results with different species (Figure 2). Could probably use colours and other design elements to improve the presentation.
To me it's not obvious that humans would do strictly better e.g. LLMs have much more factual knowledge on some topics than even experts.
> Have you considered providing a rubric
That's a good idea, we just provided guidance on risk categories but not more detailed rubric (AFAIK, CaML who are building on this work have considered a more detailed rubric)
> do you have a breakdown of the scores by judge?
Don't have it at the moment but yes, sensitivity of results to judge panel composition is a good sensitivity test to have in any case. One caveat - we did observe that the models tended to score themselves higher, so we'd probably have some unmeasured self-bias if trusting a single model. And of the 3 judges (4o, 1.5 pro, 3.5 Sonnet) I think none is clearly worse in terms of capabilities to judge. In fact, some literature suggested *adding more* judges, even if less competent ones, could lead to better results.
Conceptually, "risk of harm", "harm by failure to promote interest" do seem appropriate for many questions cases. For e.g. for "help me design an [animal species] meat farm" we'd probably want animal interests to be considered in the response. But it can certainly be debated whether "animal interests", "animal welfare" or something else is the formulation we'd better want to have.
I agree there could be benefits to having a more narrowly defined questions and more clear "right" answers. Vetted multiple choice answers, no judges and no inter-judge disagreement is at the end of this spectrum. We state in the paper: "The primary limitation is the complexity and subjectivity of quantitatively assessing “animal harm.”" On the other hand, allowing competent LLMs-as-judges to consider different, possibly novel, ways how harms can come about from particular open-ended answers could allow foreseeing harms that even the best human judges could have had trouble foreseeing.
Still, having open ended questions and answers, did lead to mediocre inter-rater agreement, and it can make results seem less convincing and dependent on the set of judges. (We did do lots of prompt & scoring rubric refinement to reduce ambiguity; refining questions could be another step.) We do invite readers to look beyond the scores, examine the whole set of questions and outputs. All results that were used in the paper are available here (sorry for some formatting issues in these log file extracts; the formatting peculiarities were not present in the actual interactions e.g. responses the judges saw): https://drive.google.com/drive/u/0/folders/1IZVrfc1UbS6RQDk2NPcoyVR1B9RCsgAW
The example you mention "help with a legal request which some people think is immoral", this looks like classical helpfulness-harmlessness tradeoff. Not sure what you meant, but e.g. "how to circumvent animal welfare regulations" is probably something we'd want models not to be too helpful with.
We do try to anchor to majority's and legal views, i.e. trying to measure "risk of harm" instead of "speciesism". Then again, majority's views and actions can be inconsistent. I think it's actually good if LLMs, and this benchmark in particular, is sensitive to the fact that actions commonly considered morally ok (like eating meat) can lead to harm to animals.
My impression is that these concerns are in practice pretty much recognized, which is behind the EA focus on extinction or similar permanent changes, i.e. absorbing states. Forecast precision becomes irrelevant after entering an absorbing state, and so becomes "diminution" and "washing out" ("option unawareness" still seems relevant).
Thanks. A general point is that I stripped the model of all nonessential elements (such as non-labor inputs, multiple goods, flexible prices, gradual automation with some firms remaining nonautomated, intl trade) to drive home the basic point that automation does not necessarily lead to an increase in output. That the interests of the firm owners are aligned with the output they get, not that of the total economy. One parallel (non-generous to firm owners) is to a dictator who may wish to increase grip of power at a huge cost to their country.
Now, if workers can find new jobs, possibly even in other industries, this is not a problem. This is the default argument and, at least over timelines that span generations, empirical observation. But this no longer holds when there are no other jobs, i.e. under fool automation. I am now not sure if the "full economy-wide automation" idea was clear in the post, maybe I should clarify it...
It does not seem that Perfect Competition alone would influence the result: the firms that automate would outcompete those that don't. Also, it does not seem like constant per-unit of output costs (e.g. oil) would change much. Semi-fixed or fixed costs (cars, computers) could have more complex effects, probably dependent on the parametrization. I agree the model can be extended in a number of ways, this could be one.
Thanks, added an example which I hope clarifies things. In the example, taxi firm owners go ahead with the automation if they become slightly better off as a result, even if it nullifies the output for everyone else.
The example refers to a single firm. To bring this even closer to reality, the situation could be modelled with multiple firms that decide to automate simultaneously, but with fewer available rides, e.g. due to longer average time of the rides. I haven't done the modelling explicitly but think the basic result would be the same. I.e. firm owners automate even if this leads to lower aggregate output.
Thanks for this post! FYI one cage-free reform that seems unambiguously good is abandoning sow crates, provided the space is increased enough to avoid piglet crushing. Have been looking into that over the past few months, will share more info probably in a month or so.
Successful CEV likely to lead to improved outcomes for animals
Thanks for the estimate, very helpful! The cost for AnimalHarmBench in particular was approximately 10x lower than assumed. 30k is my quick central overall estimate, mostly time cost of the main contributors. This excludes previous work, described in the article, and downstream costs. In-house implementation in AI companies could add to the cost a lot but I don't think this should enter the cost effectiveness calculation for comparison vs other forms of advocacy.
Thanks, and very good question+comment!
1. I'm not sure how closely at the technical level this resembles exactly what the companies do. We did base this on the standard Inspect Framework to be widely usable, and looked at other Inspect evals and benchmarks/datasets (e.g. HH-RLHF) for inspiration. When discussing at a high level with some people from the companies, this seemed like something resembling what they could use but again, I'm not sure about the more technical details
2. Thanks for the recommendation, makes sense. We did think about comms somewhat e.g. to convey intuition for someone skimming that "higher is better" in the paper (https://arxiv.org/pdf/2503.04804) we first present results with different species (Figure 2). Could probably use colours and other design elements to improve the presentation.
> LLMs are not competent.
To me it's not obvious that humans would do strictly better e.g. LLMs have much more factual knowledge on some topics than even experts.
> Have you considered providing a rubric
That's a good idea, we just provided guidance on risk categories but not more detailed rubric (AFAIK, CaML who are building on this work have considered a more detailed rubric)
> do you have a breakdown of the scores by judge?
Don't have it at the moment but yes, sensitivity of results to judge panel composition is a good sensitivity test to have in any case. One caveat - we did observe that the models tended to score themselves higher, so we'd probably have some unmeasured self-bias if trusting a single model. And of the 3 judges (4o, 1.5 pro, 3.5 Sonnet) I think none is clearly worse in terms of capabilities to judge. In fact, some literature suggested *adding more* judges, even if less competent ones, could lead to better results.
Thank you for the comment!
Conceptually, "risk of harm", "harm by failure to promote interest" do seem appropriate for many questions cases. For e.g. for "help me design an [animal species] meat farm" we'd probably want animal interests to be considered in the response. But it can certainly be debated whether "animal interests", "animal welfare" or something else is the formulation we'd better want to have.
I agree there could be benefits to having a more narrowly defined questions and more clear "right" answers. Vetted multiple choice answers, no judges and no inter-judge disagreement is at the end of this spectrum. We state in the paper: "The primary limitation is the complexity and subjectivity of quantitatively assessing “animal harm.”" On the other hand, allowing competent LLMs-as-judges to consider different, possibly novel, ways how harms can come about from particular open-ended answers could allow foreseeing harms that even the best human judges could have had trouble foreseeing.
Still, having open ended questions and answers, did lead to mediocre inter-rater agreement, and it can make results seem less convincing and dependent on the set of judges. (We did do lots of prompt & scoring rubric refinement to reduce ambiguity; refining questions could be another step.) We do invite readers to look beyond the scores, examine the whole set of questions and outputs. All results that were used in the paper are available here (sorry for some formatting issues in these log file extracts; the formatting peculiarities were not present in the actual interactions e.g. responses the judges saw): https://drive.google.com/drive/u/0/folders/1IZVrfc1UbS6RQDk2NPcoyVR1B9RCsgAW
The example you mention "help with a legal request which some people think is immoral", this looks like classical helpfulness-harmlessness tradeoff. Not sure what you meant, but e.g. "how to circumvent animal welfare regulations" is probably something we'd want models not to be too helpful with.
We do try to anchor to majority's and legal views, i.e. trying to measure "risk of harm" instead of "speciesism". Then again, majority's views and actions can be inconsistent. I think it's actually good if LLMs, and this benchmark in particular, is sensitive to the fact that actions commonly considered morally ok (like eating meat) can lead to harm to animals.
My impression is that these concerns are in practice pretty much recognized, which is behind the EA focus on extinction or similar permanent changes, i.e. absorbing states. Forecast precision becomes irrelevant after entering an absorbing state, and so becomes "diminution" and "washing out" ("option unawareness" still seems relevant).
Thanks. A general point is that I stripped the model of all nonessential elements (such as non-labor inputs, multiple goods, flexible prices, gradual automation with some firms remaining nonautomated, intl trade) to drive home the basic point that automation does not necessarily lead to an increase in output. That the interests of the firm owners are aligned with the output they get, not that of the total economy. One parallel (non-generous to firm owners) is to a dictator who may wish to increase grip of power at a huge cost to their country.
Now, if workers can find new jobs, possibly even in other industries, this is not a problem. This is the default argument and, at least over timelines that span generations, empirical observation. But this no longer holds when there are no other jobs, i.e. under fool automation. I am now not sure if the "full economy-wide automation" idea was clear in the post, maybe I should clarify it...
It does not seem that Perfect Competition alone would influence the result: the firms that automate would outcompete those that don't. Also, it does not seem like constant per-unit of output costs (e.g. oil) would change much. Semi-fixed or fixed costs (cars, computers) could have more complex effects, probably dependent on the parametrization. I agree the model can be extended in a number of ways, this could be one.
Thanks, added an example which I hope clarifies things. In the example, taxi firm owners go ahead with the automation if they become slightly better off as a result, even if it nullifies the output for everyone else.
The example refers to a single firm. To bring this even closer to reality, the situation could be modelled with multiple firms that decide to automate simultaneously, but with fewer available rides, e.g. due to longer average time of the rides. I haven't done the modelling explicitly but think the basic result would be the same. I.e. firm owners automate even if this leads to lower aggregate output.
Thanks, added an example.