U.S. Regulatory Updates to Benefit-Cost Analysis: Highlights and Encouragement to Submit Public Comments

DannyBressler

U.S. Regulatory Updates to Benefit-Cost Analysis: Highlights and Encouragement to Submit Public Comments

DannyBressler

7 min readMay 18, 2023

Comments

U.S. Regulatory Updates to Benefit-Cost Analysis: Highlights and Encouragement to Submit Public Comments — EA Forum

[anonymous]

Noting that I think that making substantive public comments on this draft (including positive comments about what it gets right) is one of the very best volunteer opportunities for EAs right now! I plan to send a comment on the draft before the deadline of 6 June.

Toby_Ord

Thanks Danny!

For those who are unaware, Benefit Cost Analysis (BCA) is the main form of quantitative evaluation of cost-effectiveness in the US, UK, and beyond. Two of the biggest problems with it as a method are its counting of the value of a dollar equally for all people (which leads to valuing people themselves in proportion to their income) and a high, constant, discount rate. So the action on both of these are big improvements to quantitative priority setting in the US!

DannyBressler

FYI - OMB has extended the comment period deadline for Circular A-4 to June 20th https://twitter.com/jacklienke/status/1665818966818799617?s=20

However, the Circular A-94 comment deadline is still today.

A-4 comment submission link: https://www.regulations.gov/document/OMB-2022-0014-0001

A-94 comment submission link: https://www.regulations.gov/document/OMB-2023-0011-0001

Larks

Thanks very much for writing this up! It seems like some very exciting changes.

My understanding is that US regulatory cost-benefit analysis are often quite weak in practice, because the Clinton administration changed the rules away from "benefits exceed costs" to a much more subjective "benefits justify costs". It sounds strange but apparently benefits can justify costs without exceeding them! Do you think it is worthwhile providing feedback that it would be good to return to the prior Reagan administration standards that benefits had to exceed costs?

DannyBressler

I think you're referring to the difference between Executive Order 12866 (from the Clinton Administration in 1993) and Executive Order 12291 (From the Reagan Administration in 1981).

The Office of Management and Budget is only asking for comment on Circular A-4 and Circular A-94, not on Executive Order 12866, so I would not suggest making comments on that.

Also, the administration released a new Executive Order 14094 on the same day that the A-4 and A-94 updates were released, which reaffirmed executive order 12866 but made some important changes, for instance increasing the definition of significant regulatory action from $100 million to $200 million, which in my view is a reasonable/helpful thing to do to save administrative capacity. Executive order 12291 required OIRA to look over every regulatory impact assessment regardless of the size, which in consequence meant that they were in inundated with reviewing ~2400 rules a year and therefore it was more difficult to carefully review proposed regulations. OMB is not seeking comment on Executive Order 14094, so I would not suggest making comments on that either.

The part of Executive Order 12866 that is highlighted in the draft A-4 update is: "...in choosing among alternative regulatory approaches, agencies should select those approaches that maximize net benefits (including potential economic, environmental, public health and safety, and other advantages; distributive impacts; and equity), unless a statute requires another regulatory approach." So they are interested in maximizing net benefits. You're right that there is one line in 12866 (which is a 10-page order) that says "Each agency shall assess both the costs and the benefits of the intended regulation and, recognizing that some costs and benefits are difficult to quantify, propose or adopt a regulation only upon a reasoned determination that the benefits of the intended regulation justify its costs." My sense is that this was written because there are often important categories of benefits and costs that are difficult to place a reliable value on (or at least one that would get through OIRA review and taken seriously if the regulation is challenged in court). But it's still important to state and discuss important unmonetized in BCA (this is covered on pages 43-47 of the new draft A-4.). Catastrophic impacts that have highly uncertain probabilities often fall in this category (see my discussion in the main post around how break-even analysis is useful in this context). 12866 is making sure that analysts have the option to account for these unmonetized benefits and costs.

In practice, my sense is that agencies do typically want to show that monetized benefits exceed costs, because this makes it more likely that things will pass OIRA review and that the regulation will make it through the courts. I'd highly recommend this interview with OIRA chief Ricky Revesz if you're curious on this: a typical administration has ~70% of its rules upheld when challenged in courts, and a strong benefit cost analysis supporting the rule certainly helps with that process. And in any case, 12866 and the draft A-4 does direct agencies to choose the option that maximizes net benefits. My sense is that the one line of language around benefits justifying costs was intended to address situations where important benefits or costs cannot be reasonably monetized in a way that would pass OIRA review/make it through litigation but are still important. But even so, my sense is that it is rare for monetized costs to exceed benefits in BCAs, and that there was not some type of step-change when 12866 was passed in 1993 where a bunch of regulations were proposed with monetized costs exceeding monetized benefits.

So to summarize, I actually think that small change in language from 12291 to 12866 thirty years ago was on net a good thing and not a bad thing, and in any case they are not asking for comment on the executive orders. But if there are things in the draft A-4/A-94 you like (or not) I'd highly recommend writing a comment.

Larks

Thanks for explaining!

Global health & development

Cost-effectiveness analysis

Cost-benefit analysis

US policy

Frontpage

On April 6, 2023, the U.S. Office of Management and Budget released a draft of the first update to the Federal benefit-cost analysis (BCA) guidelines in 20 years. I saw a nice article in Vox Future Perfect and a nice EA forum post that covered this. These posts cover some of the key points, but I think there are other important updates that might be overlooked. I will highlight some of those below.

The key new documents stipulating the new draft BCA guidelines are:

Why is the update important? Since 1993, U.S. agencies have been required to conduct a regulatory analysis of all significant regulatory actions (the definition of which was just revised from a rule with an annual impact of more than $100 million to $200 million), which includes an assessment of the costs and benefits of the action. Essentially, all major regulatory actions in the U.S. are subject to BCA, guided by Circular A-4.

However, these updated guidance documents are still drafts. They are subject to public comment and peer review and may be changed significantly in light of the feedback received in this process.

If you think that some of these highlights or other parts of the new A-4 and A-94 are a good idea (or if you don’t) I’d highly recommend submitting a public comment via the Regulations.gov system (A-4 link and A-94 link). The deadline for public comments is June 6^th.

Positive comments that support the approach taken in the document are equally and often more useful/impactful than critical comments. If everyone who dislikes something criticizes it, and everyone who supports something doesn’t bother mentioning their support, it looks like everyone who had an opinion opposed it! So, if you like the approach taken (or don’t), please write a comment! Also, note that comments supported with the analytical reasons why the approach is (or is not) justified are generally more useful and taken more seriously.

Now on to the highlights:

Short-Term Discount Rate

As the Vox article mentioned, the new update to Circular A-4 significantly lowers the discount rate to a 1.7% near-term discount rate. This of course is a large change from the previous 3% rate, but this comes from just using a similar method to the previous 2003 A-4 method with more recent Treasury yield data. The preamble has a deeper dive into this calculation and asks the public a number of questions about whether there is a better approach, for those who are interested.
The draft Circular A-4 continues to take a “descriptive” approach to discounting in which market data is used to determine the observed tradeoffs people make between money now and money in the future. The discount rate is now lower simply because yields have been steadily declining for the last 20 years.
- There are good reasons to believe that rates will continue to be low, but it’s also important to emphasize that if rates are not low in the future, then this near-term discount rate will go up again. This is why from the perspective of placing more weight on the future, the next bullets may be more important.

Long-Term Declining Discount Rate

A related important change (and more robust to future interest rate fluctuations) is that A-4 and A-94 endorse the general concept of declining discount rates, and the A-4 preamble proposed and asked for comment on a specific declining discount rate schedule, which discounts the future at progressively lower rates to account for future interest rate uncertainty. This is in line with the approach recommended in the literature based on the best available economics, and also ends up placing larger weight on the future. Since the proposed schedule was specifically placed in the preamble and not A-4, those who think it should be (or should not be) included in the final A-4 should make sure to comment on the schedule in particular.

Temporal Scope of Analysis

Discount rates are important, but only those benefits and costs that are chosen to be within the temporal scope of analysis are available to be discounted. E.g., if one analyzes a long-run policy and only includes impacts on people already born and ignores impacts on future generations that don't yet exist, this ends up placing comparatively little value on the future even when using a low discount rate.
The new A-4 has a nice brief section that makes it clear that the time frame of the analysis must be long enough to encompass all of the important benefits and costs likely to result from the policy. For instance, it emphasizes the importance of accounting for the impact that a catastrophic event would have on future generations (even if they are indirectly and not directly impacted).
- Note that the U.S. had previously been accounting for the impacts of climate change out to 500 years in BCA through the use of the Social Cost of Greenhouse Gases, so impacts on future generations had been taken into account in that context. However, it is nice that the new A-4 spells out clearly that impacts on future generations should be taken into account whenever they are relevant. This is consistent with President Biden’s Day 1 presidential memorandum that started this process and which emphasizes promoting the interests of future generations among other values.

Spatial Scope of Analysis

There is a robust discussion around the conditions under which it is important for analysts to include global impacts, such as "regulating an externality on the basis of its global effects supports a cooperative international approach to the regulation of the externality by potentially inducing other countries to follow suit or maintain existing efforts."
- This recognizes that global externalities/risks like pandemics, biosecurity, climate change, etc. require international regulatory cooperation. If, e.g., a new global pandemic is caused by poor biosecurity regulatory oversight in other countries, this harms Americans. Both because Americans will get sick and die, but also because disruptions in other countries will cause significant harm to the global economy, global trade, and global stability that will in turn harm Americans. Just as poor biosecurity regulatory oversight in the U.S. causing a pandemic will cause harm in other countries for the same reasons.
- Note that this is also the approach that the U.S. Government has taken to account for climate change in BCA through the Social Cost of Greenhouse Gases by considering global damages from climate change and not just domestic damages. As that document states, global externalities like greenhouse gas emissions and biosecurity policy imply that the whole world enjoys the benefit of one country’s decisions to reduce the harm from the global externality, and therefore “the only way to achieve an efficient allocation of resources for emissions reduction on a global basis—and so benefit the U.S. and its citizens and residents—is for all countries to consider estimates of global marginal damages” from the externality as well as the global marginal benefits of taking actions to reduce the externality.

Distributional Weighting

The draft A-4 and A-94 discuss how traditional benefit-cost analysis often assumes that everyone values a marginal dollar of costs or benefits equally. I.e., it counts a marginal dollar to a billionaire the same as a marginal dollar to someone on the poverty line. The new guidance allows analysts to account for diminishing marginal utility using distributional weights (which have also been called welfare weights and equity weights, among other terms). The weight is constructed based on estimates for the curvature of the utility function (which controls how much marginal utility diminishes). The preamble considers multiple lines of evidence to estimate that the utility function has a curvature parameter of 1.4 (the U.K., for instance, uses a curvature parameter of 1.3).
- Note also that Open Philanthropy's Cause Prioritization Framework also accounts for diminishing marginal utility, although they estimate that marginal utility diminishes at a slightly lower rate (they estimate a curvature parameter of 1, i.e. log utility).

Valuing Lives Equally

When valuing lives, the draft A-4 directs agencies to value lives at the same value as opposed to valuing the lives of rich people more than poor people (because rich people are willing to pay more to avoid mortality risk than poor people): “… it is appropriate to use a value for mortality risk reductions (sometimes referred to as the value of a statistical life, or VSL) that does not depend on the income of the sub-population to which the mortality risk reduction benefits accrue…”
- Note that, as mentioned in the draft A-4, if you value lives using a VSL that varies with income, but then distributionally weight those values, you end up valuing lives equally. This gets a bit technical, but I co-authored a paper that discusses this for those interested (unpaywalled).

Break-Even Analysis

On the topic of BCA related to catastrophes, break-even analysis is another important tool suggested by the draft A-4. Break-even analysis provides a way forward when there is a high degree of uncertainty/ambiguity in important parameters, and you need to determine what those parameters would need to be for the policy to still yield positive net benefits. This is particularly relevant for catastrophic events: “For example, there may be instances where you have estimates of the expected outcome of a type of catastrophic event, but assessing the change in the probability of such an event may be difficult. Your break-even analysis could demonstrate how much a regulatory alternative would need to reduce the probability of a catastrophic event occurring in order to yield positive net benefits or change which regulatory alternative is most net beneficial.”
- There has been a lot of focus on estimating the probability of catastrophic risks, e.g., Table 6.1 in the Precipice, and various estimates are referred to in Carl Shulman and Elliot Thornley’s recent paper on BCA and catastrophic risks. For the purposes of BCA for U.S. regulatory impact assessments, my sense is that these probabilities may be viewed as too speculative (at least for anthropogenic risks) and may be challenged in getting through the various processes that BCAs for regulations must go through including public comment, OIRA review, etc. Fortunately, break-even analysis provides a way forward here.

Accounting for Risk Aversion

The draft A-4 instructs BCA practitioners to explicitly account for uncertainty when doing BCA as well as risk aversion across that uncertainty. This ends up placing more weight on the benefits that help to avoid catastrophes, and thus would tend to favor policies that reduce the probability of catastrophes (e.g. pandemic preparedness).

Mentioned in

EA Organization Updates: July 2023