2. My comments (4 June 2023)

Comments on Circular A4 [Draft for Public Review – 6 April 2023] – Docket ID OMB-2022-0014.

The focus of my comments is on Section 4, Developing an Analytic Baseline (pp 12–15). I have five suggestions.

First, the guidance in Section 4 on an Analytic Baseline needs to be explicitly linked with the guidance on distributional effects in Section 10 (pp. 61–66). In order to analyze the distributional consequences of a proposed regulation, an analyst must use a baseline that describes the condition of each of the affected parties specified in the distributional analysis. For example, if an analyst wants to estimate the welfare effects of a regulation to mitigate flood damages on different income groups, the baseline without the regulation needs to forecast the flood risks by income group. Or consider the example on page 12: “if a harm addressed by a regulation is expected to become more severe over time, the baseline should reflect this trend.” If the analyst is required to do distributional analysis, then the baseline should reflect how much more severe the harm will become over time for the different groups used in the distributional analysis.

Second, the guidance should clarify that multiple baselines may be required for two conceptually different reasons. Multiple baselines may be needed because future conditions are inherently uncertain, and multiple plausible baselines can be used as part of the uncertainty analysis. In this case, the uncertainty analysis involves testing the sensitivity of the results to three different parts of the benefit–cost analysis:

1. (i) different forecasts of baseline conditions without the regulation,

2. (ii) changes in parameters used in the benefit–cost calculations (e.g. the discount rate, the value of a statistical life, the value of time savings), and

3. (iii) changes in the treatment effect of the regulation.

Multiple baselines also may be required because different groups may contest what they perceive the future to hold based on different assumptions about property rights, different interpretations of the science (e.g. around climate change or mortality and morbidity effects of pollution), or different ideological, cultural, or religious expectations about the future (Whittington, Reference Whittington2022). Contested baselines may require that different stakeholders have “their” baseline included to ensure that the analysis has legitimacy for all affected parties.

Third, the term “analytic baseline” does not convey the main point of the guidance that baseline conditions without the proposed regulation need to be forecast into the future. I suggest the term “dynamic baseline” instead. The term “analytic” also implies that there is a technical, uncontested answer as to what the baseline should be, which often will not be the case.

Fourth, the draft guidance in Section 4 introduces several terms and recommendations regarding the baseline that the analyst should use to deal with the dynamic evolution of regulations and compliance with changing regulations:

1. (i) “pre-statutory baseline” and “post-statutory baseline,”

2. (ii) a “dual-baseline approach” that “allows for assessment relative to both a previous regulation and any subsequent guidance.”

3. (iii) Finalization of an interim final rule (IFR) should be assessed with two baselines: (a) a state of the world without the IFR; and (b) a state of the world that “isolates changes in the subsequent finalization of the final interim rule relative to the IFR (if any).”

4. (iv) “If a previous policy has been clarified, delayed, or otherwise revised by a new regulatory or sub-regulatory action, then among the factors needing careful accounting are costs associated with past compliance activity that have already been incurred.”

This section of the guidance would benefit from an explicit statement of the problem that OIRA is trying to solve and an explanation of how this new guidance will solve this problem. In particular, a reasonable interpretation of the guidance to include costs that have already been incurred is that, in some circumstances, OIRA is asking for both ex-ante and ex-post benefit–cost analysis. This would be a substantial departure from previous guidance.

Fifth, the forecast of the state of the world without the regulation will typically require that the analyst make assumptions about population and economic growth, shifts in the demographic structure of the population, technological innovation, and climate change. OIRA should ensure that federal agencies use the same basic assumptions in the construction of a dynamic baseline. For example, it would not be appropriate for the Department of Health and Human Services to use a dynamic baseline constructed based on one assumption about population growth or temperature increases due to climate change, and for the US Environmental Protection Agency to construct a dynamic baseline using different assumptions. Just as OIRA provides agencies guidance on the discount rate, it should provide guidance on the assumptions underpinning the forecast(s) of baseline conditions.

3. Supplemental remarks on OIRA’s revisions to Circular A4 in response to public comments

OIRA made only a few minor changes to the baseline section of the draft Circular A4 in response to public comments. I believe that all five of my comments above are still relevant for analysts conducting RIAs. In these remarks, I want to expand upon my fourth comment above, i.e., that OIRA’s guidance to analysts on the construction of the baseline “would benefit from an explicit statement of the problem that OIRA is trying to solve and an explanation of how this new guidance will solve this problem.” In my opinion, the final A4 baseline guidance still does not clearly explain the two main regulatory situations about which OIRA is concerned. For the benefit of agency analysts and their consultants, I will attempt to clarify the two main regulatory situations that I believe OIRA is trying to address.

I use an example of the problem of controlling the emissions of an unspecified pollutant. In the past, this pollutant has not been regulated. Figure 1 shows the past time trend of this pollutant, the current level of emissions (E _t = 0), and three forecasts (high, medium, and low) of the emission levels in the future if this pollutant continues to be unregulated. All three baseline forecasts show increasing emissions over time (and are thus dynamic), but the quantity of emissions in the future is uncertain. If no regulatory action is taken and the low forecast materializes, in time t = 1 emissions will be E _t = 1^low. If no regulatory action is taken and the high forecast materializes, emissions in time t = 1 will be (E _t = 1^high).

Figure 1. Dynamic baseline: high, medium, and low forecasts.

Now, assume that in time t = 0, the regulator imposes a cap on emissions to ensure that emissions do not exceed the current level in t = 0 (Cap 1 in Figure 2). If the regulator imposes Cap 1, and we assume the medium forecast of the dynamic baseline, then the change in emissions in t = 1 due to the regulatory action (Cap 1) is ΔE _t = 1^{Cap 1}, and the change in emissions in time t = 2 due to the regulatory action (Cap 1) is ΔE _t = 2^{Cap 1}. In the benefit–cost analysis, the analyst will monetize this time stream of changes in emissions to estimate a time stream of economic benefits of the regulatory action to cap emissions at Cap 1.

Figure 2. Regulatory cap on emissions at time period 0 (Cap 1).

Now suppose that in time t = 1, the regulator is considering tightening the cap on emissions from Cap 1 to Cap 2 (Figure 3). There are now two different questions that may be of policy interest. First, what is the incremental effect of tightening emissions from Cap 1 to Cap 2? The incremental effect in time t = 1 is shown in Figure 3 as ΔE _t = 1^{Cap 1 → Cap 2}. The incremental effect in time t = 2 of tightening emissions from Cap 1 to Cap 2 is ΔE _t = 2^{Cap 1 → Cap 2}. Second, one could ask, what is the effect of the more restrictive emissions limit (Cap 2) compared to the dynamic baseline. In this case, the treatment effect of the intervention in time t = 1 is ΔE _t = 1^{Cap 1} + ΔE _t = 1^{Cap 1 → Cap 2} and in time t = 2 is ΔE _t = 2^{Cap 1} + ΔE _t = 2^{Cap 1 → Cap 2}.

Figure 3. More restrictive cap on emissions at time period 1 (Cap 2).

The situation depicted in Figure 3 is closely related to the two main regulatory situations with which I believe OIRA is concerned in its baseline guidance in Circular A4. The first issue concerns how to distinguish between a statutory requirement and a discretionary regulatory action. Suppose that legislative action requires that emissions be capped at a statutory requirement E ^SR (Figure 4). Compared to the dynamic baseline, the effect on emissions of the statutory requirement in time t = 1 is ΔE _t = 1^SR. In time t = 2, the effect of the statutory requirement is ΔE _t = 2^SR. From an agency’s perspective, there may seem to be no point in conducting a benefit–cost analysis of this statutory requirement because this cap on emissions is mandated by legislation, and there is no regulatory decision to be made.

Figure 4. How much discretion does the regulator have? Statutory requirement versus agency discretionary action.

However, suppose the agency has discretion to reduce emissions more than required by the statutory requirement. Assume the agency is considering a more restrictive cap on emissions E ^AD (Figure 4). Now, the agency may propose to do an analysis of the costs and benefits of this discretionary action. In this case, the reduction in benefits in time t = 1 of moving from the statutory requirement E^SR to the reduced level of emissions E ^AD would be ΔE _t = 1^SR → AD. In effect, the statutory requirement can be viewed as the new baseline from which to measure the incremental effect of the agency’s discretionary action.Footnote ¹

On the other hand, other stakeholders may want to know what the benefits and costs of the agency’s discretionary emissions standard E ^AD are compared to the dynamic baseline (i.e. a “without statute” baseline). In this case, the change in emissions in time t = 1 would be ΔE _t = 1^SR + ΔE _t = 1^SR → AD. In time period t = 2, the change in emissions would be ΔE _t = 2^SR + ΔE _t = 2^SR → AD.

A second, closely related issue with which OIRA is concerned involves the analysis of interim and final rules (Figure 5).Footnote ² Suppose that in time t = 0, the agency issues an interim regulation to cap emissions to E ^IR. Then, in time t = 1, the agency is considering tightening emissions beyond the limit set in the interim rule. In the final rule, the agency proposes to restrict emissions to E ^FR. Again, one can ask two different questions: (i) what emission reduction would result from changing the interim rule to the final rule (ΔE _t = 1^IR → FR in time t = 1 in Figure 5)? and (ii) what emissions reduction would result from the final rule compared to the dynamic baseline, which in time t = 1 would be ΔE _t = 1^IR + ΔE _t = 1^IR → FR?

Figure 5. What if final regulatory rule is different (more restrictive) than an interim rule?

The key difference between the example of a statutory requirement versus agency discretionary action (Figure 4) and the interim versus final rule (Figure 5) regards the timing of the regulatory actions. In the example of a statutory requirement versus agency discretionary action, the analyst could conduct an ex-ante analysis in time t = 0 of the benefits of the reduction in emissions due to both actions. The time stream of estimated benefits from both of these interventions could start in time t = 0. In this case, no retrospective analysis is required.

In the example of the interim versus final rule, there is a sequencing issue: the interim rule by definition comes first, followed by the final rule. The analyst could conduct an ex-ante analysis at t = 0 of the interim rule. However, any ex-ante analysis of the final rule must start when the final rule is assumed to be implemented (time t = 1 in Figure 5). The early benefits of the interim rule would have already occurred before the final rule is promulgated. Suppose the analyst wanted to compare the benefits of the interim and final rules if both had been initiated at time t = 0. The analyst would then ask, “what would the benefits of the final rule have been if it had been implemented in time t = 0?” They could use as relevant evidence what actually happened from time t = 0 to time t = 1, from the implementation of the interim rule, but this would require retrospective analysis.

Alternatively, the analyst could compare the benefits of continuing the interim rule and implementing the final rule, and initiate the benefit–cost analysis in time t = 1. In this case, the benefits achieved by the interim rule from time t = 0 to time t = 1 would be ignored.

In both situations (statutory requirement versus agency discretion and interim versus final rule), the two comparisons described above will likely be of interest to some stakeholders. In both situations, I think the agency should prepare two benefit–cost analyses in order to clearly show the difference between the incremental effect of the second regulatory change and the total effect of the most restrictive regulation compared to the dynamic baseline. These two regulatory situations provide an excellent illustration of the need for multiple dynamic baselines. In the first situation, OMB would consider the world with the statutory requirement as a “secondary” dynamic baseline. In the second situation, the world with the interim rule would be considered as a “primary” dynamic baseline. It is important to ensure that the same baselines are used for any given comparison of costs and benefits. For example, it would be a mistake to compare the benefits of an interim rule to the costs of a final rule.

I have three final reflections. First, the terminology used to refer to the state of the world that is forecast to unfold in the absence of the policy intervention is important because ambiguity over the baseline can be the source of considerable confusion.Footnote ³ I am unconvinced by OIRA’s argument in support of its terminology of “analytic baseline” and hope that this term does not gain widespread currency in the field of benefit–cost analysis. The reason I prefer the term “dynamic baseline” to “analytic baseline” is that it better conveys the key point that the baseline is an uncertain forecast of future conditions without the policy intervention. In its response to public comments, OIRA notes that status quo conditions may continue in the future as a justification of its preference for the term “analytic baseline.” It is unlikely that status quo conditions will continue in the future, but if the analyst thinks current conditions will remain unchanged in the future, this is also a forecast of baseline conditions.

Second, OIRA’s final baseline guidance still leaves agencies with too much discretion to make their own assumptions about such basic parameters in the construction of a dynamic baseline as population and economic growth, technological innovation, and climate change. Circular A4 suggests that agencies can consult with other Federal agencies that have specific data or models that would be helpful in the construction of the baseline. However, just as OIRA requires that agencies adhere to guidance on the discount rate, it should insist on consistency in the parameters used in the construction of the dynamic baseline across federal agencies.

Third, the use of multiple dynamic baselines has a benefit that is often not acknowledged. One way that analyst bias may creep into a benefit–cost analysis is in the construction of the dynamic baseline. An analyst may choose a dynamic baseline for their analysis, not because it is their assessment of the most likely forecast of the state of the world without a regulatory or policy action, but because it makes the results of the analysis conform to their desired outcome. If they want the assessment of the regulatory action to be positive, they may select a dynamic baseline that makes the benefits large (or the costs small). If they want the assessment of the regulatory action to be negative, they may select a dynamic baseline that makes the benefits small (or the costs large). The use of multiple dynamic baselines should be standard practice because it calls attention to the baseline assumptions and makes them more transparent. Thus, to some extent, the use of multiple baselines mitigates the risk of analyst bias.