Hostname: page-component-cc8bf7c57-77pjf Total loading time: 0 Render date: 2024-12-10T02:44:34.994Z Has data issue: false hasContentIssue false

Consumer’s Guide to Regulatory Impact Analysis: Ten Tips for Being an Informed Policymaker

Published online by Cambridge University Press:  27 July 2017

Susan Dudley*
Affiliation:
George Washington University, GW Regulatory Studies Center, 805 21st St, NW, Suite 612, Washington, DC 20052, United States, e-mail: sdudley@gwu.edu
Richard Belzer
Affiliation:
Regulatory Checkbook, United States
Glenn Blomquist
Affiliation:
University of Kentucky, United States
Timothy Brennan
Affiliation:
University of Maryland, Baltimore County, Public Policy; Resources for the Future, United States
Christopher Carrigan
Affiliation:
George Washington University, Trachtenberg School of Public Policy and Public Administration, United States
Joseph Cordes
Affiliation:
George Washington University, United States
Louis A. Cox
Affiliation:
Cox Associates; University of Colorado, United States
Arthur Fraas
Affiliation:
Resources for the Future, United States
John Graham
Affiliation:
Indiana University Bloomington, United States
George Gray
Affiliation:
George Washington University, School of Public Health, United States
James Hammitt
Affiliation:
Harvard University (Center for Risk Analysis) and Toulouse School of Economics, United States
Kerry Krutilla
Affiliation:
Indiana University Bloomington, United States
Peter Linquiti
Affiliation:
George Washington University, United States
Randall Lutter
Affiliation:
Resources for the Future, University of Virginia, FrankBatten School of Leadership and Public Policy, United States
Brian Mannix
Affiliation:
George Washington University Regulatory Studies Center, United States
Stuart Shapiro
Affiliation:
Bloustein School of Planning and Public Policy, Rutgers University, United States
Anne Smith
Affiliation:
NERA Economic Consulting, Washington DC, United States
W. Kip Viscusi
Affiliation:
Vanderbilt University, United States
Richard Zerbe
Affiliation:
University of Washington, Public Policy and Governance, United States
*
Rights & Permissions [Opens in a new window]

Abstract

Regulatory impact analyses (RIAs) weigh the benefits of regulations against the burdens they impose and are invaluable tools for informing decision makers. We offer 10 tips for nonspecialist policymakers and interested stakeholders who will be reading RIAs as consumers.

  1. 1. Core problem: Determine whether the RIA identifies the core problem (compelling public need) the regulation is intended to address.

  2. 2. Alternatives: Look for an objective, policy-neutral evaluation of the relative merits of reasonable alternatives.

  3. 3. Baseline: Check whether the RIA presents a reasonable “counterfactual” against which benefits and costs are measured.

  4. 4. Increments: Evaluate whether totals and averages obscure relevant distinctions and trade-offs.

  5. 5. Uncertainty: Recognize that all estimates involve uncertainty, and ask what effect key assumptions, data, and models have on those estimates.

  6. 6. Transparency: Look for transparency and objectivity of analytical inputs.

  7. 7. Benefits: Examine how projected benefits relate to stated objectives.

  8. 8. Costs: Understand what costs are included.

  9. 9. Distribution: Consider how benefits and costs are distributed.

  10. 10. Symmetrical treatment: Ensure that benefits and costs are presented symmetrically.

Type
Articles
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.
Copyright
© Society for Benefit-Cost Analysis 2017

Regulatory impact analyses (RIAs) weigh the benefits of regulatory proposals against the burdens they impose. They are invaluable tools for informing decision makers about the effects of regulatory choices; even regulatory decisions that are ultimately made on political, legal, ethical, or other grounds will benefit from the structured evaluation of trade-offs and alternatives that a good RIA provides.

Although RIAs are a core feature of regulatory practice in the United States and other countries (Kirkpatrick & Parker, Reference Kirkpatrick and Parker2007; OECD, 2016) they can be difficult to interpret. Key methodologies, assumptions, limitations, and uncertainties can be obscured by dense, technical writing, and RIAs sometimes appear to be “used to justify decisions already made, rather than to inform those decisions” (Carrigan & Shapiro, Reference Carrigan and Shapiro2016; see also Harrington, Heinzerling & Morgenstern, Reference Harrington, Heinzerling and Morgenstern2009). RIAs often serve as legal documents, running to hundreds or even thousands of pages, prepared by agencies in anticipation of litigation.

U.S. regulatory agencies develop RIAs before issuing significant new regulations, and nongovernmental interests may also present their own analyses of how different policies will affect outcomes. Dense or complex RIAs can be challenging for policy officials and interested parties to comprehend and interpret, making it difficult to distinguish facts from conjecture and to understand the likely consequences of alternative policy choices.

While numerous technical guidelines exist to aid development of RIAs (OMB, 2003, 2010; OECD, 2008), none are geared toward nonspecialist policymakers and interested stakeholders who will be reading RIAs as consumers. This guide attempts to fill that gap. It first reviews the purpose of an RIA, and then offers policy makers and other consumers of RIAs 10 tips for asking informed questions when reviewing and interpreting them.

The guide is designed to (1) explain why RIAs are valuable, review their key elements, and describe best practices; (2) point out the ways in which RIAs might fall short of achieving these best practices; (3) help readers to better judge the quality of information provided in RIAs, and make discerning assessments about the methods employed; and (4) improve readers’ capacity to critically evaluate the justification offered to support regulatory actions.

What are RIAs and when are they used?

Before issuing new regulations, governments across the world conduct RIAs to understand possible consequences of alternative decisions (OECD, 2015). Executive branch agencies in the United States have for more than 35 years been required to conduct RIAs before issuing economically significant regulations (those with impacts of $100 million or more in a year), and to rely on those analyses in designing regulations. Other developed nations also rely on impact assessments to support regulatory actions, although they are often less quantitative than those conducted in the United States.

In the simplest terms, the goal of an RIA is to present information to decision makers to help them to ensure that proposed regulations do more good than harm. In the United States, presidential Executive Orders 12866 (Clinton, Reference Clinton1993) and 13563 (Obama, Reference Obama2011) set forth principles of good regulatory decision-making; the Office of Management and Budget’s (OMB’s) Circular A-4 (2003) provides detailed guidance for developing RIAs, and its RIA Checklist (2010), FAQs (2011a ) and Primer (2011b ) highlight key elements and principles. Agencies have also developed detailed guidance specific to their regulatory mandates (for example, EPA, 2000).

In general, the level of analysis should be pertinent to the decision in question. The OMB reviews hundreds of draft regulations each year, most of which are accompanied by some form of analysis, and, on average, 80 or more per year are deemed to be economically significant and thus require a quantitative RIA (RegInfo.gov).

Per OMB’s RIA Primer (2011b ), the three basic elements that each RIA should include are (1) “a statement of the need for the regulatory action,” (2) “a clear identification of a range of regulatory approaches,” and (3) “an estimate of the benefits and costs – both quantitative and qualitative – of the proposed regulatory action and its alternatives.” Thus, RIAs often involve considerations and analysis that go beyond benefit-cost analysis (BCA, also called cost-benefit analysis or CBA). Nevertheless, BCA, which examines economic welfare differences among alternative policies, is an important component of the RIA framework (Arrow et al., Reference Arrow, Cropper, Eads, Hahn, Lave, Noll, Portney, Russell, Schmalensee, Kerry Smith and Stavins1996). For rules aimed at protecting public health and safety, risk assessment is another key component, attempting to evaluate the risks posed under certain conditions, and the potential changes in risk achievable due to different policy options (OIRA & OSTP, 2007). Cost-effectiveness analyses may also be relevant for some policy questions (OMB, 2003).

President Trump has recently required agencies to offset the costs of new regulations by removing or modifying existing regulations (E.O. 13771). The OMB’s guidance on this order states that it does not negate longstanding requirements to consider benefits or to conduct RIAs (OMB, 2017). It may lead to an increased use of RIAs to examine the effects of modifying existing regulations, as well as prospective regulations.

In the United States, administrative regulatory agencies develop RIAs both to inform decision makers within the executive branch and to inform the public and Congress. There may be tension between these goals, especially in instances where decision makers have a politically driven reason to prefer a regulatory option with lower net benefits than an alternative, or when authorizing legislation constrains policy options (Lutter, Reference Lutter1999; Graham, Reference Graham2008).

The 10 tips presented below are the result of a consensus effort of a diverse group of experts in RIA, organized by the George Washington University Regulatory Studies Center.Footnote 1 We recognize that not all of these tips will apply to every RIA, and RIAs need not include all of the elements discussed here to provide valuable information for evaluating policy options. We hope that these 10 tips can help policy makers and other interested readers to appreciate the value of RIAs, ask appropriate questions of an RIA, understand what the analysis really means, and judge its implications for regulatory policy.

1 Core problem: Determine whether the RIA identifies the core problem (compelling public need) the regulation is intended to address

Regulatory impact analyses should clearly articulate the “need for government action” (OMB, 2011b ), the core problem that requires federal regulatory intervention. Generally, this should be a description of the “material failures of private markets” (E.O. 12866, Section 1), although, as discussed below, regulations may be justified by other goals.

1.1 Appreciate the role of markets when assessing regulatory policies

The concept of “market failure” is an important one in regulation. In a market economy disciplined by competition, the exchange of goods and services between willing buyers and sellers uses price signals to allocate scarce resources to their most valued uses, to encourage innovation, and to meet consumer needs. Regulation and other forms of government intervention can disrupt those signals, making the market less efficient and harming social welfare. Thus, the first question when reviewing an RIA will often be whether it explains why market forces will not achieve an outcome at least as efficient as what government reasonably would be expected to accomplish through regulation.

1.2 Recognize when market forces may be inadequate

Markets may not efficiently allocate resources for several reasons. First, efficient markets need an adequate infrastructure, including the rule of law, well-defined property rights, and a system of exchange. Open-access fisheries, for example, may be vulnerable to collapse without either an effective system of property rights or a suitable regulatory alternative. Second, existing policies that were poorly designed may impede the functioning of markets. For example, economic regulation of private sector prices, entry, and exit tends to distort market signals and has historically kept prices of some goods and services higher than necessary, harming consumers rather than protecting them (OMB, 2003, p. 6). These problems might fit into the category of “failures of … public institutions,” described in E.O. 12866 (Section 1.b.1).

Third, markets may not perform efficiently due to classic “market failures” or deficiencies inherent in the market itself. OMB Circular A-4 breaks these into three categories: (1) externalities, public goods, and common property resources; (2) market power; and (3) inadequate or asymmetric information. (To learn more about market failures, see OMB, 2003, pp. 4–5, and Viscusi, Vernon & Harrington, Reference Viscusi, Vernon and Harrington2005.)

In some cases, a regulation may be initiated not in response to a failure of private markets but to improve the efficiency of government programs, or to implement a legislative mandate of one kind or another where the underlying objective is something other than improvement of efficiency, such as to redistribute incomes (OMB, 2003, p. 5), protect civil rights, or provide for “universal” access to services deemed to be important. In these cases, the RIA can identify which alternative would achieve the regulatory goal in the most cost-effective way.

1.3 Question anecdotal or unrealistic justifications

In evaluating the RIA’s expressed need for regulatory action, anecdotal observations that may illustrate symptoms of a problem without articulating the underlying cause of those symptoms may deserve scrutiny. Regulatory actions that do not explicitly point to a failure of private markets or public institutions underlying the need for action are likely to produce lower net benefits than those that correctly identify and seek to remedy the fundamental problem.

Regulations that derive most of their benefits from providing private monetary gains that individuals can achieve without further government intervention, such as fuel savings from driving energy-efficient cars, require a particularly demanding burden of proof (Gayer & Viscusi, Reference Gayer and Viscusi2013). Does the RIA provide evidence that individuals behave irrationally (and do not learn) in the specific situation covered by the proposed regulation? Some researchers have found that carefully formed “nudges” can help individuals to overcome heuristics and biases to make choices that improve their well-being (Thaler & Sunstein, Reference Thaler and Sunstein2008). What insights do regulators have that make them better able to judge other people’s preferences, or be more faithful agents of their interests, than the people themselves (Mannix & Dudley, Reference Mannix and Dudley2015)?

2 Alternatives: Look for an objective, policy-neutral evaluation of the relative merits of reasonable alternatives

Regulatory impact analyses should examine human welfare differences among alternative policies (OMB, 2003). Thus, it is important to look at whether the RIA considers plausible alternatives or whether it only presents the preferred regulatory approach (perhaps with some straw-man alternatives). Do alternatives vary in their stringency? Are different regulatory instruments considered? Is evidence presented that allows for evaluation of alternatives and their relative effect on human welfare? Or, instead, does the RIA seem to focus on justifying a particular regulatory action?

Are the alternatives likely to target the identified failure of private markets or public institutions? For example, if the problem is localized, are nonfederal solutions considered? Or, if the identified market failure is asymmetric information, do the alternatives serve to address those asymmetries and improve the information available to those apparently lacking it? If consumer decision-making errors support the need for regulation, do alternatives attempt to address the identified bias while respecting the importance to consumers’ individual welfare of being able to make decisions reflecting their own preferences and resources (OMB, 2011c )? There should be a well-defined documentation of the economic basis for attributing benefits to regulatory options that diminish individual choice without considering alternatives that provide information or establish different defaults to improve information processing (OMB, 2011b ). The set of people granted standing – i.e., those whose benefits and costs count in the RIA – should generally be U.S. citizens and residents, although RIAs may analyze and report separately effects on others (OMB, 2003).

The RIA should “consider the alternative of not regulating” (E.O. 12866, Section 1.a). A large body of regulations already addresses market failures. Because regulatory proposals are, by nature, incremental extensions of the existing regulatory framework, they must provide benefits that exceed costs for small changes at the margin. Theory and empirical evidence show that incremental (marginal) costs tend to rise and benefits to fall with intensifying regulation. Given this reality, the presumption should be against additional regulation without a plausible case that existing regulations are not adequately addressing the identified market failure.

3 Baseline: Check whether the RIA presents a reasonable “counterfactual” against which benefits and costs are measured

One key component of regulatory impact analysis is identification of the assumed state of the world in the absence of the regulation (the “counterfactual,” or “baseline”). The difference between this baseline and the state of the world with the regulation is the incremental change that the regulation makes, and measurement of the benefits and costs of this incremental change is what an RIA must do. Thus, the specification of a realistic baseline is a critical part of determining the incremental benefits and costs of a regulatory proposal.

In reviewing an RIA, one should evaluate whether the baseline is a reasonable reflection of the way the world would look in the absence of the proposed action. Does it take into account the effect of other regulations, the evolution of the market in the absence of regulation, and other external factors (OMB, 2003, p. 15)? Because the future state of the world, with or without the regulation, is uncertain, an RIA might use sensitivity analysis with alternative baselines to determine the degree to which baseline assumptions affect the overall conclusions of the analysis. (See also tip 5.) In all cases, one must be sure that the baseline from which costs and benefits are measured is consistently applied; if not, the resulting estimates cannot be reliably compared.

4 Increments: Evaluate whether totals and averages obscure relevant distinctions and trade-offs

For a rule with multiple components (for example, one that both sets permissible exposure levels and requires technology controls), an RIA that estimates the benefits and costs of the rule as a whole, without presenting the marginal impacts of the key elements, will not reveal the merits of individual requirements. For example, the RIA may suggest that a proposed regulation would yield net benefits, but most of those benefits may derive from one requirement, while most of the costs derive from another. This would be revealed if the estimated benefits and costs of each component were estimated separately. In such a case, this incremental analysis would show that one component had much greater net benefits, and might point to a different preferred policy.

Marginal (or incremental) analysis is also important if different degrees of stringency are considered. Knowledge of the estimated incremental benefits and costs of successively more stringent alternatives (in addition to the total benefits and costs of each option) can be informative. For example, when considering alternative emissions limitations, the RIA should measure the benefits and costs of each alternative from the next most stringent alternative, as well as from the baseline.

The “law of diminishing returns” is the common generalization that marginal benefits tend to decrease, and marginal costs tend to increase, as alternatives get more stringent. The most cost-effective actions are generally taken first, so the unit compliance costs presented in the RIA should increase as incremental regulation becomes harder and harder to achieve. Analysis that predicts that the incremental benefits of reducing emissions by an additional unit will be greater than those of the previous unit, for example, might indicate that there are economies of scale to warrant reducing emissions further (or at least to examine further reductions), or that there is a problem with the underlying assumptions.

5 Uncertainty: Recognize that all estimates involve uncertainty, and ask what effect key assumptions, data, and models have on estimates

All estimates involve uncertainty, so one should be skeptical of overly precise estimates of costs or benefits. The OMB directs agencies to “provide expected-value estimates as well as distributions about the estimates, where such information exists” (OMB, 2003, p. 45). Expected value is the most important summary statistic when the government seeks to act in a risk-neutral fashion (as opposed to a risk-averse or risk-seeking manner) in the face of uncertainty. Thus, one should look to see whether the RIA presents unbiased “expected values,” as well as ranges for costs and benefits.

Further, one should realize that assumptions that claim to be “‘public health protective,’ which err on the side of overstating risk when data are lacking,” (Gray & Cohen, Reference Gray and Cohen2012) can inflate estimates of certain risks, benefits, or costs relative to others, and lead to misaligned priorities because the degree of precaution differs across risks (Nichols & Zeckhauser, Reference Nichols and Zeckhauser1988; Hamilton & Viscusi, Reference Hamilton and Viscusi1999). Moreover, distortions in different parts of the analysis can interact and multiply, rendering the conclusions unreliable (Hamilton & Viscusi, Reference Hamilton and Viscusi1999; Fraas & Lutter, Reference Fraas and Lutter2012).

Analyses that do not provide information on how sensitive the primary estimate is to assumptions, data, and models, and the range of outcomes possible under reasonable alternative analytic assumptions should raise questions. Sensitivity analysis examines different “what if” scenarios to see how changes in key assumptions (or combinations of assumptions) influence estimated outcomes. Because many uncertain factors determine the impact of any regulation, one should look for a convincing justification regarding which uncertain parameters have the most consequential effects on outcomes, and a sensitivity analysis that varies these factors over a reasonable range to gauge their effects on the rule’s net benefits.

The OMB requires a quantitative uncertainty analysis for regulations with likely impacts greater than $1 billion (OMB, 2003, pp. 38–42). One valuable way in which agencies inform readers about uncertainty is through Monte Carlo simulations that use probability distributions for important analytic inputs and repeated random sampling to generate a probability distribution for decision-relevant outputs (such as net benefits or lives saved) (Krutilla, Good & Graham, Reference Krutilla, Good and Graham2015). Formal elicitation of experts’ judgments can be used to develop probability distributions for analytic inputs when adequate data are unavailable (OMB, 2003, p. 41). The computation of a break-even level of effectiveness at which the rule’s benefits would justify the costs might also add useful perspective.

The problems that regulations target often have multiple interacting causes, making it difficult to cleanly attribute the effect of the regulation in isolation. For example, commercial vehicle crashes are usually caused by a combination of weather, technology, human behavior, and infrastructure conditions. An hours-of-service regulation designed to reduce crash risks by mitigating driver fatigue will reduce one risk factor in this larger cluster. In such a case, where the fatigue–accident relationship is uncertain, does the RIA use sensitivity analysis to explore alternative assumptions about the relationship of hours-of-service rules to fatigue, and the impact of fatigue on crash risks?

Uncertainty analysis can also highlight what additional information would be most valuable for making a decision. If expected outcomes hinge on the value assumed for a particular uncertain variable, it might be appropriate to gather more information regarding that variable prior to making a decision (OMB, 2003, p. 39), or to ask what policies would generate the information necessary to reduce that uncertainty (Greenstone, Reference Greenstone, Moss and Cisternino2009).

6 Transparency: Look for transparency and objectivity of analytical inputs

OMB guidance requires that significant information disseminated to the public be “capable of being substantially reproduced, subject to an acceptable degree of imprecision” (OMB, 2002, p. 8460). This standard can be difficult for a nonexpert reviewer to assess. A precondition of reproducibility, however, is transparent presentation of the underlying data, assumptions, and models relied on to draw conclusions. Disclosure of underlying data and computer code has become standard among the more prestigious scientific and technical journals, which allow for data sharing agreements when individually identifiable information prevents public disclosure. These disclosure policies appear to improve the reproducibility of the results of published papers (Lutter & Zorn, Reference Lutter and Zorn2016). Without transparent presentation of inputs used and alternative inputs considered, reviewers will find it difficult to judge the objectivity or accuracy of the resulting estimates.

Often, less is understood about the effects of regulatory intervention (such as reductions in health risks, for example) than the valuation of those effects, so one should look particularly for a clear presentation of alternative plausible models and assumptions used to predict regulatory outcomes. Precise-sounding predictions not only can hide considerable uncertainty about risks, benefits, and costs, as noted in tip 5, but also can mask a reliance on biased inferences and assumptions.Footnote 2 While some judgments are necessary to translate scientific evidence into risk assessment, if they are not policy-neutral, they can lead to distorted risk estimates and false precision in the presentation of scientific information (Dudley & Gray, Reference Dudley, Gray and Johnston2012). Furthermore, if they are not transparent, reviewers will be unable to judge their objectivity.

If an RIA for a significant regulation aimed at reducing health, safety, or environmental risks does not provide a discussion of alternative interpretations of the scientific basis, it may not be providing a full and accurate picture of potential risks (Sutherland, Spiegelhalter & Burgman, Reference Sutherland, Spiegelhalter and Burgman2013). Moreover, if it does not provide the information necessary to assess how different inputs (assumptions, data, models, etc.) would change estimated outcomes, it will not provide the information needed to evaluate the consequences of the regulatory action.

7 Benefits: Examine how projected benefits relate to stated objectives

Does the RIA clearly explain how regulatory objectives will be achieved, or is the preferred regulatory outcome simply presumed to work as intended? The analysis should lay out causal linkages between regulatory requirements and desired outcomes, discuss the evidence supporting these linkages, and show how they differ across alternatives (Newcomer, Hatry & Wholey, Reference Newcomer, Hatry and Wholey2015). If regulatory outcomes depend on factors beyond the regulation’s control (e.g., energy prices), one should look for a presentation of how variations in those factors would affect outcomes.

7.1 Look for evidence for and against a causal relationship

It should be remembered that correlation does not always imply causation, so one should look for a persuasive demonstration of causation rather than relying on indicators of correlation or association to predict changes in health risks, costs, or benefits (Cox & Popken, Reference Cox and Popken2008). One should be alert for words such as “linked to” or “associated with.” Does the RIA present information indicating whether a change in the variable to be regulated (e.g., emissions) precedes and causes a change in the targeted outcome (e.g., public health) as well as the effect other possible factors might have on the outcome (Cox, Reference Cox2016)? Do the underlying studies linking harm (e.g., mortality or morbidity) to exposure to some hazard use statistical strategies appropriate to the identification of causal relationships, such as regression discontinuity designs, or quasi-experiments (Dominici, Greenstone & Sunstein, Reference Dominici, Greenstone and Sunstein2014)?

7.2 Ask whether the analysis accurately characterizes indirect benefits and costs

Some RIAs present large “co-benefits” (or “ancillary benefits”). These may need closer inspection, particularly if the co-benefits are much larger than the direct benefits, if the direct benefits on their own are significantly less than the estimated costs, or if the co-benefits appear to materialize “for free.” The presence of co-benefits almost always signals that the agency is counting costs and benefits that arise outside of the specific statutory authority that the regulation operationalizes. That is not a problem per se; indeed, all significant benefits and costs should be counted. However, in such cases, one might ask why this regulation is the best way to achieve those co-benefits. Generally, one would expect that regulation targeted directly at a particular outcome can achieve it more cost-effectively than one that achieves it circuitously as a side effect (co-benefit) of an unrelated regulation, and a sound analysis must make a thorough inventory of both the harmful and the beneficial consequences of each alternative.

8 Costs: Understand what “costs” are included

Ideally, the RIA will estimate the “opportunity cost” of the regulatory action (the lost value of the best alternative forgone). However, opportunity costs can be difficult to measure, so RIAs often rely instead on the costs of compliance, measured by the expenditures businesses make on technology or methods used to fulfill the regulatory requirements. While business compliance costs are not welfare changes, they are often a reasonable proxy for welfare changes that are passed through to consumers, to employees, and to business owners (which, for publicly traded companies, often include large mutual funds and pension funds owned by many individuals) (Mannix, Reference Mannix, Coglianese, Finkel and Carrigan2014). If compliance costs are used, they should exclude taxes, which are mostly transfers to other members of society.

The costs of some types of regulatory actions, such as prohibitions on certain actions or products, cannot be approximated by compliance costs, however. For example, a rule prohibiting a specific product may not involve compliance costs, but it would have an opportunity cost because users would not able to enjoy the product’s benefits.

9 Distribution: Consider how benefits and costs are distributed

Those who bear the costs of a regulation and those who enjoy its benefits often are not the same people. Does the RIA present evidence on the incidence of benefits and costs so that one can understand how they affect different people?

While some government programs are designed to redistribute wealth (e.g., food stamps), others do so inadvertently (e.g., regulations that raise food prices might have disproportionate impacts on low-income Americans, or regulatory compliance might burden small businesses more than large ones). The OMB calls on agencies to “distinguish … between real costs and transfer payments, [which are] monetary payments from one group to another that do not affect total resources available to society” (OMB, 2003, p. 38).

Does the RIA reveal whether a regulation will have different impacts on different subpopulations, including those living in different regions of the country, businesses of different sizes, individuals of different ages, and people with different incomes or ethnic and socioeconomic characteristics? It might be interesting to know whether the benefits are “global,” accruing to foreign countries, but the costs are borne domestically. If so, is that the program’s purpose, and what is the net effect on the United States (Fraas et al., Reference Fraas, Lutter, Dudley, Gayer, Graham, Shogren and Viscusi2016)?

The OMB advises agencies, “Where distributive effects are thought to be important, the effects of various regulatory alternatives should be described quantitatively to the extent possible, including the magnitude, likelihood, and severity of impacts on particular groups” (OMB, 2003, p. 14). If an RIA ignores distributional effects – implying that they are not “thought to be important” by the promulgating agency – one should look for a compelling explanation, based on logic and evidence, that costs and benefits generally fall on the same groups of people.

10 Symmetrical treatment: Ensure that benefits and costs are presented symmetrically

One should pay attention to how benefits and costs are measured. For example, both should be measured from the same baseline and over the same time frame. Similarly, the discount rate used to convert the future streams of benefits and costs to present values should generally be the same (OMB, 2003). The choice of discount rate can have a very large impact on the present value of estimates, so an RIA should clearly defend the use of different discount rates applied for costs and benefits (and present the effects of alternative choices in sensitivity analysis).

Are the boundaries of the analysis framed symmetrically? No analysis will ever be complete, of course. However, major elements should not be missing on one side of the equation, or overemphasized on another. For example, if the analysis presents evidence of co-benefits, are ancillary costs or countervailing risks examined to the same extent?

Final thoughts

Regulatory impact analysis can be an invaluable method for transparently evaluating contentious policy choices before they are put in effect. The OECD states that its “most important contribution to the quality of decisions is not the precision of the calculations used, but the action of analyzing – questioning, understanding real-world impacts and exploring assumptions” (OECD, 2002, p. 47). Thus, an RIA’s purpose is not to compel decisions, but rather to provide policy makers responsible for making decisions with the information needed to think through the possible consequences of different regulatory actions. Despite this informational purpose, RIAs can be opaque, complex, and even intimidating. Wittingly or unwittingly, they may be written in a way that obfuscates important information or skews the analysis to support a particular outcome.

Savvy policymakers and other consumers need to be aware of the basic elements involved in a good RIA and the kinds of questions to ask to get the most from reading one. The preceding 10 tips should help readers of RIAs to interpret what they read and ask appropriate questions.

Footnotes

1 These tips represent the professional recommendations of the authors and not necessarily the views of any organizations with which they may be affiliated. As with any consensus document, individual authors might have written a somewhat different document, but all of the authors broadly support the 10 tips presented here.

2 Former EPA scientist Robert Lackey cautions against what he calls “normative science,” or “information that is developed, presented or interpreted based on an assumed, usually unstated, preference for a particular policy choice” (Reference Lackey2013). He finds that “too often … scientific information presented to the public and decision-makers is infused with hidden policy preferences.”

References

Arrow, Kenneth J., Cropper, Maureen L., Eads, George C., Hahn, Robert W., Lave, Lester B., Noll, Roger G., Portney, Paul R., Russell, Milton, Schmalensee, Richard, Kerry Smith, V. & Stavins, Robert N. (1996). Is There a Role for Benefit-Cost Analysis in Environmental, Health, and Safety Regulation? Science, 272, 221222. April 12, 1996.Google Scholar
Carrigan, Christopher & Shapiro, Stuart(2016). What’s Wrong with the Back of the Envelope? A Call for Simple (and Timely) Benefit-Cost Analysis. Regulation & Governance. Accessed January 16, 2017. doi:10.1111/rego.12120.Google Scholar
Clinton, William(1993). Executive Order 12866, Regulatory Planning and Review, and Amendments. Accessed January 16, 2017. https://www.whitehouse.gov/omb/inforeg_regmatters#eo12866.Google Scholar
Cox, Louis Anthony(2016). A Causal Analytics Toolkit (CAT) for Assessing Potential Causal Relations in Data. The George Washington University Regulatory Studies Center. Last modified May 10, 2016. https://regulatorystudies.columbian.gwu.edu/causal-analytics-toolkit-cat-assessing-potential-causal-relations-data.Google Scholar
Cox, Louis Anthony & Popken, Douglas A. (2008). Overcoming Confirmation Bias in Causal Attribution: A Case Study of Antibiotic Resistance Risks. Risk Analysis, 28(5), 11551172. doi:10.1111/j.1539-6924.2008.01122.x.Google Scholar
Dominici, Francesca, Greenstone, Michael & Sunstein, Cass (2014). Science and Regulation. Particulate Matter Matters. Science. April 18, 2014.Google Scholar
Dudley, Susan E. & Gray, George M. (2012). Improving the Use of Science to Inform Environmental Regulation. In Johnston, Jason (Ed.), Institutions and Incentives in Regulatory Science. Lanham, MD: Lexington Books.Google Scholar
Environmental Protection Agency (2000). Guidelines for Preparing Economic Impact Analysis. 240-R-00-003. https://nepis.epa.gov/Exe/tiff2png.cgi/P1004DN9.PNG?-r+75+-g+7+D%3A%5CZYFILES%5CINDEX%20DATA%5C00THRU05%5CTIFF%5C00001350%5CP1004DN9.TIF.Google Scholar
Fraas, Art & Lutter, Randall (2012). Uncertain Benefits Estimates for Reductions in Fine Particle Concentrations. Risk Analysis, 33(3), 434449. Accessed January 16, 2017. doi:10.1111/j.1539-6924.2012.01883.x.Google Scholar
Fraas, Art, Lutter, Randall, Dudley, Susan E., Gayer, Ted, Graham, John, Shogren, Jason F. & Viscusi, W. Kip (2016). Social Cost of Carbon: Domestic Duty. Science. February 5, 2016. Accessed January 16, 2017. http://science.sciencemag.org/content/351/6273/569.2.Google Scholar
Gayer, Ted & Viscusi, W. Kip (2013). Overriding Consumer Preferences with Energy Regulations. J. Regul. Econ., 43, 248264. Accessed January 16, 2017. doi:10.1007/s11149-013-9210-2.Google Scholar
Graham, John D. (2008). Saving Lives through Administrative Law and Economics. University of Pennsylvania Law Review, 157(2), 395540.Google Scholar
Gray, G. M. & Cohen, Joshua T. (2012). Rethink Chemical Risk Assessments. Nature, 489, 2728.Google Scholar
Greenstone, Michael (2009). Toward a Culture of Persistent Regulatory Experimentation and Evaluation. In Moss, David & Cisternino, John (Eds.), New Perspectives on Regulation (pp. 111125). Cambridge: The Tobin Project.Google Scholar
Hamilton, James T. & Viscusi, W. Kip (1999). Calculating Risks: The Spatial and Political Dimensions of Hazardous Waste Policy. Cambridge: The MIT Press.Google Scholar
Harrington, Winston, Heinzerling, Lisa & Morgenstern, Richard D.(Eds.) (2009). In Reforming Regulatory Impact Analysis. Washington, DC: Resources for the Future.Google Scholar
Kirkpatrick, Colin & Parker, David(Eds.) (2007). In Regulatory Impact Assessment: Towards Better Regulation? London, UK: Edward Elgar Publishing.Google Scholar
Krutilla, Kerry, Good, David H. & Graham, John D. (2015). Uncertainty in the Cost-Effectiveness of Federal Air Quality Regulations. Journal of Benefit-Cost Analysis, 6(1), 66111. Accessed January 16, 2017. doi:10.1017/bca.2015.7.Google Scholar
Lackey, Robert T. Normative Science. Terra Magazine, 8(2), 36. Accessed January 16, 2017. http://terra.oregonstate.edu/2013/01/normative-science/.Google Scholar
Lutter, Randall (1999). The Role of Economic Analysis in Regulatory Reform. Regulation, 22(2), 3846. Accessed January 16, 2017. https://www.cato.org/pubs/regulation/regv22n2/econanalysis.pdf.Google Scholar
Lutter, Randall & Zorn, David (2016). Reinforcing Reproducibility: What Role for the Federal Government? Regulation, Winter 2015–16, 1516. Accessed January 16, 2017. https://object.cato.org/sites/cato.org/files/serials/files/regulation/2015/12/regulation-v38n4-8_4.pdf#page=10.Google Scholar
Mannix, Brian (2014). Employment and Human Welfare: Why Does Benefit-Cost Analysis Seem Blind to Job Impacts? In Coglianese, Cary, Finkel, Adam M. & Carrigan, Christopher (Eds.), Does Regulation Kill Jobs? (pp. 190206). Philadelphia: University of Pennsylvania Press.Google Scholar
Mannix, Brian & Dudley, Susan E. (2015). The Limits of Irrationality as a Rationale for Regulation. Journal of Policy Analysis and Management, 34(3), 705712. doi:10.1002/pam.21841. Accessed January 16, 2017. http://onlinelibrary.wiley.com/doi/10.1002/pam.21841/full.Google Scholar
Newcomer, Kathryn E., Hatry, Harry P. & Wholey, Joseph S. (2015). In Handbook of Practical Program Evaluation. (4th ed.). San Francisco: Jossey-Bass.Google Scholar
Nichols, A. L. & Zeckhauser, Richard J. (1988). The Perils of Prudence: How Conservative Risk Assessments Distort Regulation. Regulatory Toxicology and Pharmacology, 8(1), 6175. Accessed January 16, 2017. doi:10.1016/0273-2300(88)90007-4.Google Scholar
Obama, Barack(2011). Executive Order 13563, Improving Regulation and Regulatory Review. Accessed January 16, 2017. https://www.whitehouse.gov/omb/inforeg_regmatters#eo13563.Google Scholar
Office of Information and Regulatory Affairs and Office of Science & Technology Policy (OIRA & OSTP) (2007). M-07-24, Updated Principles for Risk Analysis. Susan E. Dudley and Sharon Hays. Last modified September 19, 2007. https://www.whitehouse.gov/sites/default/files/omb/assets/regulatory_matters_pdf/m07-24.pdf.Google Scholar
Office of Management and Budget (OMB) (2002). Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies. Last modified February 22, 2002. https://www.whitehouse.gov/sites/default/files/omb/assets/omb/fedreg/reproducible2.pdf.Google Scholar
Office of Management and Budget (OMB) (2003). Circular A-4, Regulatory Analysis. Last modified September 17, 2003. https://www.whitehouse.gov/sites/default/files/omb/assets/omb/circulars/a004/a-4.pdf.Google Scholar
Office of Management and Budget (OMB) (2010). Agency Checklist: Regulatory Impact Analysis. Last modified October 28, 2010. https://www.whitehouse.gov/sites/default/files/omb/inforeg/regpol/RIA_Checklist.pdf.Google Scholar
Office of Management and Budget (OMB) (2011a). Circular A-4, Regulatory Impact Analysis: Frequently Asked Questions (FAQs). Last modified February 7, 2011. https://www.whitehouse.gov/sites/default/files/omb/assets/OMB/circulars/a004/a-4_FAQ.pdf.Google Scholar
Office of Management and Budget (OMB) (2011b). Circular A-4, Regulatory Impact Analysis: A Primer. Last modified August 15, 2011. https://www.whitehouse.gov/sites/default/files/omb/inforeg/regpol/circular-a-4_regulatory-impact-analysis-a-primer.pdf.Google Scholar
Office of Management and Budget (OMB) (2011c). M-11-10, Executive Order 13563, Improving Regulation and Regulatory Review. Last modified February 2, 2011. https://www.whitehouse.gov/sites/default/files/omb/memoranda/2011/m11-10.pdf.Google Scholar
Office of Management and Budget (OMB) (2017). Guidance Implementing Executive Order 13771, Titled “Reducing Regulation and Controlling Regulatory Costs.” Last modified April 5, 2017. https://www.whitehouse.gov/sites/whitehouse.gov/files/omb/memoranda/2017/M-17-21.pdf.Google Scholar
Organisation for Economic Co-operation and Development (OECD) (2002). Regulatory Policies in OECD Countries: From Interventionism to Regulatory Governance. Paris: OECD Publishing; http://dx.doi.org/10.1787/9789264177437-en.Google Scholar
Organisation for Economic Co-operation and Development (OECD) (2008). Introductory Handbook for Undertaking Regulatory Impact Analysis (RIA). Paris: OECD Publishing; https://www.oecd.org/gov/regulatory-policy/44789472.pdf.Google Scholar
Organisation for Economic Co-operation and Development (OECD) (2015). OECD Regulatory Policy Outlook 2015. Paris: OECD Publishing; http://dx.doi.org/10.1787/9789264238770-en.Google Scholar
Organisation for Economic Co-operation and Development (OECD) (2016). Regulatory Impact Analysis. Accessed January 16, 2017. http://www.oecd.org/gov/regulatory-policy/ria.htm.Google Scholar
Sutherland, W. J., Spiegelhalter, David & Burgman, Mark A. (2013). Twenty Tips for Interpreting Scientific Claims. Nature. November 20, 2013. Accessed January 16, 2017. http://www.nature.com/news/policy-twenty-tips-for-interpreting-scientific-claims-1.14183.Google Scholar
Thaler, Richard H. & Sunstein, Cass R. (2008). Nudge: Improving Decisions about Health, Wealth, and Happiness. New Haven: Yale University Press.Google Scholar
Trump, Donald J.(2017). Executive Order 13771, Reducing Regulation and Controlling Regulatory Costs. Accessed March 24, 2017. https://www.federalregister.gov/documents/2017/02/03/2017-02451/reducing-regulation-and-controlling-regulatory-costs.Google Scholar
Viscusi, W. Kip, Vernon, John M. & Harrington, Joseph E. Jr. (2005). Economics of Regulation and Antitrust. (4th ed.). Cambridge: The MIT Press.Google Scholar
Zerbe, Richard O., Davis, Tyler, Garland, Nancy & Scott, Tyler (2010). Towards Principles and Standards in the Use of Benefit-Cost Analysis. In MacArthur Foundation Power of Measuring Social Benefits Initiative. Seattle: Benefit-Cost Analysis Center, University of Washington. Accessed January 16, 2017. https://evans.uw.edu/sites/default/files/public/Final-Principles-and%20Standards-Report.pdf.Google Scholar