Electricity restructuring has been a far more convoluted process than is generally acknowledged. Electricity restructuring is the proper term, because even in regions where both wholesale and retail markets were “deregulated,” there is plenty of regulation, market rules to prevent the exercise of market power and delineate participation in electricity markets, reliability rules to ensure the security of the grid, and rules governing the behavior of retail electric providers. There are other regulations in place to encourage energy efficiency and expansion of the use of renewable energy, environmental regulations on power plants and the siting of transmission lines, etc. And in most areas, “deregulation” has been only partially accomplished, with a mixture of integrated utilities, independent power producers, and publicly owned utilities.
Regulated electricity companies were not the abysmal performers that many of the proponents of deregulation claimed. The two biggest factors in higher electricity prices were poor performance of nuclear power and overpriced QF contracts. Nuclear power was an example of management hubris. While some utilities were prudent in insisting on turnkey or fixed-cost contracts, and built the capabilities to manage the new technology, too many utilities bought the hype of the nuclear power vendors. However, we have seen similar foolishness in deregulated markets, such as the behavior that lead to the fiscal crisis of 2007.1 The spasm of overinvestment that accompanied deregulation in many markets was also a waste of resources that had long-term consequences, driving up the cost of financing merchant power after the boom. QF contracts were independent of market structure as a government-mandated subsidy, and had little correlation with utility management competence.
This is not to deny that there were real efficiency gains from restructuring, rather that these improvements in efficiency were incremental, and some of these improvements in resource allocation were achieved through pooling and more stringent regulation. Gains from trade could have been achieved with existing market structures by providing vertically integrated utilities with incentives to make efficiency-enhancing trades. Investment in transmission across traditional boundaries improved allocative efficiency. Additional gains were achieved by establishment of regional electricity markets, postage stamp transmission pricing, and SCED. LMP pricing provided better signals to the market for locating generation (and to regulators concerning generation expansion) and provided efficiencies from better congestion management.
If there is one lesson to be learned from the last few decades of electricity restructuring, it is that with complex interlinked systems like the electric grid, incremental change is more effective than radical restructuring. Incremental change has received a bad name because of the illusion in economics and management decision science that large complex problems can be modeled and solved. This is impossible to do in the real world. Muddling through is a way of implementing a global plan (say, a new electricity market structure), by accepting that such plans are inherently flawed. The optimal strategy may be to implement incrementally a global plan while retaining the option to change course as new information becomes available. The rush to build and implement a market in California not only illustrates the pitfalls of a comprehensive approach to change but also, through its exposure of institutional weakness at the FERC, shows how secondary actors can be overwhelmed by “big steps.”
The gradual evolution from reliability regions to strong power pools to ISO's allowed the building of effective institutions, by permitting staff to develop expertise and providing staff with time to learn how to integrate numerous control areas under their authority. The least effective ISOs were the ones such as MISO, which were hastily cobbled together and governed numerous entities that had limited experience interacting with peers. It also helps to have excess capacity during the change in regimes as a buffer against policy errors. Texas benefited both from a deliberate effort to expand transmission in anticipation of deregulation and an investment boom that bought years to develop a resource adequacy mechanism. The Northeast ISOs made their share of mistakes but had more robust structures that allowed them to learn and evolve without serious mishaps.
The second lesson is “don't drink the Kool-Aid.” The claims of dramatic cost savings and improvements from deregulation proved to be so much hype and hot air. Building effective markets has turned out to be a trial-and-error process all over the world, and most of the successful markets have evolved over time. Efficiency gains from competition are driven by long-term dynamics, but transition costs and transaction costs (including the cost of building the institutions to operate electricity markets) are incurred early in the process.
The third lesson is that most of the potential gains from restructuring are created by the development of competitive wholesale markets combined with stronger regulation of transmission. These go hand in hand, because wholesale competition requires opening transmission to third-party transactions. The ideal solution is separation of generation and wires; the second-best solution is transferring control of the wires to an independent transmission operator such as a RTO or Transco; the third-best solution is heavy-handed regulation to enforce functional separation. But electricity markets are vulnerable to the exercise of market power, and there is no efficient market solution to the “missing money” problem. Rational regulation is required to preserve these efficiency gains and prevent investment in rent-seeking activities designed to transfer revenues from consumers to generators and power marketers.
Retail competition has been grossly overrated. In the one market where it has been a “success,” Texas, market penetration was forced through regulatory action, and a large proportion of residential and small commercial customers have not taken advantage of lower prices. Nor has there been any evidence of widespread innovation or an increase in price responsiveness. Many of the REPs in Texas buy power at fixed prices from large generators, then turn around and sell it to consumers with a markup, simply replacing the sales force of the formerly integrated utilities. In fact, it is becoming clear that the vision of an electricity market that would be self-regulating, with customers reacting to price as in most markets, is decades away, if it ever emerges. This vision will require technology advancements that sufficiently reduce the cost of automating control of residential electricity consumption (sensors, controls, software) to make it feasible to provide value-added services to residential customers. Lowering costs means moving from hard-wired systems that require truck rolls and “ownership” of customers (through regulatory fiat or long-term contracts) to wireless communication systems employing embedded controls in appliances and HVAC systems.
When it comes to industrial and large commercial customers, competition – and especially regulatory requirements for time of use pricing – has accelerated some technological innovation. However, many customers have contracted their way out of the uncertainty posed by dynamic pricing. As the cost of demand management decreases, and companies become more educated about the benefits of managing energy costs, there will be an increase in both price-responsive load and resources available as demand-response resources. But this has proven to be an incremental process, evolutionary, not revolutionary.
One wonders if electricity deregulation would have triumphed had the claims in the 1990s reflected reality. Maybe it required bogus promises of huge consumer savings to push politicians toward a policy that would provide long-term benefits, though often with short-run costs such as increased price volatility, large transition costs, and the occasional market meltdown. Overcoming lock-in and institutional inertia required bringing political pressure to bear on legislators and regulators, and it is difficult to maintain sustained pressure to force continual incremental movement from the status quo. Legislation tends to come in spasms of activity, followed by political exhaustion. The three major pieces of congressional legislation that resulted in the restructuring of the electricity industry – National Energy Act of 1978, EPAct 1992, and EPAct 2005 – all were omnibus bills that tried to balance and log roll across the energy industries. A similar pattern was seen with the Clean Air Act, with an initial bill in 1970, amendments in 1977 and 1990, and legislative gridlock preventing rationalization or adjustment for more than two decades. In both cases, legislation was honeycombed with inconsistencies caused by the need to compromise to get a bill passed.
Yet in the end, despite the rush to deregulation, the reality has been gradual evolution, led by FERC. Regulatory agencies have to work within the confines of the often ambiguous mandates provided by legislation. FERC was mandated to open up electricity markets to competition without the authority to override state regulators when it came to transmission, generation, and retail sales. I'm not sure FERC has always known where it was going, but if you look at FERC policies over the last three decades, there has been steady progress toward integrated markets employing LMP pricing, multi-settlement markets with a financially binding day-ahead market, scarcity pricing based on reserve demand curves, and capacity markets that provide additional revenues for generators in return for constraints on the exercise of market power. All seven markets have converged to this design. Even though Texas has resisted implementing a formal capacity mechanism, it has resorted to an ad hoc market collusion mechanism with revenue-related price caps to attempt to provide generators with the equivalent of capacity payments.
In the regions where integrated utilities employed their political power to thwart regional integration, FERC has gradually tightened the screws, increasing the regulatory burden and shifting the burden of proof to the integrated utility to show they are not engaged in discriminatory practices. The decision by Entergy to join MISO after a decade of prevarication suggests this strategy is having an effect. The holdouts have been reduced to the southeast utilities and WECC.
Muddling through may not be the ideal solution for social engineers or academic economists, who draw up plans for optimal market and regulatory structures, then criticize politicians and special interests when the real world diverges from their vision. However, regulators and policymakers must deal with numerous vested interests, consumer advocates, environmentalists, politicians, corporate lobbyists, and other parties pursuing their own vision of the ideal solution, engaging in turf wars or protecting their piece of the pie. While slicing the Gordian knot of political inertia is tempting, patiently unraveling its strands is often the better strategy.
