Over the last 30 years there has without question been a massive increase in the use of performance information in the health sector. This growth has been driven partly by the increased technological capacity to collect, analyse and disseminate such information, and partly by the demand for increased transparency manifest in all sectors of society. The information revolution has given rise to many opportunities and challenges, and has provided a fertile domain for academic and policy enquiry.
The paper by Wadmann et al. (Reference Wadmann, Johansen, Lind, Okkels Birk and Hoeyer2013) gives an admirably broad view of the academic literature on what they term performance-based management (PBM). They examine not the instruments or outcomes of PBM, but the analytic approach adopted in the studies. Three distinct perspectives are formulated: the functionalist, under which PBM is considered a ‘tool aimed at improving health services by market-based mechanisms’; the interpretive, which treats PBM as a ‘consequence of institutional and individual striving for public legitimacy’; and the postmodern, a rather elusive concept that the authors relate to forms of governance, but which I find most helpfully expressed as ‘what the management systems produce in terms of power effects’ (Wadmann et al., Reference Wadmann, Johansen, Lind, Okkels Birk and Hoeyer2013: 519).
The first issue to address is terminological. It can be argued that the expression PBM is tautological – what should managers do if not manage performance? In any case, the authors define PBM very broadly as a ‘form of governance in which: (1) desired results are specified in advance in measurable form; (2) some system of monitoring measures performance against that specification; and (3) feedback mechanisms are linked to measured performance’. In the context of much of the work included from the health sector, this function is much more than organizational management. My own preference would therefore be to describe the field of study as ‘performance-based governance’, as it refers not only to managerial responses to performance measurement, but also to how those measures are selected, and how institutions, managers and practitioners are held to account for the reported measures. This is not mere semantics – there is a world of difference between internal management of an institution and how that institution is held externally to account. Nevertheless, to maintain consistency with the original, I shall continue to use the expression PBM in this commentary.
The functionalist perspective represents a well-developed field of investigation (Smith et al., Reference Smith, Mossialos, Papanicolas and Leatherman2010). It assumes that the opportunities provided by new information and communication technology should be seized in order to improve the effectiveness of health services. This can be achieved by reducing unwarranted variation in treatment and focusing relentlessly on outcomes. Analytic advances have led to steady improvements in the breadth, sophistication and timeliness of reported performance. There is also increased attention to accountability mechanisms that force institutions and practitioners to reflect on the measures and where necessary take corrective action. The authors focus on market mechanisms, but there is no reason why accountability could not also embrace direct hierarchical incentives from a payer, such as ‘pay-for-performance’, indirect electoral consequences or professional peer review (Smith et al., Reference Smith, Anell, Busse, Crivelli, Healy, Lindahl, Westert and Kene2012). The results from the PBM revolution have not always been as unambiguously positive as its advocates might have hoped, but there is a general assumption that – given further time and refinement – it will yield the expected gains in increased effectiveness.
The interpretive perspective presumes that performance measurement and reporting is a response to the increasing demands for external legitimacy and accountability. From this perspective, there has been a reduction in levels of trust in institutions and professionals that can be addressed only by making explicit the achievements of the institution or individual practitioner. Furthermore, many tools of the ‘new’ public management, such as competitive tendering, may require explicit performance reporting. It is perhaps worth noting that many of these new management approaches have become attractive only when mass electronic data collection became feasible, so the potential for performance reporting may have caused reduced levels of trust and reformed public services – the reverse of that usually assumed.
The postmodern perspective reflects the belief that an ‘audit culture’ has infused public life. Here the interest is in how the performance reporting systems have emerged, whose interests they serve, and their impact on power relations. PBM then gives rise to allegedly ‘new’ problems, such as lack of competition. A central insight from this perspective is that the emphasis on PBM may fundamentally alter professional culture, substituting audit and control for traditional professional characteristics of autonomy, patient focus and trust.
Where does this typology leave us? PBM has without question nourished a rich and diverse literature that offers many insights into the behaviour of systems, institutions, professions and individuals. Perhaps the biggest contribution of the Wadmann paper is to highlight the various possible motivations for introducing PBM, either tacit or explicit, and their attendant consequences. Given the wide range of plausible hypotheses, it is possible to offer a persuasive argument for adopting any or all of the perspectives when studying a specific innovation.
Regardless of the perspective adopted, it has become clear that the introduction of PBM is not a straightforward undertaking, and that it has given rise to many unanticipated consequences and conundrums. To give just a few:
∙ public reporting of physician performance can give rise to worse outcomes for patients and payers if risk adjustment mechanisms are inadequate (Dranove et al., Reference Dranove, Kessler, McClellan and Satterthwaite2003);
∙ providers and practitioners may distort reported performance if their perceived levels of attainment and associated remuneration might be affected (Gravelle et al., Reference Gravelle, Sutton and Ma2010);
∙ many widely accepted measures of performance, such as emergency hospital readmission rates, may be wholly inadequate as indicators of organizational performance (Laudicella et al., Reference Laudicella, Li Donni and Smith2013);
∙ patients and the general public seem to be highly resistant to seeking out and using performance data (Faber et al., Reference Faber, Bosch, Wollersheim, Leatherman and Grol2009).
In short, many of the ‘self-evident’ benefits of PBM have failed to materialize, or have produced troublesome unintended side-effects.
From a functional perspective, the early experience with PBM in health has revealed an enormous potential research programme. For example, it has underlined that many traditional measures of clinical outcome (such as mortality) are completely unfit for purpose and stimulated the search for alternatives, such as patient-reported outcome measures. It has highlighted the importance of statistical risk adjustment, to account for differences in patient characteristics, not just as a technical improvement, but as a prerequisite for securing clinical acceptance of PBM. It has demonstrated the importance of whether and how the performance information is disseminated to the various stakeholder groups. Moreover, it has exposed how complex it is going to be to design performance incentives schemes that have the intended effects on providers.
The interpretive perspective underlines the need to focus on accountability relationships and how they are expected to operate. It is impossible to envisage a return to an era when professionals were ‘trusted’ to behave in the interests of patients, and allowed to practice with little or no requirement to account for their actions and outcomes. No service that is in receipt of public finance can expect to enjoy continued freedom from some sort of accountability to its payers and its users. However, it is also clear that PBM as implemented hitherto may not fulfil its intended accountability role. In particular, a crude reliance on transparency does not necessarily lead to better use of public funds. For example, publication of imperfect performance measures may lead to physicians refusing to treat sicker patients who would on average benefit from treatment, but for whom there is a higher risk of adverse outcome.
The postmodern perspective examines the potential of PBM to rebalance power relations in the health sector. Indeed adherents of this approach might argue that our way of thinking about the health system itself is reshaped because of the adoption of PBM norms. Debates undertaken from this perspective are sometimes difficult for outsiders to follow, and their implications for policy not always immediately apparent. However, it is without question the case that health services remain predominantly the outcome of an exchange between managers, clinicians and patients. How these relations are affected by PBM thinking are of profound importance for all of those parties, as well as payers. This will be especially true if it leads to fruitless efforts, for example, to measure the unmeasurable, to write contracts for unknowable contingencies, or to shift risk onto parties who do not have the capacity to manage that risk.
In spite of the broad spectrum of research reported by Wadmann et al., there remains a vast terrain of enquiry yet to be addressed. In my view, the key focus of research attention should be on the many accountability relationships within the health system, and on how PBM may affect those relationships. These could include (but are not limited to) relationships between: physicians and patients; governments and provider organizations; citizens and health insurers; and health practitioners and their professions. A functionalist viewpoint may then be interested in the consequent outcomes for patients and payers. The interpretive perspective may be more interested in whether PBM addresses the accountability concerns of legitimate stakeholders in the health system. The postmodernist may focus on how the PBM focus changes the perceptions and preoccupations of all the relevant parties, and in particular on the challenge to traditional views about the nature of the health professions.
Irrespective of the perspective adopted, there will be a continued – and sometimes very rapid – increase in the availability and use of performance information in all health systems, whether designed or accidental. Its impact on the various parties discussed above will often be profound, and the outcomes not straightforward to predict. The task for policymakers, regulators, managers, professionals, patients and citizens is to grasp the opportunities provided, while seeking to counteract any adverse consequences. Only with a secure and wide-ranging evidence-based understanding of PBM will it be possible to do so.
Over the last 30 years there has without question been a massive increase in the use of performance information in the health sector. This growth has been driven partly by the increased technological capacity to collect, analyse and disseminate such information, and partly by the demand for increased transparency manifest in all sectors of society. The information revolution has given rise to many opportunities and challenges, and has provided a fertile domain for academic and policy enquiry.
The paper by Wadmann et al. (Reference Wadmann, Johansen, Lind, Okkels Birk and Hoeyer2013) gives an admirably broad view of the academic literature on what they term performance-based management (PBM). They examine not the instruments or outcomes of PBM, but the analytic approach adopted in the studies. Three distinct perspectives are formulated: the functionalist, under which PBM is considered a ‘tool aimed at improving health services by market-based mechanisms’; the interpretive, which treats PBM as a ‘consequence of institutional and individual striving for public legitimacy’; and the postmodern, a rather elusive concept that the authors relate to forms of governance, but which I find most helpfully expressed as ‘what the management systems produce in terms of power effects’ (Wadmann et al., Reference Wadmann, Johansen, Lind, Okkels Birk and Hoeyer2013: 519).
The first issue to address is terminological. It can be argued that the expression PBM is tautological – what should managers do if not manage performance? In any case, the authors define PBM very broadly as a ‘form of governance in which: (1) desired results are specified in advance in measurable form; (2) some system of monitoring measures performance against that specification; and (3) feedback mechanisms are linked to measured performance’. In the context of much of the work included from the health sector, this function is much more than organizational management. My own preference would therefore be to describe the field of study as ‘performance-based governance’, as it refers not only to managerial responses to performance measurement, but also to how those measures are selected, and how institutions, managers and practitioners are held to account for the reported measures. This is not mere semantics – there is a world of difference between internal management of an institution and how that institution is held externally to account. Nevertheless, to maintain consistency with the original, I shall continue to use the expression PBM in this commentary.
The functionalist perspective represents a well-developed field of investigation (Smith et al., Reference Smith, Mossialos, Papanicolas and Leatherman2010). It assumes that the opportunities provided by new information and communication technology should be seized in order to improve the effectiveness of health services. This can be achieved by reducing unwarranted variation in treatment and focusing relentlessly on outcomes. Analytic advances have led to steady improvements in the breadth, sophistication and timeliness of reported performance. There is also increased attention to accountability mechanisms that force institutions and practitioners to reflect on the measures and where necessary take corrective action. The authors focus on market mechanisms, but there is no reason why accountability could not also embrace direct hierarchical incentives from a payer, such as ‘pay-for-performance’, indirect electoral consequences or professional peer review (Smith et al., Reference Smith, Anell, Busse, Crivelli, Healy, Lindahl, Westert and Kene2012). The results from the PBM revolution have not always been as unambiguously positive as its advocates might have hoped, but there is a general assumption that – given further time and refinement – it will yield the expected gains in increased effectiveness.
The interpretive perspective presumes that performance measurement and reporting is a response to the increasing demands for external legitimacy and accountability. From this perspective, there has been a reduction in levels of trust in institutions and professionals that can be addressed only by making explicit the achievements of the institution or individual practitioner. Furthermore, many tools of the ‘new’ public management, such as competitive tendering, may require explicit performance reporting. It is perhaps worth noting that many of these new management approaches have become attractive only when mass electronic data collection became feasible, so the potential for performance reporting may have caused reduced levels of trust and reformed public services – the reverse of that usually assumed.
The postmodern perspective reflects the belief that an ‘audit culture’ has infused public life. Here the interest is in how the performance reporting systems have emerged, whose interests they serve, and their impact on power relations. PBM then gives rise to allegedly ‘new’ problems, such as lack of competition. A central insight from this perspective is that the emphasis on PBM may fundamentally alter professional culture, substituting audit and control for traditional professional characteristics of autonomy, patient focus and trust.
Where does this typology leave us? PBM has without question nourished a rich and diverse literature that offers many insights into the behaviour of systems, institutions, professions and individuals. Perhaps the biggest contribution of the Wadmann paper is to highlight the various possible motivations for introducing PBM, either tacit or explicit, and their attendant consequences. Given the wide range of plausible hypotheses, it is possible to offer a persuasive argument for adopting any or all of the perspectives when studying a specific innovation.
Regardless of the perspective adopted, it has become clear that the introduction of PBM is not a straightforward undertaking, and that it has given rise to many unanticipated consequences and conundrums. To give just a few:
∙ public reporting of physician performance can give rise to worse outcomes for patients and payers if risk adjustment mechanisms are inadequate (Dranove et al., Reference Dranove, Kessler, McClellan and Satterthwaite2003);
∙ providers and practitioners may distort reported performance if their perceived levels of attainment and associated remuneration might be affected (Gravelle et al., Reference Gravelle, Sutton and Ma2010);
∙ many widely accepted measures of performance, such as emergency hospital readmission rates, may be wholly inadequate as indicators of organizational performance (Laudicella et al., Reference Laudicella, Li Donni and Smith2013);
∙ patients and the general public seem to be highly resistant to seeking out and using performance data (Faber et al., Reference Faber, Bosch, Wollersheim, Leatherman and Grol2009).
In short, many of the ‘self-evident’ benefits of PBM have failed to materialize, or have produced troublesome unintended side-effects.
From a functional perspective, the early experience with PBM in health has revealed an enormous potential research programme. For example, it has underlined that many traditional measures of clinical outcome (such as mortality) are completely unfit for purpose and stimulated the search for alternatives, such as patient-reported outcome measures. It has highlighted the importance of statistical risk adjustment, to account for differences in patient characteristics, not just as a technical improvement, but as a prerequisite for securing clinical acceptance of PBM. It has demonstrated the importance of whether and how the performance information is disseminated to the various stakeholder groups. Moreover, it has exposed how complex it is going to be to design performance incentives schemes that have the intended effects on providers.
The interpretive perspective underlines the need to focus on accountability relationships and how they are expected to operate. It is impossible to envisage a return to an era when professionals were ‘trusted’ to behave in the interests of patients, and allowed to practice with little or no requirement to account for their actions and outcomes. No service that is in receipt of public finance can expect to enjoy continued freedom from some sort of accountability to its payers and its users. However, it is also clear that PBM as implemented hitherto may not fulfil its intended accountability role. In particular, a crude reliance on transparency does not necessarily lead to better use of public funds. For example, publication of imperfect performance measures may lead to physicians refusing to treat sicker patients who would on average benefit from treatment, but for whom there is a higher risk of adverse outcome.
The postmodern perspective examines the potential of PBM to rebalance power relations in the health sector. Indeed adherents of this approach might argue that our way of thinking about the health system itself is reshaped because of the adoption of PBM norms. Debates undertaken from this perspective are sometimes difficult for outsiders to follow, and their implications for policy not always immediately apparent. However, it is without question the case that health services remain predominantly the outcome of an exchange between managers, clinicians and patients. How these relations are affected by PBM thinking are of profound importance for all of those parties, as well as payers. This will be especially true if it leads to fruitless efforts, for example, to measure the unmeasurable, to write contracts for unknowable contingencies, or to shift risk onto parties who do not have the capacity to manage that risk.
In spite of the broad spectrum of research reported by Wadmann et al., there remains a vast terrain of enquiry yet to be addressed. In my view, the key focus of research attention should be on the many accountability relationships within the health system, and on how PBM may affect those relationships. These could include (but are not limited to) relationships between: physicians and patients; governments and provider organizations; citizens and health insurers; and health practitioners and their professions. A functionalist viewpoint may then be interested in the consequent outcomes for patients and payers. The interpretive perspective may be more interested in whether PBM addresses the accountability concerns of legitimate stakeholders in the health system. The postmodernist may focus on how the PBM focus changes the perceptions and preoccupations of all the relevant parties, and in particular on the challenge to traditional views about the nature of the health professions.
Irrespective of the perspective adopted, there will be a continued – and sometimes very rapid – increase in the availability and use of performance information in all health systems, whether designed or accidental. Its impact on the various parties discussed above will often be profound, and the outcomes not straightforward to predict. The task for policymakers, regulators, managers, professionals, patients and citizens is to grasp the opportunities provided, while seeking to counteract any adverse consequences. Only with a secure and wide-ranging evidence-based understanding of PBM will it be possible to do so.