Is there any evidence that the NHS is not operating productively or efficiently?

In 2018, NHS Improvement published findings from a large-scale project assessing variations in a broad range of factors across community health services and mental healthcare providers.² This highlighted substantial differences across services in a number of areas. For example, the average cost of in-patient treatment between the most and least expensive was found to vary by up to 20%. Variations in sickness absence rates were as high as 116%, with staff engagement, retention, direct care time and frequency all varying markedly between organisations. The report concluded that reducing such variation could save £1 billion/year, if best practice was consistently followed.

Thus, NHS Improvement's report highlighted the degree of variation across services, often built to identical specifications, serving similar populations. However, the sources of this variability remain obscure. Without an understanding of the reasons underlying such differences how can we discern between ‘warranted’ and ‘unwarranted’ variation? And unless we are able to accurately define and measure the productivity and efficiency of mental health services, the potential sources of such variation will remain unknown.

What are ‘productivity’ and efficiency?

The terms ‘productivity’ and ‘efficiency’ are often used in debates regarding healthcare service performance, though rarely in their correct technical sense. Indeed, they are often used as shorthand for clinical activity levels. But does ‘looking busy’ equate to high levels of either productivity or efficiency? David Maguire, a senior policy analyst at the King's Fund, recently commented on this issue in a blog perceptively entitled: ‘The NHS needs to be more productive – or is it more efficient?’ (https://www.kingsfund.org.uk/blog/2019/03/nhs-productive-or-efficient). So, what do these terms mean, and how should they be applied to mental health services?

‘Productivity’ measures the quantity of output produced from given inputs, expressed as the ratio of the two. Thus, in healthcare, productivity should relate to the amount of ‘care’ produced, given the resources available (e.g. funds). However, the ultimate aim is not just to provide care, but to improve health. Therefore, particularly in the context of mental health, productivity must also encompass something greater. Patients seek help not to experience the care itself, but to feel better. Moreover, people who require support from mental health services do not all require the same type of care in order to recover, even if they have similar conditions. Therefore, it is inadequate to simply measure the quantity of patients who have had care delivered without considering its quality. The nature of care should be such that it supports recovery, allowing people to live fulfilling and meaningful lives. Thus, productivity measures must encapsulate both the quantity and quality of the care delivered to each patient.

In contrast to productivity, ‘efficiency’ is concerned with whether the process is completed using the optimal combination of inputs to produce the maximum outputs, with the least cost and amount of waste. The difference between productivity and efficiency is illustrated with an example in Fig. 1. The solid curve in Fig. 1 depicts a production function, which captures all combinations of inputs and outputs that are ‘technically efficient’ given the current technological process. Further, the illustrated curve suggests diminishing marginal productivity, i.e. each additional unit of input produces progressively less output. This is the same as saying that the production function exhibits decreasing returns to scale, or the more inputs used, the lower the return in terms of outputs. Assume now that we have two organisations (P₁ and P₂) producing one single output (O) each using a single input (I), given the production function depicted. In terms of the production function, the more inputs used, the lower the return in terms of outputs. Organisation P₁ has a higher level of productivity compared with P₂, as measured by the ratio of outputs to inputs, which for P₁ (O ₁/I ₁) is larger than that for P₂ (O ₂/I ₂). However, P₁ is not technically efficient as it is not operating on the production function. In fact, it would be feasible for P₁ to produce even more output by producing on the production function at point P*₁ in Fig. 1, yielding an output of O* ₁ (>O ₁). The degree of ‘technical inefficiency’ is thus defined by the vertical distance, denoted by the dashed line between P₁ and P*₁. This example shows that it is possible for an organisation to achieve a high level of productivity while operating at a suboptimal level of efficiency.

Fig. 1

The solid curve depicts a ‘technically efficient’ production function. The degree of ‘technical inefficiency’ is thus defined by the vertical distance, denoted by the dashed line between P₁ and P*₁. This illustrates that a high level of productivity can be achieved while operating at a suboptimal level of efficiency.

Towards the measurement of productivity and efficiency in mental health services

One major hurdle in working towards measuring productivity and efficiency in a mental health context is the challenge in identifying the key components of the production process. Inputs are the easiest part of the process to define, consisting of labour (medical, nursing and other healthcare staff), intermediate goods and services (clinical and pharmaceutical supplies, utilities, hotel services, etc.) and capital (non-labour input such as land and building with an asset life greater than 1 year). Ideally, outputs (in terms of care delivered) should capture the whole course of treatment provided to patients across different care settings. In practice this is often impossible to achieve, given the inability of healthcare systems to track patients across primary, secondary and tertiary care settings. Thus, one often measures the number of patients treated (discharged) and/or the amount of care provided (interventions/activities), usually adjusted to allow for patient case-mix (e.g. the nature, severity and complexity of presentation).

Outcomes, which ideally should reflect the effectiveness of the outputs (care) delivered, are the most challenging element of the process to measure. In fact, selecting and measuring relevant outcomes is not straightforward. Mental health providers have a contractual obligation to collect and report routine outcome measures (ROMs), although completion rates are often low in practice. The potential for ROMs to contribute to a mental health service productivity index has been explored.^{Reference Jacobs3} It was found that multiple barriers existed in relation to the use of ROMs. These included a lack of IT support as well as the perceptions among clinicians as to their usefulness (or otherwise). Given these factors, the report concluded, at the time, that it would be ‘premature’ to utilise these measures to evaluate service productivity.

It is also the case that different stakeholders may not agree on which outcomes matter the most. We need a lexicon that defines what outcomes are important from the differing perspectives of commissioners, clinicians and, most crucially, patients and carers. For example, the ‘recovery’ movement emphasises the use of the subjective, individual CHIME factors (connectedness, hope, identity, meaning and empowerment).^{Reference Leamy, Bird, Boutillier, Williams and Slade4} This contrasts with the currently widely adopted symptom-based questionnaires and rating scales. Indeed, it is well recognised that symptom severity does not always correlate well with functioning and is also at odds with the philosophy of the recovery movement. Moreover, it is unknown as to whether the scores from such symptom measures are associated with ‘hard outcomes’, such as the risk of compulsory readmission to hospital.

In a recent King's Fund report, Collins sets out the challenges of conceptualising mental health outcomes, writing that many physical health traumas have a clear path to recovery and a reasonably clear shared sense of priorities. ‘In the care of people with mental health conditions, and perhaps people with other chronic conditions, there is a much broader range of sometimes contested objectives.’^{Reference Collins5} We need to ensure that these different perspectives are accommodated within the process of evaluating productivity and efficiency. Inevitably this means that our optimising problem becomes multidimensional in nature. It is therefore quite possible that allocating resources in particular ways may improve outcomes in one domain while degrading those in other areas. However, there may also be situations characterised by ‘pareto-dominance’, whereby outcomes can be improved in one area without a negative impact elsewhere.

The way forward

To assess whether the limited funds available are being put to best use, we need a clear framework with which to define and measure productivity and efficiency in the context of mental health services. Moreover, increasingly, workforce availability, as much as monetary resources, are constraining how care can be delivered. There are also important issues related to the optimum (most efficient) stage within the evolution of a mental disorder at which to intervene. Indeed, ‘an ounce of prevention is often better than a pound of cure’, although identifying such ‘sensitive’ or ‘critical periods’ to target for early intervention is not straightforward. We also need an agreed basket of meaningful outcome measures with which to evaluate the effectiveness of care. If the mental health productivity puzzle is to be solved, we need to begin by identifying each piece clearly and put them together, one by one.