OUR NEW INSPECTORATE

If you are working in mental health, you get used to being inspected and commented upon — the Mental Health Act Commission, Audit Commission, Social Services Inspectorate, Health Advisory Service (HAS, as was) and public inquiries into patient homicides. Working in multidisciplinary teams has made us used to operating with a variety of perspectives and for most of us this is a necessary and welcome part of the job. Few psychiatrists, however, are so sanguine about the former HAS or about homicide inquiries. The repeated complaint has been their inconsistency. Their quality and tone (potentially as damaging as their findings) have varied to quite an indefensible degree.

The Commission for Health Improvement (CHI) is clearly a major undertaking. Established just over a year ago, it has over 200 full-time staff and plans to conduct 500 reviews within the next 3 years (CHI, 2001a ). It expects to visit all trusts every 4 years and ‘red-light’ trusts every 2 years. For mental health trusts the reviews occupy 24 weeks — 15 weeks reviewing paperwork and consulting with relevant partners, a 1-week site visit and 8 weeks writing up the report. Each panel comprises six or seven members plus administrative support. Not only is the investment substantial but the impact is tangible — as evidenced by the sudden departure of a Chief Executive a week before a scathing report on his trust (CHI, 2001b ). Will this new inspectorate establish a reputation that commands respect and authority within the professions to match its undoubted power?

The CHI mental health reports do not attempt to monitor National Service Framework achievements in any detail but focus on two areas. The first is the local processes for clinical governance — what structures exist for the trust management to monitor clinical activity and influence it to follow best practice? The second (a braver, but more challenging goal) is to get a sense of service user and carer experience in the trust. They have chosen also to focus exclusively on the services for working-age adults.

THE NORTH BIRMINGHAM REPORT

July 2001 saw the first two CHI reports into single-speciality mental health trusts — North Birmingham (CHI, 2001c ) and Wrightingon, Wigan and Leigh (CHI, 2001d ). North Birmingham Mental Health Trust has national prominence because the Department of Health has vigorously promoted its configuration and, not surprisingly, it has attracted media attention:

These media reports focus on the level of aggression and violence on the in-patient units and a total absence of risk assessment or clinical risk strategy — a grim set of headlines, accompanied in the Health Service Journal by the Chief Executive's predictable expression of gratitude and plans for improvement.

MEDIA COVERAGE

Is it really that bad? Did North Birmingham get fair treatment? Reading the two mental health reports together probably does show North Birmingham in a poorer light than Wrightingon, Wigan and Leigh. The level of violence and having no risk management strategy are serious problems that need to be addressed. Overall, however, Wrightingon, Wigan and Leigh have four areas for ‘urgent action’ and three for ‘action’; North Birmingham has three areas for ‘urgent action’ and four for ‘action’. For both trusts, two of these ‘urgent action’ areas are ‘User and carer involvement’ and ‘Staffing and staff management’ (not surprising in Wrightingon, Wigan and Leigh, with only two substantive appointments to the 18 established medical posts). Was the Wrightingon, Wigan and Leigh Mental Health Trust judged by the same standards as the North Birmingham Health Trust? Reading these two reports it is simply impossible to know, and it is important to know the answer.

Does the CHI have any responsibility for how its reviews are reported? Speculation in anticipation of the North Birmingham report ranged from a conviction that it would be ‘softened’ by the Department of Health because of its policy importance, through to a belief that as self-confident high-flyers they would be judged more harshly. Responses to the report also varied, ranging from “there but for the grace of god…” to shameless schadenfreude. It would be disingenuous to suggest that such high-profile reports have no responsibility for how they are presented and reported; we have lived through 10 years of watching colleagues' careers being destroyed by the reporting of homicide inquiries. The CHI needs to think very hard about how its reports are presented.

THE CHALLENGES FOR THE CHI

The CHI has certainly done well in its first year to get this far (Reference Day and KleinDay & Klein, 2001). Its current major challenge is to clarify its role. Is it to be “the quality police or a midwife of change?” (Reference Day and KleinDay & Klein, 2001). In mental health trusts it has focused on clinical governance. Mental health trusts are, however, complex systems and clinical governance strategies are agreed at board level. These strategies and policies need to be understood within the broader context of the service. For instance, a risk management strategy that effectively covers the interface needs of general adult, child and adolescent psychiatry plus extensive forensic services may look odd when viewed entirely from how it applies to community mental health teams. The same is true for staffing and training policies. Some systems-level analysis is essential.

Language and style are clearly crucial (Reference Day and KleinDay & Klein, 2001). The current reports aim to be of value beyond the trust reviewed. To do so, they need to be shorter and more transparent in their structure. Both of these reports were over 40 pages long, plus appendices. This level of detail, peppered with ‘service user comments’ and ‘examples’, is of little interest to those outside the trust and makes them more, not less, difficult to compare and contrast.

I remain profoundly sceptical that having a shifting panel of reviewers will give any of them enough experience to speak with real authority. Panel members need extensive experience of reviewing services in order to exercise convincing judgement. It is this experience as a reviewer that matters — not how experienced they are as a psychiatrist or a nurse or a user. It is remarkable that in neither homicide inquiries nor CHI reports (just as in HAS visits before) do you have to serve an apprenticeship before you contribute. How many of us would be happy to be operated on by a surgeon who had simply been on an induction course but never assisted in theatre before being put in charge?

Nowhere do these concerns matter more in giving “a fair picture of the trust” (CHI, 2001c ) than in the interviews with users and carers. This is a fraught area, as any social scientist would attest. How do you choose who to talk to, or who chooses you? Are quoted opinions representative or illustrative? Given that virtually every satisfaction survey (irrespective of the quality of the service) returns more negative views of in-patient than out-patient care, how do you make allowance for this in your judgements, or do you simply report it? How do you select your quotes? What, for instance, does the CHI panel consider is the significance of the nationally reported quote:

A LESSON FROM THE JOURNAL

At a time when we are being exhorted to use evidence-based practice, would it be out of place to propose a more rigorous use of scientific methods by those investigating us? If asked to review the North Birmingham report as a paper for this journal, I would have to recommend its rejection. I would point out that the conclusions proposed go far beyond the data presented and there are several conditions that would have to be met before any meaningful conclusions can be drawn.

It needs to state explicitly at the outset what it intended to examine, why its questions are important and how it would obtain the required information. More importantly, it must explain how different outcomes would be ranked or evaluated. Sampling procedures need to be explicit and genuine efforts made to reduce bias. This does not have to be random sampling — there are well-developed techniques of purposive sampling in qualitative research.

Making the questions explicit and publicising the measures that will be used can only be positive. If they really are clinically important questions and if the measures are appropriate, then it is all to the good if trusts alter practice to meet known standards. That really would be a “midwife of change”. The standards measured can be increased over time but remain within honest striking distance. If questions that are relevant to most trusts cannot be stated simply, then we must question the value of an inspectorate. If the same questions are not asked and their answers are not reported in a consistent manner, then we seriously mislead ourselves in comparing across reports and trusts. To achieve this, means that the inspectorate must exercise much more discipline in setting the questions and reporting briefly. As with a scientific paper (in words reiterated by generations of editors), the report should be “as long as it needs to be and as short as it can be”.

A PROPOSAL

It is all too easy to criticise. The CHI has clearly worked very hard with these reports and their ambitions are laudable (and they will be inspecting me soon!). It is, however, unrealistic to expect a system to generate sufficient authority from a shifting army of reviewers, no matter how able each is in their own right. It is the process itself that must command respect. The CHI's approach needs to be leaner and simpler, with greater transparency about what is to be asked and how it is to be assessed. Twenty-four weeks of gestation and 50-page reports are not the way forward. I would propose that nobody should put pen to paper in a CHI report (or a homicide inquiry for that matter) until they have sat through a minimum of three full reviews as a ‘trainee’. That way, we will get more experienced and balanced opinions — and undoubtedly pressure for a simpler, shorter process.