Performance concerns in psychiatrists referred to the National Clinical Assessment Service

Aims and method Relative to workforce numbers, the National Clinical Assessment Service (NCAS) has reported high rates of referrals for performance concerns in psychiatrists. Our aim was to see how these concerns would be distributed across good medical practice (GMP) domains. Such specification would help identify appropriate training and development interventions. Concerns were obtained from consecutive case reports (anonymised) of psychiatrists referred to NCAS between October 2004 and April 2006.

Results The proportion of psychiatrists with concerns in any domain was highest (just over 50%) in the domains of good clinical care and working with colleagues. Two-thirds had concerns across multiple domains.

Clinical implications Training and development needs of psychiatrists referred to NCAS are more complex and more prevalent in the GMP domain of good clinical care than previously reported.


Cohort selection and case management
The NCAS database was used to recruit a consecutive series of referred psychiatrists for the period 1 October 2004 to 31 March 2006. On referral, NCAS advice reception staff record preliminary details for the case in the database (including a case number, specialty and nature of performance concerns) and forward them to the NCAS case advisor. The advisor obtains more details on the concerns from the referrer, records the details using a standardised template -the case report -and forwards the advice to the referrer using another template, the advice summary.

Practitioners' baseline categories and characteristics
Practitioners' categories and characteristics at baseline were obtained from the NCAS database and the anonymised case files, and included the following information: . sociodemographic: age, gender and ethnicity; . professional: place of qualification, grade, specialty; . employer: contract type, trust type, country; . referral source: employer or self.

Identification and typing of concerns
Source and duration of concerns Information on the source and duration of the reported concerns was obtained from the case reports. Sources were Aims and method Relative to workforce numbers, the National Clinical Assessment Service (NCAS) has reported high rates of referrals for performance concerns in psychiatrists. Our aim was to see how these concerns would be distributed across good medical practice (GMP) domains. Such specification would help identify appropriate training and development interventions. Concerns were obtained from consecutive case reports (anonymised) of psychiatrists referred to NCAS between October 2004 and April 2006.

Results
The proportion of psychiatrists with concerns in any domain was highest (just over 50%) in the domains of good clinical care and working with colleagues. Two-thirds had concerns across multiple domains.
Clinical implications Training and development needs of psychiatrists referred to NCAS are more complex and more prevalent in the GMP domain of good clinical care than previously reported.
Declaration of interest None.

Performance concerns in psychiatrists referred to the National Clinical Assessment Service
Denis O'Leary, 1 Pauline McAvoy, 2 John Wilson 1 categorised as staff, patient and 'other'. The last included data from clinical governance and/or performance management systems from within (e.g. suicide review) and external (e.g. coroner's inquest) to the referring trust.
Typing across NCAS domains At the time this study was undertaken NCAS typed performance concerns into the categories of behaviour, clinical capability and health (the number of reporting categories has subsequently been increased to seven). 6 These were obtained from the database for each case. For ease of comparison with previous NCAS reports published at the time this study was reporting, the results are presented as the percentage of practitioners with concerns in any domain and across each domain. 4 Typing across GMP domains The concerns recorded by the NCAS advisor in each case report were listed and typed across the seven GMP performance domains: good clinical care; relationships with patients; working with colleagues; maintaining good medical practice; teaching and training; appraising and assessing; probity; and health. 14 Concerns pertaining to inappropriate behaviour with patients and/or colleagues were typed in the probity domain and also in the other relevant domain: for example, a sexual remark to a colleague was typed in the domains of probity and working with colleagues. Other probity concerns (e.g. falsifying expenses) were recorded in the probity domain only.

Factors associated with performance concerns
To identify factors possibly linked to performance concerns, a chi-squared analysis was undertaken of baseline characteristics (independent variables) across NCAS and GMP performance domains (dependent variables) using SPSS version 10 for Windows. Statistical significance was set at a probability of 5% (P50.05). Fisher's exact test was used when the expected number of cases within any of the two-by-two table cells was less than 5. As the analysis was exploratory, neither Bonferroni's modification nor a multiple regression analysis was undertaken.

Construction of a GMP profile of performance concerns
To allow for greater specification of the performance concerns, a GMP profile of performance concerns is presented. First, a performance framework was constructed that specified performance aspects within each GMP domain: for example, the domain of good clinical care is underpinned by six sub-domains (e.g. care provision, supporting self-care). Within the sub-domain of care provision there are 12 underpinning components. For three of these components (prescribing, record-keeping and availability) the required performance is underpinned by two elements. The naming of the levels within each domain is not specified by the GMP guidelines, but is done here for communication purposes. Second, the profile of performance concerns was constructed by calculating for each level within the framework the percentage of practitioners with concerns in each performance aspect at that level. Aspects were included in the profile only if there were concerns in 10% or more of referred practitioners at that level.

Data release and coding
Data collection and release were approved by NCAS under its Data Collection and Release policy, 17 which aims to balance the confidentiality needs of referred practitioners with the needs of NCAS to account for its casework activity and to inform the development of NHS training and development systems. Within the framework of the National Research Ethics Service the analysis is coded as service evaluation rather than research. 18

Data completeness
A consecutive series of 123 cases was recruited. Complete data were available for all the employer and referral source characteristics and for gender and grade. Data completeness was 77% and 82% on the sample's age bands and ethnicity respectively; for place of qualification and specialty it was 88% and 94% respectively.

Baseline characteristics
Most practitioners (72%) were male, with an even distribution between White and other ethnic groups ( Table 1). The most frequent age band was 50-54 years. Slightly more had obtained their primary medical qualification outside the UK (53% v. 47%) and three-quarters were in career-grade posts. All grades were represented, with consultants being the largest subgroup (57%). Although all specialties were represented, most (71%) were from the adults of working age specialty. Nearly all (98%) worked in the NHS (across 67 NHS trusts). Almost a fifth had fixed term (locum) contracts and two-thirds were employed by mental healthcare trusts.

Source and duration of concerns
Details of source and duration of concerns were available for 118 and 116 cases respectively. There was a single source of concern in 86% of cases as follows: staff 46%, 'other' 31%, patients 9%. Of the 14% cases with multiple sources of concerns, all involved staff. The modal duration was 1-4 weeks. In total, 38 (33%) had concerns of up to a month's duration and 89 (77%) had concerns of up to 1 year in duration.

NCAS domains
Two practitioners working in the independent sector were excluded on confidentiality grounds from the typing analysis. Based on the initial employer report to NCAS, four-fifths (80%) of practitioners had behavioural concerns; half (48%) and a fifth (22%) had concerns in the clinical and health domains respectively. Sixty-seven practitioners (55%) had concerns in a single domain.

GMP domains
Among the 121 NHS-employed practitioners, the NCAS advisors had recorded 395 performance concerns in total. The percentages of practitioners with concerns in each domain are shown in Figure 1. More than half had concerns in the domains of good clinical care (56%) and colleague relationships (55%); about a third had concerns in the probity and patient relationship domains (40% and 34% respectively). The majority of practitioners (66%) had concerns across several domains as follows: 38% across two domains, 16% in three, 10% in four and 2% in five. The mode (and median) was two domains.

Relationships between GMP domains
The nature of the relationships between the four most frequently presenting domains (good clinical practice, colleague relationships, probity and patient relationships) was explored. As concerns in the probity domain included some that were secondary to poor conduct with patients and/or colleagues, this exploration focused initially on the relationships between the three domains of good clinical practice, colleague relationships and patient relationships. Of the 121 referred practitioners, 103 (85%) had concerns in at least one of these three domains. Within this trio of domains, the most common combination of concerns was from the good clinical practice and colleague relationships domains (24% of practitioners). Of the remaining 18 practitioners, 8 had no concerns in a GMP domain, 8 had probity concerns only and 2 had probity concerns secondary to health concerns.

Profile of GMP performance concerns
Within the domain of good clinical care, most of the concerns were in care provision (

Discussion
Although original in describing workplace performance concerns across GMP domains nationally in psychiatrists, our study has several limitations. Methodologically it is retrospective and restricted to those working in the NHS (independent providers were not included in NCAS referrals at the time). Furthermore, in the absence of recent NHS data, it is difficult to know whether the sample is representative of psychiatrists in the NHS workforce.
Existing NCAS data for all specialties show that male gender, consultant grade, ethnicity, older age and primary medical qualification outside the UK are associated with increased risk of referral. 5,6 Conceptually, any referred performance concerns may not have been upheld by subsequent investigation, and the paper does not address any contributory factors. Concerns may be due primarily to work context rather than individual competency, for example. 6,19 Finally, GMP guidance is not a performance or competency framework. 14 However, its use can be justified on the grounds that it has been the basis for the psychiatry postgraduate curriculum and NHS appraisal. 15,16 Before discussing the findings in relation to GMP, we will consider the findings in relation to referral source and previous NCAS data on concerns in psychiatrists.

Source of referral
Such data have not been published previously by NCAS. Concerns about psychiatrists' performance were raised most commonly by other staff members, reflecting possibly the multidisciplinary nature of the specialty. If replicated, the results highlight some important implications for the identification of performance concerns: the importance of having a variety of mechanisms in place; the role of informal (e.g. staff verbal report) in addition to formal (e.g. clinical governance) processes; and the importance of patient feedback. A related issue is whether other specialties (such as surgery) would have a similar distribution of referral source. This is as yet unknown; however, the present findings can be compared with previous NCAS data on psychiatrists. [3][4][5][6] Comparison with previous NCAS data The distribution of concerns across the domains of clinical capability, health and behaviour is similar to previous NCAS reports on performance concerns in psychiatrists. [3][4][5] More recently, NCAS has reported across an increased number of categories (seven) and reported that clinical difficulties were present in 50%, behavioural concerns in 70% (misconduct and other behaviours combined) and concerns about health, including alcohol or drug misuse, in 31%. Reported in this way, concerns about behaviour remain high. At employer level it would appear that cases are referred for advice earlier than previously. This may reflect cohort effects as the duration of concerns in those referred to NCAS has been decreasing annually. 5,6 This earlier involvement in concerns' handling by NCAS is one of its key objectives. 1 Nevertheless, the consistency of concerns across these data-sets, spanning up to 8 years since NCAS inception (2001-9), suggests the need for a more systematic response to addressing these concerns. Any response can be guided by more clarification on the nature of these concerns, specifically their distribution across GMP domains.

Performance concerns across GMP domains
Using the GMP framework and after further exploration and discussion of concerns by the NCAS advisor with the referrer, the referred cases are seen to be more complex. Concerns about good clinical care are the most common, along with concerns about working with colleagues. About  two-thirds have concerns in at least two GMP domains, compared with less than half (46%) using the NCAS domains; 85% of practitioners had concerns in at least one of the good clinical care, colleague and patient relationship domains; and the most common pattern was a combination of concerns in the domains of good clinical care and working with colleagues. Most of the probity concerns were due to improper relationships with patients and or staff. Managers appear to refer the more complex cases. This paper throws further light on presenting concerns through exploration of characteristics that may be associated with increased numbers of concerns and construction of the GMP performance profile. Although exploratory and preliminary, the results tentatively suggest that male, career-grade psychiatrists over 50 years old may be experiencing problems with colleague relationships, whereas their trainee counterparts may be at increased risk of probity concerns.
The GMP performance profile is helpful in two respects. It identifies those performance aspects across several levels that underpin concerns at domain level, thus providing more detailed training needs analysis. Additionally, the profile indicates that there are training needs in areas that are more specific to psychiatry (e.g. diagnosis and management) and those that are generic to all specialties (e.g. respect for colleagues). Intriguingly, it is the more generic type of concern that appears most often in the profile, thus supporting efforts to base training across the range of good medical practice, as in the curricular innovations of the Modernising Medical Careers programme. 13 Do the findings have any further implications for addressing performance concerns in psychiatrists?

Further implications
Psychiatrists, relative to workforce numbers, have a high rate of referral to the NCAS and a high prevalence of behavioural concerns relative to most other specialties. 5,6 Modelling of referral data to NCAS suggests that the referral pattern might be partly due to effects of ethnicity and place of qualification within the psychiatry workforce. Concerns that are secondary to health problems might respond to initiatives aimed at health improvement. 9 From a training and development perspective it is unclear from our findings whether poor performance is due to inadequate transfer of skills from training, attrition of skills once in the workplace, or other factors affecting performance such as health and workload. In terms of a systematic response by the College, the options include reviewing the CPD programme and developing relevant specialty-specific and generic interventions. Generic interventions should focus on teamworking skills and interpersonal behaviours that affect trust in relationships, both with staff and with patients. Specialty-specific interventions should focus on diagnosis and management, prescribing and record-keeping. Helping career-grade psychiatrists adapt to cultural shifts in team working may be relevant in addition.
For trainees, our findings on the source of concerns supports the College's approach of using staff and patient feedback in workplace-based assessments. The results also show that a more valid workplace-based assessment requires the integration of both patient and staff feedback with results obtained from clinical governance and performance management processes. Attributing increased weight within the curriculum to inclusion of competencies in the domains of working with colleagues and patient relationships is supported by these data. There may also be a role for employers through incorporation of clear guidelines on performance management issues, including expected behavioural norms towards patients and staff, in trainee workplace induction programmes.