Selecting staff for a personality disorder service : report from the field *

Aims and method To select staff for a personality disorder service by exposing applicants to anticipated challenges. Applicants took part in a role play, an unstructured group with fellow applicants and numbers of the interviewing panel, and a panel interview. A service user representative was involved from the initial planning stage.

Results Multiple assessment methods enabled fine discrimination between applicants. Appointed staff said they felt the interviews were a valid test of required skills and have been well regarded by patients in the programme.

Clinical implications Selecting staff for a personality disorder service benefits from using multiple interview methods. The service user representative and role-play actor can contribute crucially by representing the patient’s perspective. Key domains to assess include the applicant’s psychological qualities, psychiatric skills and ability to work in a team.

Aims and method Mental Health Recovery Star is a multifaceted 10-item outcomes measure and key-working tool that has been widely adopted by service providers in the UK.We aimed to explore its factorial validity, internal consistency and responsiveness.Recovery Star readings were conducted twice with 203 working-age adults with moderate to severe mental health problems attending a range of mental health services, and a third time with 113 of these individuals.
Results Mental Health Recovery Star had high internal consistency and appeared to measure an underlying recovery-oriented construct.Results supported a valid twofactor structure which explained 48% of variance in Recovery Star ratings data.Two Recovery Star items ('relationships' and 'addictive behaviour') did not load onto either factor.There was good statistically significant item responsiveness, and no obvious item redundancy.Data for a small number of variables were not normally distributed and the implications of this are discussed.
Clinical implications Recovery Star has been received enthusiastically by both mental health service providers and service users.This study provides further evidence for its adoption in recovery-focused mental health services and indicates that items relating to addictive behaviour, responsibilities and work could be further developed in future.
Declaration of interest J.W. and Y.O. were employed by the Mental Health Providers Forum, the commissioners of Mental Health Recovery Star, at the time of the study.
to use Recovery Star has been provided by the enthusiasm of mental health service users and their key workers, who value its role in understanding the current position, planning the actions needed to progress the recovery journey and demonstrating progress across ten domains (Box 1).As such it has potential to motivate users and workers in striving towards recovery and to play an active role in it, and thus is not intended simply as an inert measurement tool.Recovery Star is predicated on an underlying model of a 'ladder of change' comprising five stages: being stuck, accepting help, believing, learning and self-reliance.The tool has been widely adopted by mental health service providers in the UK, 12 although some concerns have been raised about the lack of data on its psychometric properties. 13,14

Study aims
The aim of our study was to explore the psychometric properties of Recovery Star to inform training and further development.Specific objectives were to ascertain whether items on the tool measured a single underlying construct relating to recovery (internal consistency); to identify the nature of any underlying factors (factor validity); to identify any item redundancy; and to identify whether Recovery Star detects reported change over time (responsiveness).

Measures
Recovery Star comprises ten dimensions that were identified as central to the recovery journey (Box 1).Each item is rated jointly by service user and key worker on a 10point scale, which relates to the underlying five-stage 'ladder of change' model and specifically to the service user's own subjective sense of their position in relation to the ladder rather than as an objective measure of severity.Each stage relates to two steps on the ladder of change (i.e.scores 1 and 2 relate to the stuck stage, 3 and 4 to the accepting help stage, and so on).Recovery Star is intended both as an outcomes measure, in that it aims to quantify the service user's movement along the recovery pathway across the course of service contact, and a key-working tool, in that it guides and informs that collaborative work.

Setting and participants
Participating organisations entered routinely collected and anonymised Recovery Star and demographic data into an internet-based database between November 2008 and July 2010.Recovery Star projects were mainly run by small to medium-size community-based teams (e.g.community mental health teams) in NHS mental health trusts, private healthcare organisations and charitable agencies offering a range of services including day centres, supported housing, psychological therapies, social activities and substance misuse services.Service users were mainly adults of working age presenting with moderate to severe mental health problems.

Procedure
Recovery Star and demographic data were extracted from the MHPF database of those individuals aged 18-65 years who had completed two or more readings of the Recovery Star, either alone or in collaboration with a project worker, at least 42 days apart (n = 203).Of these, 113 (55.9%) had completed a third Recovery Star at least 84 days after the first reading.A total of 113 completed Stars were omitted from the analysis because they had been completed by a project worker alone.

Analysis
Data were transferred into SPSS 16.0.1 for Windows.Item and total mean scores and standard deviations were calculated.Descriptive data were generated to describe the sample, including significant deviations from normal Gaussian distribution.Cronbach's a was calculated as an indicator of internal consistency; exploratory factor analysis (principal axis factoring, varimax rotation with Kaiser

Descriptive data
In total, 203 pairs of Recovery Star ratings were gathered from 27 participating projects (range 1-43 pairs, median = 29).Item response rate was 100%.Mean age at completion of the first reading was 46.7 years (range 21.2-64.0,s.d.= 10.2).There were 110 males (54.2%) and 93 females (45.8%) in the sample.The ethnic background of 92% of people was recorded as 'White'.At baseline, Recovery Star was usually completed jointly by a service user and project worker (94.6%), and occasionally by a service user alone (5.4%).Mean baseline ratings for each item and for total score are presented in Table 1.Item means lay close (s.d.= 2.5) to the midpoint (5.5), although the mean score for 'responsibilities' approached the upper limit of this boundary.The score for all ten items ranged from 1 to 10.The skewness statistic indicated that the distribution of two items ('responsibilities' and 'addictive behaviour') was significantly negatively skewed (two standard errors of skewness), with scores bunched on the high end of the scale, and that of one ('work') was significantly positively skewed, with scores bunched on the low end.

Internal consistency and factor validity
All items correlated at statistically significant levels (range r = 0.50-0.77)with the total score.Cronbach's a = 0.85 represented very good internal consistency.Exploratory factor analysis indicated that two underlying factors comprising five and three items accounted for 48.0% of the variance in scores (Table 2); two items ('relationships' and 'addictive behaviour') did not load on to either factor.Eigen values showed that factor 1 explained 41.1% of the variance in the data and factor 2 explained 6.9%.Item loadings in the two extracted factors all exceeded 0.5.Internal consistency was very good for factor 1 (Cronbach's a = 0.85) and acceptable for factor 2 (Cronbach's a = 0.73).

Item redundancy
High internal consistency on a scale can indicate item redundancy if individual items simply measure the same facet through repetition, which is usually indicated by itemitem correlation in excess of 0.7.In contrast, inter-item correlation of less than 0.3 may indicate redundancy due to the non-homogeneity of test items. 15In this study almost all items correlated with one another at levels exceeding chance, but no item-item correlation exceeded the 0.7 threshold.The exceptions related to the 'addictive

Responsiveness
The skewness statistic indicated distribution within the normal range for seven items and for these, responsiveness was measured using the parametric repeated measures ttest; the non-parametric equivalent was used for items with non-Gaussian distribution.Table 1 shows that there was statistically significant change (indicating positive movement on the recovery journey) between the first two readings of the Recovery Star for seven out of ten items.
The available third readings of the Recovery Star indicated similar positive results (Table 3) for the majority of items.However, the item relating to 'responsibilities' showed no change at either second or third reading.Where significant, outcome effect size was typically small between first and second measurement (Cohen's d = 0.1-0.26)and medium between first and third readings (Cohen's d = 0.2-0.4).Those who completed the Recovery Star on three occasions were less far along their recovery journey in relation to 'self-care' (x ¯= 6.1) than those who only repeated the Star on two occasions (x ¯= 5.4, t = 72.34,d.f.= 201, P50.05), but there was no other significant difference between the two groups.

Discussion
The routinely collected data presented in this study represent the first quantitative investigation of the psychometric properties of the Recovery Star, and the results are encouraging.The tool had very good internal consistency (Cronbach's a40.8), little obvious item redundancy, and most item scores moved in a positive direction over time.However, as would be expected from a tool that was designed from the bottom-up to meet user need and to serve a dual purpose as a key-working tool, there were some issues related to the expectations that researchers may have in terms of the psychometric properties of the tool.This study was performed on routinely recorded data with few demographic or other research data to draw on and therefore has some important limitations.We hope that discussion of these will inform any future development of the Recovery Star.

Limitations
Cronbach's a of 0.85 for the instrument suggested a single underlying recovery-related construct, and exploratory factor analysis indicated that this comprised two factors with good (factor 1) or acceptable (factor 2) internal consistency of their own.These factors did not map precisely onto previous, narrative descriptions of the key components of recovery, for example those by the Sainsbury Centre for Mental Health. 3However, discussion among the researchers led us to believe that there was some rational connection between the grouped items, with factor 1 representing 'internal management and personal relationships' and factor 2 representing 'external management and external relationships'.4]16,17 Two items did not load onto either factor ('relationships' and 'addictive behaviour').The 'relationships' item might, superficially, be reasonably expected to fall under the 'internal management and personal relationships' factor but it is not intended to refer to the individual's personal relationships in general, which are adequately covered under 'social networks', but rather to a single dyadic and essentially intimate  relationship.Other authors have noted that intimate relationships may belong to a separate aspect of recovery distinct from socialisation within society because people with mental illness can find that establishing a life partnership is a challenge of a different order to that of managing employee and peer relationships. 18Recovery Star was designed with users to reflect their experience of recovery and not to adhere to a predetermined statistical factor structure.A lack of precise concordance between the actual factor structure of Recovery Star and a theorised structure of the wider recovery concept was therefore predictable.Our analysis may aid in developing or devising items to more accurately reflect the underlying statistical properties of recovery for research purposes.Data distribution for three items was non-Gaussian (Tables 1 and 3) suggesting that there may need to be some calibration if a version of the tool for research purposes were to be developed.Although 'addictive behaviour' and 'responsibilities' were negatively skewed because a relatively large number of participants identified themselves as being at an advanced stage on this part of the recovery journey, significant numbers still identified as having needs in these areas (25% of participants 'being stuck' or 'accepting help' on the 'addictive behaviour' item, and 15% at a similar stage on the 'responsibilities' item).Therefore, the tool did retain clinical utility and is not redundant in that sense.Similarly, distribution of scores for the item 'work' was positively skewed, indicating a preponderance of individuals rating themselves as at an early stage of their recovery in this area.We note that the pinnacle of the ladder for this item refers to self-reliance and that the accompanying descriptor refers to 'work' rather than to 'meaningful activity' as in some other recovery-oriented measures (e.g.Recovery Assessment Scale 19 ).Some may view this as politically motivated, for example as an attempt to move people through the healthcare system and back into paid employment.However, far from this, the item apex was oriented to paid employment as opposed to voluntary or other non-paid activities during tool development precisely because user consultees wanted recognition for their desire to be in meaningful, paid employment with appropriate support rather than be diverted into voluntary activities.The 'work' item proved to be one of the least amenable to change, particularly over the longer term, indicating that a number of people have long-term needs for ongoing support and, for them, there may be a need to clarify the item descriptor to allow for a range of interpretations of 'work'.
Our study was based on routinely collected data from users of community mental health services and it is possible, therefore, that items with non-Gaussian distribution within this sample may not be replicated elsewhere.For example, forensic mental health service users frequently have needs related to reoffending, 20 violence and selfharm [21][22][23] and substance misuse. 24There is growing demand for recovery-oriented practice in these settings 25 and a need to translate recovery principles into practice for these populations. 26Items such as responsibilities and addictive behaviour could be supplemented with additional items which reflect these needs, although it will be important to maintain congruence with the positive orientation of the recovery approach, including the underlying ladder of change model, in any future developments.
The responsiveness of items was demonstrated by statistically significant change of small effect size between first and second readings of the Recovery Star.This may simply represent the relatively short period between ratings (mean 157 days) and a consequent lack of time for service users to recognise their own progress in this aspect of their recovery.This hypothesis is consistent with data from individuals who completed a third rating an average of 9 months after baseline, when slightly greater medium effectsized changes were demonstrated.The data probably fairly reflect the slow gains made by individuals with chronic problems living in the community, and longer-term studies are required.
The study used Recovery Star and brief demographic data that were collected during the course of routine clinical practice across multiple organisations and thus indicated a potential lack of standardisation in data collection.However, most participating organisations received training in completion of the Recovery Star provided by MHPF.Little is currently known about the retest or interrater reliability of the tool or about its external validity, and we await progress on these issues.Importantly, Recovery Star was able to detect statistically significant reported change between measurements and the change was greater (medium effect size) over longer periods.No other measures were collected that could have tested convergent and divergent validity and clinical details about the participants were unavailable.Further research on the psychometric properties of Recovery Star is clearly warranted, but this study provides good preliminary evidence for its adoption in recovery-focused mental health services.Selecting the 'right' person for a personality disorder service is difficult.The applicant may have all the credentials on paper, and may even interview very well, but whether they will do the job well is another question.In response to local need and national priorities, a new service was commissioned by the East London NHS Foundation Trust in Tower Hamlets from September 2007.DeanCross Personality Disorder Service forms part of the adult mental health services within the Trust. 1 It is a dedicated, non-forensic service for people with severe and moderate Aims and method To select staff for a personality disorder service by exposing applicants to anticipated challenges.Applicants took part in a role play, an unstructured group with fellow applicants and numbers of the interviewing panel, and a panel interview.A service user representative was involved from the initial planning stage.

About the authors
Results Multiple assessment methods enabled fine discrimination between applicants.Appointed staff said they felt the interviews were a valid test of required skills and have been well regarded by patients in the programme.
Clinical implications Selecting staff for a personality disorder service benefits from using multiple interview methods.The service user representative and role-play actor can contribute crucially by representing the patient's perspective.Key domains to assess include the applicant's psychological qualities, psychiatric skills and ability to work in a team.

Declaration of interest None.
Selecting staff for a personality disorder service: report from the field* Living skills.This is about the practical side of being able to live independently-shop and cook for yourself, deal withneighbours and people who visit, keeping your place clean and tidy.4Social networks.This is about your social networks and being part of your community including volunteering or classes, being part of a club or society, school or faith organisation, or groups of friends.
normalisation) was conducted; tests of item redundancy (item-item correlation using Pearson product-moment statistic (r)) and responsiveness (repeated measures t-tests, Cohen's d) were performed.

Table 2
Recovery Star factor analysis (n = 203) behaviour' item, which did not correlate significantly with three items ('managing mental health', 'relationships', and 'identity and self-esteem').
Geoff Dickens, Head of Nursing Research & Research Manager, St Andrew's Academic Centre, King's College London, Institute of Psychiatry, Northampton; Judy Weleminsky, (previously) Chief Executive, Yetunde Onifade, Recovery Development Manager, Mental Health Providers Forum, London; Philip Sugarman, CEO & Medical Director, St Andrew's Academic Centre, King's College London, Institute of Psychiatry, Northampton.