Skip to main content Accessibility help
×
×
Home

Psychometric properties of the Mental Health Recovery Star

  • Helen Killaspy (a1), Sarah White (a2), Tatiana L. Taylor (a3) and Michael King (a3)
Abstract
Background

The Mental Health Recovery Star (MHRS) is a popular outcome measure rated collaboratively by staff and service users, but its psychometric properties are unknown.

Aims

To assess the MHRS's acceptability, reliability and convergent validity.

Method

A total of 172 services users and 120 staff from in-patient and community services participated. Interrater reliability of staff-only ratings and test–retest reliability of staff-only and collaborative ratings were assessed using intraclass correlation coefficients (ICCs). Convergent validity between MHRS ratings and standardised measures of social functioning and recovery was assessed using Pearson correlation. The influence of collaboration on ratings was assessed using descriptive statistics and ICCs.

Results

The MHRS was relatively quick and easy to use and had good test–retest reliability, but interrater reliability was inadequate. Collaborative ratings were slightly higher than staff-only ratings. Convergent validity suggests it assesses social function more than recovery.

Conclusions

The MHRS cannot be recommended as a routine clinical outcome tool but may facilitate collaborative care planning.

  • View HTML
    • Send article to Kindle

      To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

      Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

      Find out more about the Kindle Personal Document Service.

      Psychometric properties of the Mental Health Recovery Star
      Available formats
      ×
      Send article to Dropbox

      To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

      Psychometric properties of the Mental Health Recovery Star
      Available formats
      ×
      Send article to Google Drive

      To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

      Psychometric properties of the Mental Health Recovery Star
      Available formats
      ×
Copyright
Corresponding author
Helen Killaspy, MBBS, PhD, FRCPsych, Mental Health Sciences Unit, University College London, Charles Bell House, 67–73 Riding House Street, London W1W 7EJ, UK. Email: h.killaspy@ucl.ac.uk
Footnotes
Hide All

Declaration of interest

None.

Footnotes
References
Hide All
1 Department of Health. Liberating the NHS. Transparency in Outcomes – A Framework for the NHS. Department of Health, 2011.
2 Appleby, J. Patient reported outcome measures: how are we feeling today? BMJ 2012; 344: d8191.
3 HM, Government. No Health without Mental Health: A Cross-Government Mental Health Outcomes Strategy for People of all Ages. Department of Health, 2011.
4 Shepherd, G, Boardman, J, Slade, M. Making Recovery a Reality. Sainsbury Centre for Mental Health, 2008.
5 MacKeith, J, Burns, S. Mental Health Recovery Star. Mental Health Providers Forum and Triangle Consulting, 2008.
6 Burgess, P, Pirkis, J, Coombs, T, Rosen, A. Assessing the value of existing recovery measures for routine use in Australian mental health services. Aust NZ J Psychiatry 2011; 45: 267–80.
7 Anthony, WA. Recovery from mental illness: the guiding vision of the mental health service system in the 1990s. Psychol Rehab J 1993; 16: 1124.
8 Mental Health Providers Forum. Recovery Star Online Analysis – The Growing Picture Continued. Mental Health Providers Forum, 2011.
9 Beazley, PI. The Recovery Star: is it a valid tool? (letter). Psychiatrist 2011; 35: 196–7.
10 Parker, G, Rosen, A, Emdur, N, Hazipavlov, D. The Life Skills Profile: psychometric properties of a measure assessing function and disability in schizophrenia. Acta Psychiatr Scand 1991; 83: 145–52.
11 Young, SL, Bullock, WA. The Mental Health Recovery Measure. University of Toledo, 2003.
12 Najim, H, McCrone, P. The Camberwell Assessment of Need: comparison of assessments by staff and patients in an inner-city and a semi-rural community area. Psychiatr Bull 2005; 29: 13–7.
13 Burgess, P, Pirkis, J, Coombs, T, Rosen, A. Assessing the value of existing recovery measures for routine use in Australian mental health services. Aust N Z J Psychiatry 2011; 45: 267–80.
14 Flinn, S. Reliability and Validity of the Recovery Assessment Scale for Consumers with Severe Mental Illness Living in Group Home Settings. Kent State University, 2005.
15 Hasson-Ohayon, I, Roe, D, Kravetz, S. The psychometric properties of the Illness Management and Recovery Scale: client and clinician versions. Psychiatr Res 2007; 160: 228–35.
16 Porter, M. What is value in health care? N Engl Med J 2010; 363: 26.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

The British Journal of Psychiatry
  • ISSN: 0007-1250
  • EISSN: 1472-1465
  • URL: /core/journals/the-british-journal-of-psychiatry
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×
Type Description Title
PDF
Supplementary materials

Killaspy et al. supplementary material
Supplementary Table S1

 PDF (33 KB)
33 KB

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed

Psychometric properties of the Mental Health Recovery Star

  • Helen Killaspy (a1), Sarah White (a2), Tatiana L. Taylor (a3) and Michael King (a3)
Submit a response

eLetters

Evaluating recovery star - flawed study design

Geoffrey L Dickens, Research Manager
05 September 2012

Killaspy and colleagues have concluded categorically that Mental Health Recovery Star cannot be recommended as a routine clinical outcome tool. This is unfortunate as it has proved a popular and welcome innovation in UK mental health services in recent years. While the authorsacknowledge study limitations caused by a lack of resources, it is apparent that a major limitation relates to study design, which resulted in inter-rater reliability being judged on staff only ratings of service-user recovery. The paper suggests that lack of staff-staff inter-rater reliability is 'a serious problem' (p.69). However, as pointed out in other responses, and by ourselves (Dickens et al, 2012a), this method contradicts explicit instructions in published MHRS ratings guidance (MacKeith & Burns, 2011).

Ideally this significant methodological limitation would have been tackled in the peer review process, and addressed more fully in the paper,to allow BJPsych to publish a study with a discussion which grasped the contentious nature of the conclusion drawn from the data. In our view theonly conclusion about reliability that can be drawn is that the MHRS is not reliable when rated in a way which is explicitly contrary to the tool's guidelines. It is interesting that Killaspy and colleagues view therecommended collaborative nature of rating as 'unusual' (p.69), and do nottake the opportunity to embrace it is as a welcome change in clinical practice. In our view, MHRS is a very promising instrument (Dickens et al,2012b) that is immersed in the spirit of staff-service user collaboration and recovery. It clearly requires further development, not least to validate the underlying ladder of change model, but does not deserve criticism for faults that lie in the design of an independent evaluation.

References

Dickens, G., Onifade, Y., Sugarman, P. & Weleminsky, J. (2012a) Authors' response to The Mental Health Recovery Star: great for care planning but not as a routine outcome measure.. The Psychiatrist, 36, 194.

Dickens, G., Onifade, Y., Sugarman, P. & Weleminsky, J. (2012b) Recovery Star: Validating user recovery. The Psychiatrist, 36, 45-50.

MacKeith, J. & Burns, S. (2011) Mental Health Recovery Star. London: Mental Health Providers Forum.

http://www.mhpf.org.uk/sites/default/files/documents/publications/mhpfuserguide_v2.pdf

... More

Conflict of interest: Philip A. Sugarman is a Trustee of Mental Health Providers Forum, publisher of the Mental Health Recovery Star.

Write a reply

Square pegs and round holes: assessing the suitability of the Killaspy et al Recovery Star validation approach

Joy MacKeith, Director
21 July 2012

I write in response to Killaspy, White and King's piece on the psychometric properties of the Recovery Star. Although this is clearly a thorough study, there are a number of important issues which should be born in mind when interpreting the results.

1) Suitability of setting: over a third of the service users whose data is reported in this study were in secure settings at the time of the research. The version of the Recovery Star that was used in the study is not well suited to secure settings where the choices and opportunities available to service users are reduced and issues relating to safety and behaviour management are greater. A new version specifically designed forthese settings is in development.

2) Suitability of the method: As the authors point out the Recovery Star is designed to be used collaboratively, but the inter-rater analysis is based on staff only readings. Whilst staff might be expected to give areasonably consistent assessment of domains such as Managing Mental Health, Living Skills, Social Networks and Work, they are much less likelyto be able to report consistently on aspects such as Identity and Self-esteem or Trust and Hope without the service users being present as they relate to aspects of the service users' internal experience which the service user may not have discussed with staff. The findings back this up with relatively good inter-rater results for the former (Managing Mental Health 0.69, Living Skills 0.67, Social Networks, 0.67 and Work 0.77) and lower results for the latter (Self-esteem 0.58 and Trust and Hope 0.62). More broadly it does not seem logical to draw conclusions about the suitability of a tool if it has not been used in the recommended manner.

3) Suitability of comparative measures: The authors explore convergent validity of the Recovery Star by correlating Recovery Star scores with the Mental Health Recovery Measure (MHRM). Because the two measures do not show a linear correlation they conclude that the Recovery Star is not a valid measure of recovery. However, whilst the MHRM measures primarily how positively the service users feels about life, their mental illness and the future, the Recovery Star measures the service user's position on a journey towards self-reliance which starts with not acknowledging issues and progresses through accepting help to believing and learning. As the person moves along this journey they beginto acknowledge the difficulties they are facing, take more responsibility for addressing them and learn skills and strategies to do this. This can be very hard at times and the model predicts that there will be points when they will face setbacks and feel discouraged. It is therefore not surprising that the Recovery Star and MHRM do not show a strong linear correlation

4) Suitability of framework of understanding: In their discussion theauthors suggest that the Recovery Star mixes the patient experience of recovery with social functioning and imply that this may be problematic. Recovery depends on both feelings and functioning. The Recovery Star reflects this by bringing these aspects together rather than splitting them into different tools. In the same way it brings together the service user and worker views rather than separating these into Patient Reported Outcome Measures (PROMs) and clinician measures. These are strengths rather than weaknesses but ones which may not fit neatly into establishedways of understanding tools and carrying out validation.

In view of these facts I believe that these results should be seen astentative and that concluding that they do not support the use of the Recovery Star as a clinical outcomes measure is premature. The Recovery Star is a work in progress and there are plans to further develop the tooland the training to improve its qualities as both a key-work and an outcomes tool. Perhaps the existence of this new type of tool also challenges us all to continue to develop and improve approaches to validation so that new tools are not rejected on the grounds that they do not fit the existing mould.

... More

Conflict of interest: co-author of the Recovery Star

Write a reply

The OutcomeStarTM approach: an objective outcome measure or a tool for co-creating health?

Theresa E Eynon, NIHR CLAHRC Clinical Translation Fellow
21 July 2012

In attempting to determine the psychometric properties of the 'MentalHealth Recovery Star' the authors (Killaspy, White, Taylor, & King, 2012) have gone to some lengths to assess whether or not the tool is an objective measure of 'recovery'. The paper appears to be a professional response to pressure from the Department of Health. Perhaps therein lies the confusion?

As the authors point out, the 'collaborative component of the measurepresents methodological difficulties'. It is odd therefore, but understandable that, for a tool designed to be used collaboratively, they chose to compare staff-only ratings one with another in order to assess inter-rater reliability.

In attempting to treat the Mental Health Recovery Star as if it can or should provide an objective measure of 'recovery' and be rated independently of the key-worker/patient relationship, it becomes clear that this paper has lost its way in a clash of philosphical paradigms.

The OutcomeStarTM family of tools, of which Recovery Star is a member, are designed to enable the service-user, in a collaborative relationship with a service-provider, to assess their own position on a journey from disengagement and dependency towards active ownership and self-management.

In comparing it with the Mental Health Recovery Measure (Bullock, 2009), the study treats the Recovery Star tool as if it should be an objective, single-measure, snap-shot of a defined concept - 'recovery'. This is very different, philosophically, from its use as a means of co-creating health.

While,rightly, dismissing the Recovery Star as an alternative to an objective 'outcome measure', the study does agree that, as a collaborativetool, it has test-retest reliability and also convergent validity with measures of social functioning. Further research could be done along this paradigm, to determine whether an OutcomeStarTM approach can measure reliable and significant change in patient self-perception over time and whether or not this correlates with other ratings of service user satisfaction, self-efficacy, social engagement and empowerment.

At a deeper level, however, comparing the RecoveryStar with an objective Recovery Measure and finding it wanting is like complaining thatapples are not oranges. From the perspective of community oriented co-creation (De Maeseneer & Boeckxstaens, 2012), it is more important that the study confirms that the family of tools that includes Recovery Star may be acceptable for care-planning.

In a 'reformed' NHS, professionals are being encouraged to take greater account of the patient voice (National Voices, 2011). From a professional perspective, dismissing Recovery Star from routine use as an alternative 'outcome measure' is understandable. It would be a great lossto service-users and commissioners if this dismissal also meant that it lost its role as a means of achieving organisational change.

A key measure of the usefulness of the OutcomeStarTM approach will bewhether or not, by supporting co-creation, such tools can change the power relationship between service users and providers (Wallace, et al., 2012) and liberate the NHS from its paternalistic legacy. Oranges are not the only fruit.

Bibliography

Bullock, W. A. (2009). The Mental Health Recovery Measure (MHRM); updated normative and psychometric properties. Retrieved July 13, 2012, from http://psychology.utoledo.edu/images/MHRM_-_Bullock_-_Updated_Normative_and_Psychometric_Data_12-09.pdf

De Maeseneer, J., & Boeckxstaens, P. (2012). James MacKenzie Lecture 2011: multimorbidity, goal-oriented care, and equity. British Journal of General Practice , 62, 384-5.

Killaspy, H., White, S., Taylor, T., & King, M. (2012). Psychometric properties of the Mental Health Recovery Star. British Journal of Psychiatry , 201, 65-70.

National Voices. (2011, October). Principles for Integrated Care. Retrieved March 07, 2012, from www.nationalvoices.org.uk: http://www.nationalvoices.org.uk/sites/www.nationalvoices.org.uk/files/principles_for_integrated_care_final_20111021.pdf

Wallace, L., Turner, A., Kosmala-Anderson, J., Sharma, S., Jesuthasan, J., Bourne, C., et al. (2012). Evidence: co-creating health: evaluation of first phase. London: The Health Foundation.

... More

Conflict of interest: I provide consultancy to West Leicestershire Clinical Commissioning Group who are adopting the Elderly Person's Outcome Star as a case-management framework. I am paid by AssuraLLP Leicestershire to provide GP-engagement to Rethink Mental Illness for their 'Good Thinking' primary care mental health service. I am an elected Labour Party Parish Councillor. I have received honoraria from Eli Lilly and Pulse to speak on primary care mental health. I am a member of the East Midlands Commissioning Network which is supported by Lundbeck

Write a reply

Reliability is a dimension, not a category

Alastair J Macdonald, Clinical Advisor, Outcomes Team
21 July 2012

Dear Editor

So an intra-class correlation of 0.77 means that a scale is reliable,whereas one of 0.69 means that it is not? Says who? Never mind- it doesn't matter: this ignores two obvious facts. First, rating scale unreliability is a dimension. The categories "reliable" and "unreliable" are arbitrary; they maintain a pseudoscientific fog that has hung over psychiatric research for too long. The second fact is that a scale's (inevitable) unreliability will depend on who is using it and in what context; it is not set in stone forever.

A much more scientific approach to the use of rating scales would be to measure reliability whenever, wherever and by whom the scales are used.The resulting variance can then be factored into the analysis of the results of the study. The question is not 'did they use a "reliable" scale?' but 'did their results hold true controlling for the unreliabilityof the scales, as they used them in that study of those groups?'. An example of this approach can be found in Macdonald & Trauer (2010).

Surely it is time that we moved beyond 1950s research technology?

Yours sincerely

A J D MacdonaldVisiting ProfessorKing's College London, Institute of Psychiatry

Reference

Macdonald AJ, Trauer T. (2010) Objections to routine clinical outcomes measurement in mental health services: any evidence so far? J Ment Health. Dec;19(6):517-22.

... More

Conflict of interest: None declared

Write a reply

×

Reply to: Submit a response


Your details


Conflicting interests

Do you have any conflicting interests? *