Dickens et al's paper^{Reference Dickens, Weleminsky, Onifande and Sugarman1} reporting on the internal validity of the Mental Health Recovery Star provides evidence for its internal consistency and factor structure. The authors state that it is assessing a single underlying recovery-related construct. However, there is a problem with this statement, since recovery in this context is, by definition, a subjective construct. For this reason, the application of any predetermined constructs (the ten domains of the Recovery Star) can only be considered to be assessing an individual's recovery if those domains happen to coincide with an individual's own priorities. A separate study (currently under review for publication) has investigated the external validity of the Recovery Star and found interrater reliability of nine of the ten domains to be below the generally accepted level (intraclass correlation coefficient >0.7).

Dickens et al present findings from routinely collected data and suggest these are evidence of the Recovery Star's sensitivity to change in an individual's progress over time (i.e. its responsiveness). The problem is that unless the same member of staff was involved in repeat ratings, these findings are likely to be invalid given the issues with interrater reliability. In addition, responsiveness to change needs to be corroborated by an established measure. Finally, if earlier ratings were discussed between the staff and service user before re-rating (as is encouraged through the training and manual accompanying the Recovery Star), then neutrality is likely to have been reduced, as both may have an investment in showing that progress has been made. One further, fundamental issue is that the ‘ladder of change’ used to assess progress in each of the ten domains has not been validated psychometrically.

The Recovery Star is very popular and has merit as a tool to enhance discussion of recovery goals between staff and service users. However, although Dickens et al's findings have helped with understanding some of the Recovery Star's psychometric properties, they do not provide evidence for its adoption as a routine outcome measure.