Introduction
Researchers, health professionals, policy makers and administrators have been grappling with how to measure quality of care for some time, and there remains no agreed ‘gold standard’ for measurement in the long-term care setting (Donabedian, Reference Donabedian1985; Mor, Reference Mor2007; Burke and Werner, Reference Burke and Werner2019). Useful models and frameworks exist to help describe and identify key components to consider in the measurement of quality care, such as the Donabedian model (Donabedian, Reference Donabedian1985). The model proposes that indicators of the quality of care can be divided into three categories including the ‘structure’ of care (e.g. the organisational structure and governance of the services), ‘process’ of care (e.g. the processes undertaken to provide care) and the ‘outcomes’ of care (e.g. the effect of the care on the recipient). Concerns have been raised that ‘structural’ indicators are crude measures of care quality, and focus has shifted to measuring the ‘process’ and ‘outcomes’ of care (Noelker and Harel, Reference Noelker, Harel, Noelker and Harel2000).
When measuring ‘process’ or ‘outcome’ quality of care indicators, a number of different approaches and measures also exist. The Minimum Data Set set of quality indicators, developed in the United States of America (USA), is one example of ‘process’ and ‘outcome’ quality of care drawn from mandated routinely collected resident assessment data (Zimmerman, Reference Zimmerman2003). Measures drawn from resident assessment data have advantages in offering value for money and streamlining systems through repurposing data that are already being collected. However, such measures are often focused on clinical indicators, such as prevalence of falls, inappropriate medications, weight loss and use of physical restraints. Whilst important aspects of quality of care, relying on assessment data neglects other aspects of care provided to residents, such as the quality of the care environment and how well it supports the quality of life of residents.
Measuring and valuing the quality of care experience and the quality of life of residents have become increasingly important endeavours. This is especially true among the large numbers of people living with dementia in residential aged care, as without a cure for dementia and few effective pharmacological therapies available, best practice treatment focuses on managing the symptoms of the condition at an individual level, through psychosocial and supportive interventions (Laver et al., Reference Laver, Cumming, Dyer, Agar, Anstey, Beattie, Brodaty, Broe, Clemson, Crotty, Dietz, Draper, Flicker, Friel, Heuzenroeder, Koch, Kurrle, Nay, Pond, Thompson, Santalucia, Whitehead and Yates2016). Supportive models of care such as person-centred care have been proposed, which refer to certain principles which underpin the approach to caring for people with dementia (Kitwood and Bredin, Reference Kitwood and Bredin1992; Edvardsson et al., Reference Edvardsson, Winblad and Sandman2008). This includes acknowledging the human value of people living with dementia, their individuality, prior life experiences and perspectives in care (National Institute for Health and Social Care Excellence, 2018). Implementing this approach into such a large and complex system as long-term care for older people is a challenge, but practical models exist, including larger-scale organisation-wide approaches such as the ‘Green House’ model (Zimmerman et al., Reference Zimmerman, Bowers, Cohen, Grabowski, Horn and Kemper2016), or changes to the physical environment of the facilities (Castle, Reference Castle2010). While the evidence on the benefits of these models of care is still variable, studies have indicated that measuring these psychosocial and environmental components of how care is provided is important for residents, in addition to clinical aspects of the quality of care (Kane et al., Reference Kane, Lum, Cutler, Degenholtz and Yu2007; Anderiesen et al., Reference Anderiesen, Scherder, Goossens and Sonneveld2014; Milte et al., Reference Milte, Shulver, Killington, Bradley, Ratcliffe and Crotty2016; Dyer et al., Reference Dyer, Liu, Gnanamanickam, Milte, Easton, Harrison, Bradley, Ratcliffe and Crotty2018). Broadening the quality of care indicators used in the sector to include ‘process’ and ‘outcome’ indicators which include the voice of the resident themselves will be crucial to this endeavour.
However, choosing a quality of care indicator for self-report by residents, including those living with dementia, is not a straightforward task. A recent systematic review identified 29 self-report quality-of-life instruments which had been used in aged care, but of those only seven were designed for use with people living with dementia (Siette et al., Reference Siette, Knaggs, Zurynski, Ratcliffe, Dodds and Westbrook2021). Whilst several quality-of-life measures, including the DEMQOL and the Quality of Life in Alzheimer's Disease (QOL-AD), have been developed and validated in populations of older people with dementia, very few process-focused quality of care measures have been developed for application in residential aged care (Cleland et al., Reference Cleland, Hutchinson, Khadka, Milte and Ratcliffe2021; Vassimon-Barroso et al., Reference Vassimon-Barroso, Pantoni, Zazzetta, Ferreira, Vasilceac and Ansai2021). Those that have been developed have not tended to include the voices of residents themselves during the process. A further challenge to measuring quality of care from the perspective of the person within the residential care sector is the high prevalence of dementia. Over half of aged care residents in Australia have a diagnosis of dementia, which is itself a likely underestimate of the true prevalence due to known underdiagnosis (Australian Institute of Health and Welfare, 2020).
Crucially, understanding and measuring quality of care could be used as a way to drive improvements within the long-term care system, as a way for providers to identify areas of care which need improvement, and quantify improvements made when new models of care or interventions are implemented (Castle, Reference Castle2010). Sharing information on the quality of care provided within facilities with the general public more broadly could also help older people and their families make more informed decisions about where they would like to live when choosing a residential aged care service (RACS). Therefore, reliable and meaningful measurement of quality of care is the foundation for a range of mechanisms of improvement in the system. Significantly, only a few studies have investigated the relationship between the quality of care residents receive and their level of cognitive impairment. Two previous studies have indicated that there is a negative relationship between quality of care and cognitive impairment, and that the relationship varies between facilities, indicating that quality of care for people with cognitive impairment is modifiable (Bravo et al., Reference Bravo, De Wals, Dubois and Charpentier1999; Nazir et al., Reference Nazir, Arling, Perkins and Boustani2011). Neither of these studies used self-report measures from the person living in care themselves.
In response to these unique pressures and needs, a measure of the quality of care of residential long-term care services has been designed to be undertaken by residents themselves, the CCI-6D instrument (Milte et al., Reference Milte, Ratcliffe, Bradley, Shulver and Crotty2019). This instrument was developed from the ground up with the opinions of people living with dementia and their family members, and as such reflects their perceptions of what good quality of care within a residential aged care facility entails. The instrument is unique in its focus on quality of care from the perspective of the person, where quality of care is defined by six dimensions reflecting the physical and social care environment. These six dimensions were drawn from the perceptions of people living with dementia and what they considered to be important for a high-quality care home. They are the (a) time staff spend attending to individual care recipient needs (Care Time), (b) home-like and non-institutional shared spaces in the RACS (such as dining rooms or sitting rooms) (Shared Spaces), (c) home-like and non-institutional room of the person's own (Own Room), (d) easily accessible outside and garden spaces (Outside and Gardens), (e) support for participation in meaningful activities (Meaningful Activities), and (f) flexibility in the time that care activities are undertaken (Care Flexibility).
An important component of the development of a new instrument such as the CCI-6D is consideration of the validity of the measure in different settings and with different populations. The CCI-6D measure has demonstrated good content and construct validity during its development among a small sample of 68 residents including a majority (44) without cognitive impairment, and a minority (only 24) with mild or moderate cognitive impairment answering on their own behalf, as well as a larger sample (185) family members answering on behalf of residents with more severe cognitive impairment (Milte et al., Reference Milte, Ratcliffe, Bradley, Shulver and Crotty2019). The measure has a weighted scoring algorithm developed (Milte et al., Reference Milte, Ratcliffe, Chen and Crotty2018) and has been applied in practice to understand the quality of care of home-like clustered domestic and standard residential care in Australia which included 541 participants with the CCI-6D self-completed by 150 residents (including 57 with cognitive impairment and 93 participants without cognitive impairment) (Gnanamanickam et al., Reference Gnanamanickam, Dyer, Milte, Liu, Ratcliffe and Crotty2019).
However, its external validity in a large-scale sample of residents of a large range of cognitive abilities (including severe dementia) has not been tested before. Understanding in detail the validity of an instrument for measuring quality of care self-reported by residents themselves, including those with different levels of cognitive impairment, is critical for the widespread application (i.e. generalisability) of the measure in quality assessment programmes for long-term care systems. Therefore, the aim of this study was to assess the external validity in terms of assessing convergent and known-group validity of the CCI-6D measure in a new sample of residents which includes a large proportion of residents with cognitive impairment.
This paper will focus on providing further evidence in a new sample for using the CCI-6D for self-completion in practice in a large sample of residents including a significant proportion with cognitive impairment and including evidence for the construct validity of the dimensions of the CCI-6D and the preference weighted scoring algorithm among this group. We have a number of expected relationships between the CCI-6D and other socio-demographic and clinical variables. The associations presented were tested systematically using the approach as described below (Table 1).
Variables included and expected associations
Notes: ‘+’ indicates the variables are expected to exhibit a positive relationship; ‘−’ indicates the variables are expected to exhibit a negative relationship; ‘None’ indicates there is expected to be no relationship between the variables. CCI-6D: Consumer Choice Index Six Dimension. QOL-AD: Quality of Life in Alzheimer's Disease.
Where the hypothesised relationships are upheld, this provides evidence of the construct validity of the CCI-6D overall, and the preference weighted scoring algorithm (for which the validity has not been separately assessed previously). Secondly, where the hypothesised relationships hold across different severity levels of dementia, this provides important new evidence that the measure is valid across the range of people living with dementia and cognitive impairment in residential aged care.
Methods
This study is part of a larger study aiming to investigate the health outcomes and frailty status of older people living in residential aged care over a three-year period (Jadczak et al., Reference Jadczak, Robson, Cooper, Bell and Visvanathan2021). The methods have previously been described in detail for the larger study, but will be briefly described below. The study received approval from the University of Adelaide Human Research Ethics Committee, the South Australian Department for Health and Wellbeing Human Research Ethics Committee, the Department of Human Services External Request Evaluation Committee, and was registered with the Monash University Human Research Ethics Committee. All residents (or their substitute decision maker) provided written informed consent.
Participants
Older people living in 12 RACS (all belonging to one aged care provider) across South Australia were invited to participate. They were eligible if they were permanent residents who had been living in the facility for at least eight weeks. Residents who were medically unstable, currently receiving end-stage palliative care or not fluent in English were excluded. Where possible, informed consent was sought from the residents themselves to participate. The ability of the resident to provide informed consent was assessed. Where residents were not able to consent to the study on their own behalf (due to severe cognitive impairment as indicated by a Psychogeriatric Assessment Scale cognitive assessment score ≥ 16, or less commonly via information or judgement of clinical staff members), consent was sought from a substitute decision maker (e.g. a family member or close friend). For the current study, only the reports of residents who completed the questionnaires on their own behalf are included.
Data collection
Trained registered research nurses collected data from residents, their service records and nurses on site with good knowledge of the resident during the period March to October 2019.
The quality of care provided was assessed using the CCI-6D instrument (Milte et al., Reference Milte, Ratcliffe, Bradley, Shulver and Crotty2019). The CCI-6D is a six-item instrument designed to measure quality of care across six domains identified as important for people with dementia living in RACS. The CCI-6D has a scoring algorithm which can be applied based on the preferences of a sample of residents living in RACS, providing a score between 0 and 1, where 0 indicates the worst possible quality of care and 1 indicates the best possible quality of care (Milte et al., Reference Milte, Ratcliffe, Chen and Crotty2018).
Data collected from service records included socio-demographic information, weight and height of the resident. Activity of daily living (ADL) status was assessed using the Katz ADL Scale where a score of 6 indicates full function, 4 indicates moderate impairment and a score of 2 or less indicates severe functional impairment (Katz et al., Reference Katz, Ford, Moskowitz, Jackson and Jaffe1963). The frailty status of the residents was assessed using the Frailty Index, which is a 60-item index constructed from the baseline characteristics of the residents using a standard methodology as described previously (Jadczak et al., Reference Jadczak, Robson, Cooper, Bell and Visvanathan2021). Variables included in the index included items relating to co-morbid conditions, deficits in ADL and functioning, nutritional status, pain and dementia severity. The index is scored such that where a resident experiences a deficit, the Frailty Index variable is coded as 1 (and 0 for where the resident does not experience that deficit), divided by the total number of Frailty Index variables available; scores fall between 0 and ≤0.1 (non-frail), >0.1 and ≤0.2 (vulnerable), >0.21 and <0.45 (frail) and ≥0.45 (most frail). The severity of dementia experienced by the residents was assessed using the 12-item Dementia Severity Rating Scale (DSRS) (Clark and Ewbank, Reference Clark and Ewbank1996). Scores can be categorised as no dementia (0–11), mild dementia (12–18), moderate dementia (19–36) and severe dementia (37–54) (Moelter et al., Reference Moelter, Glenn, Xie, Chittams, Clark, Watson and Arnold2015). The Personal Wellbeing Index is used to assess quality of life across seven domains: standard of living, health, achieving in life, relationships, safety, community-connectedness and future security (International Wellbeing Group, 2013). Scores range from 0 to 70 with higher scores indicating higher satisfaction. Quality of life was also assessed using a dementia-specific quality-of-life instrument, the Quality of Life in Alzheimer's Disease (QOL-AD) (Logsdon et al., Reference Logsdon, Gibbons, McCurry and Teri2002). While a range of potential quality-of-life instruments are available which could be used, well-known measures such as the EQ-5D-5L, or Short Form (SF-12 or SF-36) suite of measures (Brazier et al., Reference Brazier, Ratcliffe, Salomon, Tsuchiya, Brazier, Ratcliffe, Salomon and Tsuchiya2017) are often focused narrowly on health status, despite quality of life encompassing a larger range of potential domains, including psychosocial wellbeing, relationships or engagement with broader society. These measures are also often developed using expert opinion or with members of the general population, who may also have different opinions about what is important to include in a quality-of-life measure, compared to older people with lived experience of dementia or RACS. Relatively few have been applied or validated for use in RACS, or with people living with dementia (Bulamu et al., Reference Bulamu, Kaambwa and Ratcliffe2015; Cleland et al., Reference Cleland, Hutchinson, Khadka, Milte and Ratcliffe2019; Siette et al., Reference Siette, Knaggs, Zurynski, Ratcliffe, Dodds and Westbrook2021). The QOL-AD is one quality-of-life measure which was developed specifically for use with people living with dementia, which covers a range of domains identified to be of importance to people living with dementia (e.g. social connection, environment, personhood, autonomy) as well as physical health. Given the high proportion of people living with dementia in residential aged care, a dementia-specific quality-of-life instrument was needed for this study, and the QOL-AD is among the most widely used instrument for this purpose (Ayton et al., Reference Ayton, Gardam, Pritchard, Ruseckaite, Ryan, Robinson, Brodaty, Ward and Ahern2021; Siette et al., Reference Siette, Knaggs, Zurynski, Ratcliffe, Dodds and Westbrook2021). Secondly, the QOL-AD covers a diverse range of domains including psychosocial domains which were considered to be relevant for the validation given the foundation of the CCI-6D in concepts of person-centred care. The QOL-AD is also a ‘generic preference-based instrument’, meaning it is also able to be used to generate a standardised outcome measure used in economic evaluations to consider the cost-effectiveness of health and social care interventions called a quality-adjusted life year or QALY for short. The QOL-AD utility score can be generated from five dimensions of the question: physical health, mood, memory, living situation and ability to do things for fun, valued with 1,999 members of the general population using a discrete choice experiment, on a scale between 0 and 1, where 0 indicates a health state equal to death and 1 indicates full health (Comans et al., Reference Comans, Nguyen, Ratcliffe, Rowen and Mulhern2020).
Statistical analysis
Descriptive statistics including frequency of responses across the categories for categorical variables and means and standard deviations or median and interquartile range were estimated for continuous variables. Any differences in demographic and clinical characteristics between the participants with and without a self-completed CCI-6D questionnaire was assessed using the Mann–Whitney U test for continuous variables and the chi-squared test for categorical variables. The normality of the distribution of the preference-weighted scores for the CCI-6D and QOL-AD were assessed using the Shapiro–Wilk test, and the results rejected the hypothesis that the utilities were normally distributed at the 5 per cent level. The responses to the CCI-6D are presented for the entire sample and for the sample split according to the Severity of Dementia in Respondents, as indicated by the DSRS split according to the categories no dementia, mild dementia, moderate dementia and severe dementia. The chi-squared test was used to determine differences in the response to the CCI-6D items across the subgroups. The Kruskal–Wallis test was used to compare the utilities for subgroups according to DSRS with pairwise comparisons undertaken with Bonferroni correction for multiple tests as appropriate. The strength of the correlations between the individual items of the CCI-6D and the QOL-AD and for the utilities of each instrument were assessed using the Spearman correlation coefficients. Following Cohen, a correlation coefficient of less than 0.3 was considered weak, 0.3–0.5 was considered moderate, 0.5–0.9 strong and more than 0.9 very strong (Cohen, Reference Cohen1988). The distribution of the CCI-6D index scores across the levels of the QOL-AD and vice versa were also explored using the Kruskal–Wallis test with pairwise comparisons undertaken with Bonferroni correction for multiple tests as appropriate. Statistical analysis was undertaken using SPSS version 27 (IBM Corp., Armonk, NY, USA).
Results
Sample demographics
In total, 446 residents living in aged care completed the CCI-6D measure and are included in this analysis; 115 residents who did not self-complete a CCI-6D were not included in this analysis.
Table 2 presents the demographic information for the 446 residents who completed a CCI-6D compared with the 115 residents who did not, usually because the cognitive impairment of the resident was too severe to be able to self-respond to the questionnaire (e.g. they were not able to utter words). For the participants who completed a CCI-6D, the majority of the participants were female (73%, N = 327) and were born in Australia (77%, N = 345). The highest proportion were aged between 85 and 94 (59%, N = 58.7), but significant proportions were also aged between 75 and 84 (19%, N = 83) and 95 years and older (16%, N = 70). In terms of cognitive impairment, the highest proportion (38.4%, N = 171) were experiencing moderate dementia based on the DSRS category, while a significant proportion were not experiencing dementia (30%, N = 132) or had mild dementia (20.4%, N = 91). A small proportion (N = 51, 11.5%) of the 446 who completed the CCI-6D were experiencing severe dementia. Of the 128 participants who participated in the study who were living with severe dementia, 51 (40%) completed the CCI-6D themselves, while 77 (60%) were not able to complete the CCI-6D. A high proportion of the participants were classified as frail and vulnerable (84%, N = 370) (according to the Frailty Index), while 16.3 per cent (N = 72) were classified in the ‘most frail’ category. When compared to those without a self-completed CCI-6D, the participants were more likely to have a lower Body Mass Index (chi-squared, 12.806, p = 0.005), have a lower total score on the QOL-AD (0.360, p ≤ 0.001), lower level of cognitive impairment (0.526, p ≤ 0.001), and higher level of frailty (128.066, p ≤ 0.001), but were similar in age, the proportion of females and proportion of those born in Australia in the sample.
Socio-demographic characteristics of participants
Notes: CCI-6D: Consumer Choice Index Six Dimension. BMI: Body Mass Index. QOL-AD: Quality of Life in Alzheimer's Disease. SD: standard deviation. IQR: interquartile range. DSRS: Dementia Severity Rating Scale. PWI: Personal Wellbeing Index.
Significance level: p < 0.005, except gender and age.
Responses to the CCI-6D items
Table 3 provides the responses to the CCI-6D for the entire sample, and for participants according to their level of cognitive impairment. A high proportion of participants reported that they felt very at home in their own room (82%, N = 367). Responses to other items were more varied. For example, over half of the participants reported that they could get access to outside and gardens whenever they wanted (64.8%, N = 289), and they felt very at home in the shared spaces (61%, N = 272). A lower proportion of participants reported that care staff were always able to spend enough time attending to their individual needs (48.7%, N = 217), or that they could do things that made them feel valued often (36.5%, N = 163). A large proportion of the participants reported that the care routines were very flexible (51.3%, N = 229).
Responses to the Consumer Choice Index Six Dimension (CCI-6D) questionnaire by the total sample and sample split according to dementia severity
Note: N = 446.
Significance level: p < 0.005, except Care Time.
Testing of expected associations with dementia
Table 3 also provides the responses for the CCI-6D across categories of dementia severity. As outlined in Table 1, we expected a negative relationship between quality of care measured by the CCI-6D and the severity of dementia experienced by the resident, as the increasing severity of dementia may make it more likely that poor-quality care is delivered to the resident (Bravo et al., Reference Bravo, De Wals, Dubois and Charpentier1999). There was a significant association between the DSRS categorisation and responses to the CCI-6D items Shared Spaces (p = 0.017), Own Room (p = 0.007), access to Outside and Gardens (p = 0.001), Meaningful Activities (p = 0.004) and Care Flexibility (p = 0.012), with those with moderate or severe dementia more likely to report poorer quality of care. For example, rare or occasional access to things to do that made the person feel valued was more likely to be reported by those with moderate (42.1%, N = 72) or severe dementia (31.4%, N = 16) than those without dementia (19.7%, N = 26) or with mild (27.5%, N = 25) dementia.
Table 4 presents the CCI-6D index score for subgroups of participants split according to severity of dementia as determined by the DSRS. The Kruskal–Wallis test provided strong evidence of a significant difference in distribution across the categories of cognitive impairment (p ≤ 0.001). Pairwise comparisons using the Bonferroni correction for multiple tests indicated that the CCI-6D scores for the moderate dementia group were significantly lower than for the mild dementia group and the no dementia group (p ≤ 0.001). There were no significant differences in CCI-6D scores between the other groups (p > 0.05).
Mean values and standard deviations (SD) and median values and interquartile ranges (IQR) of the Consumer Choice Index Six Dimension (CCI-6D) index according to the Dementia Severity Rating Scale
Notes: N = 446. Kruskal–Wallis test: 18.942 (degrees of freedom = 3), p < 0.000.
Testing expected associations with overall quality of life and self-concept
As outlined in Table 1, we expected a positive relationship between the overall score of the QOL-AD and quality of care for each of the dimensions of the CCI-6D and the overall CCI-6D weighted score.
As expected, the Spearman correlation coefficient between the index score of the CCI-6D and utility score of the QOL-AD was 0.459 (p ≤ 0.001), indicating moderate correlation.
Table S1 in the online supplementary material presents the mean QOL-AD utility score across the dimensions of the CCI-6D measure. The mean QOL-AD utility score also generally follows an expected progression across the levels of each dimension of the CCI-6D measure, with the lowest mean utility for the item level identified as the poorest quality care, and the highest mean utility for the item identified as the best quality care. For example, the highest mean QOL-AD utility was for the ‘Care staff are always able to spend enough time attending to my individual needs’ (0.689) as compared to the level ‘Care staff are rarely able to spend enough time attending to my individual needs’ (0.355). The lowest mean QOL-AD utility score was reported for participants indicating that they felt at home in their own room only rarely (0.219), followed by those who felt at home in shared spaces of the aged-care home rarely (0.311). The largest difference in mean QOL-AD utility scores between the levels indicating the poorest quality of care and the best quality of care for the CCI-6D was similarly seen in the items of feeling at home in their own room (a difference of 0.421) and in the shared spaces of the facility (0.340). The differences in the mean QOL-AD scores across the CCI-6D items reached statistical significance for all of the dimensions (p ≤ 0.001), excepting for Meaningful Activities, which approached statistical significance (p = 0.054).
Table S2 in the online supplementary material provides the responses to the items for the QOL-AD questionnaire for respondents who completed the CCI-6D and the distribution of mean CCI-6D index score across the dimensions of the QOL-AD measure. The mean CCI-6D index score generally follows an expected progression across the levels of each dimension of the QOL-AD, with the lowest value for the ‘poor’ level of the dimension, increasing until the highest value is for the ‘excellent’. As expected, there were significant relationships between the QOL-AD items of Whole Self and Life as a Whole and the CCI-6D index score (p ≤ 0.001).
Table 5 presents the correlation coefficients for the dimensions of the QOL-AD and the CCI-6D. As expected there was a weak positive correlation between the QOL-AD dimension of Whole Self and the CCI-6D dimensions of Care Time (0.230, p ≤ 0.001) and Meaningful Activities (0.218, p ≤ 0.001). There was a moderate positive correlation between the QOL-AD dimension of Life as a Whole and the CCI-6D item of Shared Space (0.323, p ≤ 0.001), and weak positive correlations between the QOL-AD dimension and all the other CCI-6D items.
Correlations between items of the Consumer Choice Index Six Dimension (CCI-6D) and Quality of Life in Alzheimer's Disease (QOL-AD)
Significance levels: * p < 0.05, ** p ≤ 0.001.
Testing expected associations with psychosocial wellbeing – living situation, mood and fun
As outlined in Table 1, we expected a positive relationship between the QOL-AD item of Mood and all items of the CCI-6D, due to the impact a higher-quality care environment is hypothesised to have on the mood of the participant. As expected, there was a moderate positive correlation between the QOL-AD Mood dimension and the CCI-6D dimension of Staff Time (0.301, p ≤ 0.001), and weak positive correlations with the CCI-6D dimensions of Shared Space (0.277, p ≤ 0.001), Own Room (0.273, p ≤ 0.001), Meaningful Activities (0.216, p ≤ 0.001) and Flexibility in Care Routines (0.216, p ≤ 0.001). Unexpectedly, the relationship between the Mood dimension and the CCI-6D dimension Outside and Gardens was small, although it did reach statistical significance (0.099, p < 0.05).
We expected a positive relationship between the QOL-AD item of Living Situation and the CCI-6D items of home Shared Spaces and Own Room, and Outside and Gardens, due to the expected positive relationship between having a home-like environment and access to outside and gardens and overall perception of their living environment. As expected, there was a moderate positive relationship identified between the QOL-AD Living Situation dimension and the CCI-6D dimension of Shared Space (0.374, p ≤ 0.01), and a weak positive relationship with CCI-6D dimensions Own Room (0.289, p ≤ 0.001) and Outside and Gardens (0.152, p < 0.05).
We also expected a positive relationship between the QOL-AD item of Fun and the CCI-6D item of Meaningful Activities, due to the expected impact of having access to meaningful activities familiar to and enjoyed by the participant on their perception of their ability to do things for fun. As expected, we identified a moderate positive relationship between the QOL-AD item of Fun and the CCI-6D item of Meaningful Activities (0.333, p ≤ 0.001).
There was a strong relationship between the CCI-6D index scores and the QOL-AD items of Mood, Life as a Whole and Fun (p ≤ 0.001). The largest difference in mean CCI-6D scores between the ‘excellent’ and ‘poor’ response levels were seen in the dimensions of Living Situation (difference of 0.403) and Life as a Whole (difference of 0.320). There were three occasions where the mean CCI-6D utility value for the ‘excellent’ level of the item was lower than that for the preceding item (i.e. ‘good’) for the items Money, Friends and Memory.
Testing expected associations with physical impact of ageing
We expected a negative relationship between older age and physical function and quality of care, as we expected increasing age and poorer physical function (as measured by the Katz ADL Scale) may be associated with increasing complexity of the care required by the individual, making it more difficult to achieve quality care. Despite this, there was no significant correlation between the CCI-6D index score and age (0.013, p = 0.780) or physical function measured by the total Katz score (0.042, p = 0.380).
We expected a positive relationship between the QOL-AD item of Physical Health and the CCI-6D Outside and Gardens and Meaningful Activities items due to the positive impact of having better physical health on being able to independently access outside and gardens and participate in activities. As expected, we saw a weak positive relationship (0.218, p ≤ 0.01) for both the CCI-6D Outside and Gardens item and the CCI-6D Meaningful Activities item (0.153, p ≤ 0.01) and the Physical Health item of the QOL-AD. Unexpectedly, we saw statistically significant relationships also between the Physical Health item and the other CCI-6D dimensions, although these correlations were also weak in magnitude. As outlined in Table S2 in the online supplementary material, the mean CCI-6D index score showed a positive relationship with the Physical Health item of the QOL-AD (p < 0.001).
Discussion
We were able to apply the CCI-6D in a sample of 446 people living in residential aged care. Significantly, this sample included a large proportion of people experiencing mild (38.4%) and moderate (20.4%) dementia who self-reported their quality of care using the instrument. Table 2 includes the demographic information for the 115 participants who did self-complete the CCI-6D. Of the 128 living with severe dementia, 51 (40%) completed the CCI-6D themselves, while 77 (60%) were not able to complete the CCI-6D. The majority of those responding with severe dementia to the CCI-6D had scores in the lower range of the category closest to the moderate category (i.e. a DSRS between 37 and 45, N = 37 of the 51, 72%). It should be noted that those with more severe dementia, as rated by the DSRS, experience limitations such as being completely unaware of time and place, inability to responding to speech, full assistance with feeding, etc. For this group, self-report via an instrument such as the CCI-6D is not likely to be possible, and therefore other means are necessary to include their perspective, such as using a proxy respondent (such as a close family member) to respond on their behalf. Proxy responses were not included in this study, as the focus was on self-completion among people with dementia specifically.
We expected to find a relationship between quality of care, and physical function and age, but did not find a relationship in our sample. We did identify a relationship as expected with physical health and the CCI-6D items of Outside and Gardens and Meaningful Activities. We also identified relationships between the other CCI-6D items and physical health, although these relationships were weak in nature. We did identify a positive relationship between the physical health item of the QOL-AD and the CCI-6D index score.
We also demonstrated a strong relationship between self-reported quality of care and quality of life in the current study. As expected, we identified a moderate correlation between the CCI-6D index score and the QOL-AD utility score, and between some individual items of the QOL-AD and the CCI-6D and the CCI-6D index score and QOL-AD utility score, respectively, as expected. Particularly, we identified relationships as expected between quality of care measured by the CCI-6D index score and QOL-AD dimensions of Mood, Living Situation, Whole Self, Fun and Life as a Whole. We also identified expected relationships between quality of life measured by the QOL-AD utility score and CCI-6D items relating to care time with residents, home-like shared spaces and own room, access to outside and gardens, and care flexibility. We also identified relationships between the CCI-6D index score and the QOL-AD dimensions of Family and Marriage which we did not outline in an expected hypothesis. The reason for this is unclear, but it could be that having better quality of care takes away some stress or additional burden upon family members or spouse to fulfil a formal ‘caring’ role, leading to them being able to better fulfil their informal role as a family member.
At the dimension level, we identified as expected moderate correlations between the CCI-6D item of Care Time and the QOL-AD dimension of Mood. We also identified relationships between the CCI-6D item of Care Time and QOL-AD dimensions of Chores and Fun. While not outlined as an expected hypothesis, these relationships make logical sense, where additional time that care staff spend with residents supports daily chores in their rooms being done and that having staff spend adequate time to support personal care and dressing, or mobilising, means residents feel more comfortable or have the time to access other activities.
We also identified expected relationships between the CCI-6D item of Shared Spaces and QOL-AD dimensions of Living Situation and Life as a Whole, and the CCI-6D item of Meaningful Activities and the QOL-AD dimension of Fun. Also identified were statistically significant moderate correlations between the CCI-6D item of Outside and Gardens and the QOL-AD dimensions of Chores and Fun which were not outlined as hypotheses. These findings indicate that while quality of care and quality of life are related, they are also separate entities.
There were significant differences in responses to the CCI-6D by participants according to the level of dementia experienced for the items of Shared Spaces, Own Room, Outside and Gardens, Meaningful Activities and Care Flexibility. As expected, the CCI-6D index scores were significantly lower for the group with moderate dementia, as compared to the mild or no dementia groups. Although the CCI-6D score for participants with severe cognitive impairment appeared to be slightly higher than for those with moderate cognitive impairment, this did not reach statistical significance. This could be related to the methods of the study – only those participants able to self-respond to the CCI-6D were included in the study, which was a minority of people living with severe dementia in the facility. It may be that when the quality of care of those with severe dementia who were not self-reporting their own quality of care is also considered, a more representative view of the quality of care provided to the entire population is provided, which may be more in line with the expected hypothesis. This highlights the importance of including the perspectives of people with severe cognitive impairment (including those who are unable to answer on their own behalf) in assessing quality of care, perhaps via relevant proxies (such as family members who have regular contact with the person). There remain significant research questions to be answered regarding proxy rating of important but subjective outcomes such as quality of care, such as who is the best proxy and the agreement between the proxy and how the person themselves would rate the care (Hutchinson et al., Reference Hutchinson, Worley, Khadka, Milte, Cleland and Ratcliffe2022).
The participants reporting that they felt at home in their own room and the shared spaces of the service only rarely, reported to the poorest mean quality of life using the QOL-AD utility score. This indicates that those who do not feel at home in their service are at the greatest risk of poor quality of life. Opportunities to ensure residents feel at home in the facility should be investigated as a key potential promoter of quality of life, e.g. whether improvements to the built environment or psychosocial support provided could assist residents to feel more at home and improve their quality of life overall.
Respondents indicated relatively high proportions of quality care in some domains. A high proportion (65%) indicated they could get outside whenever they wanted, and that they felt very at home in their own room (82%). By comparison, a lower proportion (49%) indicated care staff were always able to spend enough time attending to their individual needs, and they could do things that made them feel valued often (37%). There is some consistency with previously published estimates of quality of care in Australian RACS using the CCI-6D. A previous study of quality of care in 68 residents of aged-care services similarly indicated a high proportion of residents felt very at home in their own room (87%) and had access to outside and gardens whenever they wanted (62%), while a lower proportion indicated they were able to do things to make them feel valued often (52%) or the care staff were always able to spend enough time attending to their individual needs (49%) (Milte et al., Reference Milte, Ratcliffe, Bradley, Shulver and Crotty2019). Previous studies have indicated residents receive as little as 30 minutes per day of direct care time from staff (Bostick et al., Reference Bostick, Rantz, Flesner and Riggs2006; Qian et al., Reference Qian, Yu, Hailey, Zhang, Davy and Nelson2014), with this predominantly taken up by task-based physical care (e.g. assistance with personal care, eating and drinking, or medical care). This leaves little time for staff to spend engaging with residents in a meaningful way to improve their wellbeing. Increasing the time staff are able to spend engaging with residents could improve their access to meaningful activities and promote their levels of wellbeing and improve overall quality of care from the perspective of residents.
A study applying the CCI-6D in a sample of 540 residents (including a subsample living in a clustered domestic model of care specialised for residents with dementia) showed a higher proportion indicated the care routines were very flexible (59.6%) compared to the current study (51%) (Gnanamanickam et al., Reference Gnanamanickam, Dyer, Milte, Liu, Ratcliffe and Crotty2019). By comparison, a lower proportion of respondents indicated they could get outside whenever they wanted (49.6% versus 65% in the current study) and they felt at home in their own room (73.5% versus 82.3% in the current study). Responses to other items were similar. Thus, there appears to be some consistency in the responses to the CCI-6D in the current study, which included a large proportion of respondents with mild or moderate cognitive impairment who self-responded to the questionnaire, to comparison previous studies, which have included relatively high levels of proxy respondents.
Similarly to previous studies which used facility-level or clinically focused measures of care quality (Bravo et al., Reference Bravo, De Wals, Dubois and Charpentier1999; Nazir et al., Reference Nazir, Arling, Perkins and Boustani2011), the current study has demonstrated a relationship between a person's own perception of the quality of care they are receiving and their level of cognitive impairment. Improving information on quality of care among people with dementia living in residential care is critical; despite forming a large proportion of residents in RACS, people with dementia are often routinely excluded from self-reporting their own wellbeing and quality of care (Tolson et al., Reference Tolson, Rolland, Andrieu, Aquino, Beard, Benetos, Berrut, Coll-Planas, Dong, Forette, Franco, Franzoni, Salva, Swagerty, Trabucchi, Vellas, Volicer and Morley2011, Reference Tolson, Dewar and Jackson2014). If residents with cognitive impairment are not included in the process, the current assessment of quality of care is likely to be flawed. Caution needs to be used when extrapolating the assessments of care quality from people without cognitive impairment as compared to those with cognitive impairment, given available evidence that those with cognitive impairment are particularly vulnerable to poor quality of care and quality of life (Bravo et al., Reference Bravo, De Wals, Dubois and Charpentier1999; Edelman et al., Reference Edelman, Kuhn and Fulton2004; Tolson et al., Reference Tolson, Dewar and Jackson2014).
When considering the Donabedian Model of quality of care, it has long been hypothesised that an improvement in the ‘processes’ of care (such as a high-quality care environment and care staff) should lead to an improvement in the ‘outcomes’ of care (e.g. improved quality of life among residents). However, this has often been difficult to demonstrate in practice. Kim et al. (Reference Kim, Park, Kim, Nakagawa, Lung, Choi J, Ryu, Min, Shin, Kim and Yoo2014) failed to demonstrate a relationship between quality of care indicators and quality of life in their study of 316 residents without cognitive impairment in the USA. This could be related to the type of quality of care indicator used in this study, which was the Nursing Home Compare 5-Star Quality System, which includes factors such as the rate of hospitalisations and emergency department stays, falls, pressure ulcers, urinary tract infections, use of antipsychotics, catheters, decline in mobility and ADL. While high-quality clinical care is essential to maintain the health status of residents, it has been proposed that clinical measures of care are inadequate to reflect the quality of care in a setting such as long-term care where the care incorporates the majority of day-to-day life experiences of residents for an extended period of time, in a way that is inherently different from acute care settings (Noelker and Harel, Reference Noelker, Harel, Noelker and Harel2000). By comparison, the current study was able to demonstrate an association between the quality of the care environment, and the interactions between the care staff and residents and quality of life among residents, including a large proportion with cognitive impairment. This supports a previous study which also found a strong relationship between quality of life and the processes of care delivery measured by the CCI-6D (Milte et al., Reference Milte, Ratcliffe, Bradley, Shulver and Crotty2019). This highlights the importance of considering a resident's own viewpoint in quality of care assessment.
Strengths and limitations
Despite its strengths, this study has some limitations which should be considered. For this study, we were unable to access CCI-6D responses for proportion of participants with very severe dementia who were unable to answer the questionnaire on their own behalf. We were unable to access assessments via a proxy report (e.g. a close family member) on behalf of the residents for this study. Therefore, we are unable to draw any conclusions on using the CCI-6D with participants with very severe cognitive impairment from this study. A significant proportion of residents with more severe cognitive impairment (N = 185) were included in the original validation of the CCI-6D with proxy respondents (usually family members) answering on their own behalf (Milte et al., Reference Milte, Ratcliffe, Bradley, Shulver and Crotty2019). This study demonstrated strong to moderate evidence of construct validity of the various items of the CCI-6D. Additionally, the current study applied the CCI-6D at one time-point. Therefore, we are unable to comment on potential changes in the CCI-6D scores within individuals over time. The ability of the CCI-6D to reflect changes in an individual resident's reported quality of care over time is an important point to consider for the routine evaluation of the quality of care received in RACS and should be investigated in future studies.
Conclusion
This study has provided further evidence supporting the use of the CCI-6D with older people living with dementia to report the quality of care they are receiving. Concerningly, although as expected, we identified that those with moderate dementia reported experiencing poorer quality care than those with no or mild dementia. Given the critical impact that a poor-quality RACS could have on the wellbeing of a person living with dementia, future assessment of quality of care should involve the voice of people living with dementia with lived experience.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S0144686X23000247.
Acknowledgements
We would like to acknowledge Terry Shortt for supporting the study co-ordination and contributing to the study design, as well as Hilary Ashby who supported the study co-ordination. We would also like to thank our study nurses, Jean Zhang, Victoria Foy and Kim Testi, who conducted the data collection and collected informed consent. We would also like to acknowledge Tina Cooper and Leonie Robson for contributing to the study design, and thank Tina Cooper and Merridy Baylis for critically reviewing the manuscript.
Author contributions
RKM was involved in study design, undertook data analysis and wrote the manuscript. ADJ was involved in study design, project management, data collection and revised the manuscript. JR was involved in study design, interpretation of the data and revised the manuscript. RV was involved in study design, project management, acquisition of funding, data collection and revised the manuscript.
Financial support
This work was supported by the South Australian Department for Innovation and Skills; the Hospital Research Foundation; the National Health and Medical Research Council of Australia through the Centre of Research Excellence Scheme (Project ID 1102208); and Resthaven Inc. through the GTRAC-Resthaven Research Grants Scheme. Apart from Resthaven Inc., the funders did not have a role in the study design, operations or analysis.
Competing interests
RV was previously a board member of Resthaven Inc. and is currently on the clinical governance committee. In the recent past, she has received honorarium, speaker and educational grants in various combinations from Nutricia, Abbott and Nestle. JR and RKM have received funding from organisations ECH, Helping Hand, Uniting Age Well, Uniting ACT NSW and Presbyterian Aged Care as part of the Australian Research Council Linkage Project (grant number LP170100664). The authors declare that they have no other competing interests.
Ethical standards
The study received approval from the University of Adelaide Human Research Ethics Committee (HREC-2018-247), the South Australian Department for Health and Wellbeing Human Research Ethics Committee (HREC/20/SAH/15), the Department of Human Services External Request Evaluation Committee (EREC/RMS0432), and was registered with the Monash University Human Research Ethics Committee. All residents (or their substitute decision maker) provided written informed consent. All necessary institutional review boards gave ethical approval to conduct the study.
Open access
