Introduction
Nursing homes are striving to become more person-centred (Koren, Reference Koren2010). When providing person-centred care (PCC), nursing staff connect with older adults as unique individuals with their own personal experiences and preferences, rather than viewing them as patients and emphasising illness (Edvardsson et al., Reference Edvardsson, Winblad and Sandman2008; Edvardsson, Reference Edvardsson2015). Providing PCC can have important positive outcomes, like improved resident wellbeing (McCormack and McCance, Reference McCormack and McCance2010; Rathert et al., Reference Rathert, Wyrwich and Boren2013). However, it can be difficult to implement PCC innovations in the highly pressured residential aged-care settings (Cooper et al., Reference Cooper, Marston, Barber, Livingston, Rapaport, Higgs and Livingston2018). Progress towards providing PCC can be monitored by regularly assessing and documenting resident wellbeing. The current study investigates how well nursing staff are able to make assessments of resident wellbeing.
Wellbeing can be defined in various ways. According to the hedonic tradition, wellbeing is conceptualised as happiness: experiencing balanced positive and negative emotions, and being satisfied with life (Diener et al., Reference Diener, Suh, Lucas and Smith1999). In this tradition wellbeing is thus mostly concerned with feeling good. Second, the eudaimonic approach describes wellbeing as an active process of living well: living a complete life and realising human potentials (Ryan and Deci, Reference Ryan and Deci2001). The concept of engagement fits in this tradition, in which a person is absorbed in an activity, to the point of forgetting time, fatigue and everything else but the activity itself, described as an experience of flow (Csikszentmihalyi et al., Reference Csikszentmihalyi, Nakamura, Abuhamdeh and Csikszentmihalyi2015). A comprehensive definition of wellbeing includes both feeling good (e.g. happiness) and living well (e.g. engagement) (Keyes, Reference Keyes2002).
Wellbeing is seen as an important variable, however, nursing documentation is currently almost exclusively dedicated to describing physical care (Broderick and Coffey, Reference Broderick and Coffey2013). Residents may be the most valid source of their subjective experiences of happiness and engagement, but dementia and other physical disorders can impede introspection and communication. Nursing staff have daily contact with the residents, and can alternatively provide proxy assessments of resident happiness and engagement. In general, people are able to estimate the wellbeing of others to some extent, although assessments are far from perfect (Schneider and Schimmack, Reference Schneider and Schimmack2009). In the nursing home context, some studies have examined the value of proxy assessments, but mostly for the – more broad – concept of quality of life (QoL). These studies have shown inconclusive results. Low to moderate agreement is shown between proxy-assessed and self-reported QoL (e.g. Devine et al., Reference Devine, Taylor, Spencer, Diaz-Ordaz, Eldridge and Underwood2014; Usman et al., Reference Usman, Lewis, Hinsliff-Smith, Long, Housley, Jordan, Gage, Denning, Gladman and Gordon2019), nursing staff seem to both overestimate (Kane et al., Reference Kane, Kane, Bershadsky, Degenholtz, Kling, Totten and Jung2005) and underestimate residents’ QoL (Beer et al., Reference Beer, Flicker, Horner, Bretland, Scherer, Lautenschlager, Schaper and Almeida2010; Griffiths et al., Reference Griffiths, Smith, Martin, Meads, Kelley and Surr2020). Especially the more subjective components of QoL may be difficult to assess by proxies (Diaz-Redondo et al., Reference Diaz-Redondo, Rodríguez-Blázquez, Ayala, Martinez-Martin and Forjaz2014), which may be most comparable to wellbeing. However, many of these QoL assessment studies included proxy assessment instruments that did not always correspond closely to those used for self-reports and were often conducted with older adults with dementia. This leaves the possibility that high discrepancies were caused by the dissimilarity between instruments or the introspective problems of the residents with dementia (Gerritsen et al., Reference Gerritsen, Steverink, Ooms, De Vet and Ribbe2007; Hounsome et al., Reference Hounsome, Orrell and Edwards2011). However, studies in palliative care have also shown that there is systematic overestimation of physical symptoms and underestimation of psychological wellbeing by staff in comparison to patients (Murtagh et al., Reference Murtagh, Ramsenthaler, Firth, Groeneveld, Lovell, Simon, Denzel, Guo, Bernhardt, Schildmann, van Oorschot, Hodiamont, Streitwieser, Higginson and Bausewein2019).
These inconclusive findings may also rise from nursing staff differing in their ability to assess the wellbeing of residents adequately. Several nursing staff usually have daily contact with a resident, and previous studies have found both sufficient (Ettema et al., Reference Ettema, Dröes, de Lange, Mellenbergh and Ribbe2007) and insufficient agreement between colleagues in their assessments of resident QoL (Dichter et al., Reference Dichter, Schwab, Meyer, Bartholomeyczik, Dortmann and Halek2014). An important condition for valid proxy assessments of resident wellbeing is that nursing staff interpret wellbeing assessment items in a similar way (Griffiths et al., Reference Griffiths, Smith, Martin, Meads, Kelley and Surr2020). Others therefore recommended provision of training in the assessment method and a manual with clear and accurate definitions of the items (Dichter et al., Reference Dichter, Schwab, Meyer, Bartholomeyczik, Dortmann and Halek2014). It is important to investigate whether sufficient agreement between colleagues and a similar interpretation of the concepts are indeed found when using such methods.
Variation between colleagues in proxy assessments would also necessitate identifying those care-givers who are best able to assess wellbeing. While a recent systematic review revealed various nursing staff characteristics that were related to how high they assessed QoL (e.g. staff distress, having a contract) (Robertson et al., Reference Robertson, Cooper, Hoe, Hamilton, Stringer and Livingston2017), few studies investigated whether such care-giver factors could also indicate how well they assessed wellbeing in comparison to self-report. Two studies found that nursing staff age, experience in working in elder-care and function were not related to the discrepancy between QoL proxy assessments and self-reports of residents with dementia (Spector and Orrell, Reference Spector and Orrell2006; Gräske et al., Reference Gräske, Fischer, Kuhlmey and Wolf-Ostermann2012). More research is needed on potential nursing staff characteristics that are related to proxy self-report agreement for the wellbeing of residents without dementia.
Besides these issues with agreement, it can also be challenging to actually implement regular wellbeing assessments in the context of the nursing home. The instruments that are currently available for proxy assessments of QoL are often quite lengthy, and there is a need for suitable, accessible and easy-to-use measures (Hughes et al., Reference Hughes, Farina, Page, Tabet and Banerjee2019). Since excessive paperwork can be an important barrier for implementing regular wellbeing assessments in the nursing home context (Griffiths et al., Reference Griffiths, Kelley, Garrod, Perfect, Robinson, Shoesmith, McDermid, Burnley and Surr2019), we propose that short and simple methods like single questions are more suitable for regular monitoring and documentation of resident wellbeing. Single-question self-reports allow for the full expression of residents’ values and preferences (Gill and Feinstein, Reference Gill and Feinstein1994), and have been used as reliable and valid indicators for concepts such as subjective health, subjective age, QoL and life satisfaction (Gerritsen et al., Reference Gerritsen, Steverink, Ooms, De Vet and Ribbe2007; Veenhoven, Reference Veenhoven2008; Westerhof et al., Reference Westerhof, Miche, Brothers, Barrett, Diehl, Montepare, Wahl and Wurm2014; Usman et al., Reference Usman, Lewis, Hinsliff-Smith, Long, Housley, Jordan, Gage, Denning, Gladman and Gordon2019).
The current study is the first to investigate how nursing staff assessments of nursing home resident wellbeing compare to self-reports after training, with a single-question assessment method of happiness and engagement. We explore (a) the level of agreement between colleagues in their proxy assessments, (b) the level of agreement between nursing staff proxy assessments and resident self-reports, (c) potential nursing staff characteristics associated with the level of agreement between proxy assessments and self-reports, and (d) how nursing staff understand the concepts of wellbeing.
Methods
Sample and procedure
This study adopted a cross-sectional mixed-methods design. Data collection was part of a training programme to assess resident wellbeing, developed for a large care organisation in the Netherlands accounting for 17 nursing homes. Two nursing homes of comparable size were included, based on voluntary registration of the managing directors. All eight long-term care units of these nursing homes (four each) participated in this study, each unit consisting of 12–25 physically frail residents and approximately ten nursing staff providing physical care to these residents. Figure 1 shows the flowchart of participants.
Flowchart of participants and available assessments.
Residents
All 135 residents of the participating units received informational letters. Exclusion criteria for the study were: major hearing, speech or cognitive problems and dementia (N = 32), as established by nursing staff. Participation was voluntary upon giving informed consent. Participation was declined by 43 older adults and the self-reports of nine residents were excluded from analyses for various reasons (Figure 1). This resulted in a final sample of 49 residents providing self-reports of their happiness and engagement, assisted by a trained member of the research team consisting of two graduated psychologists and one psychology undergraduate.
Nursing staff
All 83 nursing staff of the participating units received a mandatory training in assessing happiness and engagement, consisting of three face-to-face group meetings of two hours each. In the first session, nursing staff discussed the concepts of happiness and engagement and practised how to observe these in nursing home residents, using video fragments. In the second session, participants practised how to provide proxy assessments of happiness and engagement using video fragments. In the subsequent weeks, nursing staff observed the residents in their unit for two weeks and provided proxy assessments of resident happiness and engagement (see nursing staff measures). Nursing staff could then voluntarily hand in their proxy assessments for one or several residents in their unit as part of the current study. Participation was declined by 28 nursing staff, and the assessments of six nursing staff were excluded from analyses for various reasons (Figure 1). This resulted in a final sample of 49 nursing staff providing proxy assessments of one or more residents. The assessments were discussed in a third return meeting, but the current study only covers the completed and submitted assessment forms.
Combinations of assessments available for analysing agreement
In total, the nursing staff provided 447 proxy assessments relating to 119 residents (see Figure 1). Proxy assessments from two or more nursing staff (maximum eight) were available for 98 residents, resulting in 860 proxy assessment pairs available for computing the level of agreement between colleagues for happiness and engagement. Proxy assessments from one or more nursing staff (maximum eight) were available for the 49 residents who provided self-reports, resulting in 49 combinations available for computing proxy self-report agreement. Nurse characteristics were available for 41 nursing staff who provided proxy assessments for computing associations between nurse characteristics and proxy self-report agreement. Finally, 42 nursing staff provided brief motivations, resulting in 325 brief motivations available for analysing nursing staff understanding of the concepts.
Nursing staff measures
Proxy assessments of happiness and engagement.
Proxy assessments of happiness and engagement were gathered using an adapted version of the Leuven Well-being and Involvement Scale (L-WIS; Laevers, Reference Laevers2005). In the L-WIS, happiness and engagement are rated on two separate 1–5 scales, with higher scores indicating better wellbeing, and a question mark indicating a lack of information to make a clear assessment. This monitoring tool was originally developed for education and child care (Laevers, Reference Laevers2005), and adapted to include detailed descriptions of feelings and behaviours of older adults corresponding to each of the happiness and engagement assessment scores (for descriptions, see the online supplementary material).
Proxy assessments were included only when both a valid happiness and a valid engagement assessment were provided (a question mark was considered an invalid assessment). When nursing staff assigned two scores for one assessment (e.g. both 3 and 4 for wellbeing for resident A), this was coded as the mean of the two scores (e.g. 3.5). The L-WIS assessment form includes space to assign both assessments per resident (up to 25 residents), and to provide some additional qualitative information to support these assessments (i.e. brief motivations) for each resident.
Nursing staff characteristics
Demographic and work-related data (i.e. age, function, hours worked per week, experience and function) were gathered as part of another study.
Resident measurements
Happiness
Self-reported happiness was measured using a single question that matched the L-WIS nursing staff assessment, and with two established happiness questionnaires. The single question of happiness (i.e. ‘All in all, how good did you feel in the previous week including today?’) was answered on a scale from 1 = not good at all to 5 = excellent. Then in line with the L-WIS, a detailed description of feelings corresponding to that score was provided (e.g. for score 5: ‘I felt great, I felt relaxed, I had confidence and I thoroughly enjoyed myself’). Residents indicated whether this described their happiness, or could choose another rating description when better fitting.
The established questionnaire measuring the balance of positive and negative emotions was the Dutch version of the Geriatric Depression Scale (GDS; Jongenelis et al., Reference Jongenelis, Gerritsen, Pot, Beekman, Eisses, Kluiter and Ribbe2007). This scale is specifically designed for nursing home residents, measuring the presence of both positive feelings (four items, e.g. ‘Do you feel happy most of the time?’) and negative feelings (four items, e.g. ‘Do you feel that your life is empty?’), answered with ‘yes’ or ‘no’. A sum score is computed, with high scores indicating more depressive feelings. The scale had a reliability of α = 0.86 in the current sample.
The established questionnaire measuring life satisfaction was the Satisfaction With Life Scale (SWLS; Diener et al., Reference Diener, Emmons, Larsen and Griffin1985; Custers et al., Reference Custers, Westerhof, Kuin and Riksen-Walraven2010). Five items (e.g. ‘I am satisfied with my life’) were scored on a scale from 1 = totally disagree to 5 = totally agree. To improve clarity, the final question was restated into a positively formulated question, without changing the content of the question. This resulted in a satisfactory item-total correlation of 0.53. A sum score was calculated, with higher mean scores indicating greater life satisfaction. The scale has shown good psychometric properties (Pavot and Diener, Reference Pavot and Diener1993). In the current sample, the scale had a reliability of α = 0.84, which is higher than was found previously with the unadjusted version of the scale with older adults (α = 0.69; Custers et al., Reference Custers, Westerhof, Kuin and Riksen-Walraven2010).
Engagement
Self-reported engagement was also measured using both a single question matching the L-WIS care-giver assessment, and an established engagement questionnaire. The single question of engagement (i.e. ‘How often were you completely absorbed in what you did?’) was answered on a scale from 1 = rarely to 5 = most often. Then in line with the L-WIS, a detailed description of feelings corresponding to that score was provided (e.g. for score 5: ‘Most of the time, I was highly concentrated and continuously involved in an activity, which made me forget the time and I could not be distracted’). Residents indicated whether they felt this described their engagement, with the opportunity to choose another rating description when suited.
The established questionnaire measuring engagement with good construct validity and reliability was the leisure scale of the Swedish Flow Proneness Questionnaire (SFPQ; Ullén et al., Reference Ullén, de Manzano, Almeida, Magnusson, Pedersen, Nakamura, Csikszentmihalyi and Madison2012). However, participants indicated that this questionnaire was too difficult, which resulted in many missing values. This measure was therefore omitted during data gathering, and results are not discussed further.
Resident characteristics
Demographic data (i.e. age, gender, marital status, number of children) and subjective health (i.e. ‘How is your health generally?’; scale 1 = poor to 5 = excellent) were additionally self-reported. Independency of activities of daily living (ADL; i.e. bathing, clothing, mobility, toileting, continence and eating) were rated by nursing staff on a scale from 1 = can do this without help to 4 = needs help with all aspects of the activity, using the Katz ADL (Katz et al., Reference Katz, Ford, Moskowitz, Jackson and Jaffe1963). Total scores range from 6 to 24, with higher scores indicating greater ADL dependency.
Analysis
Quantitative data were analysed using IBM SPSS 24, with the alpha level set to 0.05.
Agreement between colleagues
To analyse how well nursing staff assessments correspond between colleagues, the inter-rater agreement between proxy assessments was analysed. First, the proportion of exact agreement between proxy assessments was examined. For each resident, the number of matching assessments (i.e. absolute difference of 0) was divided by the total number of available assessments for that resident (Kottner and Dassen, Reference Kottner and Dassen2008). Second, relative agreement between proxy assessments was examined, which takes into account that scores that are closer together on ordinal five-point proxy scales indicate more agreement. Krippendorff's alpha was examined, using the macro of Hayes and Krippendorff (Reference Hayes and Krippendorff2007), based on 1,000 bootstrapped samples.
Agreement between proxy assessments and self-reports
To analyse how well nursing staff assess wellbeing, the agreement between proxy assessments and self-reports of happiness and engagement was tested. For resident self-reports, missing data on individual items of the established self-reported happiness questionnaires (0.6%) were replaced with the respondent's mean of that scale. Pearson correlations were computed between the single-question self-reported happiness and self-reported GDS (depressive feelings) and SWLS (life satisfaction) scores as a test of concurrent validity. Correlations r < 0.30 are indicated as weak, r < 0.50 moderate and r ≥ 0.50 as strong correlations (Cohen, Reference Cohen1988).
For the proxy assessments, the means of all available assessments of each resident (ranging from one to eight assessments) were calculated. Pearson correlations and paired sample t-tests were then conducted. Furthermore, two-way random effects intra-class correlations (ICC) of absolute consistency with multiple raters were computed. ICC ⩽ 0.50 was indicated as poor, 0.50 < ICC < 0.75 as moderate, 0.75 < ICC < 0.90 as good and ICC ≥ 0.90 as excellent (Koo and Li, Reference Koo and Li2016).
Nurse characteristics and proxy self-report agreement
Further, to analyse whether particular nursing staff are best able to assess resident wellbeing, the associations between nursing staff characteristics and both level of proxy assessment and proxy self-report agreement were tested. Three nursing staff characteristics were included: age, experience working in elder-care and number of working hours per week. For the level of proxy assessments, average assessment scores were calculated for each nursing staff, from all assessments (one to ten) that they provided. Nursing staff mean agreement with self-reports was calculated by averaging the absolute differences between proxy assessments and the corresponding residents’ single-question self-reports for each nursing staff, ranging from 0 = no discrepancy/high agreement to 4 = high discrepancy/low agreement. Pearson correlations were examined for age and experience, and Spearman's rho correlations for the number of hours worked, in relation to average assessment scores and discrepancy scores.
Nursing staff understanding of concepts
Finally, we explored whether nursing staffs’ interpretations of the concepts of happiness and engagement were in line with the definitions presented in the training and the assessment manual. An explorative qualitative analysis was conducted on the brief motivations nursing staff provided on the assessment forms. Information was first categorised as describing either happiness or engagement, based on the interpretation of the researchers. The brief motivations of 26 nursing staff were then open coded in Excel. A coding system of the happiness and engagement codes was created through deliberation between the first and the last author. Using Atlas.ti 8, each brief motivation was then assigned one happiness code and one engagement code. Finally, codes were grouped, based on three main themes of interpretation: (a) matching the definition, (b) different from definition, and (c) unknown. The content of the brief motivations that did not discuss either happiness or engagement was additionally analysed bottom-up.
Results
Participants
Residents
The 49 residents who completed self-reports had a mean age of 84.5 years (standard deviation (SD) = 7.0, range = 56–100). Most residents were female (66%), widowed (68%) and had one or more children (92%). Residents had a reasonable subjective health (mean = 2.4, SD = 0.6, range = 1–4) and moderate ADL dependency (mean = 11.1, SD = 4.6, range = 6–19).
Nursing staff
The 49 nursing staff who provided proxy assessments had a mean age of 38.1 years (SD = 12.4, range = 18–57), most were female (95%), most were licensed practical nurses (91%) and many worked 17–24 hours per week (43%), with on average 13.7 (SD = 11.5) years of experience working in nursing home settings.
Agreement between colleagues
Table 1 shows that proxy assessments about the same resident differed on average less than one point between colleagues. Exact agreement between the proxy assessments occurred in a minority of cases, and Krippendorff's alpha showed low relative agreement.
Mean scores of proxy assessments and difference and agreement between colleagues
Notes: N = 49 colleagues, comparing 860 proxy assessment pairs. CI: confidence interval. SD: standard deviation.
Agreement between proxy assessments and self-reports
Table 2 shows happiness and engagement data for the residents who provided self-reports.
Mean scores, intercorrelations and agreement between nursing staff average proxy assessments and resident self-reports
Notes: Comparing 49 proxy self-report pairs. SD: standard deviation. ICC: intra-class correlation.
Significance levels: * p < 0.05, ** p < 0.01.
Residents reported positive happiness and engagement, with a moderate interrelation. Self-reported happiness was moderately related to life satisfaction and depressive feelings (r = 0.31, p = 0.03 and r = −0.48, p = 0.001, respectively; not in Table 2), indicating concurrent validity. Resident demographics were not related to self-reported happiness or engagement scores (not in Table 2). Nursing staff proxy assessments of resident happiness and engagement also had positive mean scores, and were strongly interrelated.
Agreement
Moderate correlations were found between proxy assessments and self-reports for happiness, and no significant correlations for engagement. For both happiness and engagement, proxy assessments were significantly higher than resident self-reports. The intra-class correlation showed poor consistency between proxy assessments and self-reports for happiness and no consistency for engagement. The number of available proxy assessments for each resident was not related to proxy self-report agreement (not in Table 2).
Nurse characteristics and proxy self-report agreement
For the 41 nursing staff for whom nurse characteristics were available, the average proxy assessments of happiness and engagement had similar positive mean scores, with a wider range for engagement (see Table 3). Mean discrepancy between nursing staffs’ proxy assessments and resident self-reports was bigger for engagement than for happiness. Only the number of hours worked per week was (borderline significantly) related to average happiness assessments (r s = 0.31, p = 0.046), with more hours related to higher assigned happiness scores. None of the nursing staff characteristics were related to the discrepancy scores.
Mean scores of proxy assessments and proxy self-report discrepancy and relations with nursing staff characteristics
Notes: For 41 nursing staff. SD: standard deviation. r: Pearson correlation. r s: Spearman's rho correlation.
Significance level: * p < 0.05.
Nursing staff understanding of concepts
A total of 325 brief motivations were analysed qualitatively on the interpretation of happiness and engagement matching definitions of happiness and engagement presented in the training and the assessment manual.
Matching
Many brief motivations described happiness (54%) and engagement (22%) in line with the training and assessment manual. Some brief motivations included rather rich descriptions of situations in which happiness and engagement were experienced. Happiness was described as residents experiencing enjoyment of social contact and trips outside the nursing home, or sadness over physical problems. Engagement was only a few times described as active participation in specific activities, with focus, fascination and (lack of) concentration: ‘was actively working on knitting socks’.
Different
For happiness, very few descriptions (2%) were not in line with the definition provided in the training, describing happiness as a personal characteristic: being a positive or negative person, rather than a state: ‘is sometimes very difficult and sometimes negative’. Engagement was also described (6%) as a personal characteristic: being an active person, rather than a state. Additionally, many motivations (13%) merely described the attending activities, instead of the level of active participation in these activities and often (12%) motivations described residents expressing interest in other people: ‘always asks how I am doing’.
Unknown
A few times, care-givers merely used the term happiness (2%) or engagement (6%), without any further description.
Unrelated information
A large proportion (42%) of brief motivations included information unrelated to happiness and engagement. About a third of these described characteristics of the residents, such as being independent, talkative and able to communicate, as well as having humour, being friendly, grateful and not wanting to complain. Another third of these brief motivations described the situation of the resident: physical problems (e.g. being tired, having pain, impairments, dementia diagnosis), social contacts and the family situation (e.g. illness of family member, divorce). Residual codes included: stimulating residents to attend (organised) activities, resident autonomy, the need for a volunteer, corrigibility, trusting others and showing variable behaviour.
Discussion
The current study investigated how well nursing staff are able to assess the wellbeing of nursing home residents, using single questions of happiness and engagement. We followed previous recommendations to provide nursing staff with training and detailed scoring manuals (Dichter et al., Reference Dichter, Schwab, Meyer, Bartholomeyczik, Dortmann and Halek2014), only included self-reports of residents without dementia, and used comparable tools for proxy assessments and self-reports. Despite these adaptations, the results showed that nursing staff assessments varied considerably between colleagues, which is in line with Dichter et al. (Reference Dichter, Schwab, Meyer, Bartholomeyczik, Dortmann and Halek2014), but not with Ettema et al. (Reference Ettema, Dröes, de Lange, Mellenbergh and Ribbe2007). Moreover, nursing staff were not sufficiently able to assess resident wellbeing, in line with previous studies showing moderate proxy self-report agreement in the general population (Schneider and Schimmack, Reference Schneider and Schimmack2009), and low to moderate relations for QoL in nursing homes (Devine et al., Reference Devine, Taylor, Spencer, Diaz-Ordaz, Eldridge and Underwood2014). We found that nursing staff overestimated wellbeing compared to self-reports, which is in line with some (Kane et al., Reference Kane, Kane, Bershadsky, Degenholtz, Kling, Totten and Jung2005) but not in line with various other studies of QoL proxy self-report agreement (e.g. Griffiths et al., Reference Griffiths, Smith, Martin, Meads, Kelley and Surr2020). Nursing staff who worked more hours tended to estimate higher resident happiness. Finally, none of the included care-giver factors were related to the discrepancy between care-giver assessments and self-reports (in line with Spector and Orrell, Reference Spector and Orrell2006). There may be several explanations for the current results, which are described below.
One important potential explanation for the low agreement between colleagues lies in nursing staff's varying understanding of the concepts of wellbeing. Even though we provided a manual with a clear and detailed definition of the concepts and nursing staff practised with video fragments, happiness and especially engagement seemed to be interpreted in diverse ways. Engagement was sometimes interpreted as having interest in others, a commonly used alternative definition of engagement in Dutch (‘betrokkenheid’). Different from a recent study showing that nursing staff generally described QoL in terms of the quality of care that was provided (Robertson et al., Reference Robertson, Cooper, Hoe, Lord, Rapaport, Marston, Cousins, Lyketsos and Livingston2020), nursing staff in the current study described happiness and engagement often as resident characteristics (e.g. being a positive or active person). Furthermore, instead of wellbeing, care-givers often described rather objective aspects, like the situation of the resident (e.g. having physical problems). These results could reflect some care-givers’ difficulty with elaborating on observations in words, highlighting their unfamiliarity with discussing psycho-social aspects compared to documenting physical care (Irving et al., Reference Irving, Treacy, Scott, Hyde, Butler and MacNeela2006; Murtagh et al., Reference Murtagh, Ramsenthaler, Firth, Groeneveld, Lovell, Simon, Denzel, Guo, Bernhardt, Schildmann, van Oorschot, Hodiamont, Streitwieser, Higginson and Bausewein2019). This is also an important issue for providing PCC, because viewing wellbeing as a stable trait or part of stable objective aspects may hamper investments towards improving wellbeing. Moreover, this indicates that instead of truly assessing whether residents were feeling good or being absorbed in activities, nursing staff may have observed different concepts.
In a similar vein, residents’ understanding of the concepts may have influenced the proxy self-report agreement results (Bulamu et al., Reference Bulamu, Kaambwa and Ratcliffe2015). The validity of evaluating nursing staff assessments through comparison with residents’ self-reports depends on residents’ ability to assess their own wellbeing (e.g. Kane et al., Reference Kane, Kane, Bershadsky, Degenholtz, Kling, Totten and Jung2005). Evidence for the validity of the single question of self-reported happiness was provided by the moderate relations with questionnaires of depressive feelings and life satisfaction. However, comparable to the qualitative analysis we conducted on nursing staffs’ understanding of the wellbeing concepts, residents’ interpretation of the single questions should be further investigated (Griffiths et al., Reference Griffiths, Smith, Martin, Meads, Kelley and Surr2020), for example through cognitive interviewing.
Furthermore, even though we trained nursing staff, assessing subjective experiences of resident wellbeing may still have been too difficult (Diaz-Redondo et al., Reference Diaz-Redondo, Rodríguez-Blázquez, Ayala, Martinez-Martin and Forjaz2014), which could explain the low proxy self-report agreement found in the current study. Combining and weighting several observations to reach an overall indication of happiness and engagement requires rather advanced cognitive skills. Some nursing staff indicated they thus made assessments mostly intuitively, instead of relying on specific observations, thereby reducing the validity of assessments. Additionally, taking into account the values and preferences of the resident for a holistic wellbeing assessment (Gill and Feinstein, Reference Gill and Feinstein1994) requires extensive knowledge of the resident and careful observations, which may not always be feasible in the time-constrained environment of the nursing home. One way to support this knowledge of nursing staff is by connecting to life stories of residents, an important PCC method that is increasingly implemented in the nursing home environment (Elfrink et al., Reference Elfrink, Zuidema, Kunz and Westerhof2018).
Another possible explanation for the low proxy self-report agreement we found may be related to the fact that the two-week observation period of proxy assessments did not always perfectly overlap with the one-week self-report period in the current study. Although various measurements of nursing home resident QoL use similar time-frames (Hughes et al., Reference Hughes, Farina, Page, Tabet and Banerjee2019), the wellbeing of nursing home residents may fluctuate over time (Kolanowski et al., Reference Kolanowski, Hoffman and Hofer2007). One could imagine that proxy assessments provided after an engaging game of bingo on Friday may differ considerably from self-reports given on a quiet and lonely Monday. It could thus be that nursing staff merely differed from residents in the kind of moments they had in mind when coming to a wellbeing assessment. Future studies should examine this, e.g. by adopting an experience sampling method (Carstensen et al., Reference Carstensen, Turan, Scheibe, Ram, Ersner-Hershfield, Samanez-Larkin, Brooks and Nesselroade2011) with simultaneous proxy assessments and self-reports. This could be facilitated by incorporating integrated technology (Jensen et al., Reference Jensen, Rothrock, Dewitt, Spiegel, Tucker, Crane, Forrest, Patrick, Fredericksen, Shulman, Cella and Crane2015), such as prompting the happiness and engagement single questions regularly on a device with easy-to-understand ‘voting’ buttons (e.g. using colours and emojis) – as frequently used in retail outlets.
Limitations
Besides the above-mentioned limitations of the instrument leaving room for interpretation and the design reducing overlap between assessments and self-reports, the current study had several other strengths and limitations. It is one of the first studies to address proxy assessments of happiness and engagement in the nursing home context with single questions that corresponded closely to resident self-reports. Furthermore, analysing brief motivations helped explain some of the results. Including only residents who do not have dementia for self-reports reduced the issue of low agreement through resident introspective problems (Gerritsen et al., Reference Gerritsen, Steverink, Ooms, De Vet and Ribbe2007), although it simultaneously limits generalisability. Another limitation is that as data gathering for this study was part of a training programme, nursing staff were free to choose whom to assess, and the number of available proxy assessments varied on care-giver and resident level. More structured and homogeneous data would improve comparability, although results showed that the number of proxy assessments was not related to proxy self-report agreement. Furthermore, nursing staff had only limited time to practise the assessment method, only during training, and continued practice could potentially improve their ability. Finally, this study was conducted in only two (large) nursing homes, and in the Netherlands where care for older adults is relatively well organised and nursing staff are relatively well trained (van der Velden et al., Reference van der Velden, Francke and Batenburg2011), both limiting generalisability. One can imagine that wellbeing assessments might be even less accurate in other countries with even more profound problems of a fragmented poorly paid workforce and high turnover, but this should be investigated further.
Future studies should pay ample attention to reaching a shared understanding of the concepts, both between colleagues and between nursing staff and residents. Besides, the quality of the relationship between nursing staff and the resident should be investigated as a potential care-giver factor, as residents may express their true feelings more within a good care-giver relationship (Gräske et al., Reference Gräske, Fischer, Kuhlmey and Wolf-Ostermann2012). Furthermore, the exact wording of the single questions and rating options should be a topic of research, because asking residents how happy or satisfied they felt in the previous week may elicit very different answers than asking how good they felt. Others have used satisfactory proxy self-report agreement as a prerequisite during questionnaire development (Bergland et al., Reference Bergland, Kirkevold, Sandman, Hofoss, Vassbø and Edvardsson2014), which offers great potential as a basis for single-question measurement development. Finally, asking nursing staff to take the perspective of the resident while making the assessment may improve agreement (Leontjevas et al., Reference Leontjevas, Teerenstra, Smalbrugge, Koopmans and Gerritsen2016), suits the subjective character wellbeing and matches PCC practice (Brooker, Reference Brooker2004).
With all this in mind, it should be noted that from a clinical perspective, the variability in proxy assessments may actually be informative. Some care-givers reported specific situations or activities in which they observed happiness and engagement, and combining various observations of several care-givers during team discussions provides in-depth insight into resident preferences. Besides, the difference between care-givers was less than one point on average, which may have only limited clinical significance. A recent study showed that when implementing such a method in practice, it is essential to ensure a stable nursing home context and a clear implementation plan (Kloos et al., Reference Kloos, Drossaert, Trompetter, Bohlmeijer and Westerhof2020).
Still, when taken together, nursing staff did not sufficiently assess happiness and engagement compared to self-reports, differed considerably in their assessments, while the included nursing staff characteristics could not identify which professional care-giver could best provide wellbeing assessments. Accurate regular assessments of resident wellbeing should receive a more prominent place in documentation (Hughes et al., Reference Hughes, Farina, Page, Tabet and Banerjee2019), and using single questions of psycho-social experiences provides potential, but more research is needed. For now, in the absence of any objective measures of wellbeing, and in line with PCC assigning high value on the resident perspective (Brooker, Reference Brooker2004), proxy happiness and engagement assessments should always be combined with regular self-reports whenever possible.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S0144686X20001178.
Acknowledgements
The authors gratefully acknowledge Arnold Driessen for his help in gathering the data and Charlotte van Cleynenbreugel, Sanne Lamers and Jan Voortman for their collaboration and advice in this research project.
Author contributions
All authors made substantial contribution to the conception and design, or analysis and interpretation of the data, drafting the article or revising it, and approval of the published version.
Financial support
This work was supported by the care provider Zorggroep Sint Maarten, Denekamp, The Netherlands, who had no involvement in the study design, in the collection, analysis or interpretation of the data, nor in the decision to submit the article for publication.
Conflict of interest
The authors declare no conflicts of interest.
Ethical standards
The study was approved by the ethics committee of the Faculty of Behavioral, Management and Social Sciences at the University of Twente (number 15016).
Open access
