Introduction
Survival rates and incidence of cancer have increased due to advances in early diagnosis and treatment (Siegel et al. Reference Siegel, Miller and Fuchs2021; Sung et al. Reference Sung, Ferlay and Siegel2021). Now, cancer is considered to be a chronic disease (Bullard et al. Reference Bullard, Ji and An2019; Phillips and Currow Reference Phillips and Currow2010), and the important goals of the interventions for cancer patients have extended from survival to quality of life (Lee and Jeong Reference Lee and Jeong2019). To improve the quality of life of cancer patients, it is necessary to identify and solve the diverse needs of the patients during the full trajectory of cancer (Harrison et al. Reference Harrison, Young and Price2009) because unmet needs affect the quality of life of cancer patients (Jang and Jeong Reference Jang and Jeong2021). To manage the unmet needs of cancer patients, one of the most critical steps is to assess the patients’ needs accurately.
Valid and reliable tools are needed to identify unmet needs properly. There are various questionnaires for measuring unmet needs, and the Supportive Care Needs Survey (SCNS) is one of most widely used questionnaires globally. There are 2 types of the SCNS – long-form (LF) and short-form (SF). The SCNS was originally developed as a 59-item list, which was later named the long-form (SCNS-LF59). Later, a 34-item questionnaire, called SCNS-SF34 or short-form, was developed to lessen a responder’s burden by reducing the number of items while maintaining the psychometric properties of the long-form (Bonevski et al. Reference Bonevski, Sanson-Fisher and Girgis2000; Boyes et al. Reference Boyes, Girgis and Lecathelinais2009). Both SCNS-LF59 and SCNS-SF34 have 5 major subdomains, which are psychological, health systems and information, physical and daily living, patient care and support, and sexuality. SCNS-LF59 has 22 items in the psychological subdomain, 15 items in the health systems and information subdomain, 7 items in the physical and daily living subdomain, 8 items in the patient care and support subdomain, and 3 items in the sexuality subdomain (Bonevski et al. Reference Bonevski, Sanson-Fisher and Girgis2000). On the other hand, SCNS-SF34 has 10 items in the psychological subdomain, 11 items in the health systems and information subdomain, 5 items in the physical and daily living subdomain, 5 items in the patient care and support subdomain, and 3 items in the sexuality subdomain (Boyes et al. Reference Boyes, Girgis and Lecathelinais2009). Both SCNS forms use a 5-point response scale for each item: no need – not applicable; no need – already satisfied; low need; moderate need; or high need. The higher the score means the higher the unmet needs.
The SCNS-LF59 and SCNS-SF34 were developed in English originally. They have been translated into many languages and utilized in many countries. Bonevski et al. (Reference Bonevski, Sanson-Fisher and Girgis2000) developed SCNS-LF59 and reported that the Cronbach’s alpha ranged from 0.87 to 0.97, and the psychological subdomain presented the highest alpha and patient care and support and the sexuality subdomains showed the lowest alpha scores. In the development of SCNS-LF59, Boyes et al. (Reference Boyes, Girgis and Lecathelinais2009) reported that the alphas ranged from 0.86 to 0.96 and the highest alpha in the health systems and information subdomain compared to the lowest alpha in the physical and daily living subdomain. Both long- and short-forms have high reliability coefficient scores, but there was a difference between the 2 as to which domains have higher reliability. The studies using SCNS-LF59 or SCNS-SF34 presented various reliability coefficient scores, which ranged from 0.64 to 0.97. It is worth estimating the average reliability of SCNS-LF59 and SCNS-SF34 to give an insight to clinicians and researchers who plan to assess the unmet needs of cancer patients.
Cronbach’s alpha does not give the reliability of the tool itself but of sample-specific information. However, if reliability is reported consistently high or low over time, it informs empirical evidence for future research. Reliability generalization is an extension of the meta-analysis proposed by Vacha-Haase (Reference Vacha-Haase1998) and has been used to understand what factors affect the variability of reliability scores across the results by administrating the instruments (Vacha-Haase and Thompson Reference Vacha-Haase and Thompson2011). The purpose of this study was to examine the overall and subdomain reliability of SCNS-LF59 and SCNS-SF34 and identify moderators in reliability variability using reliability generalization meta-analysis. Study questions are as follows.
RQ1: What are the reliability scores of the SCNS long- and short-forms?
RQ2: Is there any variation in the reliability scores among studies depending on languages or study locations?
Methods
This systematic review with meta-analysis was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses 2020 guideline (Page et al. Reference Page, McKenzie and Bossuyt2021). The preestablished review protocol was registered to the International Prospective Register of Systematic Reviews database (CRD42021238584).
Search strategy
Reviewed studies for this study were searched through several databases, including PubMed, Ovid, Embase, CINAHL (Cumulative Index to Nursing and Allied Health Literature), Web of Science, Scopus, and ProQuest Dissertations & Theses. The search terms used for database query were (“Cancer”[Title/Abstract] OR (“Cancer”[Title/Abstract] AND “Patient”[Title/Abstract]) OR “Neoplasms”[MeSH Terms]) AND (“unmet need”[Title/Abstract] OR (“unmet”[Title/Abstract] AND “need”[Title/Abstract])) AND (“questionnaire”[Title/Abstract] OR “Assessment”[Title/Abstract] OR “Tool”[Title/Abstract] OR “measurement”[Title/Abstract] OR “measur*”[Title/Abstract] OR “Scale”[Title/Abstract] OR “survey”[Title/Abstract] OR “instrument”[Title/Abstract] OR “checklist”[Title/Abstract] OR “evaluation”[Title/Abstract] OR “Needs Assessment Tool”[Title/Abstract] OR “Assessment Tool”[Title/Abstract] OR “Needs Assessment”[Title/Abstract] OR (“Needs Assessment”[MeSH Terms] OR “Surveys and Questionnaires”[MeSH Terms])) in PubMed, for example. We chose the ProQuest Dissertations & Theses to search gray literature such as unpublished thesis or dissertation papers. All studies satisfied the inclusion criteria, and studies published up to December 2020 were searched. There were no language restrictions on the search.
Study selection
A librarian extracted data through electronic database search based on the inclusion criteria. Two authors (Y. Jang and H. Lee) independently screened the electronic search results. Duplicate papers were excluded by a reference management software at first and then by comparing the records screened based on title, publication year, author name, and abstract. After excluding duplicates, full-text articles were assessed for eligibility by 2 authors (Y. Jeong and H. Lee) independently. The studies published in peer-reviewed journals or theses/dissertations from a university that reported reliability from the authors’ own data were included in the analysis. Inclusion and exclusion criteria in detail are as follows:
Inclusion criteria
• A quantitative study assessing the unmet needs of cancer patients using the SCNS
• The reliability of the current study was reported
• A sample size was reported
Exclusion criteria
• The reliability of current study was not reported
• Cronbach’s alpha value was not provided
• Participants were not cancer patients
• The questionnaire was substantially modified in terms of domains and items.
• A sample size was not reported
• Only partial subdomains were used
Disagreements between the authors were resolved by discussion. When study selection was completed, the following data were extracted: author, year of publication, countries in which the study was conducted, sample size, type of sample, reliability data, the number of SCNS items used in the study, and study design.
Quality assessment of included studies
Quality assessment for systematic review was independently conducted by 2 authors (Y. Jeong and H. Lee) using the quality rating scale based on Zangaro and Soeken (Reference Zangaro and Soeken2005). The scale is a total score of 10 and consists of 7 items, including research question, subjects in sample, setting, method of data collection, response rate, measurement instrument, and reliability. The range of the scores was classified as low (0–4), moderate (5–7), or high (8–10) quality as suggested by Zangaro and Soeken (Reference Zangaro and Soeken2005). Two independent review authors (Y. Jeong and H. Lee) decided the final studies to be included in meta-analysis.
Data synthesis and analysis
This study was designed to generalize the reliability of the SCNS using a meta-analysis. All analyses were performed using R Statistical Software using R version 4.0.2 (R Core Team 2021) and RStudio version 1.4.1 (RStudio Team 2020) with “meta” (Balduzzi et al. Reference Balduzzi, Rücker and Schwarzer2019) and “metafor” (Viechtbauer Reference Viechtbauer2010) packages. To estimate the overall alpha of all the selected studies, the alpha coefficient in each study was transformed into the Fisher’s Z for normalization. After obtaining the average transformed score, it was transformed back to alpha. A random effects model was used, and the values were weighted by the inverse of the studies’ variance. To measure heterogeneity, Higgins I 2 statistics were performed. I 2 values ≥75% means high heterogeneity (Higgins et al. Reference Higgins, Thompson and Deeks2003). To explore categorial moderator, subgroup analyses were performed using the meta-analysis of variance (meta-ANOVA), which are the same with meta-regression with a categorical predictor (Harrer et al. Reference Harrer, Cuijpers and Furukawa2021, 198). According to the model selection flowchart suggested by Borenstein et al. (Reference Borenstein, Hedges and Higgins2009, 163), we used random effect model with a pooled estimate of τ 2 for meta-ANAOVA. The pooled estimate methods is “pooling Q values and degree of freedom within subgroups, estimating τ 2 from pooled values and utilizing the pool estimates of τ 2 for all subgroups” (Borenstein et al. Reference Borenstein, Hedges and Higgins2009, 162). When meta-ANOVA results were statistically significant and more than 2 subgroups existed in the moderator, post hoc tests were done by pair-wise meta-ANOVA analyses. Publication bias was evaluated by checking the level of visualized symmetricity in the funnel plot and Egger’s test.
Results
We identified a total of 12,522 records and finally selected 46 studies for a systematic review and 26 studies for meta-analysis (Figure 1). The characteristics and quality of the studies included in this systematic review and meta-analysis are listed in Table 1.
Flow diagram.
Characteristics, quality rating, and reliability scores of individual studies (k = 26)
Subdomains: C, patient care and support; D, physical and daily living; H, high; M, moderate; I, health system and information; S, sexuality; and P, psychological.
Cancer characteristics: BRCA, breast cancer; CRC, colorectal cancer; ECA, esophageal cancer; GC, gastric cancer; GICA, gastrointestinal cancer; Gyn, gynecological; HCA, hematological cancer; HNC, head and neck cancer; LCA, lung cancer; and NMSC, non-melanoma skin cancer.
Study characteristics
The 46 studies analyzed in this study were published between 2000 and 2020. Five studies used SCNS-LF59, and 41 studies utilized SCNS-SF34. The locations of studies were Amman, Australia, Canada, China, Germany, Hong Kong, Indonesia, Iran, Italy, Jordan, Korea, Malaysia, Mexico, the Netherlands, Singapore, Switzerland, Taiwan, Turkey, UAE, and the UK. The sample sizes vary between 25 and 1,106 (mean 329.13 and median 236). The reliability coefficients for SCNS of all 46 studies reported was Cronbach’s alpha. Twenty of the 46 studies did not present each reliability coefficients of subdomains but showed ranges only. Thirteen studies reported overall alpha only, while 11 studies reported each subdomain alphas. Two studies reported alphas of overall and each subdomain of SNCS-SF34. Based on the 46 studies, the lowest reliability scores reported for SCNS-LF59 and SCNS-SF34 was 0.73 and 0.56, respectively. Quality rating of 46 studies was mostly over 8 points, which was classified as high quality except for 4 studies with 7 points. Twenty-six studies reported that alpha scores of overall or each subdomain were included in final reliability generalization meta-analysis (Table 1).
Pooled results of reliability
Reliability of SCNS-LF59
Out of 5 selected studies, 2 studies presented the overall reliability of SCNS-LF59. The pooled reliability was 0.90 (95% confidence interval [CI]: 0.39, 0.99). However, high heterogeneity was observed (I 2 = 100 %, p < 0.001; see Figure 2A).
Reliability of SCNS-LF59 subdomains
Out of the 5 selected studies, 3 studies presented the reliability values of SCNS-LF59 subdomains. The pooled reliability of the subdomains ranged from 0.90 in the physical and daily living and sexuality (95% CI: 0.89, 0.91 and 0.86, 0.92, respectively) domains to 0.97 in the psychological domain (95% CI: 0.97, 0.97), as shown in Figure 2. The psychological and physical and daily living domains showed homogeneity, while the other subdomains showed high heterogeneity (I 2 = 83−98%, p < 0.001; see Figure 2B–F).
Forest plots (SCNS-LF). A, Overall; B, Psychological; C, Health systems and information; D, Physical and daily living; E, Patient care and support; F, Sexuality; and F, Sexuality.
Forest plots (SCNS-LF)
Reliability of SCNS-SF34
Thirteen studies among 21 studies presented overall reliability of SCNS-SF34. The pooled reliability was 0.92 (95% CI: 0.89, 0.94). However, high heterogeneity was observed (I 2 = 97%, p < 0.001; see Figure 3A).
Reliability of SCNS-SF34 subdomains
Ten studies presented the reliability values of the SCNS-SF34 subdomains. Li et al. (Reference Li, Lam and Shun2013) reported the results separately for each region, so the analysis was conducted separately. The pooled reliability of the subdomains ranged from 0.81 (sexuality, 95% CI: 0.74, 0.86) to 0.92 (psychological and health information), as shown in Figure 3. However, high heterogeneity was observed in all subdomains (I 2 = 93−97%, p < 0.001; see Figure 3B–F).
Forest plots (SCNS-SF). A, Overall; B, Psychological; C, Health information; D, Daily living; E, Patient care; and F, Sexuality.
Moderator analysis
Moderator analysis was done for SCNS-SF34 using meta-ANOVA. Mediators for subgroup analysis were language and location. We compared the differences in alpha coefficients between original language (English) of the SCNS and the translated version (non-English) to explore the effect of translation invariance on heterogeneity. We also compared the differences between study locations. We originally speculated culture or country as a moderator, and it was not possible to test in this study due to the small sample size. We eventually analyzed subgroup difference by continent. The results are presented in Table 2. Language and location were significantly different between subgroups in the health systems and information as well as the physical and daily living subdomains. In the physical and daily living subdomain, there was a subgroup difference only between Asia and Oceania.
Results of the moderator analyses
Note:
* p < 0.05.
Publication bias
Because the number of studies using SCNS-LF59 is too small, it is not appropriate to test publication bias. Figure 4 visualizes the level of publication bias for the analyzed studies for SCNS-SF34 through the funnel plot. According to the Egger’s regression test, the funnel plot remained symmetrical in the overall (bias = 0.9228, t = 0.24, df = 11, p = 0.812), psychological (bias = 0.310, t = 0.08, df = 9, p = 0.941), health systems and information (bias = −0.759, t = −0.16, df = 9, p = 0.874), physical and daily living (bias = 0.870, t = 0.30, df = 9, p = 0.771), patient care and support (bias = −1.698, t = −0.50, df = 9, p = 0.630), and sexuality (bias = 2.808, t = 0.63, df = 9, p = 0.544) subdomains.
Funnel plots (SCNS-SF). A, Overall; B, Psychological; C, Health information; D, Daily living; E, Patient care; and F, Sexuality.
Sensitivity test
We performed sensitivity test to explore studies contributing to influence on effect sizes. A post hoc influential analysis was done by omitting studies one by one. Sensitivity analysis showed stable results (Figure 5).
Sensitivity tests (SCNS-SF). A, Overall; B, Psychological; C, Health information; D, Daily living; E, Patient care; and F, Sexuality.
Discussion
In this study, we performed reliability generalization of SCNS-LF and SCNS-SF and tested whether there were any differences in reliability depending on language and location. In this study, we examined pooled reliability coefficients of overall and subdomains. It seems that the overall alpha value or the alpha value of the subdomain has been presented depending on whether the whole is viewed as a single structure or whether each of the multidimensional subdomains has its own focus. The authors who developed SCNS-LS and SCNS-SF did not report overall alpha scores but reported the alpha scores for each subdomain. Therefore, the authors may weigh internal consistency reliability within each subdomain more than overall reliability. However, many studies using SCNS reported overall reliability with or without subdomain reliabilities. There is an opinion that alpha should be applied when a scale is unidimensional (Dunn et al. Reference Dunn, Baguley and Brunsden2014), while another opinion exists that reliabilities of overall and subdomains may need to be reported (Cho and Kim Reference Cho and Kim2015). Which reliability to report or not is beyond the scope of this study. Therefore, we analyzed reliability coefficients of both overall and subdomains as they were reported in selected studies.
The results showed that the average Cronbach’s alpha values of overall and each subdomain of SCNS-LF were greater than 0.9. For SCNS-SF, overall Cronbach’s alpha was greater than 0.9, and the alpha values of subdomains were greater than 0.81. Although there is no definite cutoff values of alpha (Taber Reference Taber2018), reports showed that alpha values 0.7 or above are satisfactory, while a higher value over 0.90 is needed for clinical application (Bland and Altman Reference Bland and Altman1997) or the values of 0.8 or higher were reasonable (Gliem and Gliem Reference Gliem and Gliem2003). Therefore, it suggests that the average alpha coefficients of both SCNS-LF and SCNS-SF are acceptable levels. More studies have used SCNS-SF rather than SCNS-LF. One of the reasons is that the number of SCNS-SF items is small, which ultimately reduces the burden and fatigue of research participants. In general, alpha decreases as the number of items is reduced (Schrepp Reference Schrepp2020). However, the alpha values of SCNS-SF did not decrease dramatically even though the number of items decreased by 25 compared to the SCNS-LF. This might be another reason for researchers to choose SCNS-SF over SCNS-LF.
The results of this meta-analysis presented that the reliability of a specific subdomain was lower than other subdomains. Although the alpha coefficients of all subdomains in SCNS-LF and SCNS-SF were acceptable levels, the sexuality subdomain in both types of SCNS showed the lowest alpha values. Possible causes of a low Cronbach’s alpha value are small number of items, or low correlation between items, or mixed construct (Tavakol and Dennick Reference Tavakol and Dennick2011). The reason of low alpha values of sexuality may be the number of items because this domain has the smallest number of questions.
To identify whether there are any differences in reliability coefficients by language and location, we performed moderator analysis on SCNS-SF. There were no statistical differences in reliability values between original language and translated languages in version except health systems and information as well as physical and daily living subdomains. Based on this result, language may be a cause for heterogeneity in the reliability values. However, the differences were also found in the same 2 subdomains, health systems and information and physical and daily living, based on location. It is not clear that the differences are based on translation, locations, or both. Further analyses are needed in the future.
One of the limitations of this paper is that reliability generalization was performed based solely on Cronbach’s alpha. The reason we analyzed reliability based on Cronbach’s alpha was because it was the only reliability reported in selected papers. Cronbach’s alpha, also called Cronbach’s coefficient alpha or coefficient alpha, is one type of reliability. Types of reliability are classified into internal consistency, test–retest, and inter-rater reliability (Charter Reference Charter2003). Cronbach’s alpha is the most widely used measurement statistics of internal consistency reliability but is not the one and only reliability coefficient. However, most papers have reported Cronbach’s alpha without explanation of the reason to choose the alpha despite there being alternatives to alpha (Sijtsma Reference Sijtsma2009). There are controversial views of use of Cronbach’s alpha. Some researchers criticize that Cronbach’s alpha is problematic, while others support the merits of Cronbach’s alpha (Raykov and Marcoulides Reference Raykov and Marcoulides2019). In addition, there are opinions to report whether the assumption of tau-equivalence was met or not to estimate Cronbach’s alpha accurately (Graham Reference Graham2006). However, it has not been commonly reported in published papers whether the assumptions were met. It suggests that various reliability coefficients may be considered based on what to measure, and the assumptions to estimate reliability should be reported if it is required. Another limitation is that we could not analyze other moderators, such as gender, age, and disease-related characteristics of study population, due to small number of studies.
We recommend that researchers report the reliability coefficients of their own study. Surprisingly, we found some researchers did not report the alpha of their own study but described the alpha values that the original authors of the SCNS reported in their study. We also found some researchers did not report reliability coefficients at all. In addition, some papers presented the range of alpha instead of the alphas of each subdomain. When reporting alphas of subdomains, we suggest researchers to report reliability thoroughly. We also recommend in future studies to compare generalized reliability coefficients with other tools measuring unmet needs.
Conclusion
Through the reliability generalization, the pooled Cronbach’s alpha coefficients of both SCNS-LF and SCNS-SF presented acceptable reliability estimates. Language and location may be factors that affect reliability in the health systems and information and physical and daily living subdomains.
Acknowledgments
We appreciate the librarian, Ms. Yoonhee Choi for data extraction.
Conflicts of interest
None declared.
Open access
