Introduction
There is no single common definition of primary health care (PHC) in the contemporary literature. Further, the term is often not explicitly defined and is sometimes used synonymously with primary care (Henderson et al., Reference Henderson, Koehne, Verrall, Gebbie and Fuller2014; Canadian Nurses Association, 2015; White, Reference White2015). However, PHC is commonly agreed to broadly address the social determinants of health and offer essential education and health care (eg, health promotion; disease prevention, treatment and management) to communities based on principles of accessibility, equity, community participation, and intersectoral collaboration (Henderson et al., Reference Henderson, Koehne, Verrall, Gebbie and Fuller2014; White, Reference White2015; Baum et al., Reference Baum, Freeman, Sanders, Labonte, Lawless and Javanparast2016).
As a component of PHC, primary care focuses on clinical health care services and medical care provided after first contact between individuals and health care providers. Part of the confusion in terminology stems from advocacy efforts to advance the PHC philosophy, resulting in bifurcation of the term ‘primary health care’. PHC that is ‘professional’ (Levesque et al., Reference Levesque, Haggerty, Burge, Beaulieu, Gass, Pineault and Santor2011), ‘profession-centred’ (White, Reference White2015), or ‘selective’ has come to represent a primary care approach (Baum et al., Reference Baum, Freeman, Sanders, Labonte, Lawless and Javanparast2016), while PHC that is ‘community-based’, ‘societal’, and ‘comprehensive’ signals a contemporary PHC approach founded on the original vision set out in the Alma Ata Declaration on Primary Health Care (World Health Organization, 1978).
In developing the Primary Health Care Engagement (PHCE) Scale for health care providers, we sought to closely align the instrument with a contemporary PHC approach that would be relevant to ongoing PHC reform strategies across Canada (eg, Alberta Health, 2014; Saskatchewan Ministry of Health, 2016) and elsewhere (Liu et al., Reference Liu, Wang, Kong and Cheng2011; Thomas et al., Reference Thomas, Wakerman and Humphreys2014; Galdikiene et al., Reference Galdikiene, Asikainen, Rostila, Geen, Balciunas and Suominen2016). Although other instruments are available to measure primary care delivery performance based on perceptions of health care providers (eg, Schoen et al., Reference Schoen, Osborn and Huynh2006; Dahrouge et al., Reference Dahrouge, Hogg, Russell, Geneau, Kristjansson, Muldoon and Johnston2009), community-based PHC performance has not been well addressed at the provider level. Aside from the PHCE Scale, the Canadian Institute for Health Information (CIHI) PHC Provider Survey (Johnston and Burge, Reference Johnston and Burge2013) is the only other provider-level instrument that specifically targets community-based PHC performance, to our knowledge. Developed to complement the CIHI patient experience and organizational surveys, the CIHI provider survey addresses the PHC principle of intersectoral collaboration but not the principles of accessibility, equity, and community participation. Although these latter attributes may be more appropriately measured at the patient level, it is also worthwhile to gauge the degree of workplace involvement in PHC delivery according to front-line health care providers, given their central role in the actualization of PHC principles.
We were also keen that the new PHCE instrument apply to rural and remote (hereafter rural/remote) communities where effective PHC delivery is highly valued in light of persistent difficulties meeting health and social needs (Thomas et al., Reference Thomas, Wakerman and Humphreys2014; Ward et al., Reference Ward, Buykx, Tham, Kinsman and Humphreys2014). Rural/remote health care providers may work alone or in small multidisciplinary teams with members who are not co-located on a regular basis (MacLeod et al., Reference MacLeod, Kulig, Stewart and Pitblado2004), and thus it is important to make the best use of scarce health human resources in these settings (Ford, Reference Ford2016). Community participation in health service planning is also a policy expectation in rural communities, although meaningful participation can be difficult to operationalize and achieve (Kenny et al., Reference Kenny, Farmer, Dickson-Swift and Hyett2015). For these reasons, we drew on community-oriented (Levesque et al., Reference Levesque, Haggerty, Burge, Beaulieu, Gass, Pineault and Santor2011) and community-based (CIHR, 2015) conceptualizations of PHC in the development of the PHCE Scale. Community-oriented PHC is understood as including ‘… a wide range of professionals who deliver a broad spectrum of health and social services’ (Levesque et al., Reference Levesque, Haggerty, Burge, Beaulieu, Gass, Pineault and Santor2011: 23). Further, community-based PHC is person-centred, population-centred, coordinated, and integrated care delivered by multiple health care providers ‘in a range of community settings’ that covers the continuum from primary prevention to palliative care (CIHR, 2015).
The 40-item PHCE Scale was developed in a three-phase process reported previously (Kosteniuk et al., Reference Kosteniuk, Wilson, Penz, MacLeod, Stewart, Kulig, Karunanayake and Kilpatrick2016); 10 dimensions of community-oriented PHC most relevant to community-based and rural/remote PHC were endorsed by our 16-member research team and 19-member advisory board, collectively representing the 13 Canadian provinces and territories. The 40-item PHCE Scale consisted of 10 subscales: accessibility/availability, equity, community participation, intersectoral team, patient-centred care, continuity, population orientation, interdisciplinary collaboration, comprehensiveness, and quality improvement. Based on initial psychometric testing in a pilot survey of 89 nurses in current practice in Canada with nursing experience in rural/remote communities, the 40-item PHCE Scale (3–5 items per subscale) demonstrated good reliability (α=0.91). Although Cronbach’s α coefficients below 0.70 indicated evidence of low reliability for three subscales (equity, comprehensiveness, and patient-centred care), these were nonetheless retained in the final 40-item version given their theoretical importance to community-oriented PHC delivery in rural/remote communities.
The purpose of the present study was to provide validity evidence for the PHCE Scale, by evaluating the scale’s internal structure based on factor analysis and reliability findings from a large national survey of rural/remote regulated nurses in Canada. Recent years have seen major health care reforms targeting PHC in Canada (Strumpf et al., Reference Strumpf, Levesque, Coyle, Hutchinson, Barnes and Wedel2012) and other countries (Liu et al., Reference Liu, Wang, Kong and Cheng2011; Thomas et al., Reference Thomas, Wakerman and Humphreys2014; Galdikiene et al., Reference Galdikiene, Asikainen, Rostila, Geen, Balciunas and Suominen2016). We anticipate the PHCE Scale will be useful to researchers and health care planners in assessing the level of PHC delivery in health care settings, identifying the degree to which PHC elements are being enacted and the elements that may require intervention, and tracking progress in PHC reform.
Methods
Design
This study uses data from the pan-Canadian study ‘Nursing Practice in Rural and Remote Canada II (RRNII)’, which included a cross-sectional survey with a target sample of 10 072 regulated nurses [registered nurses (RNs), nurse practitioners (NPs), licensed practical nurses, and registered psychiatric nurses] residing in rural/remote Canada. Further details of the RRNII survey are reported elsewhere (MacLeod et al., Reference MacLeod, Stewart, Kulig, Anguish, Andrews, Banner, Garraway, Hanlon, Karunanayake, Kilpatrick, Koren, Kosteniuk, Martin-Misener, Mix, Moffitt, Olynick, Penz, Sluggett, Van Pelt, Wilson and Zimmer2017).
Participants
The RRNII survey population included a stratified systematic sample of regulated nurses in every Canadian province, as well as all regulated nurses working in the Territories and all rural and remote NPs. Eligible nurses practiced in a rural or remote community at the time of the survey, or had been on leave for six months or less. For the purposes of eligibility, communities with a core population of less than 10 000 were considered rural (du Plessis et al., Reference du Plessis, Beshiri, Bollman and Clemenson2001), and northern communities in the Territories (Nunavut, Northwest, and Yukon) were considered remote.
Regardless of their primary place of employment, we chose to include all survey participants in the exploratory factor analysis (EFA) for two reasons. First, community-based PHC is delivered ‘in a range of community settings’ outside of clinics and public health settings (CIHR, 2015). Second, institutional settings such as hospitals have a long history of health and wellness promotion in both urban and rural communities (Olden and Hoffman, Reference Olden and Hoffman2011). Therefore, participants were included whose primary place of employment was a community-based health care site [ie, directly accessible by patients and outside of an inpatient setting (Gibson et al., Reference Gibson, Lisy, Davy, Aromataris, Kite, Lockwood, Ritano, McBride and Brown2015)], hospital/rehabilitation or convalescent centre/integrated facility, nursing home/long-term care facility, or other setting (Table 1).
Characteristics of Nursing Practice in Rural and Remote Canada II survey respondents included and excluded from analysis
a Calculated by the χ 2 test unless otherwise noted.
b Mann–Whitney U test.
c Defined as primary health care settings that are directly accessible by patients and outside of an inpatient setting (Gibson et al., Reference Gibson, Lisy, Davy, Aromataris, Kite, Lockwood, Ritano, McBride and Brown2015), includes mental health centre/crisis centre, community health centre, home care agency, public health department/unit, private nursing/self-employed, occupational health, physician’s office/family practice unit or team, nurse practitioner led clinic, and multidisciplinary primary health care clinic.
d Includes hospital, rehabilitation/convalescent centre, and integrated facility (ie, acute and long-term care).
e Includes educational institution, professional association/government, and other.
A total of 3822 participants completed the RRNII survey. Of 10 072 sampled, 450 were ineligible (eg, incorrect addresses, duplicate registrations, retired) and 9622 were eligible, resulting in a response rate of 40% (3822/9622). Of these participants, 1587 completed all 40 items in the PHCE Scale and were included in the factor analysis; 2235 participants completed fewer than 40 items and were excluded to avoid artificially high correlations that may result from imputing missing values. Compared with excluded participants, those included were more highly educated and held higher registration status, and were more likely to work in smaller communities under 10 000 population (Table 1).
Data collection
RRNII survey data were collected from April 2014 to August 2015, with surveys distributed by the RRNII’s research centre at the University of Northern British Columbia as well as by provincial and territorial nursing associations. Participant recruitment relied on the Dillman method (Dillman et al., Reference Dillman, Smyth and Christian2014) to improve the response rate (eg, participants were offered a chance to win an iPad as an incentive). Online and paper survey packages were provided in English and French; non-respondents received a total of four contacts that included a first survey package, first reminder, second reminder, and final survey package.
Measures
Development of the original 40-item PHCE Scale consisted of a literature review and expert consultation, content evaluation and item revision, and testing in a small pilot survey of 89 nurses (Kosteniuk et al., Reference Kosteniuk, Wilson, Penz, MacLeod, Stewart, Kulig, Karunanayake and Kilpatrick2016). Items developed for the original PHCE Scale drew from multiple indicators of quality and performance [World Health Organization, 1986; Flocke, Reference Flocke1997; Shi et al., Reference Shi, Starfield and Xu2001; Canadian Institute for Health Information (CIHI), 2006; Davis et al., Reference Davis, Schoen, Schoenbaum, Doty, Holmgren, Kriss and Shea2007; Bloch et al., Reference Bloch, Rozmovits and Giambrone2011; Levesque et al., Reference Levesque, Haggerty, Burge, Beaulieu, Gass, Pineault and Santor2011; Wong et al., Reference Wong, Browne, Varcoe, Lavoi, Smye, Godwin, Littlejohn and Tu2011; Saskatchewan Ministry of Health, 2012], including nine items that were minimally adapted with permission from a study by Dahrouge et al. (Reference Dahrouge, Hogg, Russell, Geneau, Kristjansson, Muldoon and Johnston2009).
The 27-page RRNII survey employed in the present study consisted of five sections that focused on participants’ individual characteristics, work community, workplace, nursing practice, and aspects of their psychosocial health. The 40-item PHCE Scale included three to five items in each of 10 subscales, with items rated on a five-point scale as 1 (strongly disagree), 2 (disagree), 3 (neutral), 4 (agree), 5 (disagree), or 97 (not applicable). ‘Not applicable’ responses were coded as ‘missing’. Participants were instructed to respond in relation to the primary workplace where they spent most of their work time in the past 12 months, and to the community represented by the catchment area of their primary workplace. After reverse scoring negatively worded items, higher scores for each subscale and the overall scale suggest perceptions of greater workplace engagement in primary health care delivery.
Across the 40 PHCE items, the proportion of ‘not applicable’ responses (1.9–49.2%) was substantially higher than the proportion of true missing responses (3.1–7.7%). Specifically, between 20.1 and 49.2% of participants chose the ‘not applicable’ option in each of eight items across three subscales (accessibility/availability, population orientation, and intersectoral team). Participants whose primary place of employment was a community-based health care setting were more likely to provide a valid response to all of the scale items and also significantly more likely than their counterparts in other settings (ie, hospital, nursing home/long-term care facility, and other) to be included in the present analysis (57.1 versus 35.2%; P<0.001). However, those outside of community-based settings accounted for the majority of included participants (59.9%), demonstrating the wide applicability of the PHCE Scale across health care settings.
Statistical analyses
Initial principal component analysis and descriptive statistics were performed with IBM SPSS Statistics 23 and EFA was conducted with SAS 9.3.
The sample of 1587 participants in the present study exceeded the recommended 5:1 participant-to-item ratio and 200 participant minimum (Howard, Reference Howard2016). The large sample size took into account the small number of items per factor (<5) in the 40-item PHCE scale (Gaskin and Happell, Reference Gaskin and Happell2014) as well as potentially weak item communalities (<0.40) (Costello and Osborne, Reference Costello and Osborne2005), which would indicate that the factor model accounted for a small amount of variance in those items (Tabachnick and Fidell, Reference Tabachnick and Fidell2013). Missing values were not imputed, as imputation with estimated values may result in overfit data and artificially high correlations (Tabachnick and Fidell, Reference Tabachnick and Fidell2013). Instead, listwise deletion was employed to delete every case with one or more missing items.
The Kaiser–Meyer–Olkin (KMO) measure of sampling adequacy and Bartlett’s test of sphericity were first calculated with the 1587 participants to assess whether factor analysis was warranted (Howard, Reference Howard2016). A KMO value of 0.80 or above, on a range of 0–1, indicates that the items share a common factor and factor analysis is supported (Kellar and Kelvin, Reference Kellar and Kelvin2013).
An initial principal component analysis to calculate eigenvalues was conducted, and the number of factors to extract was identified with the assistance of web-based parallel analysis (Patil, 2008). Parallel analysis compares eigenvalues calculated from the actual data to eigenvalues generated on the basis of random data matrices matched to the actual data on sample size and number of factor analysis items (Patil et al., Reference Patil, Singh, Mishra and Donavan2008). The number of factors to extract is equal to the number of eigenvalues in the actual data that are greater than the corresponding eigenvalues in the random data.
A first EFA was then conducted using unweighted least squares with polychoric correlations, a method recommended when extracting factors from ordinal data (Basto and Pereira, Reference Basto and Pereira2012; Gaskin and Happell, Reference Gaskin and Happell2014). We selected promax oblique rotation given that oblique rotation methods allow the derived factors to be correlated (Costello and Osborne, Reference Costello and Osborne2005), consistent with our assumption of correlations between factors in the present study.
We compared several factor solutions to identify the ‘cleanest factor structure’ in terms of data and theoretical fit (Costello and Osborne, Reference Costello and Osborne2005: 3). Items were initially assessed for retention on the basis of the 0.40–0.30–0.20 rule: primary factor loadings ⩾0.40, secondary factor loadings <0.30, and ⩾0.20 difference between primary and secondary loadings (Howard, Reference Howard2016). Any remaining items with low communalities between 0.20 and 0.40 (Gaskin and Happell, Reference Gaskin and Happell2014) indicated that the factor solution accounted for a low proportion of variance in these items (Tabachnick and Fidell, Reference Tabachnick and Fidell2013). These items were evaluated for their theoretical contribution to the overall model, and considered for removal (Costello and Osborne, Reference Costello and Osborne2005). Furthermore, factors that consisted of only two items were considered for removal on the basis of their correlation with each other as well as with other items in the factor model (Tabachnick and Fidell, Reference Tabachnick and Fidell2013).
A second EFA was conducted with the items remaining after the first EFA. We report the mean item scores and standardized Cronbach’s α coefficients for the final PHCE subscales and overall scale, using a cut-off point of 0.70 as evidence of adequate internal consistency reliability (Nunnally and Bernstein, Reference Nunnally and Bernstein1994).
Results
Sample characteristics
As shown in Table 1, the majority of included participants were female, RNs, staff nurses, had attained a diploma as their highest level of education, were primarily employed outside of community-based health care settings, and were employed in communities with less than 10 000 population. Included participants were 47.0 years of age on average (SD=11.6) and had been registered to practice in Canada an average of 21.0 years (SD=13.0).
Exploratory factor analysis
Based on principal component analysis with data from 1587 participants, the KMO measure (0.91) and significant Bartlett’s test of sphericity [χ 2 (780)=23 859.80, P=0.000] pointed to a strong association among the scale items, which supported the use of factor analysis. Parallel analysis suggested an eight-factor solution, with eigenvalues generated by principal component analysis exceeding random values (in brackets) for the first eight factors: 9.68 (1.34), 2.65 (1.31), 2.27 (1.27), 1.60 (1.25), 1.49 (1.23), 1.47 (1.21), 1.38 (1.19), 1.21 (1.17), 1.10 (1.16).
In the first EFA (n=1587), we produced solutions of 5, 6, 7, 8, 9, and 10 factors for comparison. An eight-factor solution was optimal in terms of data fit as well as conceptual fit with the majority of the original subscales in the 40-item PHCE scale (Table 2). From the original 40 items, a total of 12 items were removed based on the 0.40–0.30–0.20 rule (Howard, Reference Howard2016). First, 11 items with primary factor loadings <0.40 were removed, as well as one item (C3) with a low factor loading of 0.41 and a low communality of 0.19 (Table 3). Second, the secondary factor loadings of all 40 items were <0.30 and therefore no items were removed on the basis of this criterion. Third, of the 12 items removed, 10 also demonstrated a difference of <0.20 between their primary and secondary factor loadings.
Factor loadings of items included in the eight-factor solution based on first exploratory factor analysis
Bold font indicates primary factor loadings.
Factor loadings of items removed from the 8-factor solution based on first exploratory factor analysis
Bold font indicates primary factor loadings.
With respect to the remaining 28 items with factor loadings ranging from 0.42 to 0.89, the majority demonstrated communalities in the recommended range (0.42–0.79). However, five items had communalities below 0.40, namely G1 and G4 (factor 3), F1 and F4 (factor 4), and C4 (factor 8). We made the decision to retain these five items given their theoretical contributions to the final scale.
The eight-factor solution (Table 2) included five factors (1, 2, 3, 5, and 7) that retained all of the items from five original dimensions (community participation, patient-centred care, intersectoral team, accessibility/availability, and interdisciplinary collaboration). Factor 4 included items that loaded from two separate original dimensions (population orientation and equity). Given that the population orientation items (D3 and D5) had higher loadings than the equity items (F1 and F4), and the equity items measured barriers between the community population and the participant’s workplace, we preserved the label ‘population orientation’ for Factor 4 and dropped the label ‘equity’. Factors 6 and 8 consisted of a reduced but sufficient number of items supporting two original dimensions (quality improvement and continuity) and were therefore retained. Although Factor 6 consisted of only two items, it was retained because these items were highly correlated with one another (>0.70) and uncorrelated or moderately correlated (<0.40) with other items. The factor loadings of items from the last original dimension (comprehensiveness) were <0.40, and as a result this factor was not included in the eight-factor solution. Therefore, the two original dimensions that were not included in the final eight-factor model were equity and comprehensiveness.
Based on the second EFA performed with the final 28 items, the total variance explained by the final eight-factor solution was 14.8%, with the proportion of variance explained by each factor 2.4% (community participation), 2.4% (patient-centred care), 2.1% (intersectoral team), 2.1% (accessibility/availability), 1.6% (interdisciplinary collaboration), 1.5% (quality improvement), 1.5% (population orientation), and 1.2% (continuity). The eight factors demonstrated moderate correlation (>0.30) between one another with the exception of: patient-centred care and quality improvement (0.27), continuity with all other factors except patient-centred care, and accessibility/availability with all other factors (Table 4).
Factor correlation matrix of eight-factor solution based on second exploratory factor analysis
Reliability and summary statistics
Internal consistency reliability was estimated by the Cronbach’s α coefficient, which was 0.89 for the final 28-item PHCE Scale (Table 5). Six of the final eight subscales demonstrated α estimates ranging from 0.77 to 0.88, which exceeded the recommended standard of 0.70 (Nunnally and Bernstein, Reference Nunnally and Bernstein1994). The α estimates for two of the final eight subscales fell below 0.70, specifically continuity (α=0.61) and population orientation (α=0.61).
Summary statistics of final 28-item Primary Health Care Engagement (PHCE) scale and subscales
Scale scoring: 1=strongly disagree; 2=disagree; 3=neutral; 4=agree; 5=strongly agree.
The mean item scores for the final 28-item PHCE Scale and eight subscales demonstrated low (3.1–4.0) to high agreement (4.1–5.0) among rural/remote nurses on workplace engagement in key dimensions of primary health care (Table 5). Mean scores ranged from a low of 3.1 (SD=0.7) on intersectoral team indicating a low level of agreement that collaboration exists between the primary workplace and community sectors, to a high of 4.2 (SD=0.6) on patient-centred care indicating high agreement that respect, safety, and confidentiality toward patients is present in the primary workplace. Overall, the mean PHCE Scale score of 3.6 (SD=0.5) indicated low agreement that the primary workplaces of rural/remote nurses were engaged in primary health care delivery.
Discussion
The present study describes an evaluation to provide validity evidence for an instrument to measure workplace engagement in primary health care delivery. Evidence for the construct validity of conclusions that may be drawn from an instrument is generally derived from five sources, including the scale’s content and internal structure (Cook and Beckman, Reference Cook and Beckman2006). In an earlier study, we provided content evidence to demonstrate that the PHCE Scale represents community-based/oriented primary health care (Kosteniuk et al., Reference Kosteniuk, Wilson, Penz, MacLeod, Stewart, Kulig, Karunanayake and Kilpatrick2016). Findings from the present study provide evidence of the internal structure of the PHCE Scale, based on an eight-factor structure identified in EFA and good internal consistency reliability (α=0.89).
Other provider-level instruments are available to assess health care sites on performance indicators of primary care (Schoen et al., Reference Schoen, Osborn and Huynh2006; Dahrouge et al., Reference Dahrouge, Hogg, Russell, Geneau, Kristjansson, Muldoon and Johnston2009) and PHC delivery (Johnston and Burge, Reference Johnston and Burge2013). However, the 28-item PHCE Scale differs from previous provider-level instruments in that it includes most of the principles common to contemporary community-based PHC approaches (eg, accessibility, community participation, and intersectoral collaboration) (Henderson et al., Reference Henderson, Koehne, Verrall, Gebbie and Fuller2014; White, Reference White2015; Baum et al., Reference Baum, Freeman, Sanders, Labonte, Lawless and Javanparast2016).
Based on the results of EFA, the original 40-item PHCE Scale across 10 subscales was reduced to 28 items across eight factors. Five of the final eight factors retained all of the items proposed in the original subscales (community participation, patient-centred care, intersectoral team, accessibility/availability, and interdisciplinary collaboration). The factor loadings of these items were well above the recommended cut-off, and reliability estimates of the five factors were satisfactory (α>0.70). Two of the final eight factors (continuity and quality improvement) lost items from the original subscales, however, the remaining items were sufficient in number and demonstrated adequate loadings for the factors to be retained. The last of the eight factors consisted of items loading from two separate original subscales (population orientation and equity), and therefore the label that was consistent with the highest loading items was retained (population orientation). Lower than expected reliability estimates (α<0.70) were demonstrated by two of the final factors (continuity and population orientation), which may be due to low correspondence between items or heterogeneity of concepts within the factors (Tavakol and Dennick, Reference Tavakol and Dennick2011).
The overall 28-item PHCE score provides a broad measure of workplace engagement in primary health care delivery, while the eight separate 2–4 item subscales allow measurement of particular aspects of PHC engagement. Higher scores on the PHCE Scale indicate a higher level of workplace participation in PHC delivery. The final 28-item scale directly represents three of four principles common to a contemporary approach to PHC (Henderson et al., Reference Henderson, Koehne, Verrall, Gebbie and Fuller2014; White, Reference White2015; Baum et al., Reference Baum, Freeman, Sanders, Labonte, Lawless and Javanparast2016) based on the original Alma Ata Declaration on Primary Health Care (World Health Organization, 1978): accessibility (accessibility/availability), community participation, and intersectoral collaboration (intersectoral team). The fourth element (equity) is indirectly represented in the population orientation subscale, indicating possible overlap in these two concepts or in the design of the items representing these two concepts. The remaining dimensions in the final PHCE Scale directly represent most elements of community-oriented (Levesque et al., Reference Levesque, Haggerty, Burge, Beaulieu, Gass, Pineault and Santor2011) and community-based PHC models (CIHR, 2015): a wide range of providers (interdisciplinary collaboration), person-centredness (patient-centred care), coordination and integration (continuity), population-centredness (population orientation), and quality improvement. An important element missing from the final PHCE Scale is comprehensiveness, the delivery ‘of a broad spectrum of health and social services’ (Levesque et al., Reference Levesque, Haggerty, Burge, Beaulieu, Gass, Pineault and Santor2011: 23) considered essential to community-based/oriented PHC. Although it is possible that as a conceptual category, comprehensiveness may not be an essential dimension of PHCE, it is more likely that the items did not adequately capture the complexity of this concept in the context of diverse rural/remote patient populations.
Some study limitations must be acknowledged. First, the proportion of participants that responded ‘not applicable’ was greater than 20% and as high as 49% for eight of the original 40 items across three subscales. We chose not to impute these ‘not applicable’ responses and other missing values, since including these values may have resulted in artificially high correlations (Tabachnick and Fidell, Reference Tabachnick and Fidell2013). Six of these items remained in the final 28-item scale (A2, A3, and A4 in accessibility/availability; G1, G2, and G3 in intersectoral team). The high proportion of ‘not applicable’ responses to the accessibility/availability items indicates that some participants may view questions regarding workplaces that ‘open’ and ‘close’ as irrelevant to their practice context. These items may be more relevant to community-based settings that offer regular office hours and appointment times than to institutional settings that offer 24/7 care. The high proportion of ‘not applicable’ responses to the intersectoral team items may reflect a ‘tyranny of the acute’ organizational culture that places a low emphasis on population health issues in some workplaces, as well as challenges to working intersectorally without formal supporting structures in place (CIHI, 2014). However, accessibility/availability and intersectoral collaboration are principles of a contemporary approach to PHC and therefore it is important and appropriate to retain these items in the final 28-item PHCE Scale.
A second limitation is that we applied a stringent listwise deletion criterion to exclude cases with at least one missing item, resulting in a large number of excluded cases (n=2235). Participants employed in community-based health care settings were more likely to provide a valid response to all of the scale items and to be included in the analysis than their counterparts in institutional settings (eg, hospital, long-term care). However, the majority of included participants were employed outside of community-based health care sites, indicating that the overall PHCE Scale may be applicable across a range of health care settings. We recommend the use of a case mean imputation strategy (El-Masri and Fox-Wasylyshyn, Reference El-Masri and Fox-Wasylyshyn2005) when employing the PHCE Scale for purposes other than psychometric testing, so as to obtain the maximum number of valid responses (see the PHCE Scale in the Appendix for specific imputation guidelines). Third, the study sample included only regulated nurses working in rural/remote Canadian communities, and as such this limits the generalizability of our findings in terms of professional discipline and geography. Future research should evaluate the instrument outside of these populations. Finally, the comprehensive 27-page RRNII survey provided an opportunity to administer the original 40-item PHCE Scale to regulated nurses working across rural/remote Canada. However, the purpose of the survey was to gather data on the overall nature of rural/remote nursing practice across five wide-ranging content domains such as work community and nursing practice (MacLeod et al., Reference MacLeod, Stewart, Kulig, Anguish, Andrews, Banner, Garraway, Hanlon, Karunanayake, Kilpatrick, Koren, Kosteniuk, Martin-Misener, Mix, Moffitt, Olynick, Penz, Sluggett, Van Pelt, Wilson and Zimmer2017) and as such we were unable to include other measures that may be expected to correlate with the PHCE Scale. Therefore, future research should focus on assessing the relations between the 28-item PHCE Scale and other variables, to provide further evidence of the construct validity of conclusions drawn from the scores (Cook and Beckman, Reference Cook and Beckman2006).
Conclusion
The most important finding of the present study is that the results supported an eight-factor structure of the PHCE Scale with 28 items in total, offering a comprehensive measure of workplace engagement in primary health care that is short and easy to administer. The scale is intended to be administered by researchers and health care planners to any health care provider likely to be involved in PHC delivery.
The 28-item PHCE Scale reflects the high priority placed on community participation in rural/remote PHC programs and services (Preston et al., Reference Preston, Waugh, Larkins and Taylor2010), and the importance of community participation in PHC more broadly as emphasized in recent position papers (Henderson et al., Reference Henderson, Koehne, Verrall, Gebbie and Fuller2014; Canadian Nurses Association, 2015; White, Reference White2015; Baum et al., Reference Baum, Freeman, Sanders, Labonte, Lawless and Javanparast2016) and PHC reform strategy reports (Alberta Health, 2014; Saskatchewan Ministry of Health, 2016). Although the scale may be more relevant to community-based than institutional health care settings such as hospitals and nursing homes, findings from the present study indicate that it is relevant to rural/remote providers from both settings. Further psychometric testing with a diversity of health care providers and settings is recommended.
Acknowledgements
The article stems from the study: ‘Nursing Practice in Rural and Remote Canada II’, led by Martha MacLeod, Norma Stewart and Judith Kulig (http:// ruralnursing.unbc.ca). The authors are grateful to the nurses who responded to this survey and to Nadine Mix, Leana Garraway, and Larine Sluggett for their assistance with data analysis.
Financial Support
The full study was funded by the Canadian Institutes of Health Research (MOP 130260).
Conflicts of Interest
None.
Ethical Standards
The ethics committees of the research team approved the RRNII survey: University of Northern British Columbia Research Ethics Board (E2013.0320.037.02), University of Saskatchewan (Behavioural Research Ethics Board Certificate of Approval), University of Lethbridge (Certificate of Human Participant research), Aurora College (Scientific Research License), University of Montreal (Hospital Maisonneuve-Rosemont), and Dalhousie University (Health Science Research Ethics Board letter of approval).
