Introduction
The central premise of health equity that propelled the primary health care (PHC) movement initiated by the Declaration of Alma-Ata has resulted in health system reforms across many countries in the last few decades [World Health Organization (WHO), 2008]. The WHO proposed four key social values underpinning PHC, namely health equity, people-centred care, reliable health authorities, and promotion and protection of health within communities (WHO, 2008).
‘Primary health care’ encompasses delivery of ‘basic medical and curative care at the first level’, that is, ‘primary care’, and further includes activities related to health promotion, illness prevention, and determinants of health (eg, social, behavioural, and environmental; Canadian Nurses Association, 2005). Although primary care and PHC are often used to refer to the same concept, PHC is a holistic approach that involves multiple disciplines focused on the numerous factors associated with health, whereas primary care focuses mainly on basic medical and health maintenance services (Saskatchewan Ministry of Health, 2002). The Canadian Institutes of Health Research, Canada’s federal health research funding agency, recently introduced the term ‘community-based primary health care’ to refer to a continuum from primary prevention and health promotion to home care and palliative care, delivered in a range of locales [Canadian Institutes of Health Research (CIHR), 2014]. Interprofessional and interdisciplinary in nature, community-based PHC is coordinated across settings (eg, schools, homes, clinics, workplaces) and health care professionals (eg, nurses, pharmacists, social workers, physicians). This conceptualization of community-based PHC guided the present study.
In rural/remote areas of Canada and elsewhere, PHC reform involves introducing innovations in the organization of health service delivery to address geographic inequities and meet population health needs (eg, health promotion, chronic disease management; Banner et al., Reference Banner, MacLeod and Johnston2010). Although rural–urban differences vary by country, rural communities worldwide generally have poorer accessibility to health care services and resources than urban communities (Farmer et al., Reference Farmer, Prior and Taylor2012). Rural Canadians also typically exhibit poorer health outcomes than their urban counterparts (DesMueles et al., Reference DesMeules, Pong, Lagacé, Heng, Manuel, Pitblado, Bollman, Guernsey, Kazanjian and Koren2006; Williams and Kulig, Reference Williams and Kulig2011; White, Reference White2013), a situation partially rooted in inequities in social determinants of health including sociodemographics (eg, lower income and education), lifestyle (eg, higher rates of smoking and obesity, poorer dietary practices, lower physical activity levels), and geography (eg, degree of rurality) (DesMueles et al., Reference DesMeules, Pong, Lagacé, Heng, Manuel, Pitblado, Bollman, Guernsey, Kazanjian and Koren2006; White, Reference White2013). Rural residents thus require additional time, travel, and finances to meet their health care needs (Grzybowski and Kornelsen, Reference Grzybowski and Kornelsen2013). Internationally, rural/remote communities are becoming hubs of innovation in PHC delivery to address these issues (Wakerman and Humphreys, Reference Wakerman and Humphreys2011), encouraging the growth and integration of services across acute care and community sectors, with accompanying advanced and expanded practice roles for rural/remote nurses, paramedics, and other health professionals providing PHC services (Mitton et al., Reference Mitton, Francois, Masucci, Wong and Law2011).
Nurses fill a range of roles in the context of PHC, working in settings where care is individual/family focused (eg, home care), community focused (eg, public health), and integrated (eg, general practice) (Banner et al., Reference Banner, MacLeod and Johnston2010). One challenge to PHC reform shared by most countries is the redefinition of practice roles and functions to meet reform demands, specifically the resistance offered by traditional models of physician-centred care (Mitton et al., Reference Mitton, Francois, Masucci, Wong and Law2011; Mable et al., Reference Mable, Marriott and Mable2012). This resistance can result in some health care professionals, particularly advanced practice nurses, being underutilized and undervalued as integral members of collaborative PHC teams (Lavis, Reference Lavis2011). Barriers that hinder the integration of nurse practitioners (NPs) in particular within the Canadian PHC system include differences in legislation across provinces and territories (eg, policies restricting NPs from prescribing and referring to medical specialists) and variations regarding the educational preparation requirements of NPs (Donald et al., Reference Donald, Martin-Misener, Lukosius, Kilpatrick, Kaasalainen, Carter, Harbman, Bourgeault and DiCenso2010).
Primary Health Care Engagement
The current period of PHC reform requires unambiguous constructs plus reliable and valid indicators of those constructs, to assess ongoing changes in the PHC system (Williams, Reference Williams2011). To this end, Haggerty and colleagues developed definitions of 24 PHC attributes considered relevant in Canada as well as internationally, in consultation with Canadian health care providers, decision-makers, and academics (Haggerty et al., Reference Haggerty, Burge, Gass, Lévesque, Beaulieu, Pineault and Santor2007). Levesque et al. (Reference Levesque, Haggerty, Burge, Beaulieu, Gass, Pineault and Santor2011) further characterized these attributes as essential to either professional or community-oriented models, or both. Professional models represent the traditional physician-centred care model (ie, primary care), staffed by predominantly fee-for-service family physicians serving patients’ general medical needs. Community-oriented models involve multiple health and social professionals delivering services aimed at improving individuals’ health as well as serving their medical needs, in community- or public-administered organizations (Levesque et al., Reference Levesque, Haggerty, Burge, Beaulieu, Gass, Pineault and Santor2011). This conceptualization of community-oriented models aligns with the community-based PHC definition that guided the present study.
A number of instruments are currently available to evaluate dimensions of PHC delivery. However, many of these instruments were developed to evaluate patient rather than provider experiences (Flocke, Reference Flocke1997; Safran et al., Reference Safran, Kosinski, Tarlov, Rogers, Taira, Lieberman and Ware1998; Shi et al., Reference Shi, Starfield and Xu2001; Wong and Haggerty, Reference Wong and Haggerty2013). Further, many of these tools were developed for use in primary care rather than PHC settings. Fewer instruments are available to assess key PHC dimensions from the perspective of physicians (Schoen et al. Reference Schoen, Osborn and Huynh2006) and other health care providers, including NPs, physiotherapists, pharmacists, and others (Dahrouge et al., Reference Dahrouge, Hogg, Russell, Geneau, Kristjansson, Muldoon and Johnston2009; Johnston and Burge, Reference Johnston and Burge2013).
Health care professionals are well placed to observe many of the activities and functions that characterize PHC delivery, for instance, the activities that promote and maintain accessibility, interdisciplinary collaboration, and comprehensive care. Workplaces that are involved in these functions to a greater degree may be said to exhibit a higher level of PHC engagement on the key dimensions being assessed. There is significant merit in developing a provider-focused instrument that is relevant to multiple disciplines (eg, nurses, physicians, pharmacists, and occupational therapists) given that strengthening the interprofessional team-based nature of health care delivery is one of the key principles of PHC reform believed to underpin improved service access, quality, and equity (McPherson and McGibbon, Reference McPherson and McGibbon2010).
The purposes of this project were to (1) develop a new scale to measure the perceptions of rural/remote nurses regarding the engagement of their workplaces in key dimensions of PHC, (2) conduct a content evaluation of the newly developed Primary Health Care Engagement (PHCE) Scale, including item-by-item verification, (3) conduct an assessment of the psychometric properties of the PHCE Scale using data from a pilot survey of nurses with nursing experience in rural/remote Canada, and (4) use the findings from the psychometric assessment to refine the number of items in the PHCE Scale. The refined PHCE Scale has been included in a larger Canada-wide survey of rural/remote nurses; data collection with a sample of ∼10 000 nurses began in April 2014.
Methods
Design
The first of the three phases of this study focused on instrument development. The first phase consisted of a literature review to identify published measures of PHC, followed by expert consultation with our 16-member research team to identify essential dimensions of PHC in rural/remote settings. Our research team included 13 registered nurses (RNs)/NPs (10 of whom are nursing faculty), representing six provinces and one territory of Canada’s 13 provinces and territories. This phase concluded with the generation of six items for each of the dimensions by the scale developers (J.G.K. and E.C.W.).
The second phase involved an iterative process of content evaluation and item revision of a draft version of the new scale, by our research team and 19-member advisory team. Members of the advisory team represented nine provinces and territories as well as the federal level of public health services governance. In the final phase, a pilot survey was undertaken for the purpose of psychometric evaluation of the new instrument. On the basis of psychometric assessment, all subscales in the instrument were retained but each was subsequently trimmed to three to five items. The refined scale was later included in a larger nationwide survey, the Nursing Practice in Rural and Remote Canada II Study. This larger survey will investigate the nature of nursing practice in rural/remote Canada, with a goal to assist health service planners to improve service quality and access in rural/remote areas.
The aim of the three-phase design employed in the present study was to create an instrument that was comprehensive enough to reflect the essential dimensions of PHC yet included the smallest possible cluster of items (between three and five) within subscales that exhibited acceptable internal consistency. As noted by Furr and Bacharach (Reference Furr and Bacharach2008: 173), an instrument ‘…might not cover every conceivable facet of the construct, but hopefully the selected items reflect a fair range of elements relevant to the construct’, and is not so lengthy and time-consuming as to deter potential respondents. The length of the new instrument was of concern given the fact that it would be included in a subsequent wide-ranging 27-page survey of nurses in different professional roles [RNs, NPs, registered psychiatric nurses (RPNs), and licensed practical nurses (LPNs)]. Further psychometric testing based on data from the larger survey will involve exploratory factor analysis to test the proposed factor structure of the refined instrument. Convergent and discriminant evidence (Furr and Bacharach, Reference Furr and Bacharach2008) will also be gathered based on correlations with constructs believed to be related and unrelated to PHC engagement.
Instrument development
This process involved review by our 16-member research team of the 24 PHC attributes and their definitions developed by Haggerty et al. (Reference Haggerty, Burge, Gass, Lévesque, Beaulieu, Pineault and Santor2007). As shown in Table 1, our research team identified 14 attributes as most relevant to rural/remote PHC and grouped these into 10 dimensions for the purpose of subscale development, namely (1) accessibility/availability, (2) patient–provider relationship, (3) continuity, (4) population orientation, (5) community participation, (6) equity, (7) intersectoral team, (8) interdisciplinary collaboration, (9) quality improvement, and (10) comprehensiveness.
Dimensions (subscales) of the Primary Health Care Engagement (PHCE) Scale, source attributes, and definitions
a Attributes and definitions reproduced from Haggerty et al. (Reference Haggerty, Burge, Gass, Lévesque, Beaulieu, Pineault and Santor2007). Adapted with permission from Haggerty et al. (Reference Haggerty, Burge, Gass, Lévesque, Beaulieu, Pineault and Santor2007). Copyright© 2007 American Academy of Family Physicians. All rights reserved.
b Term ‘Patient–Provider Relationship’ derived from Hogg et al. (Reference Hogg, Rowan, Russell, Geneau and Muldoon2008).
The process of identifying the attributes that were most relevant to rural/remote PHC was guided by the work of Levesque et al. (Reference Levesque, Haggerty, Burge, Beaulieu, Gass, Pineault and Santor2011). Levesque et al. ranked the 24 PHC attributes developed by Haggerty et al. (Reference Haggerty, Burge, Gass, Lévesque, Beaulieu, Pineault and Santor2007) on a 5-point scale from ‘somewhat important’ to ‘essential’ with respect to: (a) community-oriented models of PHC, (b) professional models of PHC, and (c) both models (Levesque et al., Reference Levesque, Haggerty, Burge, Beaulieu, Gass, Pineault and Santor2011). The attributes that we found most relevant to rural/remote PHC were those attributes that Levesque et al. considered essential or very important to community-oriented models of PHC, given that these models aligned best with the community-based PHC definition guiding the present study. For four of the 10 dimensions depicted in Table 1, Levesque et al. ranked the original attributes as essential to both professional and community-oriented PHC models (accessibility/availability, patient–provider relationship, comprehensiveness, and continuity). Levesque et al. rated the original attributes of four dimensions as essential to community-oriented models alone (population-orientation, community participation, equity, and interdisciplinary collaboration). Quality improvement was considered very important to both models, while intersectoral team was classified as very important to community-oriented models, but only somewhat important to professional models. Of the 24 PHC attributes developed by Haggerty et al., the 10 attributes that our team considered least relevant to rural/remote PHC were informational continuity, technical quality of clinical care, clinical information management, system integration, advocacy, family-centred care, whole-person care, accountability, availability, and efficiency/productivity. These attributes were not included in our instrument.
Guided by the definitions in Table 1, the scale developers (J.G.K. and E.C.W.) generated a 60-item PHCE Scale that consisted of six items for each of the 10 dimensions. We limited the number of items to a maximum of six with the understanding that the items with the lowest item-total correlations within each dimension, based on psychometric evaluation of pilot survey data, would be removed to create a refined version of the scale for a subsequent larger nation-wide survey of rural/remote nurses. The generated items were informed by key studies that addressed one or more of the dimensions [WHO, 1986; Flocke Reference Flocke1997, Shi et al., Reference Shi, Starfield and Xu2001; Canadian Institute for Health Information (CIHI), 2006; Davis et al., Reference Davis, Schoen, Schoenbaum, Doty, Holmgren, Kriss and Shea2007; Dahrouge et al., Reference Dahrouge, Hogg, Russell, Geneau, Kristjansson, Muldoon and Johnston2009; Bloch et al., Reference Bloch, Rozmovits and Giambrone2011; Levesque et al., Reference Levesque, Haggerty, Burge, Beaulieu, Gass, Pineault and Santor2011; Wong et al., Reference Wong, Browne, Varcoe, Lavoi, Smye, Godwin, Littlejohn and Tu2011; Saskatchewan Ministry of Health, 2012]. Nine items across four dimensions (accessibility/availability, patient–provider relationship, continuity, and population orientation) were minimally adapted with permission from a survey administered to primary care physicians and NPs (Dahrouge et al., Reference Dahrouge, Hogg, Russell, Geneau, Kristjansson, Muldoon and Johnston2009).
Content evaluation and item revision
A two-day in-person session was held in June 2012 with our 16-member research team, to review the pilot survey measures. During the two-day session, members of the research team evaluated the content, wording, and format of every item in the first version of the 60-item PHCE instrument to reflect the context of PHC in rural/remote communities. Based on the definitions in Table 1, team members were specifically requested to flag for removal or revision those items that were irrelevant to each dimension and to ensure that the full range of content relevant to each dimension was included (Cook and Beckman, Reference Cook and Beckman2006; Furr and Bacharach, Reference Furr and Bacharach2008). In addition, region- and program-specific terminology were revised to broaden the scale’s appeal to nurses across Canada; language was revised to be inclusive with regard to gender, sexual orientation, and vulnerable groups; items were modified to reflect an orientation to health promotion as well as to medical care; and formatting was revised to include italics for emphasis and examples for illustrative purposes. The 16-member research team received updated versions of the instrument and participated in two team teleconferences. Subsequent versions of the instrument underwent an iterative process of review by the research team during seven teleconference meetings, until consensus was achieved on the inclusion of every item in the preliminary 60-item PHCE instrument.
Pilot survey
The preliminary 60-item PHCE Scale was included in a pilot survey of nurses with nursing experience in rural/remote settings. Psychometric evaluation in the pilot survey phase was undertaken to refine the 60-item scale to a 40-item version.
The survey consisted of four parts (demographics, current employment, work community, and work setting), plus one adapted 12-item Work Satisfaction Scale (Williams et al. Reference Williams, Konrad, Linzer, McMurray, Pathman, Gerrity, Schwartz, Scheckler, Van Kirk, Rhodes and Douglas1999) and three newly developed scales (60-item PHCE Scale, 42-item Practice Resources Scale, and 60-item Practice Demands Scale). The central purpose of the pilot survey was to conduct a psychometric evaluation of each of the four scales. These analyses were used to reduce the number of items in each of the longer scales, in order to include refined versions in a subsequent larger nation-wide survey of rural/remote nurses. Pilot testing also involved evaluation of online administration of the survey.
Participants
The pilot survey population consisted of all RNs, NPs, LPNs, and RPNs in current practice in Canada with nursing experience in rural or remote locations. Eligible participants had current or previous experience working in rural or remote locations and met one of the following criteria: (1) were currently employed in nursing, (2) on leave from nursing for fewer than six months or (3) retired but occasionally employed in nursing. The target sample was 100 participants based on power analysis to determine the minimum sample size necessary for internal consistency reliability testing of a 60-item scale (Bonett, Reference Bonett2002).
Participants were recruited using a snowball sampling method. Research team members each received six paper survey packages as well as a recruitment email containing a link to the online version of the survey (and access code) to distribute to peers and colleagues across rural/remote Canada. In addition, requests were made to a small number of national nursing organizations to forward the recruitment email to membership, advertise the study in paper and electronic newsletters, and post the recruitment advertisement on social media sites.
Data collection
Data were collected by cross-sectional English mail and online questionnaire from September to December 2013. Participants had the option of completing either an online or paper version of the survey. Both versions included an information sheet, feedback form, and survey questionnaire. The paper package also included a cover letter and self-addressed stamped envelope.
Preliminary instrument
The preliminary 60-item PHCE Scale included in the pilot survey contained 10 subscales: accessibility/availability, patient–provider relationship, continuity, population orientation, community participation, equity, intersectoral team, interdisciplinary collaboration, quality improvement, and comprehensiveness (Table 1). Participants were instructed to respond to the items in relation to their primary workplace (where they spent most of their time in the past 12 months) and the catchment area served by their primary workplace. Each subscale consisted of six items, two of which were negatively worded and four positively worded. The negatively worded items were randomly distributed within each subscale. The items used a 5-point Likert scale of 1 (strongly disagree), 2 (disagree), 3 (neutral), 4 (agree) 5 (strongly agree) plus a ‘not applicable’ option. Negatively worded items were reverse scored. Higher subscale scores indicated perceptions of a higher degree of workplace engagement in 10 key dimensions of PHC.
Statistical analyses
All data were analysed using SPSS 20.0. Demographic characteristics were investigated with descriptive statistics, including frequencies, mean, SD, and range. Case mean imputation was performed for every participant’s subscale that was missing 25% or less of the items (ie, one item) in the preliminary 60-item instrument, and in the subscales with four or more items in the refined 40-item instrument (El-Masri and Fox-Wasylyshyn, Reference El-Masri and Fox-Wasylyshyn2005). Where a subscale was missing more than 25% of the items (ie, two or more) in the preliminary 60-item instrument, or one item in the three-item subscales in the refined 40-item instrument, that participant’s subscale was discarded.
After performing case mean imputation for missing values, reliability analysis was conducted with each subscale. Reliability assessment is an important element of the process of developing and refining subscales (Furr and Bacharach). Using the item discrimination method (Furr and Bacharach, Reference Furr and Bacharach2008) in addition to judgement regarding the theoretical value of each item, the corrected item-total correlation of each item within each subscale was assessed. A low item-total correlation suggests inconsistency between an item and the test (ie, subscale) as a whole. Item-total correlations lower than 0.20 were considered very weak, 0.20 to 0.39 weak, 0.40 to 0.59 moderate, 0.60 to.79 strong, and 0.80 and above very strong (Swinscow and Campbell, Reference Swinscow and Campbell2002). The item with the lowest item-total correlation in each subscale was removed and the Cronbach’s α coefficient for the subscale with the remaining items was evaluated. This process was repeated for each subscale as necessary. A Cronbach’s α coefficient of 0.70 suggested modest and acceptable internal consistency reliability for the subscales, considering the early stage of this research (Nunnally and Bernstein, Reference Nunnally and Bernstein1994). Cronbach’s α coefficient can be computed on the basis of one test administration, as in the current cross-sectional pilot survey, and as such is generally used more than other statistical tests to demonstrate internal consistency reliability (DeVon et al. Reference DeVon, Block, Moyle-Wright, Ernst, Hayden, Lazzara, Savoy and Kostas-Polston2007; Tavakol and Dennick, Reference Tavakol and Dennick2011).
Results
Sample characteristics
The sample included 89 participants recruited by snowball sampling method. The majority of participants practiced in the British Columbia/Alberta region (n=60; 69.0%), followed by Saskatchewan/Manitoba (n=15; 17.2%). As shown in Table 2, 92.1% of participants (n=82) were women and the average age was 44.8 years (range 24–82, SD=12.4). Most nurses indicated their primary position as staff nurse/direct care provider (n=60; 67.4%) and their registration status as RN (88.8%). The most frequently reported place of employment was community health centre (n=29; 32.6%), followed by hospital (n=24; 27%) and public health department/unit (n=16; 18%). The majority of participants worked in communities with populations under 5000 (n=49; 55.0%) and one in three nurses lived outside of their work community (n=29; 32.6%).
Demographic characteristics of survey respondents (n=89)
a Non-response (n=1).
b Non-response (n=1).
c May hold more than one registration.
Psychometric evaluation
As shown in Table 3, item-total correlations within the subscales of the preliminary 60-item PHCE Scale ranged from −0.13 to 0.76. Four of the 60 items correlated very weakly (r<0.20) with the total subscale correlations. A further 16 items correlated weakly (0.20⩽r<0.40), 25 items correlated moderately (0.40⩽r<0.60), and 15 items correlated strongly (0.60⩽r<0.80). Within each subscale, the item with the lowest item-total correlation was removed one at a time, and the contribution of each remaining item to the internal consistency reliability for that subscale was evaluated. After item removal, 10 refined subscales in the 40-item PHCE Scale remained (Table 4).
Item analysis of the preliminary 60-item Primary Health Care Engagement (PHCE) Scale (n=89)
Note: Italics appeared in original items; bolded items were retained in the final 40-item PHCE Scale; r=corrected item−total correlation.
a Reverse scored.
Mean scores and internal consistency reliability of subscales in the preliminary 60-item Primary Health Care Engagement (PHCE) Scale and final 40-item PHCE Scale
a All subscales in the final 40-item PHCE Scale contained four items, with the exception of Population Orientation (five items), Equity (five items), Interdisciplinary Collaboration (three items), and Comprehensiveness (three items).
As indicated in Table 4 by comparing the 60-item subscales and the 40-item subscales, the α estimates for seven subscales increased when the number of items in those subscales was reduced. The α estimates decreased for the remaining three subscales. Satisfactory Cronbach’s α coefficients ranging from 0.70 to 0.83 were estimated for seven of the 10 refined subscales (accessibility/availability, continuity, population orientation, community participation, intersectoral team, interdisciplinary collaboration, and quality improvement). Alpha values for three of the 10 refined subscales fell below 0.70, ranging from 0.61 to 0.64 (equity, comprehensiveness, and patient–provider relationship). The Cronbach’s α estimate was 0.93 (n=63) for the preliminary 60-item PHCE Scale. After refining the subscales, the α value was 0.91 (n=66) for the refined 40-item PHCE Scale.
Refined instrument
The refined 40-item PHCE Scale consisted of two five-item subscales (population orientation and equity), six four-item subscales (accessibility/availability, patient–provider relationship, continuity, community participation, intersectoral team, and quality improvement), and two three-item subscales (interdisciplinary collaboration and comprehensiveness). Before including the 40-item PHCE Scale in a larger nation-wide survey of rural/remote nurses, the tool was further revised as follows: items B1, B4, J2, and J4 (Table 3) were revised to remove the emphasis on ‘should’, since participant feedback indicated that these items caused confusion. ‘Patient–provider relationship’ was renamed ‘patient-centred care’, as this new name better reflects the overall construct. Negatively scored items that were removed from the subscales were not replaced with other negatively scored items, therefore, not all of the subscales in the refined instrument contain at least one reverse scored item. The refined 40-item PHCE Scale is available from the authors.
Discussion
The purpose of the present study was to assess the psychometric properties of a new scale developed to measure rural/remote nurses’ perceptions regarding the engagement of their workplaces in key dimensions of PHC. Higher subscale scores reflected perceptions of a greater degree of workplace engagement on 10 key dimensions of PHC (accessibility/availability, patient–provider relationship, continuity, population orientation, community participation, equity, intersectoral team, interdisciplinary collaboration, quality improvement, and comprehensiveness). The refined 40-item PHCE Scale, comprised three to five items in each of 10 subscales, exhibited good internal consistency reliability (α=0.91) when tested in a pilot survey of Canadian nurses with experience practicing in rural or remote settings. Specifically, all but three of the 10 refined PHCE subscales demonstrated acceptable reliability (α⩾0.70), namely equity, comprehensiveness, and patient–provider relationship. It is possible that the items in these three subscales were measuring ‘heterogeneous constructs’, that is, more than one dimension, thus leading to lower α estimates (Tavakol and Dennick, Reference Tavakol and Dennick2011).
Significant PHC reform has been well underway across Canada since the late 1990s, leading to the introduction of the Health Transition Fund to support pilot testing of new PHC models and capacity building across the country (Mable et al., Reference Mable, Marriott and Mable2012). However, Canada, similar to other WHO member countries, has yet to achieve a universal health care system with PHC principles at its centre as envisioned in the Alma-Ata Declaration on Primary Health Care (Gauld et al., Reference Gauld, Blank, Burgers, Cohen, Dobrow, Ikegami, Kwon, Luxford, Millett and Wendt2012). Implementing a PHC system in Canada that is integrated and coordinated has been a challenging task for numerous reasons (Wilson and Lavis, Reference Wilson and Lavis2014). These challenges include a decentralized health care delivery system across 13 provinces and territories, the complexity of alternate payment structures for non-physicians involved in the delivery of PHC services (ie, other than fee-for-service), and the slow pace of implementing electronic health records and health information systems (Gauld et al., Reference Gauld, Blank, Burgers, Cohen, Dobrow, Ikegami, Kwon, Luxford, Millett and Wendt2012).
To evaluate the quality of patient encounters with the PHC system during on-going reform, a number of different instruments may be used (Flocke, Reference Flocke1997; Safran et al., Reference Safran, Kosinski, Tarlov, Rogers, Taira, Lieberman and Ware1998; Shi et al., Reference Shi, Starfield and Xu2001; Haggerty et al., Reference Haggerty, Burge, Beaulieu, Pineault, Beaulieu, Levesque, Santor, Gass and Lawson2011; Wong and Haggerty, Reference Wong and Haggerty2013). While it is important to account for the patient perspective, it is equally important to solicit the perspectives of health care professionals who have unique insight into the system, in efforts to assess ongoing changes in the PHC system. The CIHI recently made available a survey which measures providers’ perspectives on nine PHC dimensions (information technology, quality and safety processes, accountability, health human resources, team functioning, organizational adaptiveness, provider satisfaction, coordination of care, and collaboration; Johnston and Burge, Reference Johnston and Burge2013). While there is some overlap with the refined PHCE Scale in the present study, several dimensions are missing in the CIHI instrument, namely accessibility, patient-centred care, population orientation, equity, community participation, and comprehensiveness. These particular dimensions are important to a holistic PHC approach that involves multiple disciplines focused on the numerous factors associated with health. As a recent systematic review found, the dimensions of access, comprehensiveness, population orientation, and patient-centredness (ie, patient–provider relationship) have been employed to evaluate quality in PHC system performance in several international projects in the last two decades (Simou et al., Reference Simou, Pliatsika, Koutsogeorgou and Roumeliotou2013). Obtaining the perspective of health care professionals on these particular dimensions of PHC is therefore a worthwhile endeavour.
Some study limitations should be considered when interpreting the findings. The first limitation concerns the fact that a content validity index was not calculated; rather, an iterative content evaluation process was undertaken that involved our 16-member research team and 19-member advisory team. Second, the PHCE Scale has been pilot tested solely with nurses, the majority of whom were RNs (88.8%), and has not been tested with other health care professionals. Third, reducing the number of items in the subscales may affect the viability of some subscales after exploratory factor analysis is conducted in the larger study. Further limitations concern the lack of factor analysis in the present study. For a multidimensional construct, a statistical procedure such as factor analysis is typically employed in the instrument development process to help determine relationships between items and identify factors within the construct (DeVon et al., Reference DeVon, Block, Moyle-Wright, Ernst, Hayden, Lazzara, Savoy and Kostas-Polston2007). Relying upon the item discrimination method to identify items for removal may have resulted in the loss of useful items, since high internal consistency reliability of a subscale is necessary, but not sufficient, evidence of validity (Cook and Beckman, Reference Cook and Beckman2006). A sample of 300 to 400 would be necessary for factor analysis of a 60-item scale, based on sample size parameters for this statistical technique (DeVellis, Reference DeVellis2003). However, a firm start date for the larger survey was previously established in co-operation with many registration associations across Canada, therefore, it was not feasible to recruit a sample of this size due to time constraints. Although factor analysis is a typical step in the instrument development process, it is important to first gather content evidence by searching for previously published instruments and identifying constructs (Cook and Beckman, Reference Cook and Beckman2006). We placed significant emphasis on gathering content evidence by drawing on previous research that identified essential dimensions of PHC (Haggerty et al., Reference Haggerty, Burge, Gass, Lévesque, Beaulieu, Pineault and Santor2007; Levesque et al., Reference Levesque, Haggerty, Burge, Beaulieu, Gass, Pineault and Santor2011), and further extended this line of inquiry by developing items that were informed by other relevant research (Flocke, Reference Flocke1997; Shi et al., Reference Shi, Starfield and Xu2001; Dahrouge et al., Reference Dahrouge, Hogg, Russell, Geneau, Kristjansson, Muldoon and Johnston2009; Bloch et al., Reference Bloch, Rozmovits and Giambrone2011; Levesque et al., Reference Levesque, Haggerty, Burge, Beaulieu, Gass, Pineault and Santor2011; Wong et al., Reference Wong, Browne, Varcoe, Lavoi, Smye, Godwin, Littlejohn and Tu2011) and reports (WHO, 1986; CIHI, 2006; Davis et al., Reference Davis, Schoen, Schoenbaum, Doty, Holmgren, Kriss and Shea2007; Saskatchewan Ministry of Health, 2012).
Conclusion
In the present study that solicited the views of 89 Canadian nurses with experience practicing in rural or remote communities, a new 40-item PHCE Scale demonstrated a good reliability estimate (0.91, n=66) for the overall refined 40-item PHCE Scale, and acceptable reliability estimates (α⩾0.70) in seven of 10 subscales. With their unique insight into the organization and delivery of health care, nurses provide a window into the functions and activities that characterize PHC delivery in their workplaces.
The current study provided an initial assessment of reliability and preliminary evidence of validity. The psychometric properties of the new 40-item PHCE scale will be further assessed in a larger nation-wide survey of rural/remote nurses, for which data collection began in April 2014. The larger study will provide an opportunity to test the structure of the refined 40-item PHCE Scale, specifically whether the factors (subscales) will be supported. Analysis of correlations between the subscales and variables in the larger study will provide additional evidence of validity. We will also compare the degree of PHC engagement across work setting (eg, community health centre, multidisciplinary PHC clinic, family practice unit), type of nurse (RNs, NPs, RPNs, and LPNs), region (ie, province and territory), and population of work community. This new instrument has the potential to improve our understanding of key dimensions that characterize PHC systems in numerous countries, to assess ongoing changes in these systems. Researchers may consider employing the new 40-item PHCE Scale in research with other health care professionals (eg, family physicians, occupational therapists, dietitians), in urban as well as rural practice settings, in nations where PHC reform is underway.
Acknowledgements
The authors would like to thank the nurses who participated in the Nursing Practice in Rural and Remote Canada II pilot survey. They are also grateful to Nadine Meroniuk, Jessica Place, Leana Garraway, and Larine Sluggett for their assistance with data analysis and manuscript preparation.
Financial Support
The full study is funded by the Canadian Institutes of Health Research.
Conflicts of Interest
None.
Ethical Standards
The pilot survey was approved by the ethics committees of the research team: University of Northern British Columbia Research Ethics Board (E2013.0320.037.00), University of Saskatchewan (Behavioural Research Ethics Board Certificate of Approval), University of Lethbridge (Certificate of Human Participant research), Aurora College (Scientific Research Licence), University of Montreal (Hospital Maisonneuve-Rosemont), and Dalhousie University (Health Science Research Ethics Board letter of approval).
