Introduction
Integration is an organizational strategy for connecting the health system, enhancing performance, and improving quality of care (Kodner, Reference Kodner2009). Important components of integration include communication in a standardized clinical language, interdisciplinary collaboration, integrated electronic information systems, and appropriate funding mechanisms (Suter, Oelke, Adair, & Armitage, Reference Suter, Oelke, Adair and Armitage2009). For persons with chronic and complex health conditions, enhanced integration can contribute to better health outcomes, cost-effectiveness, and quality of care (Martínez-González, Berchtold, Ullman, Busato, & Egger, Reference Martínez-González, Berchtold, Ullman, Busato and Egger2014).
The real-world application of integrated approaches has been suboptimal. Despite a large body of international literature on best practices, surveys show that only 24 per cent of Canadian primary care providers communicate with home care providers about the needs and services of their patients (Doty, Tikkanen, Shah, & Schneider, Reference Doty, Tikkanen, Shah and Schneider2019). Additionally, one American study found that 96 per cent of home care providers felt that their inability to obtain outside clinical information about their patients was problematic, and 73 per cent said that with access to outside clinical information, they would need to make fewer referrals to emergency departments (Vaidya et al., Reference Vaidya, Shapiro, Papa, Kuperman, Ali, Check and Lipton2012). The fragmentation between home care and primary care may prevent health care providers from fully appreciating a patient’s clinical complexity and, as a result, limit their ability to provide optimal care. Fragmentation is also associated with delayed care provision, repeat hospitalization, duplicate assessment, and other leading causes of adverse events (Masotti, McColl, & Green, Reference Masotti, McColl and Green2010; Porter, Herring, & Levinton, Reference Porter, Herring and Levinton2007; Toscan, Mairs, Hinton, Stolee, & InfoRehab Research Team, Reference Toscan, Mairs, Hinton and Stolee2012). However, this subject has been poorly researched in Canadian contexts. Therefore, generating evidence on innovations for enhancing the integration of care was identified as a strategic priority in the Canadian Institute of Health Services and Policy Research Strategic Plan for 2021 to 2026 (Canadian Institutes of Health Research, 2021).
One opportunity for enhancing integration between home care and primary care may be reinventing how results from the interRAI Home Care (interRAI-HC) assessment are used. The interRAI-HC is a valid and reliable comprehensive clinical assessment instrument used by trained assessors in home care to support care provision and improve health care quality (De Almeida Mello, Hermans, Van Audenhove, Macq, & Declercq, Reference De Almeida Mello, Hermans, Van Audenhove, Macq and Declercq2015; Gray et al., Reference Gray, Berg, Fries, Henrard, Hirdes, Steel and Morris2009; Landi et al., Reference Landi, Onder, Tua, Carrara, Zuccalá and Gambassi2001). It uses open- and closed-ended questions to obtain an overall picture of client health. As an instrument that is part of a suite of instruments used in multiple health care sectors, the interRAI-HC can be used to establish a shared understanding of patient needs between care settings, support care planning and transitions, reduce assessment duplication, and support the provision of high-quality integrated care (Nova, Zarrin, & Heckman, Reference Nova, Zarrin and Heckman2020b). More information on interRAI and the interRAI suite of instruments can be found at https://interrai.org.
However, the interRAI-HC is not being used to its full potential. While interRAI instruments are used across most of the Canadian health sector, many primary care providers in Ontario are unfamiliar with the interRAI-HC or are unaware of the functionalities and information available within the tool to support care planning (Nova et al., Reference Nova, Zarrin and Heckman2020b). Additionally, insufficient attention has been paid to the usability of interRAI information in clinical contexts (LUCAS KU Leuven, 2019). Usability is defined in this paper as the ability for users to learn, understand, and operate a tool or system (Nielsen, Reference Nielsen2017). The most common criticisms of the interRAI-HC among clinicians who use it are that it is delivered inconsistently and there is a disconnect between the assessment results and goals of care (Stolee et al., Reference Stolee, Steeves, Manderson, Toscan, Glenny and Berg2010).
The Patient Falls Risk Report
The Patient Falls Risk Report is a one-page report that was originally designed by the researchers of this study with knowledge from preliminary research and the Behaviour Change Wheel theoretical framework (Guthrie et al., Reference Guthrie, Pitman, Fletcher, Hirdes, Stolee and Poss2014; Michie, Atkins, & West, Reference Michie, Atkins and West2014; Nova, Zarrin, & Heckman, Reference Nova, Zarrin and Heckman2020a, Reference Nova, Zarrin and Heckman2020b). It relays information derived from the interRAI-HC assessment about home care client falls risk, particularly if the client is at moderate or high risk of future falls. This measure has high predictive accuracy and is based on a prior report of one fall (moderate risk) or multiple falls (high risk) over the last 180 days (Norman & Hirdes, Reference Norman and Hirdes2020). The original report held structured falls-related information derived from the interRAI-HC, including previous falls, cognitive impairment, pain, foot problems, inappropriate medication use, and physical activity levels. It also listed recommended interventions from the American Geriatrics Society and British Geriatrics Society Clinical Practice Guidelines for Prevention of Falls in Older Persons (Panel on Prevention of Falls in Older Persons, American Geriatrics Society, & British Geriatrics Society, 2011). Each of the concerns listed in the Patient Falls Risk Report are prevalent among home care clients (Canadian Institute for Health Information, 2018), can be addressed in primary care settings, and may go undiscussed, undisclosed, undetected, or deprioritized (AuYoung et al., Reference AuYoung, Linke, Pagoto, Buman, Craft and Richardson2016; Howland et al., Reference Howland, Hackman, Taylor, O’Hara, Liu and Brusch2018; Inouye, Reference Inouye1994; Mackenzie & McIntyre, Reference Mackenzie and McIntyre2019; Mueller et al., Reference Mueller, Klaassen-Mielke, Penner, Junius-Walker, Hummers-Pradier and Theile2010; O’Brien, Shields, Oh, & Fowles, Reference O’Brien, Shields, Oh and Fowles2017; Panel on Prevention of Falls in Older Persons et al., 2011; Schofield, Reference Schofield2018; Williams et al., Reference Williams, Blake, Cherry, Alcacer-Pitarch, Edwards and Hopkinson2017; Wilson, Kirwan, Dures, Quest, & Hewlett, Reference Wilson, Kirwan, Dures, Quest and Hewlett2017). The original Patient Falls Risk Report is shown in Figure 1. See supplementary material for a rationale for why each actionable component is included.
Original patient falls risk report.
In theory, upon implementation, untrained primary care providers would receive the report by fax via the Client Health and Related Information System (CHRIS), a Web-based electronic decision support and document management system still in use as of mid-2022 that allows for the automated exchange of records (Health Shared Services Ontario, 2017; Ontario Association of Community Care Access Centres, 2016). Recipients would then schedule an appointment with the patient to discuss their results, collect missing information, and develop a care plan, as would be expected in normal primary care practice. Since falls are highly preventable with timely screening and assessment, we believed that sharing the report with primary care providers in a usable, actionable, and context-appropriate manner could enhance falls-related care planning (Guthrie et al., Reference Guthrie, Pitman, Fletcher, Hirdes, Stolee and Poss2014; Nova et al., Reference Nova, Zarrin and Heckman2020b; Stolee et al., Reference Stolee, Steeves, Manderson, Toscan, Glenny and Berg2010). The purpose of this study was to develop and test the usability of the Patient Falls Risk Report for sharing clinical information from home care to primary care in partnership with primary care providers.
Methods
This two-part, mixed methods pilot study employed in-depth interviews and short surveys to inform the development of the Patient Falls Risk Report. Using qualitative and quantitative methods to provide complementary perspectives on the report was expected to strengthen the reliability of our findings (Carter, Bryant-Lukosius, DiCenso, Blythe, & Neville, Reference Carter, Bryant-Lukosius, DiCenso, Blythe and Neville2014). AN, GH, LG, and MA all carried out the methods of this study.
Interviews
The purpose of the interviews was to develop the Patient Falls Risk Report using the feedback of practising primary care providers. Research shows that interventions are more likely to achieve their intended outcomes when the contexts, needs, and preferences of end-users are considered (Barnum, Reference Barnum2011).
Sample
From September to December 2019, we recruited 9 self-identified, English-speaking primary care providers for interviews who were practicing as family doctors, general practitioners, or nurse practitioners. A sample size of up to 10 was considered appropriate because, according to Kushniruk and Patel (Reference Kushniruk and Patel2004), 10 participants are enough to identify up to 80 per cent of surface level issues of usability. Additionally, a sample size of up to 10 was considered attainable given recruitment challenges identified in previous studies (Johnston et al., Reference Johnston, Liddy, Hogg, Donskov, Russell and Gyorfi-Dyke2010). As a clinician and leader in his chosen field, GH can be considered an insider in the clinical sphere. Therefore, GH was better connected to key informants and led recruitment. We used snowball and maximum variation sampling methods and sought to attain maximum variation on clinical background and training. Specifically, we aimed to include at least one nurse practitioner, one rural provider, one provider not in an interprofessional team, and one provider in an interprofessional health team. There were no exclusion criteria, and recruitment continued until the maximum variation aims were met and saturation was achieved.
Data collection procedures
From December 2019 to February 2020, AN performed one-on-one qualitative interviews with primary care providers in Ontario and Alberta, Canada. As an early career researcher and Canadian graduate student, AN can be considered an outsider to the clinical research context; AN understood the topic of study but was not assumed to understand the day-to-day activities of a primary care provider. Consequently, participants were primed to provide more explanation on topics that would have been familiar to an insider (Holmes, Reference Holmes2020). Data collection was guided by usability testing methods and a constructivist theoretical approach, which posits that knowledge is jointly constructed and exists relative to social, historical, and cultural contexts (Barnum, Reference Barnum2011; Guba & Lincoln, Reference Guba and Lincoln1994).
Prior to interviews, AN shared an information letter and consent form with participants. This document explained the value of obtaining their individual perspective as a primary care provider, informed them that confidentiality would be maintained, and emphasized that they could withdraw from the study at any time. The information letter is shown in Figure 2. Once written consent was obtained, data were collected with semi-structured interviews over the telephone or at the location of the participants’ choosing.
Information letter for interviews.
The first interview questions explored participant experiences with falls prevention to prepare the participants to respond to subsequent questions. The findings from this portion of the study have been published elsewhere (Nova, Heckman, Giangregorio, & Alarakhia, Reference Nova, Heckman, Giangrogorio and Alarakhiain press). Next, AN provided participants with a copy of the Patient Falls Risk Report with mock data and asked them to propose care planning options, if necessary. No training on how the Patient Falls Risk Report should be used in practice was provided before the interview so that the researchers could better ascertain the usability of the stand-alone report. Participants were then asked to describe their individual thoughts and feelings about using the report, if they would use it in their practice, and whether they believed it would change what they normally do in a patient encounter. Following this, participants were asked about their preferences for design and delivery of the report, potential barriers to implementation, and medicolegal risk. Finally, participants’ type (family doctor, general practitioner, or nurse practitioner) and duration in practice were identified, and additional comments and questions were solicited. The interview schedule is shown in Figure 3.
Full interview schedule.
The interviews were audio-recorded by a fingerprint-locked smartphone and, following each interview, reflexive notes on researcher thoughts, insights, and assumptions were taken by AN to improve dependability of the research process (Tobin & Begley, Reference Tobin and Begley2004). Within two weeks following each interview, the data were deidentified, transcribed, and stored by AN on a password-locked computer.
Data analysis
AN analysed the transcripts with NVivo 12 using iterative thematic analysis. Each iteration of analysis began with a combination of deductive and open coding. Specifically, a coding framework based on the behaviour change wheel, usability testing, and preliminary research guided but did not constrain coding (Barnum, Reference Barnum2011; Michie et al., Reference Michie, Atkins and West2014; Nova et al., Reference Nova, Zarrin and Heckman2020a, Reference Nova, Zarrin and Heckman2020b). AN then grouped useful codes into themes and reviewed and mapped each theme to ensure a relationship to the overarching research topic. At the end of each iteration, the findings were summarized, and the Patient Falls Risk Report was revised accordingly. While reflecting on their outsider and insider perspectives, the authors jointly made decisions about changing the report based on availability of items within the interRAI-HC, critique frequency, and relevance to falls prevention in primary care. Following the final analysis, AN linked the findings to direct quotes and created a one-page infographic of the synthesized and analysed data. This easy-to-read document, shown in Figure 4, was shared with participants via e-mail for member checking, to make sure that the findings resonate with the experiences of participants and enhance trustworthiness of the findings (Birt, Scott, Cavers, Campbell, & Walter, Reference Birt, Scott, Cavers, Campbell and Walter2016).
Infographic summary of the synthesized and analysed findings.
Surveys
The purpose of the surveys was to evaluate the usability of the revised Patient Falls Risk Report, shown in Figure 5, and strengthen the reliability of the qualitative findings with a complementary quantitative perspective (Carter et al., Reference Carter, Bryant-Lukosius, DiCenso, Blythe and Neville2014).
Revised patient falls risk report.
Sample
Ongoing survey recruitment was conducted by AN, GH, and MA from March to May 2020. We aimed to recruit at least 20 primary care providers or primary care residents using voluntary response sampling via newsletter, e-mail, and Twitter. The minimum sample size of 20 was determined using the System Usability Scale Calculator (Barnum, Reference Barnum2011; Sauro, Reference Sauro2011). Additionally, our ability to recruit primary care providers was heavily limited by the coronavirus disease (COVID-19) pandemic. In the end, we concluded that a sample of at least 20 would allow for an acceptable margin of error of about 10 points with a 95 per cent confidence interval (Sauro, Reference Sauro2011).
Data collection procedures
Data collection for the surveys was led by AN and took place from March to May 2020. To evaluate the revised Patient Falls Risk Report, participants were invited to five-minute anonymous surveys. When participants opened the link to the survey on the Qualtrics XM platform, the purpose and procedures of the study, a description of the Patient Falls Risk Report, researcher contact details, and an informed consent question were displayed. Consent could be withdrawn at any time prior to survey submission. Once consent was provided, participants were shown the revised Patient Falls Risk Report with mock data and asked to identify at least two care planning options. According to usability expert John Brooke, a participant should use the subject of evaluation before reporting on its usability to improve the chances that their true perceptions are captured (Brooke, Reference Brooke1996). To test usability, we used the survey questions listed in the System Usability Scale, a robust, reliable, and valid industry standard (Bangor, Kortum, & Miller, Reference Bangor, Kortum and Miller2008; Sauro, Reference Sauro2011). The System Usability Scale is used to score the usability of products and services on a scale of 0 to 100, where 100 represents the best possible usability (Bangor et al., Reference Bangor, Kortum and Miller2008). The survey, which is shown in Figure 6, asked participants to rate 10 statements about the usability of the Patient Falls Risk Report on a Likert scale of 1 to 5 (from “strongly disagree” to “strongly agree”, respectively). If uncertain on the best response, participants were told to select the middle of the scale (Brooke, Reference Brooke1996). Following completion of the survey, participants were given the option to provide additional comments in an open-ended comment box. The survey data were stored by AN in an Excel file on a password-locked laptop.
Usability testing survey.
Data analysis
To prepare the quantitative data, individual scores on the System Usability Scale were calculated for each survey by AN, using Excel 2004. Next, AN generated a histogram, box-and-whisker, and probability plot using SAS University Edition to evaluate the distribution of the scores. AN also conducted a Shapiro-Wilk test to determine whether the sample was selected from a population with a normal distribution and, in turn, determine whether the System Usability Scale was used appropriately (Sauro, Reference Sauro2011). If the scores were not normal, then there would have been concern around reporting percentile ranks, confidence intervals, and error, and sampling would need to continue (Sauro, Reference Sauro2011). Next, the range, maximum, minimum, median, and average of System Usability Scale scores were determined, and the standard deviation and confidence intervals for the average System Usability Scale score were calculated. Finally, AN performed benchmarking of the average score with the System Usability Scale curved grading scale (Sauro & Lewis, Reference Sauro and Lewis2016). This valid and reliable scale compares the usability of an innovation to thousands of other innovations (Sauro & Lewis, Reference Sauro and Lewis2016). Our aim was to achieve a score of at least 70, as recommended by Bangor et al. (Reference Bangor, Kortum and Miller2008). Finally, responses to the care planning activity and comments were reported for descriptive purposes. They would be analysed more thoroughly if quantitative analysis indicated a need to improve usability of the Patient Falls Risk Report. In this situation, comments would be analysed with thematic analysis, similar to the interviews.
Ethical Considerations
This study was reviewed for ethics clearance through a university research ethics committee and conforms to the Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans (TCPS2).
Results
Interviews
After analysing nine interviews, which were 26 minutes in length on average, and employing two iterations of qualitative analysis, we concluded that saturation was achieved since no new information emerged from the data. Four out of nine participants were nurse practitioners, one had worked in a rural practice, most practised or had previously practised as part of an interprofessional health team, and several had worked in a practice without an interprofessional team. The sample had practised primary care for 21.7 years on average. One participant was based in Alberta, Canada, and the rest were in Ontario, Canada.
Two overarching themes were identified from the interviews. The first was “Perspectives on enhancing utility of the Patient Falls Risk Report” and the second was “Perspectives on enhancing usability of the Patient Falls Risk Report”. “Utility” is defined as the quality of having the right features to solve a user need, and “usability” is the ability for users to learn, understand, and operate a tool or system (Nielsen, Reference Nielsen2017).
Theme 1: Perspectives on enhancing utility of the Patient Falls Risk Report
“It would help me provide really good care”. All nine participants claimed that they would use the Patient Falls Risk Report in their practice, and seven said that the tool would impact how they interact with patients. To illustrate, one nurse practitioner described that the Patient Falls Risk Report could facilitate their conversations with patients: I would probably show [my patient] the assessment and say ‘I’m really concerned about this for you… Let’s work together to try and make some changes to… decrease your risk and improve your health’ (NP1). The perception of utility stemmed from several described strengths of the report: It offers novel information from the home environment, supports critical thinking in assessment, prompts providers to address key risk factors in an evidence-informed way, and reminds them of recommended interventions and community resources. Most participants indicated that they would welcome the Patient Falls Risk Report: It would help me provide really good care (NP1). When asked about the impact of potential medicolegal risks of implementation, one physician explained: I don’t feel that as a problem. I mean anytime we get anything, be it a laboratory report, a consultant report, a nursing report. You know, if you ignore what it says then [laughs] that’s not good (MD5).
“It should provide more information”. To enhance utility of the Patient Falls Risk Report, several participants requested that it be more detailed (MD1). In particular, to tease out the serious falls (MD4), some participants wanted more detail around fracture risk, injuries, circumstances of the fall (e.g., loss of consciousness), and the patient’s ability to stand, sit, and walk around. Other general suggestions included the addition of narrative notes from the home care providers and information on demographics, drug and alcohol abuse, relevant chronic diseases (e.g., heart failure), and the state of the home environment. There were also several suggestions made regarding the list of risk factors. Four participants expressed the need for more investigation (NP3) around cognitive impairment. In terms of foot problems, one physician wanted more detail because, pain versus wounds versus deformities are very different things (MD1). Similarly, there was a suggestion to list the classes of high-risk medications prescribed to the patient and to indicate the prescriber. Participants also wanted to know more about pain – specifically: Where’s the pain? When does it happen? What makes it better? What makes it worse? (MD1). Is their pain well managed? … how is it managed? [and] Does it manage through physiotherapy? (NP3). Finally, some participants suggested changes to the list of recommendations, such as including a list of local services or health providers who could be referred to, key pieces of knowledge (i.e., bone mineral density, orthostatic vital thresholds), and the actions that home care had taken. To facilitate access to outside support in particular, a nurse practitioner suggested emphasizing an eReferral management platform: … because I think people forget that you can go onto [the platform] and actually find the falls programs in our area (NP4). While all of the suggestions can be considered important, it was not possible to add all of the details that participants requested without getting bogged down in detail (MD4) and exceeding a one-page length. Changes made to the utility of the report are listed in Table 1.
Changes made to the Patient Falls Risk Report for improving utility
“It’s just another paper to file”. Despite improvements made to the original report, two participants stated that receiving it would not change what they normally do in a patient encounter. One or these individuals indicated that they already collect the information in the report with custom-built comprehensive instruments and claimed to already know about the risk factors that their patients face. Specifically, one physician working in an interdisciplinary practice expressed the preference for an internally developed case finding program. The other provider felt that information provided was insufficient to support their needs. Instead, this nurse practitioner emphasized that more support was needed in managing the health of patients with complex conditions. The health care worker explained:
I think one of the struggles is time management. Trying to have the time to put towards these patients… It’s a great tool, but the bottom line is: what are the resources that [the report is] going to get for me?… It’s just another paper to file… it’s not helping me get any resources… I didn’t need a tool to tell me there’s a problem with this person. I just need some help to figure out how I’m going to take care of them. (NP3)
In the same vein, other participants seemed to agree that being overwhelmed by a heavy workload was an important concern: If everybody gives me a report like this for cognitive impairment, for mental health things, and… I have a hundred reports and I can’t do it, then I’d rather have zero reports (MD1). Therefore, several participants called for enhanced shared care planning: I think if it is more of a community responsibility… you don’t feel completely responsible, because oftentimes… it does come back on to you (MD4). However, challenges with shared care were identified: Shared care planning and interdisciplinary care, collaborative care, means different things to different people. And I think we all think we’re doing it, but we don’t do it very well (MD2). Suggestions for preferred interventions included an automatic community referral system or a report that identified a list of actions taken by home care providers.
Theme 2: Perspectives on enhancing usability of the Patient Falls Risk Report
“It’s easy to read”. Overall, the usability of the report was evaluated positively: I like how it’s laid out… I could look at this report in less than a minute and find out whether I need to act on it (MD1). Characteristics reported to increase usability and make the report easy to read (NP1, NP2, NP4, MD2) included its one-page length, intuitive organization, simplicity of language and content, selection and emphasis of a limited number of key risk factors, and action items.
The information in the report is “not entirely clear”. Several critiques on the usability of the report itself were also identified. The few participants who commented on the falls overview section of the report sought clarification and quickly found the answers to their questions within the report:
So, did my patient [pause] have a fall? I’m assuming they probably did – ‘high risk is based on report of multiple falls’ – So, then I’m assuming my patient did have a fall at least – I guess more than one. I guess that’s not entirely clear maybe with the statements below. (MD3)
Additionally, one participant explained that they would perform their own cognitive assessment based on the mock data, despite receiving the results of a valid cognitive assessment within the Patient Falls Risk Report, because: I didn’t get that they did a full cognitive assessment, because I don’t know what they did to get that answer (NP3).
The most prominent issue of usability that could be improved upon was lack of clarity around jargon in the report. In the first round of interviews, one participant expressed confusion around the interRAI jargon “moderately impaired 1” under the section on cognitive performance. In fact, several participants indicated confusion with the numbers on the report:
I guess moderately impaired would mean something, but the 1 beside it means absolutely nothing to me and wouldn’t to most primary care providers… most primary care people do not see RAI stuff at all… Is higher score worse or better?… That might want some clarification in case people needed to know. (MD3)
The word “triggered” within the medications and physical activities sections faced the same critique. Also in this section, the term ‘Inappropriate Medications’ was labelled a ‘judgmental term’ (MD1) since it implies blame on the prescriber and ignores contextual factors that may make the medication appropriate. Additionally, the meanings of the physical activities section in terms of lifestyle, physical condition, and motivations were unclear. Due to this lack of usability, the usefulness of the Physical Activities section was doubted by several participants. One participant, in particular, indicated that having conversations about exercise preferences are essential to developing an understanding of the item. To account for each of these critiques without increasing length, the information was reworded and rearranged. The changes made based on these critiques can be found in Table 2.
Changes made to the Patient Falls Risk Report for improving usability
“Fax is fine”. Another key issue of usability was delivery of the Patient Falls Risk Report. While fax was described as a fine means of delivery by four participants (NP1, NP2, NP4, MD3), electronic medical record integration of the report would be helpful (MD5), according to those with the systems that allowed for it. One physician with decades of primary care experience summarized personal views on the matter:
There’s lots of people want to eliminate the fax. But I think the reality is it’s pretty much in common use. I like it… The fax machine I think works for quite a few physicians still… I’m not the best person to ask because I depend on faxes. I still continue to get most of my messaging from other providers by fax. I have a process in place, but I think this is how people feel: that the fax machine is out of date, and they would rather there was electronic messaging. So, if I had a fully integrated [electronic medical record]. I may choose another method, but yeah, sorry. (MD2)
While most would need to manually scan faxes into their systems, a task requiring time and effort, some participants reported using the Health Report Manager, which automatically uploads faxes into their electronic medical record: Through health report manager it actually comes in electronically. But fax is fine (MD3). In summary, for most participants, integration with electronic medical records was preferred due to easier incorporation of the tool into their workflows.
Surveys
The sample size achieved for quantitative evaluation of the revised Patient Falls Risk Report was 27 primary care providers, or primary care residents. The data from these participants were approximately normally distributed (W-Statistic = 0.94); therefore, use of the System Usability Scale was considered appropriate (Sauro, Reference Sauro2011).
The overall System Usability Scale score for the revised Patient Falls Risk Report was 83.4 (SD = 11.99) which is considered excellent on the System Usability Scale Benchmarking Scale at the 90th to 95th percentile (Sauro & Lewis, Reference Sauro and Lewis2016). Additionally, the 95 per cent confidence interval was within the range of acceptable scores (Sauro & Lewis, Reference Sauro and Lewis2016). In other words, the survey determined that the report is highly usable. Descriptive statistics for System Usability Scale scores are shown in Table 3. Additionally, all participants completed the optional step and suggested care planning options. The most popular interventions suggested by survey participants were medication reviews, pharmacy referral, and referral to an exercise or balance program. There were also nine comments on the survey. Seven were short positive evaluations (e.g., Great report!), two were questions about the report (Who would complete this? and Were the recommendations lists at the bottom just general suggestions for everyone or were they specifically recommended for my patient situation?), and one was a suggestion to present risk factors in a more concise way.
Descriptive statistics for System Usability Scale scores
Max. = maximum; Min. = minimum.
Discussion
This research shows that the Patient Falls Risk Report has the potential to support primary care providers in identifying risk factors and care planning options for patients receiving home care. The report was also determined to be usable and easy to understand. However, the participants suggested that poor shared care planning should be a key consideration for the development and implementation of frailty-related information sharing tools.
Using structured approaches to sharing information between home care and primary care may motivate “good care” by enhancing informational continuity, which refers to the ability of clinicians use information about patient medical history, conditions, context, and values to provide appropriate care (Haggerty et al., Reference Haggerty, Reid, Freeman, Starfield, Adair and McKendry2003; Nova et al., Reference Nova, Zarrin and Heckman2020b). Using interRAI-HC information in clinical practice is proven to be beneficial in supporting high-quality health care provision (De Almeida Mello et al., Reference De Almeida Mello, Hermans, Van Audenhove, Macq and Declercq2015; Gray et al., Reference Gray, Berg, Fries, Henrard, Hirdes, Steel and Morris2009; Landi et al., Reference Landi, Onder, Tua, Carrara, Zuccalá and Gambassi2001). Proven benefits are integral for innovation sustainability (Fleiszer, Semenic, Ritchie, Richer, & Denis, Reference Fleiszer, Semenic, Ritchie, Richer and Denis2015). However, in information sharing, balance is key. While many participants in this study wanted more detail to be displayed in the report or had the means to conduct their own comprehensive case finding programs, many participants also described facing significant time constraints, burdensome workloads, and lack of support with managing complexity. Most primary care providers face heavy workloads (Agarwal, Pabo, Rozenblum, & Sherritt, Reference Agarwal, Pabo, Rozenblum and Sherritt2020), and addressing the complex needs of home care patients requires an interplay of clinical judgment and analytical thinking (Dhaliwal & Detsky, Reference Dhaliwal and Detsky2013). Therefore, we adjusted the report so that only information perceived as relevant and actionable to primary care was provided (Nova et al., Reference Nova, Zarrin and Heckman2020a). Of course, this adjustment was subjective and limited to information available within the original interRAI-HC assessment.
While the communication of relevant information is a necessary component of integration, providing more responsibility without minimizing burden in other ways can lead to loss of motivation, dissatisfaction, or burn-out in primary care providers (Agarwal et al., Reference Agarwal, Pabo, Rozenblum and Sherritt2020). Since the Patient Falls Risk Report provides information without offering direct support to address falls risk, some participants felt as though work would be “dumped” on them if this report were implemented. This view is justifiable. Adding detail on the actions of home care providers would have been beneficial for providing a more holistic view of patients, reducing the number of repeated referrals, and showing that home care is addressing patient health concerns (Heckman et al., Reference Heckman, Hillier, Manderson, McKinnon-Wilson, Santi and Stolee2013). Additionally, burden could be reduced by enhancing team-based care between primary care and allied health care providers (rather than care that is dominated by one provider), defining clear and manageable scopes of responsibility, and addressing electronic medical record limitations (Agarwal et al., Reference Agarwal, Pabo, Rozenblum and Sherritt2020).
The one-page length, intuitive organization, simplicity, and actionability of the Patient Falls Risk Report were usability-related strengths described by participants. Through the interviews, we were able to address many of the key criticisms of the interRAI-HC in the Patient Falls Risk Report (Stolee et al., Reference Stolee, Steeves, Manderson, Toscan, Glenny and Berg2010). For example, we attached information on community resources to the assessment results to support improved care planning. Engaging in usability-related changes may support incorporation of the report into existing primary care processes and structures, and, in turn, enhance sustainability of the innovation (Fleiszer et al., Reference Fleiszer, Semenic, Ritchie, Richer and Denis2015). Additionally, usability of the tool among participants was improved by decreasing interRAI jargon. While the use of a common interRAI language is a key characteristic of the instruments (Gray et al., Reference Gray, Berg, Fries, Henrard, Hirdes, Steel and Morris2009), “translating” the assessment results made it easier for participants to understand the assessment results presented to them. In the end, we found that the revised report was highly usable. High scores on the System Usability Scale correlate with greater task success (Bangor et al., Reference Bangor, Kortum and Miller2008; Kortum & Peres, Reference Kortum and Peres2014), and usability itself can lead to ease of learning, ease of use, and intuitiveness, thus saving users time and increasing satisfaction with a product (Barnum, Reference Barnum2011). In future research, we intend to pilot the Patient Falls Risk Report regionally to obtain feedback from primary care providers, patients, and clinicians beyond primary care and use a more rigorous investigation to measure whether receiving the intervention has an impact on care provision. Additionally, we recommend that future researchers intending to develop sustainable interRAI-HC innovations seek the perspectives of a diverse group of potential end users throughout the development process and on an ongoing basis as needs change.
As a final comment, fax delivery may limit the usability of the Patient Falls Risk Report. Electronic records are an effective, practicable, and acceptable means of delivery due to easier integration of information into primary care workflows and enhanced decision support capabilities for improving patient outcomes (Dhaliwal & Detsky, Reference Dhaliwal and Detsky2013; Heckman et al., Reference Heckman, Hillier, Manderson, McKinnon-Wilson, Santi and Stolee2013; Martínez-González et al., Reference Martínez-González, Berchtold, Ullman, Busato and Egger2014; Nova et al., Reference Nova, Zarrin and Heckman2020a). However, electronic medical records as a delivery mechanism may also be unaffordable and inequitable since information sharing is an expensive functionality, limited to few system vendors (Canadian Institute for Health Information, 2013). Ontario’s primary care sector needs more standardized data collection and management before delivery of the Patient Falls Risk Report by electronic medical record becomes feasible (Kortum & Peres, Reference Kortum and Peres2014). Ongoing development of the tool in terms of delivery (as well as usability and general utility) will be key in ensuring that the report is a sustainable intervention (Fleiszer et al., Reference Fleiszer, Semenic, Ritchie, Richer and Denis2015).
Strengths of this study included method triangulation, end-user involvement, and overall trustworthiness. By mirroring how humans naturally collect information, the combination of qualitative and quantitative data offered rich information that would not have been possible otherwise (Wisdom & Cresswell, Reference Wisdom and Cresswell2013). In contrast, there were some notable limitations. This research may have been susceptible to volunteer bias (e.g., social desirability bias) due to nonprobability sampling. Most of the participants in this study practised in Ontario and were likely to be more interested in system integration than the general population of primary care providers (Sedgwick, Reference Sedgwick2013). There was also limited information collected on the survey sample. Therefore, as a limitation, the findings of this study may not be representative of all primary care providers in Ontario. Additionally, since most of the interview participants practised in interprofessional health teams, this study may overestimate primary care providers’ knowledge of and connectedness with community resources. As an attempt to mitigate the issues that arose from nonprobability sampling, we used maximum variation sampling and assurance of confidentiality or anonymity (Salkind, Reference Salkind2010).
Moreover, the analysis of interviews and reporting of this study were shaped by the world-views of the researchers involved (Anderson, Reference Anderson2010; Holmes, Reference Holmes2020). To improve credibility, we provided thick descriptions of themes, used data triangulation, and employed member checking to ensure that participant views were represented accurately. Finally, this study was limited by small sample sizes. Recruitment was challenging throughout this study due to limited time, resources, and motivation among primary care providers to participate in research, exacerbated by the COVID-19 pandemic (Heckman, Saari, McArthur, Wellens, & Hirdes, Reference Heckman, Saari, McArthur, Wellens and Hirdes2020). To try and mitigate this challenge, we used snowball sampling in interview recruitment, and the survey inclusion criteria were broadened to include primary care residents.
Conclusion
This research suggests that the Patient Falls Risk Report is a useful way to convey information derived from interRAI-HC assessments. It has the capacity to support primary care providers in identifying risk factors and engaging in care planning for patients with clinical complexity. The report also has the capacity to be sustainable. However, further consideration of clinician workloads, supportive resources (i.e., technological or human resources), and team-based approaches to care is needed. We also learned that, with the appropriate systems in place, sharing high-quality standardized information does not require imposing a standardized format. Utility and usability can support primary care frailty management and should be prioritized to benefit older persons with complex needs.
Supplementary Materials
To view supplementary material for this article, please visit http://doi.org/10.1017/S0714980822000228.
Open access
