Introduction
Palliative care focusses on managing pain and other symptoms for individuals with serious illnesses. Worldwide there is a significant need for palliative care, with more than 56 million individuals needing palliative care annually, 80% of whom live in low- and middle-income countries (LMICs), such as Nepal, where access to palliative care is extremely limited (Worldwide Hospice Palliative Care Alliance 2020). In Nepal, there are few health facilities providing palliative care, with most services concentrated in the Kathmandu Valley (Clark et al. Reference Clark, Baur and Clelland2020; Gautam and Adhikari Reference Gautam and Adhikari2021). Since 2009, morphine has been manufactured in Nepal and is available in both oral (syrup, immediate and sustained release tablets) and intravenous formulations (Gautam and Adhikari Reference Gautam and Adhikari2021; Munday et al. Reference Munday, Basnyat and Swarbrick2018).
Limited knowledge and training in palliative care among health-care professionals (HCP) has been identified as a major barrier to increasing the availability of palliative care in LMICs including Nepal (Donkor et al. Reference Donkor, Luckett and Aranda2018; Gautam and Adhikari Reference Gautam and Adhikari2021). Although medical schools in Nepal have started to implement palliative care into undergraduate education, a recent study among medical students study found limited knowledge about palliative care (Pandey et al. Reference Pandey, Gaire and Dhakal2015). Palliative care has not been systematically incorporated into UG or PG training for HCP in other disciplines, such as nursing and psychology (Gautam and Adhikari Reference Gautam and Adhikari2021). Opportunities for continuing professional development (CPD) in palliative care are limited, and if available, training is generally delivered in urban centers which makes it challenging for staff from rural areas to attend (Hannon et al. Reference Hannon, Zimmermann and Knaul2016; Saini and Bhatnagar Reference Saini and Bhatnagar2016).
Online learning has increasingly been reported as innovative solution to improve the availability of palliative care education in LMICs. Online education offers particular benefits, since it can reduce financial and travel barriers for HCP, and allow specialists to connect directly with HCP in rural and remote areas for training and mentorship (Frehywot et al. Reference Frehywot, Vovides and Talib2013; Salins Reference Salins2020). Project ECHO (Extensions for Community Health Care Outcomes) is novel model of online education which was developed to train and support community-level health-care providers by connecting them with subject-matter experts during regular (weekly, biweekly, or monthly) teaching and case discussion sessions by videoconferencing (Arora et al. Reference Arora, Snead and Zalud-Cerrato2017). ECHO employs a “hub-and-spoke” design where experts at a “hub” health facility are connected with participants or “spokes” (Fig. 1).
Hub-and-spoke design (Image credit ECHO Ontario, used with permission) (Lalloo et al. Reference Lalloo, Osei-Twum and Rapoport2021, 3).
The “hub” team generally includes specialist clinicians, while “spoke” sites are generally health-care providers working outside of tertiary centers, who join ECHO to learn and receive support from specialists for particular patient population (Usher et al. Reference Usher, Payne and Real2022). In Nepal where access to specialist palliative care clinicians is extremely limited, ECHO suggests a model which can reach health-care providers who may not otherwise be able to access palliative care training and mentorship (Sutcliffe et al. Reference Sutcliffe, Bains and Black2021). Furthermore, ECHO allows palliative care specialists, from outside of Nepal to participate in the learning and mentoring which can further enhance the learning program (Arora et al. Reference Arora, Snead and Zalud-Cerrato2017). ECHO emphasizes active learning through case discussions and sharing experiences, which may also support learning for participants, since they are encouraged to consider how to apply the new knowledge and skills to their specific clinical practice and cultural milieu. Considering how to adapt palliative care practices to the local clinical situation is particularly relevant in palliative care, since health care for individuals with serious illness and death is particularly influenced by sociocultural factors (Varon et al. Reference Varon, Baker and Byers2021).
Despite the potential benefits of online education and the ECHO model, there are several important challenges to implementing virtual learning in resource limited settings. First, a lack of face-to-face interaction can limit active interaction between participants which is important for learning (Frehywot et al. Reference Frehywot, Vovides and Talib2013). Second, education and training materials need to be adapted to the local health-care environment and health-care resources available (Frehywot et al. Reference Frehywot, Vovides and Talib2013; Kiss-Lane et al. Reference Kiss-Lane, Spruijt and Day2019). Third, addressing technological and language barriers for participants are key considerations which have been identified as barriers for participants in a previous ECHO program in India (Doherty et al. Reference Doherty, Modanloo and Evans2021a). Previous studies have described the importance of interactive activities and enabling interactions between learners when providing online learning and with Project ECHO specifically (Doherty et al. Reference Doherty, Rayala and Evans2021b; Scott et al. Reference Scott, Baur and Barrett2017).
Several PC CPD programs using the ECHO model have been reported in the literature, including a program on pediatric palliative care in India and on humanitarian palliative care in Bangladesh, both of these programs included interdisciplinary HCP (Doherty et al. Reference Doherty, Lynch-Godrei and Azad2022, Reference Doherty, Rayala and Evans2021b). The ECHO program in India identified key adaptions to modify the program to the needs of health-care providers in South Asia. ECHO participants in Bangladesh reported that the program improved their knowledge, comfort, and attitudes toward palliative care (Doherty et al. Reference Doherty, Lynch-Godrei and Azad2022). Several recent studies have described using online education for UG medical training in Nepal during COVID-19; however, the use of online CPD education in Nepal has not been reported (Nepal et al. Reference Nepal, Atreya and Menezes2020; Subedi et al. Reference Subedi, Hirachan and Paudel2022). Developing and evaluating online palliative care education for HCP in Nepal is an important topic which warrants further study.
The aims of this study are to evaluate the impact of an ECHO program to provide CPD training on palliative care for interdisciplinary clinicians in Nepal, assessing program acceptability and changes in learners’ self-reported knowledge, comfort, attitudes, and practice change through pre- and post-ECHO surveys. Evaluating the impact of ECHO programs will support further refinements and modifications to ECHO for CPD.
Methods
Learning program description
The ECHO program developed through an existing partnership between clinicians and educators in Nepal and Canada (Brown et al. Reference Brown, Black and Vaidya2007; Sutcliffe et al. Reference Sutcliffe, Bains and Black2021). A series of collaborative discussions with key stakeholders was conducted, leading to the formation of a leadership team of palliative care experts and educators from Nepal and Canada. The leadership included palliative care nurses, doctors, and a pharmacist, supported by a program coordinator for administrative and logistical support.
Learning needs assessment and course content
During the planning phase, an online needs assessment survey of potential ECHO participants was done to understand learning needs. The survey was distributed by email and social media to individuals interested in palliative care training and included questions about the format of training (preferences for timing and duration of sessions) and specific topics of interest. The leadership team also met with potential participants and other palliative care clinicians in Nepal to further explore topics and course design. The leadership team had experience with in-person palliative care education in Nepal and was thus well connected to potential ECHO learners. Several team members had participated in previous pediatric palliative care ECHO programs (Doherty et al. Reference Doherty, Rayala and Evans2021b; Lynch-Godrei et al. Reference Lynch-Godrei, Doherty and Sapkota2021). Teaching topics were determined through group consensus by the leadership team, with consideration of survey results, stakeholder feedback, expert opinions, and a review of the relevant literature. Table 1 shows the topics which were included.
Weekly session topics of palliative care ECHO in Nepal
Program recruitment
The course was advertised to potential participants by email, word of mouth, telephone, and social media chat groups, through team members’ established networks. All interested HCP were invited to participate and there was no course fee. Potential participants were provided with information regarding the ECHO, including the timing of sessions and the technological requirements to join (laptop, tablet, or smartphone and stable high-speed internet connection).
Program structure
The ECHO program consisted of 12 weekly 1-hour online sessions conducted between January and April 2022 using Zoom Videoconferencing Software. An information technology specialist attended each session to support rapid troubleshooting of technical issues. Each session included a didactic presentation (20–30 minutes), from a palliative care expert (local or international). For international speakers, a Nepali physician provided a summary of the specific adaptations of the topic to the local health-care situation. Didactic teaching was followed by questions from participants (10 minutes) and case presentation by a participant with subsequent case discussion (20–30 minutes).
Sessions were facilitated by the leadership team, after facilitation training which emphasized key practices to encourage interaction and sharing by participants. Participants who registered and attended all course sessions received a course certificate. Participants received weekly emails following each session which included a summary of the session’s key learning points, the didactic presentation slides, and a link to the session video recording. Participants were invited to join a social media chat group on WhatsApp for the duration of the course, where participants and faculty interacted between sessions.
Study recruitment
All learners who register for ECHO were invited to participate in this study, which included surveys at the time of program registration (“pre”) and at the end of the ECHO program (“post”). Survey recruitment was done by email and social media.
Survey development
Surveys were develop based on team members’ previous experiences evaluating ECHO programs on palliative care in South Asia and a review of the relevant literature. The surveys were intended to explore participants’ knowledge of palliative care as well as their attitudes toward key topics and their comfort in delivering key elements of palliative care. There were specific survey questions for each teaching topic. Survey questions also addressed course satisfaction, barriers and enablers of program participation, and clinical practice change as well as exploring participants’ use of the learning materials, including recordings, presentation slides, and additional reading materials. Demographic data collected included health profession, area of practice/specialty, years of experience, and contact with seriously ill patients in clinical practice. The wording of questions was reviewed and modified based on feedback from HCP in Nepal to ensure that the wording reflected local palliative care concepts.
Participants’ knowledge of specific items included the stem question “for my scope of practice, I have an appropriate level of knowledge about …” In the self-efficacy domain, there were 15 questions with the stem “within my scope of practice, I am confident in my ability to …” Studies of ECHO programs report that participants may initially overestimate their level of knowledge and comfort before the ECHO program, and this study asked participants to re-rate their baseline abilities during follow up surveys and these adjusted baseline responses were subsequently used in the analysis. Survey questions included responses as multiple choice, free text, and using a 7-point Likert scales with response options ranging from [1] “strongly agree” to [7] “strongly disagree” (Data Supplement S1).
Statistical analysis
Data were analyzed using Microsoft Excel to obtain descriptive statistics. Mann–Whitney U-tests were performed to evaluate differences between knowledge, comfort, and attitude scores of survey participants before and after participating in the ECHO program. An a priori statistical significance was set to p = 0.05. Likert scales were reverse coded to improve clarity of results in presentation. Effect size was calculated using Cohen’s d, with an effect size 0.8 or more suggesting a large effect. Likert scales were reverse coded to improve clarity of results in presentation.
Results
Program characteristics
There were 14 educational sessions, with a total of 44 ECHO participants who registered and attended at least one ECHO session, along with 13 faculty members. The needs assessment survey identified pain management, talking patients about serious illness and death, and end-of-life care as topics of most interest to participants (Table 2).
Topics of interest to participants
Participant characteristics
Forty-two ECHO learners participated in this study, including 22 nurses (19 staff nurses, 2 nursing students, and 1 nursing lecturer) and 20 physicians (17 staff physicians and 3 residents). Most participants were from urban areas (n = 38, 90%), and most were working in a government hospital (n = 29, 69%). A smaller number of participants were working in a hospice setting (n = 10, 24%) or a university hospital (n = 3, 7%). There was a wide variety of specialty areas represented among participants, with the largest numbers coming from oncology (38%), general practice (24%), and other internal medicine subspecialties (19%). There were 6 participants who reported that palliative care was the focus of their clinical work (14%). Most participants were in the first 5 years of clinical practice (67%). Further demographic details of participants are shown in Table 3.
Characteristics of participants for project ECHO on palliative care in Nepal (n = 42)
a Multiple responses permitted.
Palliative care knowledge, comfort, and attitudes
The pre-program responses from participants indicated areas where participants rated their knowledge or comfort as low, as indicated by responses of “strongly disagree,” “disagree,” and “somewhat disagree.” These areas included providing end-of-life care (n = 10, 26%), breaking bad news (n = 7, 18%), and managing anxiety and depression (n = 6, 16%). Areas where participants more frequently indicated low levels of comfort were in discussing the transition from a curative to a palliative approach with patients/families (n = 8, 21%), discussing palliative care with other health-care providers (n = 8, 21%), and providing bereavement care to family members (n = 8, 21%). Participants’ attitudes related to palliative care included 12 (31%) participants who disagreed with the following statement “withholding or stopping NG tube feeding in a patient with advanced cancer who is in the terminal phase of life and who is no longer able to eat may be good care,” including strongly disagree”, “disagree,” and “somewhat disagree” responses.
Comparisons of participant knowledge (6 areas) and comfort (12 areas) found statistically significant improvements in knowledge and comfort in all areas, as shown in Table 4. At the end of the ECHO program, participant attitudes toward palliative care improved in all 8 areas evaluated, with significant improvements in 6 of 8 areas evaluated. Subgroup analyses with duration of professional experience and profession did not show any significant differences in knowledge, comfort, or attitudes. Table 4 shows further details of the attitudes measured and the changes before and after participation.
Changes in Likert scale scores of participants’ knowledge, self-rated confidence, comfort, and attitudes about palliative care at baseline and the end of ECHO program (n = 27–29 for each individual question, full data set available on request). Item response options ranged from strongly disagree (1) to strongly agree (7)
* Mann–Whitney U-test.
** Effect size (Cohen’s d). Effects of ≥0.8 generally suggest large effect.
# Response options reverse coded to improve comprehensibility of results; strongly agree (1) to strongly disagree (7).
Items without significant effect size and/or difference between pre- and post-survey scores.
Despite completing the program almost one-third of participants felt that they needed more formal teaching/training (32%) or clinical training (29%) to be able to provide palliative care. Additional barriers to practice change included a lack of time (n = 6, 21%) and lack of palliative care training among team members (n = 6, 21%). Only 2 participants indicated that medications discussed during ECHO were unavailable in their setting.
Program acceptability and satisfaction
Upon completion of the ECHO program, most participants (93%) felt the program had become a supportive COP and was an effective way to learn (89%). Most participants would recommend this program to colleagues (89%) and noted that participating in this program was a valuable experience (89%). Participants felt comfortable participating in ECHO discussions (68%) and using video conferencing (86%), with 89% agreeing that faculty were supportive and approachable.
Further details of participants’ experiences with the program are shown in Table 5. The most frequent barriers to participating in the ECHO program included at lack of time (n = 20, 71%) and technical issues (n = 11, 39%).
Participants’ comfort and learning experience of ECHO program
a Includes strongly agree, agree, or somewhat agree.
b Multiple responses permitted.
Learning resources
Seventeen participants (60%) accessed learning resources from the shared online folder for the course; however, 25% of participants were unaware of this resource. Three quarters (n = 21) of participants reported that they reviewed the key learning points for each session some or most of the time. Video recordings were most commonly (71%) reviewed by participants after they had missed a session, and 18% of participants used video recordings to teach colleagues or learners who were not participating in the ECHO.
Discussion
This study describes an online education and mentorship program in palliative care, which used the Project ECHO model to train health-care providers. The study identifies improvements in learners’ knowledge, comfort, and attitudes toward palliative care from program participation. Most participants were highly satisfied with the program, noting that faculty provided a supportive learning environment; however, the majority felt that they would benefit from further clinical training to be able to provide palliative care.
Online learning
Online learning has been suggested as an important tool to address HCP shortages in LMICs, and increasing access to palliative care training in these settings is particularly important (Yennurajalingam et al. Reference Yennurajalingam, Amos and Weru2019). Despite the proposed benefits, online learning has not been widely implemented in LMIC (Barteit et al. Reference Barteit, Jahn and Banda2019; Karim et al. Reference Karim, Sunderji and Jalink2021) Project ECHO is a method of online learning which may be particularly effective, as several previous studies have demonstrated improvements in HCP knowledge, skills, and attitudes from participation in PC ECHO programs in India, Bangladesh, and Ireland (Doherty et al. Reference Doherty, Lynch-Godrei and Azad2022; Lynch-Godrei et al. Reference Lynch-Godrei, Doherty and Sapkota2021; Usher et al. Reference Usher, Payne and Real2022). The present study’s findings of improved comfort and attitudes toward palliative care among a diverse group of nurses and physicians in Nepal supports the previous findings; suggesting that Project ECHO is an effective learning model for training HCP in diverse settings in LMICs.
Moore’s Expanded Continuing Medical Education (CME) Framework (Table 6) describes a hierarchy of learning outcomes from participation (Level 1) to HCP knowledge (Level 3a), and patient and community health (Levels 6 and 7). Previous ECHO programs, in the fields of chronic liver disease in the USA, have described improvements in patient-level outcomes (Glass et al. Reference Glass, Waljee and McCurdy2017; Moore et al. Reference Moore, Green and Gallis2009; Su et al. Reference Su, Glass and Tapper2018). Evaluation of patient- and community-level outcomes in palliative care can be particularly challenging due to the burden of serious illness for patients and family caregivers, and no previous programs have reported on patient-level outcomes for ECHO programs in this field. Future studies should consider assessing patient level outcomes in relation to PC ECHO programs. Given that study participants identified the need for more clinical training to be able to deliver palliative care, evaluation of learning outcomes at higher levels of Moore’s Framework, such as competence (Level 4) or performance in clinical practice (Level 5), is needed.
Moore’s expanded CME framework (Reference Moore, Green and Gallis2009)
Despite the potential benefits of online education and the ECHO model, there are several important challenges to implementing virtual learning in resource-limited settings. First, a lack of face-to-face interaction can limit active interaction between participants which is important for learning (Frehywot et al. Reference Frehywot, Vovides and Talib2013). Our program mitigated this through interactive case discussions and session moderation by trained facilitators who focused on encouraging active participation from leaners. Incorporating practice activities and enabling learners and teachers to interact have been identified as evidence-based principles for online learning (Scott et al. Reference Scott, Baur and Barrett2017). Second, it is important to ensure that education and training materials are adapted to the local health-care environment and resources available (Frehywot et al. Reference Frehywot, Vovides and Talib2013; Kiss-Lane et al. Reference Kiss-Lane, Spruijt and Day2019). This was addressed through during preparation of teaching materials, by ensuring that the content focused on medications and treatments available in Nepal. During ECHO sessions, the faculty from local health-care facilities ensured that resource availability and the practicality of implementing various aspects of palliative care were discussed. Working closely with local health-care facilities is important to ensure that treatments and recommendations are relevant and feasible for learners.
Development of a learning community
Communities of practice (COP) is a social learning theory which describes how learning can occur when HCP with shared interests interact and learn together, seeking to improve patient care (Wenger Reference Wenger1999). Project ECHO facilitates opportunities for learners and faculty to establish connections and build relationships with their peers around a shared interest, in this case palliative care. The collaborative nature of the ECHO sessions encouraged participants to engage in active discussions, ask questions, and learn from each other’s experiences. Study participants noted that the program created a supportive learning environment, which supported enhanced learning. Previous studies have described how ECHO programs can lead to the development of COP, including 1 study from South Asia, where participants identified the value of safe and encouraging environment where they could share ideas about how to provide palliative care (Doherty et al. Reference Doherty, Modanloo and Evans2021a). Future studies should continue to explore the impact of COP in online learning programs, seeking to identify how COP impacts learning amongst different types of learners and in different settings.
This ECHO project seemed to enhancing collaboration, communication, and mutual support among participants. The collective problem-solving approach not only increased the participants’ knowledge and self-efficacy but also strengthened professional relationships and created a supportive COP. The creation of a COP also streamlined the referral process, as providers were able to refer patients to different centers within Nepal for improved access to palliative care services. Our findings match those of previous authors in high-income countries who have described how ECHO established acted as a hub for continuous learning, professional development, and ongoing support among the participants, with participants reporting that they provided improved palliative care services to patients and families (Lalloo et al. Reference Lalloo, Osei-Twum and Rapoport2021; Usher et al. Reference Usher, Payne and Real2022). Despite improvements in knowledge and self-efficacy, participants identified that they needed further hands-on training, suggesting that Project ECHO should be included as a component of palliative care training initiatives which also include clinical training opportunities.
Learning resources
During this ECHO, additional learning resources, including videos, presentations, and related articles, were accessed by most learners during the program, as learners reported that they watched session recordings and reviewed additional resources for self-study as well as to teach others at their health facilities. These findings suggest that combining the ECHO sessions with complementary educational resources may enhance learning for individuals and may also lead to diffusion of new ideas to other HCP who have not attended ECHO training, creating a multiplier effect from ECHO. A social media chat group via WhatsApp also provided participants with a way to connect asynchronously between ECHO sessions. Participants used the WhatsApp chat to discuss complex cases with local and international experts during the ECHO program and afterwards as well. Creating a social media group to enhance learning and facilitate asynchronous communication has not previously been described in the ECHO literature. A previous study of undergraduate medical students described the importance of social media chats in facilitating peer support (Chou et al. Reference Chou, Johnston and Singh2011). Our findings show that HCP in Nepal use social media to support clinical decision and incorporate new palliative care knowledge into practice as the connections established through ECHO-enabled participants to seek ongoing guidance and mentorship from experts, as they developed a sense of belonging to a larger community of HCP focused on a shared goal of improving palliative care in Nepal. Further studies should explore the use of social media to enhance online learning, to better understand participants’ preferences, barriers, and limitation to its use.
Strengths and limitations
The study describes the implementation of a novel online palliative care training program in Nepal. The education program showed positive outcomes on participants’ knowledge, comfort, and attitudes toward palliative care. The program included health-care professionals from nursing and medicine, with representation from oncology, general practice, as well as internal medicine and palliative care, which suggests that the findings may be transferable. Future studies should seek to incorporate assessments of changes in clinical practice as well as patient outcomes and broader outcomes for the health-care system and communities.
Project ECHO can successfully be implemented in Nepal to build local palliative care capacity. Bringing together palliative care experts and teachers from Nepal and internationally supports learning for participants through COP. Encouraging active participation from participants and ensuring that teaching addresses the availability and practicality of treatments in the local health-care context addresses key barriers of online education.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S1478951524000786.
Funding
This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.
Competing interests
The authors declare that this project and research was carried out in absence of commercial or financial relationships that could be considered a potential conflict of interest.
Ethical approval
The study was approved by the Children’s Hospital of Eastern Ontario Research Ethics Board (CHEO 17/201X).
Open access
