Hostname: page-component-78c5997874-94fs2 Total loading time: 0 Render date: 2024-10-30T15:07:53.737Z Has data issue: false hasContentIssue false

A mixed methods evaluation of a World Health Organization competency-based training package for foundational helping skills among pre-service and in-service health workers in Nepal, Peru and Uganda

Published online by Cambridge University Press:  14 August 2023

Gloria A. Pedersen*
Affiliation:
Center for Global Mental Health Equity, Department of Psychiatry and Behavioral Health, George Washington University, Washington, DC, USA
Pragya Shrestha
Affiliation:
Program Department, Transcultural Psychosocial Organization (TPO) Nepal, Kathmandu, Nepal
Josephine Akellot
Affiliation:
Uganda Country Office, Programs Department, HealthRight International, Kampala, Uganda
Alejandra Sepulveda
Affiliation:
Socios En Salud Sucursal Perú, Lima, Peru
Nagendra P. Luitel
Affiliation:
Program Department, Transcultural Psychosocial Organization (TPO) Nepal, Kathmandu, Nepal
Rosco Kasujja
Affiliation:
School of Psychology, Department of Mental Health, Makerere University, Kampala, Uganda
Carmen Contreras
Affiliation:
Socios En Salud Sucursal Perú, Lima, Peru Harvard Global Health Institute, Harvard University, Cambridge, MA, USA School of Social Work, University of South Florida, Tampa, FL, USA
Jerome T. Galea
Affiliation:
School of Social Work, University of South Florida, Tampa, FL, USA Department of Global Health and Social Medicine, Harvard Medical School, Boston, MA, USA
Leydi Moran
Affiliation:
Socios En Salud Sucursal Perú, Lima, Peru
Vibha Neupane
Affiliation:
Program Department, Transcultural Psychosocial Organization (TPO) Nepal, Kathmandu, Nepal
Damodar Rimal
Affiliation:
Program Department, Transcultural Psychosocial Organization (TPO) Nepal, Kathmandu, Nepal
Alison Schafer
Affiliation:
Department of Mental Health and Substance Use, World Health Organization, Geneva, Switzerland
Brandon A. Kohrt
Affiliation:
Center for Global Mental Health Equity, Department of Psychiatry and Behavioral Health, George Washington University, Washington, DC, USA
*
Corresponding author: Gloria A. Pedersen; Email: gapedersen@gwu.edu
Rights & Permissions [Opens in a new window]

Abstract

Health systems globally demand more competent workers but lack competency-based training programs to reach their goals. This study evaluates the effectiveness of a competency-based curriculum (EQUIP-FHS) for trainers and supervisors to teach foundational helping knowledge, attitudes and skills, guided by the WHO/UNICEF EQUIP platform, to improve the competency of in-service and pre-service workers from various health and other service sectors. A mixed-methods, uncontrolled before-and-after trial was conducted in Nepal, Peru, and Uganda from 2020 to 2021. Trainees’ (N = 150) competency data were collected during 13 FHS trainings. Paired t-tests assessed pre- to post-change in ENACT competency measures (e.g., harmful, helpful). Qualitative data was analyzed using thematic analysis. EQUIP-FHS trainings, on average, were 20 h in duration. Harmful behaviors significantly decreased, and helpful behaviors significantly increased, across and within sites from pre-to post-training. Qualitatively, trainees and trainers promoted the training and highlighted difficult competencies and areas for scaling the training. A brief competency-based curriculum on foundational helping delivered through pre-service or in-service training can reduce the risk that healthcare workers and other service providers display harmful behaviors. We recommend governmental and nongovernmental organizations implement competency-based approaches to enhance the quality of their existing workforce programming and be one step closer to achieving the goal of quality healthcare around the globe.

Type
Research Article
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0), which permits unrestricted re-use, distribution and reproduction, provided the original article is properly cited.
Copyright
© The Author(s), 2023. Published by Cambridge University Press

Impact statement

Through the World Health Organization (WHO) and United Nations International Emergency Fund (UNICEF) Ensuring Quality in Psychological Support (EQUIP) initiative, a curriculum for trainers and supervisors to teach knowledge, attitudes and skills and assess foundational helping competencies was iteratively developed in 2020. Each module includes an evidence-based competency assessment, therein supporting a trainer or supervisor in targeting, evaluating, and providing feedback on competencies throughout the training and proactively adapting the plan as needed to meet trainees’ goals. This study looks to determine if the WHO’s curriculum can improve foundational helping competencies of different in-service (obstetricians, nurses, community health workers) and pre-service (public health, social work, nursing, and psychology students) workers from Nepal, Peru and Uganda. Before training, most healthcare workers and other service providers were scored as “potentially harmful” on foundational helping competencies. After completing the training, they displayed significantly fewer harmful behaviors and significantly more helpful behaviors. Our study found that trainees and trainers valued the EQUIP-FHS curriculum in its content and approach, highlighting the use of competency-based role plays and feedback in helping trainees achieve their competency goals. Based on the qualitative and quantitative results of this study, we strongly suggest that government, non-profit, and academic organizations implement competency-based training and assessment.

Introduction

Foundational helping competencies are the core behaviors that help to strengthen relationships, build emotional well-being and support positive interactions for people providing services across the health and community service fields (Watts et al., Reference Watts, Hall, Pedersen, Ottman, Carswell, Hof, Kohrt and Schafer2021). Foundational helping competencies involve the necessary knowledge, attitudes and skills for helping, and are based upon common factors in mental health and psychological services, which have been widely researched and identified as essential and universal pre-requisite for the effective delivery of any psychosocial or psychological components in health and global mental health interventions and align with the global competency framework for Universal Health Coverage (UHC) (Imel and Wampold, Reference Imel, Wampold, Brown and Lent2008; Wampold, Reference Wampold, Duncan, Miller, Wampold and Hubble2011, Reference Wampold2015; Singla et al., Reference Singla, Kohrt, Murray, Anand, Chorpita and Patel2017; Pedersen et al., Reference Pedersen, Lakshmin, Schafer, Watts, Carswell, Willhoite, Ottman, van ’t Hof and Kohrt2020; World Health Organization, 2022). Other foundational helping competencies include rapport building, the demonstration of empathy, using culturally or age-appropriate terminology and concepts for distress, and ensuring communication of hope. Competent use in foundational helping by health and care workers improves outcomes for people accessing different fields of health services, ranging from surgery to pain clinics (Kohrt et al., Reference Kohrt, Jordans, Rai, Shrestha, Luitel, Ramaiya, Singla and Patel2015; Hojat, Reference Hojat and Hojat2016; Golshan et al., Reference Golshan, Feizy, Tavasoli and Basiri2019; Walsh et al., Reference Walsh, O’Neill, Hannigan and Harmon2019; Surchat et al., Reference Surchat, Carrard, Gaume, Berney and Clair2022).

For instance, Lambert (Reference Lambert2022) discusses the significance of communication for surgeons to support the palliative management of patients and considers the promotion of hope to be “the most necessary element of care for most patients and surgeons in the palliative setting.” Similarly, a review by Heyn et al. (Reference Heyn, Løkkeberg, Ellington, van Dulmen and Eide2023) explored how expressing positive emotions during interactions could support the relationship of the practitioner–patient (nurse–patient, physician–patient, professional–patient). They identified several strategies and mechanisms, including using open/honest communication, genuineness, empathy, contributing to a patient’s hope for healing, forming trust, offering reassurance and comfort, providing emotional support through positive affirmation or “praise and support” and creating a feeling of connectedness with the patient. Research has also shown a lack of foundational helping competencies could impact patient outcomes or affect treatment adherence. For instance, Tiwary et al. (Reference Tiwary, Rimal, Paudyal, Sigdel and Basnyat2019) provide two case reports where healthcare professionals’ poor communication, such as failing to check the patient understands the diagnosis and treatment plan or appropriately involving and communicating to the patient’s family to support treatment adherence, could be what led to life-threatening complications with patients in Nepal. Additionally, Blasi et al.’s (Reference Blasi, Harkness, Ernst, Georgiou and Kleijnen2001) systematic review found that healthcare consultations that provided cognitive care (influencing patients’ beliefs about the effects of treatment) and emotional care (being warm, empathic, or reassuring) had significant impacts on patient outcomes in decreasing pain and increasing the speed of recovery compared to neutral consultations.

Demands are increasing to improve the quality of the health, care, social, and other service workforce to reach UHC, and competency-based strategies for training and supervision have been proposed as a pathway to get there (Frenk et al., Reference Frenk, Chen, Bhutta, Cohen, Crisp, Evans, Fineberg, Garcia, Ke, Kelley, Kistnasamy, Meleis, Naylor, Pablos-Mendez, Reddy, Scrimshaw, Sepulveda, Serwadda and Zurayk2010; Cometto et al., Reference Cometto, Buchan and Dussault2020; Uribe, Reference Uribe2022). More competency-based training curricula that address foundational helping competencies are needed. Through the World Health Organization (WHO) and United Nations International Emergency Fund (UNICEF) Ensuring Quality in Psychological Support (EQUIP) initiative (Kohrt et al., Reference Kohrt, Schafer, Willhoite, van ‘tHof, Pedersen, Watts, Ottman, Carswell and van Ommeren2020) (www.equipcompetency.org), a curriculum for trainers and supervisors to teach knowledge, attitudes and skills and assess foundational helping competencies, called Teaching Foundational Helping Skills: An EQUIP Competency-Based Training Manual for Trainers and Supervisors (EQUIP-FHS) was iteratively developed in 2020. To minimize training burden and maximize effectiveness, the curriculum is designed as a modular competency-based training rather than a “one-size-fits-all” approach and intends to be feasible for implementation in low-resource settings. Each module is complemented by an evidence-based competency assessment using items from the ENhancing Assessment of Common Therapeutic (ENACT) factors tool (Kohrt et al., Reference Kohrt, Jordans, Rai, Shrestha, Luitel, Ramaiya, Singla and Patel2015), found on the EQUIP platform, therein supporting a trainer or supervisor in targeting, evaluating, and providing feedback on foundational helping competencies throughout the training (see Figure 1).

Figure 1. Steps for an EQUIP-FHS competency-based training with the ENhancing Assessment of Common Therapeutic factors (ENACT) tool.

This EQUIP competency-based approach to teaching foundational helping and assessing competencies allows trainers and supervisors to proactively adapt the training plan and provide tailored feedback based on trainees’ existing and developing competencies. Like the tools and resources on the EQUIP platform, the EQUIP-FHS curriculum is intended to be freely available for trainers and supervisors and it can potentially improve foundational helping competencies among all health professionals and other service providers.

The present research aims to evaluate the effectiveness of the EQUIP-FHS curriculum to improve competency from pre- to post-training of different pre-service and in-service healthcare workers and other service providers who have no prior experience delivering formal mental health or psychosocial interventions (e.g., obstetricians) and which are working in low-resourced settings. Additionally, the current study will assess trainees’ and trainers’ acceptability and perceived benefit of partaking in an EQUIP-FHS training.

Methods

Context

This study was conducted in Nepal, Peru, and Uganda from 2020 to 2021. Trainings were conducted remotely, in-person, or using a hybrid method according to existing COVID-19 policies within each setting. In Kampala, Uganda, EQUIP-FHS trainings were implemented in-person by a team with HealthRight Uganda and Makerere University. In Kathmandu, Nepal, trainings were implemented by the Transcultural Psychosocial Organization (TPO) Nepal using a hybrid method, wherein TPO Nepal provided conference room spaces with WIFI and Zoom connections for small groups of trainees to join remotely while staying socially distanced. In the Metropolitan Area of Lima, Peru, the implementing organization Socios en Salud (SES) delivered all trainings remotely over Zoom. Trainees in Peru without access to unlimited WIFI were lent cell phones with unlimited internet.

Design

The current study is based on the EQUIP-Foundational Helping Skills (EQUIP-FHS) study, a multi-site, mixed methods, uncontrolled before-and-after trial (ClinicalTrials.gov ID: NCT04511156). The study used an intervention mixed methods framework, wherein qualitative data were collected primarily to explain results of the training outcomes and to understand contextual factors during the intervention that could affect the outcome, including trainers’ and trainees’ perceptions of the feasibility, acceptability and perceived utility of the training (Fetters et al., Reference Fetters, Curry and Creswell2013). The objective was to demonstrate that a brief, competency-based training intervention in foundational helping knowledge, attitudes and skills will improve the competency of workers naive to mental health and psychosocial services. We integrate quantitative and qualitative data at the interpretation and reporting level through narrative using a contiguous approach wherein quantitative results are reported first, followed by qualitative results. Integration is further elaborated via in-text reference and in the discussion.

Intervention description

The EQUIP-FHS curriculum was developed in a modular format, with each module relating to a specific foundational helping competency (e.g., nonverbal communication, confidentiality). The content for the pilot-test version of the training manual used in this study was developed collaboratively and iteratively, led by EQUIP team members at the George Washington University (GWU) and the WHO with support from a review group of selected field and academic experts between August to November 2020. Description of this first phase of module development can be found in Supplementary Box S1.

The training outline implemented by the sites is in the Supplementary Material. It includes approximately 2 days of EQUIP-FHS modules and approximately a half-day of training that involves remediation, “Skills Strengthening,” of any foundational helping competencies that have been identified via the assessments or which trainers felt needed extra time for review. Topics covered in the EQUIP-FHS modules align with competencies found in ENACT. During a pilot test, trainers deemed certain competencies as more advanced in mental health and psychosocial services and, therefore, less relevant. The ENACT items related to these modules included social functioning (Item 08), explanatory models (Item 09), family involvement (item 10), coping mechanisms (Item 13), psychoeducation (Item 14) and eliciting feedback (Item 15). As such, it was agreed among the trainers across the sites that they could integrate these topics into other modules when it was seen as complementary, such as including didactic on psychoeducation and explanatory models when teaching the module on promoting hope (ENACT Item 12). The decision to integrate typically relied on how trainees were progressing in the initial modules, using trainers’ subjective judgment and objective competency assessment scores.

All trainers accessed assessment results daily to adapt training plans as they saw fit and to give competency-based feedback to trainees. Trainers had access to an EQUIP e-learning module, “Giving and Receiving Feedback” (https://equipcompetency.org), as an optional guide to support competency-based feedback. Finally, as a supplementary training tool, each site had access to brief (“mini”) roleplay videos (1–2 min each) that show helpful and unhelpful examples of a given ENACT competency. Videos were initially developed in English, then recreated in Nepali and Spanish by the partnering sites.

Cultural adaptation and translation

The EQUIP-FHS curriculum and assessment tools were translated into the respective languages that the training would be conducted: Nepali in Nepal, Spanish in Peru, and English in Uganda. Before running any trainings, each site ran a small pilot training with a group of mental health and psychosocial service experts (N = 3–6) to support contextual adaptation and ensure relevancy for healthcare workers and other service providers that do not have experience with mental health services. A charting form (by EQUIP-FHS module, per activity) was used to track adaptations made by each site to support cross-site learning and consistency. Adaptations were primarily made to meet trainees’ needs to support learning and facilitation and in line with using a modular approach to training, rather than cultural-specific adaptations.

For instance, in Uganda, there was only one trainer, so trainees were asked to volunteer in role plays with the trainer for demonstrative purposes during training activities rather than a second trainer. In Peru, they took the modular approach of introducing Module 5 “Attitudes toward helping” at the beginning of the training and offered more examples of the importance of mental health in health care to promote engagement and confidence among the in-service trainees like the CHWs and obstetricians. In Nepal, the existing acronyms were useful tools for synthesizing learning, so the trainers added more during certain modules, such as adding EAR: Empathy, Acceptance and Reflection into the verbal communication module. In Uganda and Nepal, trainers allocated more time in the verbal communication module for roleplay practice and review on changing close-ended to open-ended questions.

Instrument

ENhancing Assessment of Common Therapeutic factors (ENACT) rating tool is a competency assessment tool based on common factors in mental health and psychological support. Each item relates to a foundational helping competency. It is used in objective structured clinical examinations, that is, standardized role plays with a simulated client (actor). It is a 15-item tool with an ordinal scale; each competency item has four scoring levels: Level 1 = potentially harmful; Level 2 = not done or limited demonstration of basic competency; Level 3 = basic helping competency; Level 4 = advanced helping competency. For each competency, approximately 8–12 individual attributes can be dichotomously checked (0,1) to determine the level.

Participant recruitment

A total of 160 trainees participated in 13 EQUIP-FHS trainings across the sites (three trainings each in Nepal and Uganda, seven trainings in Peru). All trainees participating were over 18 years old, had no experience in mental health or psychosocial service delivery, and had fluency in the language in which the trainings would be conducted: Nepali in Nepal, Spanish in Peru, and English in Uganda. Trainees who were under 18 years old and had experience with mental health service delivery or similar (e.g., completed psychology degree) were excluded. In Nepal, trainees were recruited from local universities. In Uganda, trainees were recruited from the HealthRight Organization. In Peru, trainees were recruited through a partnership with the Bonilla-La Punta Health Network, health establishments of the Integrated Health Network of Lima Norte (DIRIS Lima Norte) and with the Private Universities of Lima.

Research and training design

Pre- and post-training research days were used to collect primary and secondary outcomes with support from research staff (trained raters, actors, and qualitative researchers). A structured role play ENACT assessment was measured pre-training (day before) and post-training (either on the last day of training or the following day). Each pre-post assessment was 10 min (per participant, per timepoint). Roleplay assessments were done remotely (e.g., over Zoom) for remote trainings and in-person for face-to-face trainings. All assessments were video recorded. Supplement Figure S2 shows the sequence of research and training activities.

Quantitative analysis

A summary rating was calculated per assessment at the ENACT attribute and level measures. An algorithm in Excel supported this calculation. Description of the inter-rater reliability (IRR) processes and calculation of IRR and summary scores can be found in the Supplementary Material. Statistical analysis was conducted using R Statistical Software (2022) and RStudio Integrated Development Environment (v2022.12.0.353) (Posit team, 2022; R Core Team, 2022) with the tidyverse, dplyr, lessR and irr packages (Gamer et al., Reference Gamer, Lemon and Singh2019; Wickham et al., Reference Wickham, Bryan, Chang, LD, François, Grolemund, Henry, Hester, Kuhn, Pedersen, Miller, Bache, Ooms, Robinson, Seidel, Spinu, Takahashi, Vaughan, Woo and Yutani2019; Gerbing, Reference Gerbing2021; Wickham et al., Reference Wickham, François, Henry, Müller and Vaughan2023). From an eligible sample of 160 trainees (Nepal, N = 42, Peru, N = 82, Uganda, N = 36), we conducted complete case analysis and therefore excluded those with missing data on ENACT post-tests (Peru, N = 1, 1%; Uganda, N = 9, 25%). This produced a total sample size of 150 (Nepal, N = 42, Peru, N = 81, Uganda, N = 27). We use paired t-tests to compare pre- to post-change of two measures of competency outcomes: the ENACT-item attribute measure (change in total harmful behavioral attributes (Level 1) and change in total helpful behavioral attributes (Level 2, 3 and 4) and the ENACT-item level measure: change from harmful level (Level 1) to nonharmful level (Level 2,3,4) and change in “competent” level (Level 3,4) from pre- to post-training. The number of attributes was summed across all 15 competency items per participant into a total of “harmful attributes” score and a total of “helpful attributes” score and ran comparisons within and across sites. For level comparisons, we did a total count of Level 1 scores and a total count of Level 3 and Level 4 scores per trainee at pre- and post-training, within and across sites. Data is analyzed as a total sample and broken down by the three country sites. A statistical significance level of p < .05 was used.

Qualitative analysis

We used framework analysis to allow for a combination of inductive and deductive modes of coding and analysis (Smith and Firth, Reference Smith and Firth2011). All qualitative data were coded in Dedoose, a cross-platform application for analyzing qualitative and mixed-methods research (Dedoose, 2018). To develop the codebook, we created a theme matrix informed by the interview guide. The final codebook resulted in 10 parent codes and 5 child codes. A selection of 22 excerpts across 5 transcripts was used to check for inter-coder reliability (ICR) among 3 coders using the “Test” function in the Training Center on the Dedoose platform. After coder agreement was reached (ICR, 85%), transcripts were independently coded at the paragraph level. We used a code summary template to summarize codes and prepare for analysis. We further identified linkages and patterns in the data running queries and matrices in Dedoose. To ensure consensual validation, the coders had multiple discussions throughout the process.

Ethics

The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008. The study was approved locally for each site (Nepal Health Research Council, Nepal (ERB604/2020), Universidad Peruana Cayetano Heredia, Comité Institucional de Ética en Investigatción, Peru (CIEI 19021), Research Ethics Committee (MUREC), Uganda (REF 0608-2020) and internationally by the George Washington University Committee on Human Research Institutional Review Board (IRB FWA00005945) and the World Health Organization Research Ethics Review Committee (ERC.0003437). All participants provided informed written consent to participate in the study.

Results

Participant demographics

Trainees included pre-service students (nursing, public health, and social work university students) from Nepal, in-service workers (community health workers (CHW), nurses, and obstetricians) and pre-service students (psychology students) in Peru, and in-service workers (CHWs) from Uganda – all which were naïve to mental health and psychosocial service delivery. Table 1 displays a breakdown of trainee characteristics.

Table 1. Characteristics of trainees in EQUIP-FHS “in-service” and “pre-service” trainings (N = 13) in Nepal, Peru and Uganda

Training and research implementation

Each training had an average of 12 trainees, with 1–2 trainers per training. EQUIP-FHS trainings, on average, were 20 h long, typically delivered over 4 days. Supplementary Table S2 displays implementation characteristics of an average EQUIP-FHS training and number of trainings conducted in Nepal, Peru and Uganda. Each site had at least one clinical supervisor available for monitoring.

Potentially harmful and helpful attributes

Changes in potentially harmful attribute scores and helpful attribute scores from pre-to post-training are shown in Table 2. There was a significant decrease in harmful behaviors across all sites (t = − 14.76, n = 150, p < .0001, d = −1.21). Similarly, harmful attribute scores decreased significantly from pre- to post-training within each site: Nepal: t = −9.65, n = 42 p < .0001, d = −1.49; Peru: t = −13.23, n = 81, p < .0001, d = −1.47; Uganda, t = −3.32, n = 27, p < .01, d = −.64. There were also significant increases in helpful behaviors across all sites (t = 16.14, n = 150, p < .0001, d = 1.32), and within each site: Nepal: t = 12.44, n = 42, p < .0001, d = 1.92; Peru: t = 13.93, n = 81, p < .0001, d = 1.55; Uganda, t = 3.27, n = 27, p < .01, d = .63 from pre- to post-training. Qualitative results in Box 1 further elaborate potential reasons for such effective outcomes, such as the benefits of using a competency-based approach with practice and role plays and the relevance and importance of training in foundational helping competencies for all health workers.

Table 2. Comparison of total scores for the ENACT harmful and helpful attributes pre- and post-training

a Paired t-test.

Box 1. Key highlights from qualitative interviews with trainees and trainers on FHS training

Across sites, assessing suicidal behavior (ENACT07), goal setting (ENACT11), empathy (ENACT06), and responding to feelings and normalization (ENACT05) were mentioned most often as competencies that took more time to learn:

The biggest challenge was the topic of suicide…As little knowledge as I may have had, I knew what questions could be asked based on what I had read and what we had been trained in “Noguchi” [National Mental Health Institute Honorio Delgado-Hideyo Noguchi] about suicide. So, it never occurred to me to talk about it directly with the patient…Even during role-playing it was a little bit difficult to ask those questions, which were quite sensitive; I didn’t know that getting straight to the point helped [patients] the most. – Obstetrician 1, Peru

Most challenging…setting the goal, yeah. Like [using] the actual words, you are not supposed to force someone. – CHW 2, Uganda

More challenging was like … suicidal thoughts… to deal with such patients … and we were so conscious like, will we be able to use the appropriate word during that counseling period or not? I think when that type of situation arrives, then that is the challenging one. – Nursing Student 1, Nepal

Some trainees described how they had applied their skills since finishing the training, including the changes they saw in their clients:

The training taught me to have a lot more patience and to emphasize a lot more issues. For example, I didn’t have the habit of bringing up the topic of suicide. Now, this week [at work], I’m doing it and I’m realizing that a lot of patients don’t tell you right away. But [then]…at night, or after the session, they are sending me messages asking if they can talk to me and “it’s something personal.” So, it’s helped me a lot. – Obstetrician 2, Peru

…in the hospital setting, it [the foundational helping competencies] is very helpful. During the case presentation, we take the history of patients…And while taking the history, we have to maintain the IPR [interpersonal relationship] and for that, we have to listen to them properly and talk to them in a good way. – Nursing Student 1, Nepal

…I feel like more patients are going to come to me, especially being referred by fellow patients, because once you handle someone better, they…recommend you to another person, not because you have given them a drug and they healed, but because of the way you handled them in the process…Also [I] am going to be fulfilled because I know I did the right thing, not just because of the profession, but as a human being. – CHW 4, Uganda

Trainers compared differences, and perceived benefits, of the EQUIP-FHS competency-based training to other trainings:

In other trainings, mostly we took it alongside the theory, from the background, everything [theoretical] would be included, whereas in this [training], there was more of an introduction given on the contents briefly, and then practice was given more focus… – Trainer 2, Nepal

The first few days are somehow shocking to them [trainees], because they were not used to the assessment, or the role plays, or showing them the results, or putting the skills into practice, because most of their trainings had been theoretical with a theoretical assessment… The day after the assessment they received feedback, and they sort of adapted to this methodology. [Then] they felt more prepared, more confident, compared to the first few days. – Trainer 2, Peru

As a clinical psychologist, if I had continued the way I was without FHS [competency-based training] I wouldn’t be able to help, because there are so many things that I would not bother about. The sweetness is the practical attributes and pointing out how those attributes have an interplay in the person that you’re helping [training], because it demonstrates the good and the bad behaviors…this has created awareness now that each time a bad behavior is trying to cross your way, you’re conscious. – Trainer 1, Uganda

Trainers and trainees described challenges to training, both remotely and in-person:

When it’s an online format, sometimes there is problem because of poor internet connection and sometimes the electricity gets cut off. Because of that sometimes we might miss the important things from the training. – Social Work Student 1, Nepal

My biggest challenge has been dealing with time, motivation…I am talking specifically about the nurses and the obstetricians…most of them were working, especially the nurses were constantly working, they didn’t have 100% of their time available to receive the training… if necessary, [I know we can] add more hours, more days to the training…But the issue was the willingness of the participants. – Trainer 2, Peru

Trainees recommended organizations or professions that could benefit from an FHS training:

On the first day of the assessment, I mentioned that I would like all my colleagues to take this course. These skills that we have all learned on how to treat and care for our patients should be shared with others so patients can be cared for in an adequate manner. Not just one group, but all of us, so that everyone goes through this type of training. – Nurse 2, Peru

I recommend it [EQUIP-FHS] because many people like…local leaders, the Local Councils, the politicians, they are missing the skills. And even the political leaders, the way they are handling their offices, they have to get that knowledge of helping, the foundational helping [competencies]…to help the people who come to their offices. – CHW 6, Uganda

Trainers described pathways and potential blockages to scaling up EQUIP-FHS training:

I believe one of the barriers to scaling up this training, at least in the public sector, is the issue of working hours and productivity levels…For example, in the case of obstetricians, the person in charge of the sexual health unit told me: “If she participates in your training, she will stop taking care of pregnant women. That lowers my productivity, lowers my indicators.” I told her she was right…not having staff available for [4 daily hours for 1 week] can affect users and patients, but at the same, it would generate a benefit. – Trainer 2, Peru

If one involved leadership, then the demand will get higher than the supplies… we need to have trained trainers, so we’d have to have a ToT [training of trainers] …We must do a sensitization of the leadership…you know, give a presentation to them and justification why we think it’s necessary. And then ask them to nominate people to be trained…maybe heads of department… – Trainer 1, Uganda

Post hoc analysis of harmful and helpful attributes

Post hoc analysis was conducted to assess whether there were any observable differences in outcomes based on the experience level of the trainees (e.g., in-service or pre-service), specifically with the Peru site dataset, as this was the only site that included both in-service and pre-service trainees. Paired t-tests were used to compare the pre-post change in harmful and helpful attributes by experience level, and a simple linear regression was used to test if the experience level of the trainees explained post-test scores adjusting for baseline (pre-test) scores. We found no significant difference between experience levels (e.g., pre-service being less experienced, in-service being more experienced) and outcomes. The regression results were not significant (R 2 = .02, F(2, 78) = 1.98, p = .15). A table showing post hoc paired t-tests can be found in the Supplementary Material.

Harmful and “competent” levels

Table 3 shows the change in total counts of only harmful levels (Level 1) and the change in total counts of competency in Levels 3 and 4. There was significant movement from a Level 1 (“potentially harmful”) score to a Level 2 (“not done or limited basic competency”), 3 (“basic competency”), or 4 (“advanced competency”) across all sites pre-to post-training (t = − 14.78, n = 150, p < .0001, d = −1.17). Similarly, significant movement from Level 1 scores to Level 2, 3 or 4 scores from pre-to post training was found within each site: Nepal: t = −9.87, n = 42 p < .0001, d = −1.62; Peru: t = −13.68, n = 81, p < .0001, d = −1.89; Uganda, t = −3.24, n = 27, p < .01, d = −.52). There was a significant movement from pre to post-training in Levels 3, 4 (basic competency, advanced competency) from pre- to-post training across sites (t = 3.56, n = 150, p < .001, d = .47). In Nepal, participants’ scores showed a significant movement with a large effect size (t = 4.37, n = 42, p < .0001, d = 1.76); however, in Peru, only a very slight movement of competency scores was made from pre-training (M = .74, SD = .75) to post-training (M = .75, SD = .90) and was not significant Peru (t = .11, n = 81, ns, d = .01). Similarly, Uganda showed a slight increase in competency scores from pre-training (M = .59, SD = .89) to post-training (M = 1.00, SD = .96), but was insignificant (t = 2.02, n = 27, ns, d = .46). The average number of Level 1, 2, 3 and 4 scores across participants (N = 150) pre-and post-training can be found in Supplementary Figure S2.

Table 3. Comparison of total scores for the ENACT Level 1 (Harmful) and Levels 3 and 4 (“Competent”) pre- and post-training

a Paired t-test.

Figure 2 shows the percent of participants (N = 150) that scored a Level 1 (harmful) per ENACT item at pre-and post-training. At pre-training, most participants (N = 141, 94%) scored a Level 1 (harmful) competency on assessment of harm (Item 07), of which dropped by 68% at post-training (N = 45 participants scored Level 1, 30%) at post-training. Two other areas where many participants scored Level 1 at pre-training included confidentiality (Item 03; N = 92 participants, 61%) and eliciting feedback (Item 15; N = 77 participants, 51%). These scores decreased by 82% (N = 17 participants scored Level 1, 11%) and 67% (N = 24 participants scored Level 1, 16%) at post-training. Qualitative results in Box 1 help to further explain these results wherein trainees describe more difficult competencies to achieve, regardless of their previous professional training, and those which they now feel more confident in applying in their work – particularly in relation to the assessment of harm and suicidal behaviors.

Figure 2. Percent of participants (N = 150) with Level 1(harmful) score per ENACT item, pre-and post-training.

Movement to competent (Level 3,4) scoring was not as prominent across all ENACT items; however, change was substantial in the items where movement was made. For instance, participants that scored a Level 3 or 4 (competent) in nonverbal (Item 01) increased from 11 participants to 27 participants (157%) from pre-to post-training, respectively. Only one participant scored a Level 3 or 4 on assessment of harm (Item 07) at pre-training; however, this changed to 6% (N = 9) of participants scoring as competent in assessment of harm (Item 07) at post-training. Qualitative results in Box 1 elucidate these findings, wherein trainees describe competencies that were most difficult to learn or achieve. A visual showing the percent of participants (N = 150) that scored a Level 3,4 (competent) per ENACT item at pre- and post-training can be found in the Supplementary Material.

Qualitative interviews

A sample of trainees and trainers from Nepal, Peru, and Uganda were interviewed between December 2020 and November 2021. They reported their experiences and perceptions with participating in the FHS trainings within 1 week of finishing the training. Additionally, some trainees reported how they were applying what they had learned from the FHS training into their daily work 3–4 weeks post-training. Some extracts from their accounts can be found in Box 1.

Discussion

To reach the United Nations (UN) Sustainable Development Goal (SDG) 3.8 (achieving universal health coverage, UHC) targets by 2030, organizations around the world agree that health systems need more competent and empowered workers (World Health Organization, 2019a, 2019b, 2022; Rees et al., Reference Rees, Peralta Quispe and Scotter2021; Ssengooba et al., Reference Ssengooba, Ssennyonjo, Rutebemberwa, Musila, Namusoke Kiwanuka, Kemari and Nattimba2021; Adhikari et al., Reference Adhikari, Mishra and Schwarz2022). Locally and globally, existing competency frameworks identify foundational helping skills as core to the minimum standards healthcare workers and other service providers need to ensure safe and effective care. However, as the results from this study show, these training programs may not currently be achieving their competency goals: at pre-training, 98% of healthcare workers and other service providers (N = 148) scored Level 1 (harmful) competency on at least one ENACT item, while on average, these trainees scored as potentially harmful on 3 of the 15 ENACT competencies. It is possible that the education these currently practicing healthcare workers and other service providers received before the EQUIP-FHS training either did not cover these competency topics, or if the education did cover these topics, it may have been strictly knowledge-based learning. This latter connection was identified in the qualitative results, wherein in-service workers compared their previous professional training to EQUIP-FHS training in building competency (e.g., knowledge, attitudes and skills). For instance, some pointed out that the techniques used in the EQUIP-FHS training increased their competency in the assessment of risk of harm, such as suicide risk, as compared to what was taught in the existing National Mental Health Institute Honorio Delgado-Hideyo Noguchi training in Peru that gave them “little knowledge.”

Similarly, competency may be inadequately assessed in existing healthcare worker and other service provider education programs when using unidimensional measures, like multiple-choice knowledge tests, that cannot capture complex, multidimensional constructs such as competency (DeVellis, Reference DeVellis2012). On a programmatic level, essential information to inform and improve educational curricula or training programming may be missed when using norm-referenced results, such as from knowledge tests (Tucker, Reference Tucker2015), and could increase the inaccuracy of results due to the biases in formatting, language, and generalization of indicators to meet a “norm” population (Underly, Reference Underly2021), thereby increasing ambiguity for service delivery monitoring and evaluation purposes. Experts have suggested a more multidimensional approach to measuring competency through observational roleplay assessment (Miller, Reference Miller1990; Tucker, Reference Tucker2015; Mills et al., Reference Mills, Middleton, Schafer, Fitzpatrick, Short and Cieza2020), which could be why this study captured such detail on harmful behaviors.

This study provides evidence that competency-based training in foundational helping can be implemented in a brief time (~20 h). It also showed that the EQUIP-FHS competency-based tools for assessment and providing feedback were effective in building and assessing competency from pre- to post-training and found to be feasible, acceptable, and useful among trainees and trainers from Nepal, Peru and Uganda. We contacted the EQUIP-FHS training participants up to 3 weeks post-training to qualitatively determine how they were applying the foundational helping competencies in their daily work. Specific examples were provided by nurses, obstetricians, community health workers and students on how these competencies were having a positive impact on their work. Long-term follow-up competency assessment (e.g., 3, 6, 12 mos.) is needed to understand the extent to which these competencies are retained, as well as how much and how often an EQUIP-FHS competency-based training may be needed (e.g., refresher trainings). Similarly, impacts on healthcare and other service delivery should be investigated, as one community health worker in Uganda thoughtfully requested post-training, “I would love to see my progress (from this training). Looking at the number of cases, the number of clients that I have handled and checking on to see if they are satisfied, if they are happy, the guidance I gave them, the help I gave them…

The EQUIP-FHS curriculum is modular and each site had a similar selection of modules in this study. Six ENACT competency items (social functioning (Item 08), explanatory models (Item 09), family involvement (Item 10) and coping mechanisms (Item 13), psychoeducation (Item 14) and eliciting feedback (Item 15)) were not specifically included in the training plan but deemed optional and therefore covered in less detail by the trainers. This implementation approach likely explains why only a few trainees’ competency scores change from pre-post training in these six items. Future organizations and governmental bodies implementing EQUIP-FHS training are encouraged to use a similar modular approach, mainly by optimizing competency assessment results to determine which modules to deliver. Based on results and trainer feedback, we recommend that two trainers support the implementation of any EQUIP-FHS training plan to optimize the use of observational assessments and competency-based feedback.

The findings of this study show that the current EQUIP-FHS training manual is applicable and easy to use across three different settings when translated into the language in which the training will be delivered. We strongly encourage organizations to access the translated and adapted mini roleplay videos, roleplay scripts, and assessment tools, freely available on the EQUIP platform (https://equipcompetency.org), for direct implementation or for replicating these materials into a different language. Further guidance for implementing the EQUIP-FHS training package is being developed, including how trainers can contextually adapt role plays and other activities by actively involving the trainees, like asking about their daily work activities and role playing these scenarios when practicing foundational helping competencies. This approach could strengthen learning and help trainees apply the competencies to their current or future work.

Given the brevity of the training along with perspectives highlighted in our qualitative data, it is reasonable to expect that more EQUIP-FHS training time (>20 h) could increase the number of healthcare workers or other service providers who score competently (Level 3,4) post-training. Likewise, pre-service and in-service workers with previous experience in mental health and psychosocial services may be expected to reach higher competency levels within a shorter time frame (~20 h). As such, organizations and governmental bodies implementing EQUIP-FHS or similar EQUIP competency-based approaches should consider the advantages and disadvantages of increasing the time spent training workers with various occupational backgrounds.

Recommendations

As part of progress toward achieving universal health coverage (SDG 3.8), we recommend implementing and scaling the EQUIP-FHS training for in-service and pre-service workers across healthcare and other service sectors. Based on the findings of this research, we present various implementation strategies.

  1. 1. Trainers assess competencies to strive for a “do no harm” approach among healthcare workers and other service providers

To truly verify that client safety is at the forefront of training and supervision program outcomes, focus needs to shift from using only performance measure indicators at the organizational level (e.g., incident reporting, productivity) to using behavioral assessments at the individual level (e.g., trainers assessing healthcare worker and other service provider behaviors). Results from this study suggest that competency-based training like EQUIP-FHS offers the tools organizations and governmental bodies need to ensure their programs are achieving the “do no harm” principle. Notably, results from this study show that pre-service and in-service workers naïve to mental health or psychosocial service delivery (e.g., nurses, obstetricians) should ideally demonstrate no harmful behaviors and aspire to competency in foundational helping. To integrate competency-based assessment and approaches into programs, investment is needed in training trainers and supervisors on how to use competency-based techniques like rating competency assessment tools, incorporating role plays, providing competency-based feedback, and adjusting training and supervision plans accordingly. In this way, trainers and supervisors can actively identify harmful behaviors (Level 1) during training and supervision sessions and provide tailored feedback or additional training to help trainees remove these behaviors and replace them with more helpful behaviors.

  1. 2. Integrate competency-based training and assessments into pre-service programming to strengthen the quality of a growing workforce across healthcare and other service sectors.

The qualitative findings in this study suggest that it could be challenging to find time for in-service workers to participate in an EQUIP-FHS 20-h curriculum. As such, we recommend this training during pre-service as an optimal pathway for scaling competency-based approaches and to build foundational helping knowledge, attitudes, and skills. An evidence-based competency assessment complements each module in EQUIP-FHS, and the digital tools on the EQUIP platform provide immediate results to help trainers and supervisors provide tailored feedback. Such techniques are ideal for implementing into pre-service programs and curricula. For instance, EQUIP-FHS competency-based training could be part of an initial orientation for first-year medical residents. Thereafter, competency assessment, feedback, and refreshers could continue throughout the program.

Similarly, competency-based assessments and feedback could be paired with existing program modules or curricula and are optimally used during pre-service supervision sessions. In Uganda, partners from Makerere University suggested integrating EQUIP-FHS modules into the community, counseling and clinical psychology training programs for undergraduate and graduate students as part of their pre-service curriculum (Alipanga and Kohrt, Reference Alipanga and Kohrt2022). Additionally, they plan to integrate competency assessments and feedback with their students throughout their programs and train second-year undergraduate and graduate students as competency raters to support a feasible scale-up of this approach. In Nepal, partners at TPO Nepal have begun using the EQUIP-FHS training with nonspecialists as pre-requisite course to training on manualized interventions, such as the WHO’s Thinking Healthy Program (THP) (see Supplementary Material for sample schedule) and Problem Management Plus (PM+).

Furthermore, as the identity of the healthcare and service provider workforce expands to include a range of cadres that directly engage with people and offer a helping role, such as seen through the increase in the training of police officers to appropriately and safely respond to people dealing with mental health problems, or the training of barbers to be mental health advocates, an EQUIP-FHS training could ideally be integrated into police academy training or barbershop school to support these and similar efforts.

  1. 3. Work with leaders, working groups, and public health structures to integrate competency-based assessment and feedback for strengthening in-service care delivery, monitoring and evaluation, including task-shared programs.

According to feedback from trainers and trainees in this study, EQUIP-FHS competency-based training would benefit “all walks of life,” including workers and the persons they engage with across health and other service sectors (e.g., doctors, teachers, police officers, political leaders). Additionally, trainers and trainees promoted the scale-up of EQUIP-FHS competency-based training and assessment and suggested leveraging leadership and managing perceived barriers in public health structures to do so. The competency-based approach to EQUIP-FHS supports its modularity, and as such, there is flexibility to which elements of EQUIP-FHS and its competency-based techniques are incorporated. Partners at Socios En Salud in Peru have been generating alliances with various institutions in the public and private health sector, including the Ministry of Health, to support continued implementation of EQUIP-FHS training and general use of EQUIP competency-based assessment and feedback in programs for various public health professionals. For instance, competency-based assessment can be included as part of ongoing health facility and service utilization monitoring and evaluation, whereby a program manager, facility manager or supervisor incorporates competency assessments during observation of real-world care delivery and client engagement. Such incorporation could provide invaluable, detailed feedback on the performance of healthcare workers and other service providers and the evaluation of existing training protocols.

Another tactic to help healthcare workers and other service providers retain their foundational helping competencies could be to hold single module-based refresher trainings as an approach to continuing education. Similarly, “practical” homework assignments could be integrated into continuing education. Trainers in Peru successfully assigned “practical” homework to obstetricians and nurses who were participating in EQUIP-FHS training during the evening while still working during the daytime to deliver care. The trainers asked the obstetricians and nurses to practice or apply the daily learned foundational helping competencies at work and then discussed how that application went during EQUIP-FHS training hours. In Uganda, HealthRight has adopted the EQUIP competency-based training approach and competency tools within their programming, including assessing staff delivering WHO’s PM+. They are also promoting competency assessments and competency-based training throughout their mental health and psychosocial support (MHPSS) national working groups and will conduct training for the Uganda Counsellors Association, including professional counselors working in various practice settings.

Limitations

This study cannot account for differences in trainer styles or training modes (e.g., in-person, virtual), which may contribute to competency outcomes. The uncontrolled before-and-after design lacks the robustness of a randomized experimental design and has limitations, including certain threats to validity because of several biases (e.g., regression-to-the-mean, history, instrumentation, testing, maturation, dropout, placebo, Hawthorn). We believe these biases were minimized as follows: regression-to-mean bias was low-risk because we did not restrict our recruitment of trainees based on any characteristics, but allowed any type of in-service or pre-service participant take part in the training as long as they had no experience delivering mental health or psychosocial services. Our study unlikely experienced maturation or history effects due to the short time frame in which EQUIP-FHS trainings were implemented. We had high follow-up rates (all sites: 150/160, 94%; Nepal: 42/42, 100%, Peru: 1/82, 1%; Uganda, 9/36, 25%), except for the Uganda site, of which the dropout rate was still relatively low. Regarding practice effects (test–retest), we do not anticipate that the repeated ENACT role plays (pre- and post-training) outside the training context introduced a change in competency because formal feedback from these assessments was not provided, and participants were given different role plays scenarios at each time point. Placebo and Hawthorne threats to bias were very low, as this study was not clinical in nature, and the researchers collecting the data within each site were not involved in any way with the trainings. Finally, instrumentation was consistent for pre-post data collection.

Although inter-rater reliability (IRR) exercises were conducted in each site, different strategies to measure ICCs varied depending on whether groups of raters rated the same participant. An advantage to this study was calculating a summary score across raters per site, which may be a more realistic approach to ensure the reliability of ratings with a nonparametric observational tool in real-world settings compared to controlled research environments. Finally, the follow-up period was very short across sites (1 day to 3 weeks). Longer follow-up periods on how trainees continue to apply foundational helping competencies in their daily work are needed to understand how much and how often training (e.g., refresher trainings) is needed.

Conclusion

A brief competency-based curriculum in foundational helping knowledge, attitudes and skills, delivered through pre-service or in-service training, can reduce the risk that healthcare workers and other service providers display harmful behaviors. The EQUIP-FHS training curriculum provides evidence that competency-based assessment, feedback and other competency-based approaches can help ensure that in-service and pre-service workers are meeting their competency goals and ultimately engaging with clients safely. Trainers, supervisors, program managers, leaders in non-profit, governmental, and academic organizations, and donors are encouraged to use this study’s training materials and results to guide future program planning, evaluation and funding.

Open peer review

To view the open peer review materials for this article, please visit http://doi.org/10.1017/gmh.2023.43.

Supplementary materials

The supplementary material for this article can be found at http://doi.org/10.1017/gmh.2023.43.

Data availability statement

The data that support the findings of this study are available from the corresponding author, G.A.P., upon reasonable request.

Acknowledgments

We are grateful to the team members of the EQUIP partner teams who helped: Nepal (FHS trainer: Indira Pradhan; Collected data: Bimala Shrestha, Jagat Mohan, Sunil Khanal, Kumar Poudel, Ujwal Bhandari, Niko Gautam); Peru (FHS trainer: Margot Aguilar, Janeth Santa Cruz; Collected data: Maricielo Espinoza, Margaret Rivas) and Uganda (Collected data: Samuel Wasekera, Joel Omoding, Ayio Rebecca, Emmanuel Oboi, Sharon Charity Akello). From GWU, we also thank Jenny Sabol and Lea Simms for their help on qualitative coding, and Allison Morrow for her support in cleaning quantitative data.

Author contribution

Conceptualization: G.A.P., A.S.; B.A.K.; Data curation: G.A.P., A.S., L.M., D.R.; Formal analysis: G.A.P.; Funding acquisition: A.S., B.A.K.; Investigation: P.S., J.A., A.S., L.M., V.N.; Methodology: G.A.P., P.S., J.A., A.S., B.A.K.; Project administration: G.A.P., P.S., J.A., A.S., B.A.K.; Resources: G.A.P., B.A.K.; Supervision: N.P.L., R.K., C.C., J.T.G., A.S., B.A.K.; Visualization: G.A.P.; Writing – original draft: G.A.P.; Writing – review and editing: P.S., J.A., A.S., N.P.L., R.K., C.C., J.T.G., L.M., V.N., D.R., A.S., B.A.K.

Financial support

This study was possible due to the Ensuring Quality in Psychological Support (EQUIP-1 and EQUIP-2) project, which was funded by the World Health Organization and the United Nations International Children’s Emergency Fund (UNICEF) through a grant from USAID. B.A.K. is supported by the U.S. National Institute of Mental Health (R01MH120649).

Competing interest

A.S. is employed with the World Health Organization. All other authors declare no competing interests.

Ethics standard

All authors declare to adhere to the publishing ethics of Global Mental Health.

References

Adhikari, B, Mishra, SR and Schwarz, R (2022) Transforming Nepal’s primary health care delivery system in global health era: Addressing historical and current implementation challenges. Globalization and Health 18(1), 8. https://doi.org/10.1186/s12992-022-00798-5.CrossRefGoogle ScholarPubMed
Alipanga, B and Kohrt, BA (2022) Competency-based pre-service education for clinical psychology training in low- and middle-income countries: Case study of Makerere University in Uganda [Curriculum, Instruction, and Pedagogy]. Frontiers in Psychology 13. https://doi.org/10.3389/fpsyg.2022.924683.Google Scholar
Blasi, ZD, Harkness, E, Ernst, E, Georgiou, A and Kleijnen, J (2001) Influence of context effects on health outcomes: A systematic review. The Lancet 357(9258), 757762. https://doi.org/10.1016/S0140-6736(00)04169-6.CrossRefGoogle ScholarPubMed
Cometto, G, Buchan, J and Dussault, G (2020) Developing the health workforce for universal health coverage. Bulletin of the World Health Organization 98(2), 109116. https://doi.org/10.2471/blt.19.234138.CrossRefGoogle ScholarPubMed
Dedoose (2018) Dedoose Version 8.0.35, Web Application for Managing, Analyzing, and Presenting Qualitative and Mixed Method Research Data in. Los Angeles, CA: SocioCultural Research Consultants, LLC.Google Scholar
DeVellis, RF (2012) Scale Development: Theory and Applications. Thousand Oakes, CA: Sage. Available at https://books.google.com/books?id=Rye31saVXmAC (accessed 1 May 2022).Google Scholar
Fetters, MD, Curry, LA and Creswell, JW (2013) Achieving integration in mixed methods designs-principles and practices. Health Services Research 48(6 Pt 2), 21342156. https://doi.org/10.1111/1475-6773.12117.CrossRefGoogle ScholarPubMed
Frenk, J, Chen, L, Bhutta, ZA, Cohen, J, Crisp, N, Evans, T, Fineberg, H, Garcia, P, Ke, Y, Kelley, P, Kistnasamy, B, Meleis, A, Naylor, D, Pablos-Mendez, A, Reddy, S, Scrimshaw, S, Sepulveda, J, Serwadda, D and Zurayk, H (2010) Health professionals for a new century: Transforming education to strengthen health systems in an interdependent world. The Lancet 376(9756), 19231958. https://doi.org/10.1016/S0140-6736(10)61854-5.CrossRefGoogle Scholar
Gamer, M, Lemon, J and Singh, P (2019) irr: Various Coefficients of Interrater Reliability and Agreement. R Package Version 0.84.1. Available at https://CRAN.R-project.org/package=irr (accessed 1 September 2022).Google Scholar
Gerbing, DW (2021) Enhancement of the command-line environment for use in the introductory statistics course and beyond. Journal of Statistics and Data Science Education 29(3), 251266. https://www.tandfonline.com/doi/abs/10.1080/26939169.2021.1999871.CrossRefGoogle Scholar
Golshan, S, Feizy, T, Tavasoli, S and Basiri, A (2019) Service quality and urolithiasis patient adherence. International Journal of Health Care Quality Assurance 32(1), 210. https://doi.org/10.1108/IJHCQA-08-2017-0140.CrossRefGoogle ScholarPubMed
Heyn, LG, Løkkeberg, ST, Ellington, L, van Dulmen, S and Eide, H (2023) Understanding the role of positive emotions in healthcare communication – A realist review. Nursing Open 10(6), 34473459. https://doi.org/10.1002/nop2.1604.CrossRefGoogle ScholarPubMed
Hojat, M (2016) Empathy and patient outcomes. In Hojat, M (ed.) Empathy in Health Professions Education and Patient Care. Cham: Springer International Publishing, pp. 189201. https://doi.org/10.1007/978-3-319-27625-0_11.CrossRefGoogle Scholar
Imel, ZE and Wampold, BE (2008) The importance of treatment and the science of common factors in psychotherapy. In Brown, SD and Lent, RW (eds.), Handbook of Counseling Psychology, 4th Edn. Hoboken, NJ: John Wiley & Sons, pp. 249266.Google Scholar
Kohrt, BA, Jordans, MJD, Rai, S, Shrestha, P, Luitel, NP, Ramaiya, MK, Singla, DR and Patel, V (2015) Therapist competence in global mental health: Development of the ENhancing assessment of common therapeutic factors (ENACT) rating scale. Behaviour Research and Therapy 69, 1121. https://doi.org/10.1016/j.brat.2015.03.009.CrossRefGoogle ScholarPubMed
Kohrt, BA, Schafer, A, Willhoite, A, van ‘tHof, E, Pedersen, GA, Watts, S, Ottman, K, Carswell, K and van Ommeren, M (2020) Ensuring quality in psychological support (WHO EQUIP): Developing a competent global workforce. World Psychiatry 19(1), 115116. https://doi.org/10.1002/wps.20704.CrossRefGoogle ScholarPubMed
Lambert, LA (2022) Communication in surgery: The therapy of hope. Annals of Palliative Medicine 11(2), 958968. https://doi.org/10.21037/apm-20-2590.CrossRefGoogle Scholar
Miller, GE (1990) The assessment of clinical skills/competence/performance. Academic Medicine, 65(9), S63S67. http://winbev.pbworks.com/f/Assessment.pdfCrossRefGoogle ScholarPubMed
Mills, JA, Middleton, JW, Schafer, A, Fitzpatrick, S, Short, S and Cieza, A (2020) Proposing a re-conceptualisation of competency framework terminology for health: A scoping review. Human Resources for Health [Electronic Resource] 18(1), 15. https://doi.org/10.1186/s12960-019-0443-8.CrossRefGoogle ScholarPubMed
Pedersen, GA, Lakshmin, P, Schafer, A, Watts, S, Carswell, K, Willhoite, A, Ottman, K, van ’t Hof, E and Kohrt, BA (2020) Common factors in psychological treatments delivered by non-specialists in low- and middle-income countries: Manual review of competencies. Journal of Behavioral and Cognitive Therapy 30(3), 165186. https://doi.org/10.1016/j.jbct.2020.06.001.CrossRefGoogle ScholarPubMed
Posit Team (2022) RStudio: Integrated Development Environment for R. R version 4.1.3. Posit Software, PBC. Available at http://www.posit.co/ (accessed 1 October 2022).Google Scholar
R Core Team (2022) R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing. Available at https://www.R-project.org/ (accessed 1 October 2022).Google Scholar
Rees, GH, Peralta Quispe, F and Scotter, C (2021) The implications of COVID-19 for health workforce planning and policy: The case of Peru. International Journal of Health Planning & Management 36(S1), 190197. https://doi.org/10.1002/hpm.3127.Google ScholarPubMed
Singla, DR, Kohrt, BA, Murray, LK, Anand, A, Chorpita, BF and Patel, V (2017) Psychological treatments for the world: Lessons from low- and middle-income countries. Annual Review of Clinical Psychology 13, 149181. https://doi.org/10.1146/annurev-clinpsy-032816-045217.CrossRefGoogle ScholarPubMed
Smith, J and Firth, J (2011) Qualitative data analysis: The framework approach. Nurse Researcher 18(2), 5262. https://doi.org/10.7748/nr2011.01.18.2.52.c8284.CrossRefGoogle ScholarPubMed
Ssengooba, F, Ssennyonjo, A, Rutebemberwa, E, Musila, T, Namusoke Kiwanuka, S, Kemari, E and Nattimba, M (2021) Research for universal health coverage: Setting priorities for policy and systems research in Uganda. Global Health Action 14(1), 1956752. https://doi.org/10.1080/16549716.2021.1956752.CrossRefGoogle ScholarPubMed
Surchat, C, Carrard, V, Gaume, J, Berney, A and Clair, C (2022) Impact of physician empathy on patient outcomes: A gender analysis. British Journal of General Practice 72(715), e99e107. https://doi.org/10.3399/bjgp.2021.0193.CrossRefGoogle ScholarPubMed
Tiwary, A, Rimal, A, Paudyal, B, Sigdel, KR and Basnyat, B (2019) Poor communication by health care professionals may lead to life-threatening complications: Examples from two case reports. Wellcome Open Research 4, 7. https://doi.org/10.12688/wellcomeopenres.15042.1.CrossRefGoogle ScholarPubMed
Tucker, CG (2015) Psychometric Considerations for Performance Assessment with Implications for Policy and Practice. Princeton, NJ: Educational Testing Service.Google Scholar
Underly, JK (2021) Limitations of Norm-Referenced Tests (Wisconsin Department of Public Instruction, ed.). Wisconsin: Wisconsin Department of Public Instruction.Google Scholar
Uribe, JP (2022) 4 Priorities to Achieve Universal Health Coverage. Available at https://blogs.worldbank.org/health/4-priorities-achieve-universal-health-coverage (accessed 7 March).Google Scholar
Walsh, S, O’Neill, A, Hannigan, A and Harmon, D (2019) Patient-rated physician empathy and patient satisfaction during pain clinic consultations. Irish Journal of Medical Science 188(4), 13791384. https://doi.org/10.1007/s11845-019-01999-5.CrossRefGoogle ScholarPubMed
Wampold, BE (2011) The research evidence for common factors models: A historically situated perspective. In Duncan, BL, Miller, SD, Wampold, BE and Hubble, MA (eds.), The Heart and Soul of Change: Delivering What Works in Therapy, 2nd Edn.. Washington, DC: American Psychological Association, pp. 4982.Google Scholar
Wampold, BE (2015) How important are the common factors in psychotherapy? An update. World Psychiatry 14(3), 270277. https://doi.org/10.1002/wps.20238.CrossRefGoogle ScholarPubMed
Watts, S, Hall, J, Pedersen, GA, Ottman, K, Carswell, K, Hof, E, Kohrt, BA and Schafer, A (2021) The WHO EQUIP foundational helping skills Trainer’s curriculum. World Psychiatry 20(3), 449450. https://doi.org/10.1002/wps.20880.CrossRefGoogle ScholarPubMed
Wickham, HAM, Bryan, J, Chang, W, LD, MG, François, R, Grolemund, GHA, Henry, L, Hester, J, Kuhn, M, Pedersen, TL, Miller, E, Bache, SMMK, Ooms, J, Robinson, D, Seidel, DP, Spinu, V, Takahashi, K, Vaughan, DWC, Woo, K and Yutani, H (2019) Welcome to the tidyverse. Journal of Open Source Software 4(43), 1686. https://doi.org/10.21105/joss.01686.CrossRefGoogle Scholar
Wickham, H, François, R, Henry, L, Müller, K and Vaughan, D (2023) dplyr: A Grammar of Data Manipulation. R package version 1.1.0. Available at https://CRAN.R-project.org/package=dplyr (accessed 1 February 2022).Google Scholar
World Health Organization (2019a) Global Action on Patient Safety. Geneva: World Health Organization. https://apps.who.int/gb/ebwha/pdf_files/WHA72/A72_R6-en.pdfGoogle Scholar
World Health Organization (2019b) Recovery Practices for Mental Health and Well-Being. WHO QualityRights Specialized Training. Course Guide. Geneva: World Health Organization.Google Scholar
World Health Organization (2022) Global Competency and Outcomes Framework for Universal Health Coverage. Geneva: World Health Organization.Google Scholar
Figure 0

Figure 1. Steps for an EQUIP-FHS competency-based training with the ENhancing Assessment of Common Therapeutic factors (ENACT) tool.

Figure 1

Table 1. Characteristics of trainees in EQUIP-FHS “in-service” and “pre-service” trainings (N = 13) in Nepal, Peru and Uganda

Figure 2

Table 2. Comparison of total scores for the ENACT harmful and helpful attributes pre- and post-training

Figure 3

Table 3. Comparison of total scores for the ENACT Level 1 (Harmful) and Levels 3 and 4 (“Competent”) pre- and post-training

Figure 4

Figure 2. Percent of participants (N = 150) with Level 1(harmful) score per ENACT item, pre-and post-training.

Supplementary material: File

Pedersen et al. supplementary material
Download undefined(File)
File 782.2 KB

Author comment: A mixed methods evaluation of a World Health Organization competency-based training package for foundational helping skills among pre-service and in-service health workers in Nepal, Peru and Uganda — R0/PR1

Comments

Dear Drs Judy Bass and Dixon Chabanda, Editors-in-Chief:

We wish to submit an original research article entitled “A mixed methods evaluation of a World Health Organization competency-based training package for Foundational Helping Skills among pre-service and in-service health providers in Nepal, Peru and Uganda” for consideration by Cambridge Prisms: Global Mental Health. We confirm that this work is original and has not been published elsewhere, nor is it currently under consideration for publication elsewhere.

In this article, we report quantitative and qualitative results on the evaluation of 13 trainings using a competency-based curriculum to teach 150 pre-service and in-service health providers foundational helping skills (FHS) in Nepal, Peru and Uganda. The results show significant impact in reducing harmful behaviors and increasing helpful behaviors, as assessed by the ENhancing Assessment of Common Therapeutic factors (ENACT) tool on the WHO/UNICEF Ensuring Quality in Psychological Support (EQUIP) digital platform (www.equipcompetency.org).

Programs globally are aiming to reach Universal Health Coverage in 2030, of which requires quality training of health workers to ensure they have the core skills needed to deliver supportive and safe care. Using a competency-based approach to training and supervision in FHS that is in line with global and local competency frameworks has the potential to support academic institutions, governments and organizations in reaching these goals; however, curriculums are lacking. This study is significant because it reveals the effectiveness of a WHO brief competency-based training package to reduce harmful behaviors and increase helpful behaviors related to FHS among both in-service and pre-service providers. Also, this study highlights the feasibility and acceptability participating and delivering the training, and the benefits of scaling-up competency-based programs as described by providers and trainers from three lower-resourced settings. The WHO/UNICEF EQUIP platform was launched in March 2022 to offer freely available competency assessment tools, e-Learning materials, and other resources to anyone around the world. The evaluation in this study is contributing to the final publication and global release of the manualized WHO EQUIP competency-based curriculum for trainers and supervisors to teach FHS, predicted to be released by Fall 2023.

We believe that this manuscript is appropriate for publication by Cambridge Prisms: Global Mental Health because it offers insight into potential pathways for implementing and scaling competency-based programs that offer the tools needed to assess health workers in various sectors (e.g. psychiatry, psychology, nursing, obstetrics, public health) are doing no harm. Based on our results, we highlight the application and usefulness of incorporating the EQUIP FHS competency-based curriculum into academic institutions for pre-service training of various social and health workers for a feasible approach to strengthening the future in-service workforce. We offer key recommendations for institutions for implementing competency-based approaches like the FHS curriculum into their existing programming.

We anticipate this could be a highly cited manuscript as application of the tools and resources on the EQUIP platform, and training in FHS, continues to build globally.

One author of this article is currently employed with the WHO; all other authors have no conflicts of interest to disclose. All authors confirm that they fulfill the criteria listed for authorship by ICJME recommendations, and that no authors who would reasonably be considered an author have been excluded.

For reviewers, we recommend Gilles Dussault, Freddie Ssengooba, Gareth H. Rees, and Zulfiqar A. Bhutta. This evaluation was possible with funding by the World Health Organization and the United Nations International Children’s Emergency Fund (UNICEF) through a grant from USAID. The senior author, Brandon Kohrt, is supported by the U.S. National Institute of Mental Health (R01MH120649).

On behalf of the authors,

Gloria A. Pedersen, DrPH, MSc

Senior Research Associate, The Center for Global Mental Health Equity

George Washington University School of Medicine and Health Sciences

2120 L St NW Suite 600, Washington DC 20037

gapedersen@gwu.edu

Review: A mixed methods evaluation of a World Health Organization competency-based training package for foundational helping skills among pre-service and in-service health workers in Nepal, Peru and Uganda — R0/PR2

Conflict of interest statement

I have previously worked with and published on the ENACT tool and have previously published articles on other projects with several of the authors.

Comments

GMH-23-0092

This paper describes the outcomes of a foundational helping skills training for paraprofessionals. I appreciate the inclusion of qualitative and quantitative data as well as the fact that this paper is framed around universal health coverage – these skills are important for all providers, not just those working in mental health. Additionally, the fact that data are collected across three different contexts allows this paper to highlight commonalities and differences across sites. I have made some recommendations below to further refine this paper, largely focusing on the integration of quantitative and qualitative methods.

- Abstract

o I find the use of the word educated to be a little confusing in that first sentence (this is also used in the impact statement). It leads me to expect a study focused on increasing knowledge, not skills. Maybe replacing that with skills or competence would be more appropriate?

- Introduction

o It may be helpful to provide a little more context on why it’s important to increase FHS skills among providers. What are the consequences to patients and health systems if providers do not have these skills?

o I may have missed it, but I don’t believe MHPSS is defined anywhere.

- Methods

o Was the e-learning module on giving and receiving feedback for supervisors required or optional?

o Please provide a brief explanation as to why the sample from Peru was so much larger.

o I appreciate that you focused on a clinically meaningful outcome – the change from harmful to non-harmful.

□ Did you run statistical tests for the change in levels for individual items or just calculate proportions harmful and not harmful pre and post?

o Can you speak to the cultural adaptation and/or translation decisions made in each of these three contexts? For example, why were those specific items excluded or made optional (as mentioned in the discussion)?

- Results

o Because this is a mixed methods paper, I was expecting an integration section to explicitly relate the quantitative and qualitative findings to other another. And I do think such a section would be helpful. For example, there seem to be parallel findings on the risk item across quant and qual.

□ This reference may be helpful in deciding upon a method of integrating results: https://www.bmj.com/content/341/bmj.c4587.long

o Did you observe any differences across sites based on the experience level of trainees?

- Discussion

o It was not clear to me what you were saying about the optional modules aligning with competency results (line 349). Could you rephrase?

o Again, could you discuss the need to culturally adapt this curricula based on the setting?

o I think it would be helpful to link the recommendations more clearly to the common challenges experienced across sites in the qualitative work. An integration section in the results (as I describe above) would build nicely into that.

Review: A mixed methods evaluation of a World Health Organization competency-based training package for foundational helping skills among pre-service and in-service health workers in Nepal, Peru and Uganda — R0/PR3

Conflict of interest statement

Reviewer declares none.

Comments

This is a quasi-experimental study (pre-post design) describing the use of the WHO EQUIP platform to evaluate the competency in basic ‘helping’ (counselling) skills of non-specialist mental health providers following training in 3 countries. This is a well-written manuscript and gives a good demonstration of the use of the platform. It should be interest to the global mental health audience engaged in provision of psychosocial care to people with common mental disorder.

Review: A mixed methods evaluation of a World Health Organization competency-based training package for foundational helping skills among pre-service and in-service health workers in Nepal, Peru and Uganda — R0/PR4

Conflict of interest statement

Reviewer declares none.

Comments

My congratulations to the authors, and those who have developed and delivered the training package. My feedback relates to the write-up of the training package, rather than the course itself.

Overall, I found that the paper makes some bold claims that aren’t necessarily backed up by the information provided; and it would benefit from some restructuring of the text, to support the recommendations – and perhaps identify new ones, for example the logistics or considerations in operationalizing the course. The recommendations are also framed more generalizable than FHS or this course, but as relevant to the whole of health worker education – is that intentional, and can that be justified based on the study?

I find the discussion of ‘harmful’ and competence to relate to two different concepts, and this would benefit from clarification. Was the assessment tool established to assess not only ‘not competent’ but also ‘harm’? There is a frightening level of ‘harm’ reported – 98% of learners – what is the implication for other health workers who have not completed the training? But then I wondered, is it harm, or is it not yet learned competence?

I would also encourage consistency in the language throughout. I am familiar with the many different terminologies and definitions for competence, competency and plurals of the same. Defining up front what is meant by each term would help – competency levels and competent levels are used interchangeably for example; but it also seems that competency can refer to skills. Similarly, there is reference to healthcare workers, health workers and health and social care workers, as well as in many places, providers; this again would benefit from streamlining and consistency.

More specific observations are listed here:

Line 6 – add MHPSS acronym, as this later used

Line 9 – Universal instead of United.

First para – check the intent; not accurate to state that the FHS are part of the Global Competency Framework for UHC, as they are not identified as such – worth stating ‘aligned with’ or other overlapping content? It would be worth expanding this paragraph to establish the rationale and need for FHS – as it does not come directly from the UHC framework.

First para – important to situate these for mental health services, rather than for all health services – the skill ‘communication for hope’ is not consistent across all health services, but is important for mental health.

Harmful and not competent used to describe levels 1 and 2. Conversely, I find that the opposite of competence is not always incompetence. Was the assessment designed to differentiate between ‘harm’ and ‘not competent’? If not, then I would suggest reframing the narrative after Table 2 around ‘competence’

Line 331 – first mention of social services. It may be simpler to align with UN terms and refer to ‘health and care workers’ – nb WHO doesn’t use the term ‘provider’ as it is used in some country contexts to refer to the health workers, and in order countries, to refer to the organizations.

Line 332-333 – check, the implication is that the learners who were test pre-training had completed previous training programs that are deemed to be not achieving their goals. Can the evaluation of those courses be inferred from the pre-training test of individuals?

Line 336 – only competency-based assessment is linked to verifying competence. How that learning has been acquired – whether through formal training programmes or otherwise – is not necessary here.

Lines 227-343 strike me as being conclusions, not discussions. Suggest the weblink and availability is provided elsewhere. Additional detail about the performance metrics of the assessment items, as well as post-assessment transfer to practice and retained knowledge e.g. 6 and 12 months post training, must be provided. Otherwise, the claim should be tempered to reflect what is assessed – pre- and post-test score change for example. There is also a big debate about ‘measurement’ of competence, which implies precision – the authors might thoughtfully consider their use of the term.

Lines 344-358 would benefit from clarity. The first sentences are providing new context about the course, which should be established before the discussion. But it is then unclear what is meant by ‘this’ in line 348 – what is reflected in the re4sults?

The rationale supporting recommendation 1 doesn’t seem to support or relate to recommendation 1. I would expect the discussion to reflect that ‘level 1’ is worse than not providing care, and therefore what actions to take if a learner does score Level 1. There is then discussion around the practicalities of course provision, for example training trainers and supervisors which seems irrelevant to the recommendation about do no harm, or the discussion about length of time etc. This could be considered for a new/separate recommendation 4.

Line 390 – what is meant by ‘pre-service programming’. The example in line 397 of police academy does not fit the accepted definition of ‘health worker’ used in the wording of the recommendation.

It is unclear who the recommendations are for. How can the training be integrated during in-service care – does this mean supervised practice, or in-service learning? Who should integrate competency-based assessments or work with public health structures? Are these recommendations applicable beyond FHS and in fact relevant to the whole of pre-service and in-service learning programmes?

Recommendation: A mixed methods evaluation of a World Health Organization competency-based training package for foundational helping skills among pre-service and in-service health workers in Nepal, Peru and Uganda — R0/PR5

Comments

Dear authors:

Please, check the comments of the reviewers and make the suggested corrections accordingly.

Thank you.

Decision: A mixed methods evaluation of a World Health Organization competency-based training package for foundational helping skills among pre-service and in-service health workers in Nepal, Peru and Uganda — R0/PR6

Comments

No accompanying comment.

Author comment: A mixed methods evaluation of a World Health Organization competency-based training package for foundational helping skills among pre-service and in-service health workers in Nepal, Peru and Uganda — R1/PR7

Comments

Dear Dr Dixon Chibanda,

We appreciate the feedback and are glad to submit a revision to this manuscript. We have included a tracked-changes version of the manuscript, an updated supplement file, along with a response to reviewers point-by-point response. Please let us know if you will need anything more to support this resubmission.

Review: A mixed methods evaluation of a World Health Organization competency-based training package for foundational helping skills among pre-service and in-service health workers in Nepal, Peru and Uganda — R1/PR8

Conflict of interest statement

I have worked with/published on the ENACT and with authors of this paper on different projects.

Comments

The authors have been very responsive to my comments and I have no further comments.

Review: A mixed methods evaluation of a World Health Organization competency-based training package for foundational helping skills among pre-service and in-service health workers in Nepal, Peru and Uganda — R1/PR9

Conflict of interest statement

Reviewer declares none.

Comments

None.

Recommendation: A mixed methods evaluation of a World Health Organization competency-based training package for foundational helping skills among pre-service and in-service health workers in Nepal, Peru and Uganda — R1/PR10

Comments

Dear Authors:

The reviewers have accepted your corrections to the manuscript.

The similarity anti-plagiarism software reports some problems within your text.

Please check the manuscript with your own software and correct the text accordingly.

Thank you.

Decision: A mixed methods evaluation of a World Health Organization competency-based training package for foundational helping skills among pre-service and in-service health workers in Nepal, Peru and Uganda — R1/PR11

Comments

No accompanying comment.