Introduction
In Belgium, there is problem with the inappropriate use of emergency services (Brasseur et al., Reference Brasseur, Gilbert, Servotte, Donneau, D’Orio and Ghuysen2019a; Philips et al., Reference Philips, Remmen, De Paepe, Buylaert and Van Royen2010a). In 2012, we registered 290 emergency contacts per 1000 inhabitants, which is far above the average in comparable nations (Carret et al., Reference Carret, Fassa and Domingues2009; Philips et al., Reference Philips, Michiels, Coenen and Remmen2014; Uscher-Pines et al., Reference Uscher-Pines, Pines, Kellermann, Gillen and Mehrotra2013). Besides, 71% of the patients are self-referrers. Since 2003, 80 general practitioner cooperatives (GPCs) were established offering out-of-hours care in a primary care context. Against all expectations, the number of emergency contacts did not decrease (Philips et al., Reference Philips, Remmen, Van Royen, Teblick, Geudens, Bronckaers and Meeuwis2010b). The opening hours of a GPC are restricted, there are no formal agreements between GPC and ED (emergency department) and, above all, patients are more confident with the service of an ED.
A nationwide study demonstrated that there is a need to develop an effective triage system improving the out-of-hours referring process (Van den Heede et al., Reference Van Den Heede, Devriese, Baier, Camaly, Depuijdt, Geissler, Ghesquiere, Misplon, Quentin, Van Loon and Van De Voorde2016). From a review, we also know that the introduction of telephone triage systems is a promising intervention to reduce inappropriate ED visits (Van den Heede & Van de Voorde, Reference Van Den Heede and Van De Voorde2016).
The overuse of the ED is also encouraged by free access (Carret et al., Reference Carret, Fassa and Kawachi2007). This inappropriate use of the ED undermines the functioning of GPCs for out-of-hours care (O’Kelly et al., Reference O’kelly, Teljeur, Carter and Plunkett2010; Huibers et al., Reference Huibers, Thijssen, Koetsenruijter, Giesen, Grol and Wensing2013; Philips et al., Reference Philips, Remmen, Van Royen, Teblick, Geudens, Bronckaers and Meeuwis2010b). The government installed the nationwide number 1733 for unplanned, non-live-threatening out-of-hours care. An operator directs the caller to the appropriate level of care: three levels of ambulance intervention, urgent or planned referral to out-of-hours primary care services or to planned care (https://www.health.belgium.be/en/health/need-call-doctor-call-1733). These operators are trained to follow digital protocols. These protocols are the result of a collaboration between academic partners, the Federal Public Service Health, experts in general practice and in emergency medicine.
To evaluate the efficiency and safety of the use of the initial protocols, a dry run test phase was implemented (Van der Mullen et al., Reference Van Der Mullen, Van Baelen, Crits, Wuyts and Sabbe2017). All patients were asked to call the 1733 emergency number in the presence of a researcher. The operator interrogated the patient following a flow chart and referred according to the chosen protocol to the appropriate care level. In 30% of all calls, the operator assigned the call to the protocol ‘unclear problem’. In 11.9% of all these cases, there was indeed an unclear problem (eg, communication, multiple complaints). In all other cases, there was no appropriate protocol available or there was a problem on operator level.
The initial protocols were adjusted and completed with specific primary care protocols (PCP) (attachment 1). The operators received instructions and dry run training by GP- and ED-experts to improve the use of the new protocols. The protocol ‘unclear problem’ was renamed to ‘unwell for no clear reason’. The Belgian Counsel for Urgent Healthcare approved this final version of the protocols.
In international literature, there is a lack of research on the implementation of a central, nationwide number for unplanned out-of-hours care. Since many health care systems are confronted with inefficient patient flows in unplanned or emergency care, an implementation research on this topic might contribute to improvement of this particular type of care (Ghazali et al., Reference Ghazali, Richard, Chaudet, Choquet, Guericolas and Casalino2019; Young et al., Reference Young, Wagner, Kellermann, Ellis and Bouley1996).
Therefore, the aim of this research was to determine whether the adjusted protocol ‘unwell for no clear reason’ led to a safer and more efficient referral of calls to the appropriate care level and to evaluate the efficiency and safety of the specific PCPs.
Methods
Design
The study was performed in a cross-sectional design. The earlier protocol ‘unclear problem’ was adjusted and renamed to ‘unwell for no clear reason’. The operator chooses this protocol when the caller reports complaints that do not fit into any other protocol.
A protocol is considered as efficient when it leads with the appropriate care level: three levels of ambulance intervention, urgent or planned referral to out-of-hours primary care services or to planned care. A protocol is considered safe when the rate of under-triage is <15%. Both decisions on efficiency and safety were the result of a consensus between experts, the Intermediary Report Project 1733, guidelines and national research reports (Wheeler et al., Reference Wheeler, Greenberg, Mahlmeister and Wolfe2015; Van der Mullen et al., Reference Van Der Mullen, Van Baelen, Crits, Wuyts and Sabbe2017; Van den Heede & Van de Voorde, Reference Van Den Heede and Van De Voorde2016; Uscher-Pines et al., Reference Uscher-Pines, Pines, Kellermann, Gillen and Mehrotra2013).
The 1733 operator follows a flow chart, instructed by the appropriate protocol, to question and to refer the caller. Each flow chart starts with a set of medical terms. Each protocol integrates different levels of severity leading to a particular level of care.
To determine whether the adjusted protocol ‘unwell for no clear reason’ leads to a safer and more efficient referral, a set of calls was audited by two researchers. A computerized random selection was made of all calls recognized as ‘unwell for no clear reason’ between May 4 and June 26, 2018. For the collection of data, a template inventorying time of day, patient history, protocol choice and level of care structured the assessment. The researchers individually assessed the calls, marked their protocol of choice and motivated this choice: correct protocol, missing protocol, fail to recognize keyword, fail of the operator, communication problem and impossible to refer. At this stage, the researchers were not in contact with each other.
The data were qualitatively analyzed according to the 'Grounded Theory approach'. For the qualitative part, the researchers (n = 5) discussed the results of the assessment of calls in different rounds. In case of disagreement with the operators, the call was labeled as ‘discordant’. In case the researchers did not reach consensus, the call was presented to the advisory group. If the operator initially referred to a lower care level, the call was labeled as ‘under-triage’ (with the number of levels below). If the operator initially referred to a higher care level, the call was labeled as over-triage.
In the second research question, the efficiency and safety of the specific PCPs were studied following the same procedure as described above. For each of the 18 PCPs, 15 calls were randomly selected by randomization software from January 1 until June 30, 2018. All patients referred to these protocols were assessed by the GP on duty (Weekends July 27–30, 2018 and February 8–11, 2019). The assessment took place when the patient presented at the GPC and before the consult started. The GP asked the patients the reason for encounter and then selected the level of care. This option was compared to the option the operator picked. The outcome of this assessment did not affect the provided care.
Efficiency of referral by the operator was expressed as the level of concordance between the operators’ and the GPs’ referrals. Safety was measured by the occurrence and the level of under-triage.
Ethical approval
This study was approved per research question by the Medical Ethical Committee of the University Hospitals of the KU Leuven (041C55FA505A80, 04324FEAA15C80 and 047066BAC85E80).
Results
Efficiency and safety of ‘unwell for no clear reason’
For the period of January 2018 to June 30, 2018, 11 622 calls were registered. 756 (6.6%) of these calls were allocated to the protocol ‘unwell for no clear reason’ of which a random sample of 180 calls was audited. Sixteen calls were excluded because of technical errors. In total, 164 calls were included.
The researchers assigned the label ‘fail to recognize keyword’ to 51 calls (31%), the label ‘correct protocol’ to 44 (27%) calls and the label ‘missing protocol’ to 37 (23%) calls. They assigned the label ‘fail of the operator’ to 23 (14%) calls. Nine (5%) calls were labeled as ‘communication problem’. No call was labeled as ‘impossible to refer’ (Table 1).
Efficiency and safety of ‘Unwell for no clear reason’
When assessing the efficiency of allocation to a protocol, 104 (64%) calls were labeled as ‘unwell for no clear reason’. Eight calls (5%) were allocated to the protocol ‘dizziness’, six calls to the protocol ‘skin problems’ and five (3%) calls to ‘airway problems’ (Table 1).
The operator referred 135 (82%) to the out-of-hours primary care services and 23 (14%) calls were urgently referred. Planned care was advised in 4 (3%) calls. Two (1%) calls were referred to ‘ambulance intervention’.
When assessing the safety of referral, the researchers referred 111 (68%) calls to the same level of care as the operator. 36 (22% of total) of all calls were recognized as ‘over triage’ and 17 (10% of total) as ‘under triage’ (Table 1).
The allocation to an inappropriate protocol led in 25 (15%) calls to a discordance of care level between the operator and the researcher. In 12 calls, the referral was labeled as ‘over-triage’ and 13 calls were recognized as ‘under triage’ (Table 1).
The efficiency and safety of (referral to) the specific PCPs
For each of the 18 PCPs, 15 calls were randomly selected for assessment (Table 2). For seven protocols, <15 calls were available. The researchers assessed 202 calls. The researchers assessed 126 (62%) calls as ‘correct protocol’, 13 (6%) as ‘missing protocol’, 48 (24%) as ‘fail to recognize keyword’, 12 (6%) as ‘fail of the operator’ and three (2%) as ‘communication problem’. To determine the efficiency of the PCP, the researchers assessed the allocation to a protocol. 76 (37%) calls were allocated to another protocol: 16 (8%) calls were allocated to ‘unwell for no clear reason’, 10 (5%) to ‘child with fever’, 6 (3%) to ‘wounds’, 5 (2.5%) to ‘abdominal pain’ and 4 (2.5%) to both ‘ear-nose-throat’ and ‘allergic reaction’.
The efficiency and safety of (referral to) the specific primary care protocols
To determine the safety of referral to the PCP, the researchers assessed the allocation to the level of care (Table 2). 175 calls (87%) were allocated to ‘out of hours primary care services’, 19 calls (9%) to ‘urgent out of hours primary care services’, 7 calls (3%) to ‘ambulance’ and 1 (1%) call to ‘planned care’. The researchers referred 46 (23%) calls to another care level: in 22 (11%) calls, there was under-triage and in 24 (12%) over-triage. The researchers disagreed about the level of care in 10 calls. These calls were audited and discussed until consensus: two calls remained undetermined due to a language issue and missing administration. The researchers labeled 100 (50%) calls as ‘correct protocol and correct care level’ and 27 (13%) calls as ‘inappropriate protocol and correct care level’. 57 (28%) calls were labeled as ‘incorrect protocol’ but correct care level. In 18 (8%) cases, the researchers labeled a call as ‘inappropriate protocol and inappropriate care level’. Nine (6%) calls were referred to a lower care level than appropriate and seven (3.5%) to a higher level.
To determine the safety of referral to a care level using the PCP, 325 allocated patients received an assessment by the GP (Table 3). Seven assessments were incomplete, and 68 assessments were completed after the consultation. In total, 250 assessments were included.
Discordance of allocation to care level between operator and GP on duty
In the summer weekend, the following reasons for encounter were withheld: 24 (10.5%) wound/skin problem, 15 (15%) stomach-intestines, 10 (10%) ear-nose-throat, 7 (7%) uro-genital, 6 (6%) allergic reaction/insect bite and 40 (39%) all other problems <2% prevalent. In the winter weekend, reason for encounter was in 45 cases (32%) ear-nose-throat, 19 (12%) lung, 15 (10%) stomach-intestines, 14 (9%) unwell for no clear reason, 10 (7%) uro-genital and 45 (30%) other reasons.
Of all patients presenting at the guard post, the operator allocated 181 (73%) cases to a regular consultation, 43 (19%) to regular home visit and 26 (8%) to urgent home visit. In summer versus winter 6 (5%), respectively, 13 (9%) cases concerned an urgent home visit, 25 (25%), respectively, 22 (15%) a regular home visit, 71 (70%), respectively, 113 (76%) a regular consultation.
After assessment by the GP, 236 (94.4%) calls were correctly referred. In two (0.8%) cases, the patients needed referring to a higher level of care and in 12 (5%) cases to a lower level of care.
Discussion
The efficiency and safety of the protocol ‘unwell for no clear reason’ improved after adjusting the protocols and adding PCPs. The number of calls referred to this protocol decreased as compared to the first trial. The concordance rate between allocation of a call to this protocol by operator and researcher was 2/3, which was higher than in the test phase. The rate of under-triage by allocation to ‘unwell for no clear reason’ was 1/10. By implementing new PCPs, there was a halving of allocation to ‘missing protocol’. The concordance of allocation to the new PCPs by operator and researcher was 2/3, but the safety levels for under- and over-triage were not exceeded. The GP’s on duty judged that over 9/10 of all patient encounters were correctly allocated. In <1% of all referrals, the GP marked an under-triage.
In one-third of the referrals to ‘unwell for no clear reason’, the call was labeled as ‘fail to recognize key word’. This rate was higher than that in the test phase and could be explained by the lack of experience with the new protocols. Training of the operators is therefore very important (Hansen & Hunskaar, Reference Hansen and Hunskaar2011; Huibers et al., Reference Huibers, Smits, Renaud, Giesen and Wensing2011).
By improving the PCPs, there was a manifest decrease of the number of calls assigned to ‘missing protocol’. When assessing the allocation of calls to the protocol ‘unwell for no clear reason’, the researchers now lacked the protocols ‘fever adult’, ‘breast problems’ and ‘hypertension’. By developing and implementing these protocols, the number of allocations to ‘unwell for no clear reason’ could further decrease (Smits et al., Reference Smits, Hanssen, Huibers and Giesen2016; Brasseur et al., Reference Brasseur, Servotte, Donneau, Stipulante, D’Orio and Ghuysen2019b). In addition, researchers assessed half of the allocations to the PCP as correct for protocol and care level, and this rate could benefit from adding new PCP.
The implementation of PCP seemed successful since operators allocated more than 9/10 of all calls to the GP guard post. Only 1% of all calls was referred to planned care. This number is low and probably due to a lack of experience with this newly introduced care level (Rutten et al., Reference Rutten, Vrielink, Smits and Giesen2017; Plat et al., Reference Plat, Peters, Loots, De Groot, Eckhardt, Keizer and Giesen2018; Morreel et al., Reference Morreel, Philips and Verhoeven2019).
The main reasons of discordance between operator and researcher in allocation to the PCP were a fail of the operator and a fail to recognize the keyword. This was particularly the case when the age of the caller was important to allocate to a protocol. Operators often failed to question the context and the state of emergency. Time, experience and training were lacking present. A digitized algorithm could support the operators (Smits et al., Reference Smits, Hanssen, Huibers and Giesen2016; Smits et al., Reference Smits, Rutten, Keizer, Wensing, Westert and Giesen2017; Brasseur et al., Reference Brasseur, Servotte, Donneau, Stipulante, D’Orio and Ghuysen2019b). Operators were confronted with dissatisfied callers who expected to receive a medical advice (Giesen et al., Reference Giesen, Ferwerda, Tijssen, Mokkink, Drijver, Van Den Bosch and Grol2007; Huibers et al., Reference Huibers, Smits, Renaud, Giesen and Wensing2011; Huibers et al., Reference Huibers, Giesen, Smits, Mokkink, Grol and Wensing2012). Public information campaigns are necessary to align the objectives of the 1733-system and the public expectations (Morreel et al., Reference Morreel, Philips and Verhoeven2019; Philips et al., Reference Philips, Verhoeven, Morreel, Colliers, Remmen, Coenen and Van Royen2019). Strikingly, the operators only once opted for the care level ‘planned care’, and the researchers also rarely opted this care level. Operators are not familiar with this newly introduced care level and callers not used to referral to postponed care. Language barriers and missing keywords in protocols were also reasons for incorrect referral. Probably, this is a point of attention that needs evaluation on regular base (Huibers et al., Reference Huibers, Smits, Renaud, Giesen and Wensing2011; van der Straten et al., Reference Van Der Straten, Van Stel, Spee, Vreeburg, Schrijvers and Sturms2012).
The rate of under- and over-triage was acceptable as appeared by re-listening the calls and by pre-consultation assessment at the guard post. Over-triage was more likely than under-triage and mainly due to referral to care level ‘out of hours primary care services’ instead of referral to ‘planned care’. Unlike over-triage, under-triage may result in an unsafe situation (Huibers et al., Reference Huibers, Smits, Renaud, Giesen and Wensing2011; Plat et al., Reference Plat, Peters, Loots, De Groot, Eckhardt, Keizer and Giesen2018; Brasseur et al., Reference Brasseur, Servotte, Donneau, Stipulante, D’Orio and Ghuysen2019b). Under-triage was caused by referral to ‘out of hours primary care services’ instead of ‘urgent home visit’. In two cases, there was an unacceptable under-triage by referring the patient to ‘out of hours primary care services’ instead to ‘ambulance’.
The strength of this research lies in the fact that it started in a dry run phase and, after evaluation and adjustment, continued in a real-life setting. Second, the protocols are now fully used in the 1733 operating center. The researchers (GP’s) took part in the development and fine-tuning of the protocols during the first test phase of the implementation of the protocol.
An important limitation of the study is that the assessment of calls was based upon the competence of the researchers. Indeed, the researchers were GPs, while the majority of the operators do not have a medical background (Plat et al., Reference Plat, Peters, Loots, De Groot, Eckhardt, Keizer and Giesen2018). Although a thorough reviewing and revision of the results was performed, misinterpretations cannot be ruled out.
Conclusion
This study demonstrated that the 1733-telephone triage system for out-of-hours care is successful if protocols, flow charts and emergency levels are well defined, continuously monitored and operators well trained. Further research should focus on the rate and conditions of under- and over-triage to improve the competences and skills of the operators. In practice, selection, education and training of operators are indispensable to guarantee safety and cost efficiency.
Acknowledgements
Our gratitude goes to the participating patients and GP’s and to the coordinators of the General Practitioner Cooperatives. Our special thanks go to the participating physicians taking the time to assess patients during their shifts and to the patients willing to undergo a second assessment while waiting for medical help.
Authors’ Contribution
Conception and design of the research: all authors; Acquisition of data: LD, SP, MJ; Analysis of data: all authors; Drafting the article: LD, SP, MJ; Revision the article: all authors; All authors read and approved the final manuscript.
Financial support
This research received no specific grant from any funding agency, commercial or not-for-profit sectors.
Conflicts of interest
None.
Open access
