In 1999, the Institute of MedicineReference Kohn, Corrigan and Donaldson 1 reported that in the United States 44,000 to 98,000 people die annually as a result of avoidable medical errors. Furthermore, in a retrospective study of 15,000 charts of randomly selected admissions to Utah and Colorado hospitals, Gawande et al.Reference Gawande, Thomas, Zinner and Brennan 2 reported that 66% of the adverse events they analyzed were from surgical causes, while 54% were preventable.
Among the many initiatives undertaken to try and correct these shortcomings, the World Health Organization (WHO) Surgical Safety Checklist has certainly been one of the most successful. Haynes et al.Reference Haynes, Weiser and Berry 3 demonstrated that by using this simple tool, the surgical mortality rate fell from 1.5% to 0.8%, while inpatient complications were reduced from 11% to 7%. A Canadian study recently challenged the clinical benefit we may expect following government-driven mandatory implementation of surgical checklists.Reference Urbach, Govindarajan, Saskin, Wilton and Baxter 4 , Reference Wang and Serak 5 However, and despite this lack of consistency in the literature, available data still suggests that the WHO Surgical Safety Checklist has the potential to improve mortality and complication rates after surgery.Reference Tang, Ranmuthugala and Cunningham 6 Specifically in neurosurgery, there is no literature demonstrating a direct link between neurosurgical checklists and patient safety.Reference Zuckerman, Green, Carr, Dewan, Morone and Mocco 7
Implementation of a surgical checklist may pose a wealth of different challenges and, if not done properly, may reduce the benefits derived from such a process. Many surgical units that have adopted the WHO Surgical Safety Checklist audited their performance and discussed issues relating to the implementation process.Reference Borchard, Schwappach, Barbir and Bezzola 8 - Reference Levy, Senter and Hawkins 13 However, such literature is still lacking in neurosurgery.Reference McConnell, Fargen and Mocco 14
Thus the main objective of this study was to assess compliance with a surgical safety checklist in a neurosurgical unit. We sought to assess compliance with each step of the process (Sign-in, Time-out and Sign-out) and to identify factors associated with reduced compliance.
Methods
In the Centre Hospitalier Universitaire (CHU) de Québec – Hôpital de l’Enfant-Jésus, a tertiary care university hospital, a surgical safety checklist was formally established in February 2009, with surgical verification being conducted informally before that date. Although not mandatory in the province of Québec, surgical safety checklists are now required for hospital accreditation at the national level. At the time of implementation, the checklist was introduced to all operating room (OR) staff and practical workshops were held. The checklist is also taught to every new member who joins the OR team. The checklist adopted is based on the WHO Surgical Safety ChecklistReference Tang, Ranmuthugala and Cunningham 6 and comprises three steps: Sign-in, Time-out and Sign-out (Figure 1). Moreover, compliance with the surgical safety checklist is not systematically recorded provincially nor in our institution, hence the need to perform internal audits.
Perioperative checklist
Sign-in is usually done either in the surgical ward, in the waiting area of the OR or directly in the OR. It is mainly the nursing and anesthesia teams’ responsibility and the surgical team is usually absent from the OR at the time of Sign-in. Time-out is done in the OR in the presence of the neurosurgeon or neurosurgical resident, the nursing team, the anaesthesiologist and his/her team. This step is done immediately after positioning and draping and must be completed before skin incision. Finally, Sign-out is carried out during the final stages of the surgery with the neurosurgery resident or faculty still in the room, in interaction with the nursing and anesthesia teams.
Internal audit
A prospective observational study of all consecutive neurosurgical cases was conducted at the CHU de Québec – Hôpital de l’Enfant-Jésus between July 3 and August 29, 2013. All consecutive procedures performed between 8 a.m. and 5 p.m. on weekdays (total of 42 consecutive weekdays of observation) were included. An independent, direct observer, with no role in patient care and no involvement with the surgical team, (J.-F. G.) assessed compliance with the Time-out and Sign-out steps. The staff was aware that their compliance with the checklist was being recorded by the observer. Proper documentation of Sign-in was also analyzed, but direct observation of this step was not undertaken. Neurosurgical cases were not included if the observer was not available to properly document completion of the surgical safety checklist (e.g., when recording compliance with the checklist for another case). In accordance with institutional policy, research ethics board approval was not sought, since the patient data collected was non-identifying.
The time, date and surgical intervention performed were recorded for each case. Surgeries were recorded as either elective or emergent and categorized by subspecialty: vascular (excluding endovascular), spine (including tumours and trauma), head trauma, oncology, hydrocephalus, peripheral nerves, functional and others (including intracranial pressure monitoring and external ventricular drain).
The actual checklist used by the operating room (OR) staff was used to record which Sign-in, Time-out and Sign-out elements were verified.
In this study, we defined the term “compliance” with any of the steps on the checklist as proper completion of the step (as detailed above). The term “completeness”, which applies to Time-out and Sign-out, refers to the number of elements in each of these steps verified by the surgical team and reported by the observer. Making the distinction between compliance and completeness for the checklist and reporting both measures seemed important in analyzing how thoroughly our team has adopted and implemented the surgical safety checklist. This distinction was not made for Sign-in because this step was not directly observed and each Sign-in element was not analyzed separately.
-
1) We recorded whether or not Sign-in was documented correctly in the chart and if each element in this step had been verified. For the team to be considered compliant, Sign-in had to include: (a) patient identification, (b) confirmation of surgical site, surgical procedure and allergy status with the patient, (c) verification of the presence of a signed consent form in the chart and, finally, (d) a completed checklist.
-
2) Through direct observation, we documented whether or not the Time-out was done, its timing, its leader, and which elements were completed. For the surgical team to be considered compliant with the Time-out, it had to take a clear “pause” to carry out the verifications. For an element to be considered done, it had to be spoken clearly to the whole team present in the OR.
-
3) Through direct observation, we assessed whether or not the Sign-out was done and which elements were completed. To be considered compliant for the Sign-out, the surgical workflow had to be paused to carry out the verifications. Again, for an element to be considered done, it had to be spoken clearly to the whole team.
Statistical analyses
Results are presented as percentages. Univariate analyses were conducted to identify predictors of compliance with and completeness of the checklist using the following variables: subspecialty, emergent versus elective case, order of the case on the operative schedule (1st case of the day, 2nd case of the day, etc.), timing of data collection (first versus second half of the cohort) and Time-out leader. Categorical variables were compared between cases in which compliance with and completeness of were or were not observed using contingency tables and chi-square tests. Data were analyzed using the statistical software JMP v. 9.0.1 (SAS Institute Inc., Cary, NC).
Results
During the study period, 202 neurosurgical cases were operated on at our institution, 171 of which were included in the present study. The main reasons for exclusion were performance of the surgery outside of regular hours (30 cases) and absence of the observer at a case when recording checklist compliance for a concurrent case in another OR (1 case). Case characteristics are presented in Table 1.
Case characteristics
1. Sign-in
Compliance with the Sign-in was observed in 82% of the 168 recorded cases. It was completed in the OR 38% of the time, in the ward for 15% of patients and in an unspecified location for the remaining 47% of cases. No statistically significant difference was found in subgroup analyses for this data.
2. Time-out
Compliance with and completeness of the Time-out was assessed in 163 cases (Table 2). In eight cases, the observer was not present during the Time-out. These cases were kept in the study for their Sign-in and Sign-out data. The Time-out compliance was 99%. On average, 92% of Time-out elements were verified and in 46% of cases, all the elements were checked. Table 3 presents the compliance rate for each Time-out element.
Compliance with and mean completeness of the surgical Time-out
Proportion of cases in which the Time-out and Sign-out elements were completed
2.1 Univariate analysis of Time-out
In univariate analysis, none of the variables subspecialty, emergent versus elective case, timing of data collection, and order of the case during the day were associated with the Time-out compliance. However, elective cases and cases done during the first half of the observation period were associated with a higher number of Time-outs with verification of all elements (100% completion) (p=0.0065 and p=0.0004, respectively).
Subgroup analysis of the Table 3 elements did not reveal any statistically significant differences, including with respect to the least verified elements: adequate patient positioning, availability of venous access (p=0.2434) and particular concerns: anesthesia (p=0.2571).
3. Sign-out
One hundred and thirty-two cases were assessed for Sign-out compliance with an overall compliance rate of 93% and a completeness rate of 90%. In the remaining 39 cases, the surgery was completed outside of regular hours and could not be recorded. While the subspecialty and subgroup data were not significantly different, analysis of the elective subgroup (96%) versus emergent subgroup (81%, p=0.003) and the first half of the cohort (99%) versus the second half of the cohort (88%, p=0.0146) showed a statistically significant difference in compliance. Compliance with the different elements in the Sign-out is presented in Table 3. For the least verified element, namely, important events/postoperative plan, the subgroup analysis of elective versus emergent cases showed verification rates of 72% and 54%, respectively, but this difference was not statistically significant (p=0.6494). The observation time and subspecialty subgroups were not significantly different.
Discussion
In this internal audit, compliance of our neurosurgical unit with a surgical safety checklist was high, reaching 99% for Time-out. Likewise, the Sign-in and Sign-out processes were carried out in a high, albeit lower percentage of cases, 82% and 93% respectively. These results demonstrate that there is still room for improvement and stress the importance of measuring the actual implementation of these tools in order to perfect them and enhance their use. Observance of the Sign-in process is poorly reported in the surgical and neurosurgical literature. Only one study reports compliance with Sign-in elements.Reference Spence, Goodwin, Enns and Dean 15 Even though compliance was relatively high in the present study, this was the component of our surgical safety strategy that showed the lowest rate of adherence. The surgical staff’s presence is not mandatory during this step of the surgical checklist. Moreover, the location where it was completed was not standardized, resulting in high variability. In interpreting this data, one must remember that Sign-in was not always directly observed in this audit and the compliance estimates we report actually partially reflect proper documentation of completion of this step. Nevertheless, this finding led to the recognition that two changes were in order: (1) establish the OR as the only appropriate setting for actually completing Sign-in and (2) require the presence of a member of the neurosurgical team during this step. These two changes can further standardize the Sign-in process, emphasize the importance of this step and help foster a teamwork mentality.
The 99% compliance rate with Time-out obtained during this audit compares favourably with previously reported rates from various surgical specialties that range from 39% to 100%.Reference Borchard, Schwappach, Barbir and Bezzola 8 , Reference Johnston, Tergas and Bennett 10 , Reference Logan, Cressey and Wu 12 , Reference de Vries, Dijkstra, Smorenburg, Meijer and Boermeester 16 - Reference Fourcade, Blache, Grenier, Bourgain and Minvielle 19 In the neurosurgical literature, Lyons et al.Reference Lyons 20 reported that the checklist was used in 99.5% of their cases, while McLaughlin et al.Reference McLaughlin, Winograd, Chung, Van de Wiele and Martin 21 had a 100% compliance rate. We also sought to determine how complete the Time-outs were. In the present audit, we obtained a 92% mean completeness rate for the different questions in the Time-out. Interestingly, the elements that were the least verified were adequate patient positioning, availability of venous access, availability of implants and materials and availability of appropriate imaging results. On the other hand, identity verification, side and surgical procedure verification, antibiotics administration and surgical and nursing team concerns were adhered to well. This finding may reflect the importance given to each of these questions by Time-out leaders and participants. It may also reflect adjustments made by the surgical team to each case, thus raising the question of what are acceptable practice variations in implementing the surgical safety checklist. These observations in our audit will be followed by a re-examination of the checklist per se with the intent of removing or improving some of the elements in this step. This discrepancy between the number of Time-outs being done and of Time-outs with a 100% completion rate may also be explained in part by “checklist fatigue”. Reducing the number of elements in this step could help improve its application.
During the study period, the Sign-out process had a 93% compliance rate. The component that was the least frequently verified in the Sign-out was the reporting of important events during surgery and the postoperative plan. This may be due to the erroneous belief on the neurosurgeons’ part that the nursing and anesthesia teams are already aware of any unplanned surgical events or changes in the postoperative destination. Zuckerman et al.Reference Zuckerman, France, Green, Leming-Lee, Anders and Mocco 22 previously assessed the implementation of a debriefing checklist in neurosurgery and showed that it is hindered by challenges that are different to those in the other stages of surgical verification. We found similar obstacles in our study; namely, confusion regarding when to do it (before the end of the case versus prior to removal of the patient from the OR) and “ownership” of Sign-out (surgeons versus nurses). Sign-out must be considered a collaborative communication tool similar to the surgical briefing described by Lingard et al.Reference Lingard, Regehr and Orser 23
Emergent cases were not associated with a statistically significant difference in Time-out compliance. However, we found that they were associated with a lower number of cases in which all the elements in the Time-out were verified and with reduced compliance with Sign-out. This is likely explained by the urgency and need for efficiency that the surgical team may feel when it is time to proceed with the surgical intervention. Yet, these are probably the cases in which a sound surgical safety strategy is needed most. This was a useful finding for us and was tagged as an area in which the whole team had to improve its performance.
One limitation of this study is the presence of the Hawthorne effect, as demonstrated by the significantly higher number of Time-outs with verification of all the elements and compliance with Sign-outs in the first half of the study period. This bias might have been mitigated if the data collection were conducted by video surveillance. Besides, this Hawthorne effect was an important reminder to our team that the integration of the checklist into our routine still needed to be improved. One may also propose that the process may need to be recorded constantly to achieve an effective perioperative checklist and, more importantly, to achieve the cultural shift we are aiming for. By giving regular attention to preventing and recording “near misses” and errors, behaviours may be changed and errors limited in frequency as shown by Oremakinde and Bernstein.Reference Oremakinde and Bernstein 24 In fact, by analyzing our performance in implementing the surgical checklist, our intention was to emphasize the culture of patient safety. This institutional value is at the centre of high-quality neurosurgical care.Reference Berger, Wachter, Greysen and Lau 25 Such a culture of patient safety was already present and shared by the different members of our surgical team at the time of the study. This is evidenced by the shared leadership of the Time-Out process (33% led by surgeons, 43% by nurses and 24% by residents). Although institutional leadership is paramount, empowerment of all members of the team is, in fact, central to this safety strategy.Reference Sekhar and Mantovani 26 Indeed, we hypothesize that this empowerment is key in obtaining the clinical benefit demonstrated by Haynes et al.Reference Haynes, Weiser and Berry 3 in their seminal study and that was not as substantial in more recent reports.Reference Urbach, Govindarajan, Saskin, Wilton and Baxter 4
Additionally, a selection bias may have been introduced by including only daytime cases. Excluding surgeries that were performed outside of regular hours may have resulted in a lower number of emergent cases and lower personnel variability. Recording of the Sign-out step was subject to a higher rate of missing data, which may have resulted in an additional selection bias for the same reasons. Finally, a larger cohort would have been desirable for the purposes of internal validity and certain subgroup analyses (e.g., subspecialty).
Conclusion
In this internal audit study, compliance with the preoperative checklist reached a satisfactory level. Work is still needed, however, on some aspects of our surgical strategy, namely, a relatively low compliance rate with the Sign-in process was recorded and emergent cases were associated with decreased performance. A number of modifications to our surgical safety strategy were proposed based on the findings of this audit. These include a more standardized Sign-in with the mandatory participation of a member of the surgical team, revision of the elements in each step of the checklist, clarifications concerning the timing and leadership of the Sign-out and the introduction of regular audits of compliance with the safety strategy.
On the whole, the preoperative checklist must be part of a more comprehensive safety attitude and a teamwork mentality should be encouraged and developed to derive the full benefits of its implementation.
Acknowledgements
The authors thank Simon Leblond, neurosurgery nursing team leader, Josée Migneault, OR head nurse and Andrée Langevin, Johanne Dussault and Cindy Lamarre, neurosurgery OR nurses, for their collaboration in this project and their significant assistance in conducting the study and thank all the neurosurgeons, especially Dr Pascale Lavoie, and all the staff from the Hôpital de l’Enfant-Jésus OR. This study was supported by local funding.
Part of this work was presented during the Annual Conference of the Association de Neurochirurgie du Québec held in Montreal, Quebec, Canada on November 7 and 8, 2014.
Disclosures
Jean-François Gagné, Moujahed Labidi and André Turmel do not have anything to disclose.
