Hostname: page-component-8448b6f56d-qsmjn Total loading time: 0 Render date: 2024-04-25T00:02:58.877Z Has data issue: false hasContentIssue false
  • English
  • Français

Peak and ceiling effects in final-product analysis of mastoidectomy performance

Published online by Cambridge University Press:  22 September 2015

N West ,
L Konge ,
P Cayé-Thomasen ,
M S Sørensen  and
S A W Andersen
N West
Affiliation:
Department of Otorhinolaryngology – Head & Neck Surgery, Rigshospitalet, Copenhagen, Denmark
L Konge
Affiliation:
Center for Clinical Education, Centre for HR, Capital Region of Denmark, Copenhagen, Denmark
P Cayé-Thomasen
Affiliation:
Department of Otorhinolaryngology – Head & Neck Surgery, Rigshospitalet, Copenhagen, Denmark
M S Sørensen
Affiliation:
Department of Otorhinolaryngology – Head & Neck Surgery, Rigshospitalet, Copenhagen, Denmark
S A W Andersen*
Affiliation:
Department of Otorhinolaryngology – Head & Neck Surgery, Rigshospitalet, Copenhagen, Denmark
*
Address for correspondence: S A W Andersen, Department of Otorhinolaryngology – Head & Neck Surgery, Rigshospitalet, Blegdamsvej 9, DK-2100 Copenhagen Ø, Denmark E-mail: stevenarild@gmail.com
Get access Rights & Permissions [Opens in a new window]

Abstract

Background:

Virtual reality surgical simulation of mastoidectomy is a promising training tool for novices. Final-product analysis for assessing novice mastoidectomy performance could be limited by a peak or ceiling effect. These may be countered by simulator-integrated tutoring.

Methods:

Twenty-two participants completed a single session of self-directed practice of the mastoidectomy procedure in a virtual reality simulator. Participants were randomised for additional simulator-integrated tutoring. Performances were assessed at 10-minute intervals using final-product analysis.

Results:

In all, 45.5 per cent of participants peaked before the 60-minute time limit. None of the participants achieved the maximum score, suggesting a ceiling effect. The tutored group performed better than the non-tutored group but tutoring did not eliminate the peak or ceiling effects.

Conclusion:

Timing and adequate instruction is important when using final-product analysis to assess novice mastoidectomy performance. Improved real-time feedback and tutoring could address the limitations of final product based assessment.

Keywords

Temporal BoneEducationMedicalComputer SimulationClinical Competence
Type
Main Articles
Information
The Journal of Laryngology & Otology , Volume 129 , Issue 11 , November 2015 , pp. 1091 - 1096
Copyright
Copyright © JLO (1984) Limited 2015 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

1.George, AP, De, R.Review of temporal bone dissection teaching: how it was, is and will be. J Laryngol Otol 2010;124:119–25CrossRefGoogle ScholarPubMed
2.Mills, R, Lee, P.Surgical skills training in middle-ear surgery. J Laryngol Otol 2003;117:159–63CrossRefGoogle ScholarPubMed
3.Wiet, GJ, Stredney, D, Sessanna, D, Bryan, JA, Welling, DB, Schmalbrock, P.Virtual temporal bone dissection: an interactive surgical simulator. Otolaryngol Head Neck Surg 2002;127:79–83CrossRefGoogle ScholarPubMed
4.Rhienmora, P, Haddawy, P, Khanal, P, Suebnukarn, S, Dailey, MN.A virtual reality simulator for teaching and evaluating dental procedures. Methods Inf Med 2010;49:396–405Google ScholarPubMed
5.Ungi, T, Sargent, D, Moult, E, Lasso, A, Pinter, C, McGraw, RC et al. Perk Tutor: an open-source training platform for ultrasound-guided needle insertions. IEEE Trans Biomed Eng 2012;59:3475–81CrossRefGoogle ScholarPubMed
6.Sewell, C, Morris, D, Blevins, N, Dutta, S, Agrawal, S, Barbagli, F et al. Providing metrics and performance feedback in a surgical simulator. Comput Aided Surg 2008;13:63–81CrossRefGoogle Scholar
7.Zirkle, M, Taplin, MA, Anthony, R, Dubrowski, A.Objective assessment of temporal bone drilling skills. Ann Otol Rhinol Laryngol 2007;116:793–8CrossRefGoogle ScholarPubMed
8.Laeeq, K, Bhatti, NI, Carey, JP, Della Santina, CC, Limb, CJ, Niparko, JK et al. Pilot testing of an assessment tool for competency in mastoidectomy. Laryngoscope 2009;119:2402–10CrossRefGoogle ScholarPubMed
9.Butler, NN, Wiet, GJ.Reliability of the Welling scale (WS1) for rating temporal bone dissection performance. Laryngoscope 2007;117:1803–8CrossRefGoogle ScholarPubMed
10.Martin, JA, Regehr, G, Reznick, R, Macrae, H, Murnaghan, J, Hutchinson, C et al. Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg 1997;84:273–8Google ScholarPubMed
11.Szalay, D, MacRae, H, Regehr, G, Reznick, R.Using operative outcome to assess technical skill. Am J Surg 2000;180:234–7CrossRefGoogle ScholarPubMed
12.Andersen, SA, Cayé-Thomasen, P, Sølvsten, Sørensen M.Mastoidectomy performance assessment of virtual simulation training using final-product analysis. Laryngoscope 2015;125:431–5CrossRefGoogle ScholarPubMed
13.Wan, D, Wiet, GJ, Welling, DB, Kerwin, T, Stredney, D.Creating a cross-institutional grading scale for temporal bone dissection. Laryngoscope 2010;120:1422–7CrossRefGoogle ScholarPubMed
14.Munz, Y, Moorthy, K, Bann, S, Shah, J, Ivanova, S, Darzi, SA.Ceiling effect in technical skills of surgical residents. Am J Surg 2004;188:294–300CrossRefGoogle ScholarPubMed
15.Moorthy, K, Munz, Y, Adams, S, Pandey, V, Darzi, A.A human factors analysis of technical and team skills among surgical trainees during procedural simulations in a simulated operating theatre. Ann Surg 2005;242:631–9CrossRefGoogle Scholar
16.Visible Ear Simulator. In: http://ves.cg.alexandra.dk/ [15 December 2014]Google Scholar
17.Sorensen, MS, Mosegaard, J, Trier, P.The visible ear simulator: a public PC application for GPU-accelerated haptic 3D simulation of ear surgery based on the visible ear data. Otol Neurotol 2009;30:484–7CrossRefGoogle ScholarPubMed
18.Trier, P, Noe, KØ, Sørensen, MS, Mosegaard, J.The visible ear surgery simulator. Stud Health Technol Inform 2008;132:523–5Google ScholarPubMed
19.Bech, B, Lönn, L, Falkenberg, M, Bartholdy, NJ, Räder, SB, Schroeder, TV et al. Construct validity and reliability of structured assessment of endovascular expertise in a simulated setting. Eur J Vasc Endovasc Surg 2011;42:539–48CrossRefGoogle Scholar
20.Reznick, RK.Teaching and testing technical skills. Am J Surg 1993;165:358–361CrossRefGoogle ScholarPubMed
21.Bjurström, JM, Konge, L, Lehnert, P, Krogh, CL, Hansen, HJ, Petersen, RH et al. Simulation-based training for thoracoscopy. Sim Healthcare 2013;8:317–323CrossRefGoogle ScholarPubMed
22.Kruglikova, I, Grantcharov, TP, Drewes, AM, Funch-Jensen, P.The impact of constructive feedback on training in gastrointestinal endoscopy using high-fidelity Virtual-Reality simulation: a randomised controlled trial. Gut 2010;59:181–5CrossRefGoogle ScholarPubMed
23.Brydges, R, Carnahan, H, Safir, O, Dubrowski, A.How effective is self-guided learning of clinical technical skills? It's all about process. Med Educ 2009;43:507–515CrossRefGoogle ScholarPubMed
24.Brydges, R, Nair, P, Ma, I, Shanks, D, Hatala, R.Directed self-regulated learning versus instructor-regulated learning in simulation training. Med Educ 2012;46:648–656CrossRefGoogle ScholarPubMed
25.Fann, JI, Caffarelli, AD, Georgette, G, Howard, SK, Gaba, DM, Youngblood, P et al. Improvement in coronary anastomosis with cardiac surgery simulation. J Thorac Cardiovasc Surg 2008;136:1486–91CrossRefGoogle ScholarPubMed
26.Kerwin, T, Wiet, G, Stredney, D, Shen, H-W.Automatic scoring of virtual mastoidectomies using expert examples. Int J Comput Assist Radiol Surg 2012;7:1–11CrossRefGoogle ScholarPubMed