2 results
LO84: Ready to run the show: development of a new instrument for assessing resident competence in the emergency department
- W. Cheung, W. Gofton, T. Wood, M. Duffy, S. Dewhirst, N. Dudek
-
- Journal:
- Canadian Journal of Emergency Medicine / Volume 21 / Issue S1 / May 2019
- Published online by Cambridge University Press:
- 02 May 2019, p. S38
- Print publication:
- May 2019
-
- Article
-
- You have access Access
- Export citation
-
Innovation Concept: The outcome of emergency medicine training is to produce physicians who can competently run an emergency department (ED) shift. While many workplace-based ED assessments focus on discrete tasks of the discipline, others emphasize assessment of performance across the entire shift. However, the quality of assessments is generally poor and these tools often lack validity evidence. The use of entrustment scale anchors may help to address these psychometric issues. The aim of this study was to develop and gather validity evidence for a novel tool to assess a resident's ability to independently run an ED shift. Methods: Through a nominal group technique, local and national stakeholders identified dimensions of performance reflective of a competent ED physician. These dimensions were included in a new tool that was piloted in the Department of Emergency Medicine at the University of Ottawa during a 4-month period. Psychometric characteristics of the items were calculated, and a generalizability analysis used to determine the reliability of scores. An ANOVA was conducted to determine whether scores increased as a function of training level (junior = PGY1-2, intermediate = PGY3, senior = PGY4-5), and varied by ED treatment area. Safety for independent practice was analyzed with a dichotomous score. Curriculum, Tool or Material: The developed Ottawa Emergency Department Shift Observation Tool (O-EDShOT) includes 12-items rated on a 5-point entrustment scale with a global assessment item and 2 short-answer questions. Eight hundred and thirty-three assessment were completed by 78 physicians for 45 residents. Mean scores differed significantly by training level (p < .001) with junior residents receiving lower ratings (3.48 ± 0.69) than intermediate residents who received lower ratings (3.98 ± 0.48) than senior residents (4.54 ± 0.42). Scores did not vary by ED treatment area (p > .05). Residents judged to be safe to independently run the shift had significantly higher mean scores than those judged not to be safe (4.74 ± 0.31 vs 3.75 ± 0.66; p < .001). Fourteen observations per resident, the typical number recorded during a 1-month rotation, were required to achieve a reliability of 0.80. Conclusion: The O-EDShOT successfully discriminated between junior, intermediate and senior-level residents regardless of ED treatment area. Multiple sources of evidence support the O-EDShOT producing valid scores for assessing a resident's ability to independently run an ED shift.
Contributors
-
- By Lenard A. Adler, Pinky Agarwal, Rehan Ahmed, Jagga Rao Alluri, Fawaz Al-Mufti, Samuel Alperin, Michael Amoashiy, Michael Andary, David J. Anschel, Padmaja Aradhya, Vandana Aspen, Esther Baldinger, Jee Bang, George D. Baquis, John J. Barry, Jason J. S. Barton, Julius Bazan, Amanda R. Bedford, Marlene Behrmann, Lourdes Bello-Espinosa, Ajay Berdia, Alan R. Berger, Mark Beyer, Don C. Bienfang, Kevin M. Biglan, Thomas M. Boes, Paul W. Brazis, Jonathan L. Brisman, Jeffrey A. Brown, Scott E. Brown, Ryan R. Byrne, Rina Caprarella, Casey A. Chamberlain, Wan-Tsu W. Chang, Grace M. Charles, Jasvinder Chawla, David Clark, Todd J. Cohen, Joe Colombo, Howard Crystal, Vladimir Dadashev, Sarita B. Dave, Jean Robert Desrouleaux, Richard L. Doty, Robert Duarte, Jeffrey S. Durmer, Christyn M. Edmundson, Eric R. Eggenberger, Steven Ender, Noam Epstein, Alberto J. Espay, Alan B. Ettinger, Niloofar (Nelly) Faghani, Amtul Farheen, Edward Firouztale, Rod Foroozan, Anne L. Foundas, David Elliot Friedman, Deborah I. Friedman, Steven J. Frucht, Oded Gerber, Tal Gilboa, Martin Gizzi, Teneille G. Gofton, Louis J. Goodrich, Malcolm H. Gottesman, Varda Gross-Tsur, Deepak Grover, David A. Gudis, John J. Halperin, Maxim D. Hammer, Andrew R. Harrison, L. Anne Hayman, Galen V. Henderson, Steven Herskovitz, Caitlin Hoffman, Laryssa A. Huryn, Andres M. Kanner, Gary P. Kaplan, Bashar Katirji, Kenneth R. Kaufman, Annie Killoran, Nina Kirz, Gad E. Klein, Danielle G. Koby, Christopher P. Kogut, W. Curt LaFrance, Patrick J.M. Lavin, Susan W. Law, James L. Levenson, Richard B. Lipton, Glenn Lopate, Daniel J. Luciano, Reema Maindiratta, Robert M. Mallery, Georgios Manousakis, Alan Mazurek, Luis J. Mejico, Dragana Micic, Ali Mokhtarzadeh, Walter J. Molofsky, Heather E. Moss, Mark L. Moster, Manpreet Multani, Siddhartha Nadkarni, George C. Newman, Rolla Nuoman, Paul A. Nyquist, Gaia Donata Oggioni, Odi Oguh, Denis Ostrovskiy, Kristina Y. Pao, Juwen Park, Anastas F. Pass, Victoria S. Pelak, Jeffrey Peterson, John Pile-Spellman, Misha L. Pless, Gregory M. Pontone, Aparna M. Prabhu, Michael T. Pulley, Philip Ragone, Prajwal Rajappa, Venkat Ramani, Sindhu Ramchandren, Ritesh A. Ramdhani, Ramses Ribot, Heidi D. Riney, Diana Rojas-Soto, Michael Ronthal, Daniel M. Rosenbaum, David B. Rosenfield, Durga Roy, Michael J. Ruckenstein, Max C. Rudansky, Eva Sahay, Friedhelm Sandbrink, Jade S. Schiffman, Angela Scicutella, Maroun T. Semaan, Robert C. Sergott, Aashit K. Shah, David M. Shaw, Amit M. Shelat, Claire A. Sheldon, Anant M. Shenoy, Yelizaveta Sher, Jessica A. Shields, Tanya Simuni, Rajpaul Singh, Eric E. Smouha, David Solomon, Mehri Songhorian, Steven A. Sparr, Egilius L. H. Spierings, Eve G. Spratt, Beth Stein, S.H. Subramony, Rosa Ana Tang, Cara Tannenbaum, Hakan Tekeli, Amanda J. Thompson, Michael J. Thorpy, Matthew J. Thurtell, Pedro J. Torrico, Ira M. Turner, Scott Uretsky, Ruth H. Walker, Deborah M. Weisbrot, Michael A. Williams, Jacques Winter, Randall J. Wright, Jay Elliot Yasen, Shicong Ye, G. Bryan Young, Huiying Yu, Ryan J. Zehnder
- Edited by Alan B. Ettinger, Albert Einstein College of Medicine, New York, Deborah M. Weisbrot, State University of New York, Stony Brook
-
- Book:
- Neurologic Differential Diagnosis
- Published online:
- 05 June 2014
- Print publication:
- 17 April 2014, pp xi-xx
-
- Chapter
- Export citation