Hostname: page-component-669899f699-cf6xr Total loading time: 0 Render date: 2025-05-05T15:47:30.755Z Has data issue: false hasContentIssue false
  • English
  • Français

Automated summative feedback improves performance and retention in simulation training of mastoidectomy: a randomised controlled trial

Published online by Cambridge University Press:  28 October 2021

A Frithioff ,
M Frendø ,
J Hastrup von Buchwald ,
P Trier Mikkelsen ,
M Sølvsten Sørensen  and
S Arild Wuyts Andersen
A Frithioff*
Affiliation:
Department of Otorhinolaryngology – Head and Neck Surgery and Audiology, Rigshospitalet, Copenhagen, Denmark Copenhagen Academy for Medical Education and Simulation, The Capital Region of Denmark, Aarhus, Denmark
M Frendø
Affiliation:
Department of Otorhinolaryngology – Head and Neck Surgery and Audiology, Rigshospitalet, Copenhagen, Denmark Copenhagen Academy for Medical Education and Simulation, The Capital Region of Denmark, Aarhus, Denmark
J Hastrup von Buchwald
Affiliation:
Department of Otorhinolaryngology – Head and Neck Surgery and Audiology, Rigshospitalet, Copenhagen, Denmark Copenhagen Academy for Medical Education and Simulation, The Capital Region of Denmark, Aarhus, Denmark
P Trier Mikkelsen
Affiliation:
The Alexandra Institute, Aarhus, Denmark
M Sølvsten Sørensen
Affiliation:
Department of Otorhinolaryngology – Head and Neck Surgery and Audiology, Rigshospitalet, Copenhagen, Denmark
S Arild Wuyts Andersen
Affiliation:
Department of Otorhinolaryngology – Head and Neck Surgery and Audiology, Rigshospitalet, Copenhagen, Denmark Copenhagen Academy for Medical Education and Simulation, The Capital Region of Denmark, Aarhus, Denmark
*
Author for correspondence: Dr A Frithioff, Department of Otorhinolaryngology – Head and Neck Surgery, Rigshospitalet, Blegdamsvej 9, Copenhagen Ø DK-2100, Denmark E-mail: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

Objective

This study aimed to investigate the effects of automated metrics-based summative feedback on performance, retention and cognitive load in distributed virtual reality simulation training of mastoidectomy.

Method

Twenty-four medical students were randomised in two groups and performed 15 mastoidectomies on a distributed virtual reality simulator as practice. The intervention group received additional summative metrics-based feedback; the control group followed standard instructions. Two to three months after training, participants performed a retention test without learning supports.

Results

The intervention group had a better final-product score (mean difference = 1.0 points; p = 0.001) and metrics-based score (mean difference = 12.7; p < 0.001). At retention, the metrics-based score for the intervention group remained superior (mean difference = 6.9 per cent; p = 0.02). Also at the retention, cognitive load was higher in the intervention group (mean difference = 10.0 per cent; p < 0.001).

Conclusion

Summative metrics-based feedback improved performance and lead to a safer and faster performance compared with standard instructions and seems a valuable educational tool in the early acquisition of temporal bone skills.

Keywords

Temporal BoneOtologic Surgical Procedures
Type
Main Articles
Information
The Journal of Laryngology & Otology , Volume 136 , Issue 1 , January 2022 , pp. 29 - 36
Copyright
Copyright © The Author(s), 2021. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

Footnotes

Dr A Frithioff takes responsibility for the integrity of the content of the paper

References

Reznick, RK. Teaching and testing technical skills. Am J Surg 1993;165:358–61CrossRefGoogle ScholarPubMed
George, AP, De, R. Review of temporal bone dissection teaching: how it was, is and will be. J Laryngol Otol 2010;124:119–25CrossRefGoogle Scholar
Frithioff, A, Sørensen, MS, Andersen, SAW. European status on temporal bone training: a questionnaire study. Eur Arch Otorhinolaryngol 2018;275:357–63CrossRefGoogle ScholarPubMed
Zhao, YC, Kennedy, G, Yukawa, K, Pyman, B, O'Leary, S. Improving temporal bone dissection using self-directed virtual reality simulation: results of a randomized blinded control trial. Otolaryngol Head Neck Surg 2011;144:357–64CrossRefGoogle ScholarPubMed
Andersen, SAW, Foghsgaard, S, Konge, L, Cayé-Thomasen, P, Sørensen, MS. The effect of self-directed virtual reality simulation on dissection training performance in mastoidectomy. Laryngoscope 2016;126:1883–8CrossRefGoogle ScholarPubMed
Javia, L, Deutsch, ES. A systematic review of simulators in otolaryngology. Otolaryngol Head Neck Surg 2012;147:999–1011CrossRefGoogle ScholarPubMed
Brydges, R, Dubrowski, A, Regehr, G. A new concept of unsupervised learning: directed self-guided learning in the health professions. Acad Med 2010;85:S49–55CrossRefGoogle ScholarPubMed
Brydges, R, Nair, P, Ma, I, Shanks, D, Hatala, R. Directed self-regulated learning versus instructor-regulated learning in simulation training. Med Educ 2012;46:648–56CrossRefGoogle ScholarPubMed
Frendø, M, Thingaard, E, Konge, L, Sølvsten, M, Steven, S. Decentralized virtual reality mastoidectomy simulation training: a prospective, mixed-methods study. Eur Arch Otorhinolaryngol 2019;276:2783–9CrossRefGoogle ScholarPubMed
Cook, DA, Brydges, R, Zendejas, B, Hamstra, SJ, Hatala, R. Technology-enhanced simulation to assess health professionals: a systematic review of validity evidence. Acad Med 2013;88:872–83CrossRefGoogle ScholarPubMed
Issenberg, SB, Mcgaghie, WC, Petrusa, ER, Gordon, DL, Scalese, RJ. Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Med Teach 2005;27:10–28CrossRefGoogle ScholarPubMed
Wijewickrema, S, Zhou, Y, Ioannou, I, Copson, B, Piromchai, P et al. Presentation of automated procedural guidance in surgical simulation: results of two randomised controlled trials. J Laryngol Otol 2018;132:257–63CrossRefGoogle ScholarPubMed
Kerwin, T, Stredney, D, Wiet, GJ, Shen, HW. Virtual Mastoidectomy performance evaluation through multi-volume analysis. Computer Assist Radiol Surg 2013;8:51–61CrossRefGoogle ScholarPubMed
Andersen, SAW, Mikkelsen, PT, Sørensen, MS. Expert sampling of VR simulator metrics for automated assessment of mastoidectomy performance. Laryngoscope 2019;129:2170–7CrossRefGoogle ScholarPubMed
Zirkle, M, Roberson, DW, Leuwer, R, Dubrowski, A. Using a virtual reality temporal bone simulator to assess otolaryngology trainees. Laryngoscope 2007;117:258–63CrossRefGoogle ScholarPubMed
Andersen, SAW, Konge, L, Cayé-Thomasen, P, Sørensen, MS. Learning curves of virtual mastoidectomy in distributed and massed practice. JAMA Otolaryngol Head Neck Surg 2015;141:913–18Google ScholarPubMed
Andersen, SAW, Konge, L, Mikkelsen, PT. Mapping the plateau of novices in virtual reality simulation training of mastoidectomy. Laryngoscope 2017;127:907–14CrossRefGoogle ScholarPubMed
Park, J, Shea, CH, Wright, DL, Shea, CH, Reduced-frequency, DLW, Park, J et al. Reduced-frequency concurrent and terminal feedback: a test of the guidance hypothesis. J Mot Behav 2000;32:287–96CrossRefGoogle ScholarPubMed
Hatala, R, Cook, D, Zendajas, B, Hamstra, S, Brydges, R. Feedback for simulation-based procedural skills training: a meta-analysis and critical narrative synthesis. Adv Heal Sci Educ Theory Pract 2014;19:251–72CrossRefGoogle ScholarPubMed
Sweller, J. Cognitive load during problem solving: effects on learning. Cognitive Science 1988;12:257–85CrossRefGoogle Scholar
Haji, FA, Cheung, JJH, Woods, N, Regehr, G, de Ribaupierre, S, Dubrowski, A. Thrive or overload? The effect of task complexity on novices’ simulation-based learning. Med Educ 2016;50:955–68CrossRefGoogle ScholarPubMed
Frithioff, A, Frendø, M, Mikkelsen, PT, Sørensen, MS, Andersen, SAW. Ultra-high-fidelity virtual reality mastoidectomy simulation training: a randomized, controlled trial. Eur Arch Otorhinolaryngol 2020;277:1335–41CrossRefGoogle ScholarPubMed
Van Merriënboer, JJG, Sweller, J. Cognitive load theory in health professional education: design principles and strategies. Med Educ 2010;44:85–93CrossRefGoogle ScholarPubMed
Sethia, R, Kerwin, TF, Wiet, GJ. Performance assessment for mastoidectomy. Otolaryngol Head Neck Surg. 2017;156:61–69CrossRefGoogle ScholarPubMed
Al-Shahrestani, F, Sørensen, MS, Andersen, SAW. Performance metrics in mastoidectomy training: a systematic review. Eur Arch Otorhinolaryngol 2019;276:657–64CrossRefGoogle ScholarPubMed
Vilmann, AS, Norsk, D, Bo, M, Svendsen, S. Computerized feedback during colonoscopy training leads to improved performance: a randomized trial. Gastrointest Endosc 2018;88:869–76CrossRefGoogle ScholarPubMed
Ahmed, OMA, Niessen, T, Gallagher, AG, Breslin, DS, Dunngalvin, A, Shorten, GD. The effect of metrics-based feedback on acquisition of sonographic skills relevant to performance of ultrasound-guided axillary brachial plexus block. Anaesthesia 2017;9:1117–24CrossRefGoogle Scholar
Sørensen, MS, Mosegaard, J, Trier, P. The visible ear simulator: a public PC application for GPU-accelerated haptic 3D simulation of ear surgery based on the visible ear data. Otol Neurotol 2009;30:484–7CrossRefGoogle ScholarPubMed
Andersen, SAW, Cayé-Thomasen, P, Sørensen, MS. Mastoidectomy performance assessment of virtual simulation training using final-product analysis. Laryngoscope 2015;125:431–5CrossRefGoogle ScholarPubMed
Naismith, LM, Cavalcanti, RB. Validity of cognitive load measures in simulation-based training: a systematic review. Acad Med 2015;90:24–35CrossRefGoogle ScholarPubMed
Andersen, SAW, Mikkelsen, PT, Konge, L, Cayé-Thomasen, P, Sørensen, MS. Cognitive load in mastoidectomy skills training: virtual reality simulation and traditional dissection compared. J Surg Educ 2016;73:45–50CrossRefGoogle ScholarPubMed
Leppink, J. Data analysis in medical education research: a multilevel perspective. Perspect Med Educ 2015;4:14–24CrossRefGoogle ScholarPubMed
Zirkle, M, Taplin, MA, Anthony, R, Dubrowski, A. Objective assessment of temporal bone drilling skills. Ann Otol Rhinol Laryngol 2007;116:793–8CrossRefGoogle ScholarPubMed
Davaris, M, Wijewickrema, S, Zhou, Y, Piromchai, P, Bailey, J, Kennedy, G et al. The importance of automated real-time performance feedback in virtual reality temporal bone surgery training. In: Isotani, S, Millán, E, Ogan, A, Hastings, P, McLaren, B, Luckin, R, eds. Artificial Intelligence in Education. Cham: Springer International Publishing, 2019;96–109CrossRefGoogle Scholar
Wijewickrema, S, Piromchai, P, Zhou, Y, Ioannou, I, Bailey, J, Kennedy, G et al. Developing effective automated feedback in temporal bone surgery simulation. Otolaryngol Head Neck Surg 2015;152:1082–8CrossRefGoogle ScholarPubMed
Andersen, SAW, Mikkelsen, PT, Sørensen, MS. The Effect of simulator-integrated tutoring for guidance in virtual reality simulation training. Simul Healthc 2020;15:147–53CrossRefGoogle ScholarPubMed
Andersen, SAW, Frendø, M, Guldager, M, Sørensen, MS. Understanding the effects of structured self-assessment in directed, self-regulated simulation-based training of mastoidectomy: a mixed methods study. J Otol 2020;15:117–23CrossRefGoogle ScholarPubMed
Andersen, SAW, Frendø, M, Sørensen, MS. Effects on cognitive load of tutoring in virtual reality simulation training. MedEdPublish 2020;9:1–6CrossRefGoogle Scholar
Van Merriënboer, JJG, Kester, L, Pass, F. Teaching complex rather than simple tasks: balancing intrinsic and germane load to enhance transfer of learning. Appl Cogn Psychol 2006;20:343–52CrossRefGoogle Scholar
Gawecki, W, Wegrzyniak, M, Mickiewicz, P, Talar, M, Wierzbicka, M, Gawłowska, MB. The impact of virtual reality training on the quality of real antromastoidectomy performance. J Clin Med. 2020;9:3197CrossRefGoogle ScholarPubMed
Andersen, SAW, Foghsgaard, S, Cayé-Thomasen, P, Sørensen, MS. The effect of a distributed virtual reality simulation training program on dissection mastoidectomy performance. Otol Neurotol 2018;39:1277–84CrossRefGoogle ScholarPubMed
Smith, S, Lonie, J. Mastery learning: how is it helpful? An analytical review. Adv Med Educ Pract 2017;8:269–75Google Scholar