Delayed or missed diagnosis of critical congenital CHD can result in significant morbidity and mortality. Reference Menahem, Sehgal and Meagher1–Reference Bakker, Bergman and Krikov3 Early identification of congenital CHD remains a challenge despite advances in fetal imaging and widely implemented pulse oximetry screening protocols. While prenatal detection through imaging is ideal, this is not always possible for a variety of reasons including inequitable access to care. While congenital CHD pulse oximetry screening can be used to detect hypoxia or saturation differentials in an infant with underling pathology – this does not negate the need for clinicians to consider the diagnosis of congenital CHD in an infant who may not yet be in extremis. Thus, the development of physician clinical assessment skills and inclusion of congenital CHD on the differential as a cause of clinical change remains critical.
Current exposure to training on congenital CHD is limited. This is especially true for those clinicians transitioning from residency training to work in newborn nurseries, emergency departments, and hospitals – settings in which they will likely be the frontline provider that could make the difference in early recognition of undiagnosed congenital CHD. Reference Su and Munoz5 While previous education interventions have focused on enhancing specific resident skills in reading electrocardiograms and learning cardiac auscultation, Reference Bergero, Saavedra, Guglielmino, Soto Perez, Llera and Busaniche6–Reference Mattioli, Belmont and Davis11 these skills do not address the fundamental gap in considering congenital CHD as an aetiology for decompensation followed by enacting initial diagnostic and management steps. In addition to the lack of directed training while on cardiology rotations, paediatric residents have very few opportunities to learn these skills on other rotations.
Simulation is an effective strategy for teaching clinicians, including paediatric residents, resuscitation in a safe, and standardised environment. Reference Lopreiato and Sawyer12–Reference Okuda, Bryson and DeMaria14 High-fidelity simulation has demonstrated feasibility and effectiveness in improving paediatric resident knowledge, comfort, and confidence when caring for patients with cardiac disease. Reference Harris, Adler, Unti and McBride15–Reference Subat, Goldberg, Demaria and Katz17 Prior studies have mainly used traditional debrief simulation to teach residents recognition and management strategies. Reference Harris, Adler, Unti and McBride15,Reference Mohan, Follansbee, Nwankwo, Hofkosh, Sherman and Hamilton16 Rapid cycle deliberate practice is a simulation method in which feedback is given in real time to allow for participants to stop and immediately correct their mistakes. Rapid cycle deliberate practice has been used to reinforce correct behaviour in paediatric emergencies and resuscitation, though has not been applied to training on congenital CHD. Reference Taras and Everett18–Reference Lemke20 In this study, we aimed to assess the impact of a rapid cycle deliberate practice simulation curriculum at enhancing paediatric resident recognition of congenital CHD and competency at the initiation of early diagnostic and therapeutic interventions.
Methods
We performed a single-centre prospective pilot study at a large, urban, quaternary care paediatric hospital. Approximately, 52 categorical and combined paediatric interns complete a 4-week, mandatory paediatric cardiology rotation, and split their time between the outpatient and inpatient settings. The outpatient time consists of paediatric cardiology consults and clinic. During the inpatient portion, the interns are front-line providers alongside advanced practice providers on a 29-bed acute care cardiology unit, supervised by a cardiology fellow and attending.
Prior to our work, the curriculum for first year paediatric residents on the cardiology rotation consisted of a series of lectures that rotated monthly with no specific focus on the recognition and early management of congenital CHD. Didactics did cover the main aetiologies of congenital CHD, including one lecture on single ventricle physiology and one on acyanotic heart lesions that included an overview of coarctation of the aorta.
Curriculum development
Using Kern’s six-step approach to curriculum development, we conducted an informal targeted needs assessment which revealed an opportunity to use simulation-based education to fill an experiential learning gap. Reference Thomas, Thomas, Kern, Hughes, Tackett and Chen21 We anchored our curriculum within the conceptual framework of Ericsson’s deliberate practice – the theory that learning occurs through an intentional experience partnered with expert-level feedback on performance, followed by the opportunity to apply what was learned. Reference Ericsson22,Reference Zackoff, Real, Abramson, Li, Klein and Gusic23 Rapid cycle deliberate practice is a simulation-based modality that aligns with this conceptual framework – consisting of learners performing a task in a simulated environment, being stopped to correct mistakes, followed by immediately allowing for practice of those learned skills or behaviours. Reference Ng, Primiani and Orchanian-Cheff24
We designed the curriculum to span 4 weeks, aligned with the paediatric cardiology rotation (Table 1). The paediatric interns were all participants in the curriculum but had the option of opting out by not answering the assessments or surveys. During weeks 2, 3, and 4 interns participated in one-hour sessions during which 2 simulations occurred: an assessment simulation followed by a training simulation. During the assessment simulation, interns were given a case prompt and then were allowed to navigate the simulation uninterrupted. Following the conclusion of the simulation, a structured debrief occurred through which the interns were provided feedback and core areas for improvement were identified. During the training simulation, rapid cycle deliberate practice was used to identify and mitigate gaps in knowledge, skills, or behaviour in real time. The week two assessment simulation was a case of neonatal sepsis, while the training simulation was a case of critical coarctation. This was done to demonstrate the similarity in presentation between sepsis and congenital CHD. The week three simulations were both cases of critical pulmonary stenosis while the week four simulations were both cases of hypoplastic left heart syndrome. During the assessment simulation, a checklist of observed behaviours by the intern team was completed in real time by one of the trained facilitators. At the end of week 4, interns repeated the online case-based assessment as well as a retrospective pre-post survey assessing perceived change in knowledge.
Table 1. PRICE curriculum assessments.
Survey development and outcome measures
We developed all survey instruments de novo by a multidisciplinary team of paediatric cardiologists, medical educators, and simulation experts. Our primary outcome was measured skills observed via the simulation checklists. Our secondary outcomes were perceived skill ascertainment and measured knowledge via the retrospective pre-post survey and online case scenario assessments, respectively.
Online case scenario assessment
During week 1, the interns took an online case-based assessment that consisted clinical scenarios followed by free response and multiple-choice questions (MCQs) (Supplementary figure 1). We scored the free-response questions using a checklist of expected statements adapted from the simulation checklists (Supplementary figure 3). The same case-based assessment was repeated during week 4 following completion of all simulation sessions. Participants completed all assessments independently through an online REDCap survey which ensured de-identified data, but linked data across assessments and assured secure storage. Reference Harris, Taylor, Thielke, Payne, Gonzalez and Conde25 Two members of the study team reviewed a total of 10 cases (5 pairs of week 1 and week 4 assessments) using the developed rubric and were found to have an interrater reliability across all items of 0.8. All remaining assessments were reviewed by a single study team member.
Simulation checklist
We developed a checklist of expected observable behaviours for completion by a facilitator during the assessment simulations in weeks 2, 3, and 4. We placed emphasis on actions that were deemed “cannot miss” by our multidisciplinary investigator team (Supplementary figure 3). Items ranged from recognition of distress, to asking for help, to obtaining 4-extremity blood pressures, and checking pre and post-ductal pulse oximetry. All checklists were completed directly within a secure REDCap database.
Retrospective pre/post survey
We developed a retrospective pre-post survey to characterise intern perceptions of the change in their ability to recognise and treat congenital CHD post-curriculum compared to pre-curriculum. We used this question design to minimise the impact of changes in intern self-assessment standards over time as they experience a shift in their frame of reference following an intervention. Reference Howard26 Paediatric interns completed the survey in week 4 following completion of all simulations, with responses stored within a secure REDCap database.
Data analysis
We used summary statistics including medians with interquartile range and frequencies with percentages to describe participant demographic characteristics and pre- and post-intervention case scenario assessment scores. We analysed changes of case scenario assessment scores from week 1 to week 4 using a paired samples t-test. Furthermore, we analysed differences in case scenario assessment scores between participant groups and simulation checklist scores at weeks 2, 3, and 4 using mixed effect linear regression with random intercept to account for within-subject correlation. P-values < 0.05 were considered statistically significant. We utilised SAS 9.4 for all statistical analysis.
This study was approved by the CCHMC Institutional Review Board.
Results
A total of 26 interns (100% of those eligible) participated in the curriculum between August and January 2022. The majority of participants were women (75%), with 22% having previous exposure to a paediatric cardiology rotation as medical students and 9% expressing interest in specialising in paediatric cardiology (Table 2).
Table 2. Participant demographics and background.
Online case scenario assessment
Twenty-one participants (84%) completed both the pre and post case scenario assessments; the remaining five participants were excluded due to incomplete assessments. Overall, interns’ individual median scores for the online case scenario assessment increased from week 1 to week 4 (4/18, interquartile range 3–6 versus 8/18, interquartile range 6–10, p-value < 0.0001) (Fig. 1a). There was no difference in group performance in the case scenario assessment based on the time at which they completed their paediatric cardiology rotation relative to other intern year clinical rotations and education (Fig. 1a).
Figure 1. a, b: Panel a shows box and whisker plot depicting aggregate median online case scenario assessment scores from week 1 to week 4 and then group scores for week 1 and week 4. Panel B shows retrospective pre-post aggregate survey scores between week 1 and.
Simulation checklist assessment
A total of 6 groups completed all three simulation sessions. Group A was not included due to not participating in all three simulation sessions. There was a statistical increase in the percentage of observed completed tasks from weeks 2–3 (73% versus 83%, p-value 0.0263), as well as from weeks 2 to 4 (73% versus 92%, p-value of 0.0025). There was no statistical difference between weeks 3 and 4 performance (p-value 0.193) (Fig. 2a). There was no statistical difference observed between groups (Fig. 2b).
Figure 2. A shows a box and whisker plots for simulation checklist scores by percentage complete from week 2, 3 and 4, difference between week 2 and 3 p-value 0.0115, difference between week 2 and 4 p-value 0.0006. B shows group scores for groups B through G for weeks 2, 3, and 4. This shows each intern groups’ progress over the course of the curriculum highlighting improvement in all groups.
Discussion
We conducted an education pilot study that assessed the impact of a rapid cycle deliberate practice-based simulation curriculum on paediatric interns’ knowledge, observed behaviours, and self-assessed knowledge related to recognition and treatment of paediatric clinical decompensation from undiagnosed congenital CHD. Our intervention resulted in a statistically significant increase in case scenario scores, number of behaviours performed during assessment simulations, and self-assessed knowledge. Our study adds to the literature by demonstrating the utility and impact of simulation on paediatric residents’ clinical knowledge and skill sets when evaluating a child with CHD, while specifically employing rapid cycle deliberate practice as the main intervention. Reference Harris, Adler, Unti and McBride15–Reference Subat, Goldberg, Demaria and Katz17,Reference Ericsson22
Previous studies have shown that rapid cycle deliberate practice is more effective than traditional debrief simulation in improving paediatric resuscitation skills as well as paediatric critical care skills. Reference Taras and Everett18,Reference Won, Doughty and Young19,Reference Ng, Primiani and Orchanian-Cheff24 Rapid cycle deliberate practice is intrinsically linked to the conceptual framework of deliberate practice and our study showed that with consistent practice, paediatric interns can improve their identification and treatment of a child with congenital CHD. By conducting the study in our unit, interns were able to attend without significant interruption to their clinical responsibilities and practice in a familiar setting. Using rapid cycle deliberate practice allowed for immediate correction such that interns could focus on the simulation, how to intervene, and what actions corresponded to a clinical improvement. Over the course of the month, the paediatric interns were able to apply skills they learned in each session to subsequent simulations demonstrating statistical improvement by week 3 of the rotation, without additional improvement between weeks 3 and 4. Our pilot study may have been underpowered to observe a smaller difference in the final simulation; alternatively, 3 simulations may be sufficient to achieve our objectives. Eliminating one simulation session would reduce the needed resources and infrastructure, which may promote sustainability. Our findings are consistent with a prior paediatric resuscitation study, which found a significant improvement in resident performance of key resuscitation skills using rapid cycle deliberate practice in comparison to standard simulation debrief models. Reference Hunt, Duval-Arnould and Nelson-McMillan27 These studies suggest that rapid cycle deliberate practice could have broader utility in other subspecialty settings where residents have limited exposure, yet early recognition and initiation of therapy are critical for positive patient outcomes.
In addition to rapidly teaching resident-specific skills, interns showed improvement in overall knowledge and self-assessed knowledge after participating in the simulations. We noted no difference in week 1 and week 4 case scenario assessments between the groups, despite later groups having been exposed to 5 additional months of clinical training during their first year of residency. The group comprised of the least experienced interns, group A, and the most experienced interns, group G, scored similarly suggesting that despite other skills attained throughout intern year, the identification and treatment of patients with congenital CHD required deliberate practice and targeted education. Residents are less likely to attain these skills via incidental encounters or informal and unintended contextual learning (situated learning) alone. Reference Zackoff, Real, Abramson, Li, Klein and Gusic23,Reference Lave and Wenger28 Interestingly, knowledge increase was not associated with interns’ interest in paediatric cardiology. Since postnatal congenital CHD will not often be diagnosed by paediatric cardiologists, but rather by newborn nursery clinicians, emergency medicine physicians, and primary care physicians, we believe that training all paediatric residents is important to prepare them for future practice. These front-line clinicians must have a practical learning environment to recognise congenital CHD and learn initial treatment.
Limitations
This study has several limitations. First, our study was conducted at a single paediatric institution, so may not be generalisable to other training programmes. However, our paediatric interns’ performance likely reflects that of other large academic medical centres. Second, our institution benefits from a well-resourced simulation team. Our curriculum, which incorporated several facilitated computerised manikin-based simulations during each rotation block within a clinical unit, may not be reproducible at other institutions. Third, as a pilot study, our sample size was small which may have impacted our ability to detect significant differences between simulations in weeks 3 and 4. Fourth, although this pilot was integrated into the interns’ standard curriculum, they were allowed to opt out of completing the online pre and post case-based assessments and the retrospective pre-post survey, which could bias the results, despite our high participation rate. Fifth, we did not include a control group to evaluate if the changes seen in overall performance came solely from simulation curriculum that included rapid cycle deliberate practice or the standard didactic curriculum and bedside teaching interns received during the rotation. Sixth, all our assessments and surveys were developed de novo, as there were no preexisting validated tools to use for this curriculum. Finally, we did not assess knowledge retention and application in actual patient care, which could be important areas for future work.
Conclusion
A pilot rapid cycle deliberate practice simulation curriculum focused on recognition and management of congenital CHD can improve paediatric interns’ knowledge and skills observed in simulation encounters. The pilot was also feasible in our busy academic paediatric residency training programme. Future plans include evaluating knowledge retention, application to patient care, and spreading the curriculum to other institutions to assess the generalisability of our approach and findings. We also plan to consider how this approach may be utilised to improve recognition of clinical deterioration and initiation of interventions in children with other low incidence, high-risk conditions.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S104795112400074X.
Acknowledgements
This work was conducted with support from the Heart Institute Research Core at Cincinnati Children’s Hospital.
Financial support
None.
Competing interests
None.
