BACKGROUND
Point-of-care ultrasound (PoCUS) is an important adjunct in the approach to evaluating critically ill patients presenting to the emergency department. 4 - Reference Henneberry, Hanson and Healey 6 The utility of simulation for the development and competence of ultrasound skills has recently gained attention.Reference Sidhu, Olubaniyi and Bhatnagar 7 , Reference Mendiratta-Lala, Williams and de Quadros 8 One strategy to develop skills in resuscitative PoCUS is by integrating ultrasound simulation into high-fidelity simulation (HFS) programs. There are a variety of methods of incorporating ultrasound into simulation, ranging from simple verbal descriptions of ultrasound findings to HFS devices. These devices require image generation skills and interpretation in real time, and are thus more realistic.
SUMMARY
Olszynski et al. performed a randomized crossover study to compare two PoCUS simulators during critical care simulation sessions for EM trainees.Reference Olszynski, Harris and Renihan 3 The edus2 simulator uses a modified ultrasound probe and laptop to display pre-recorded video clips when the user passes the probe over a radio frequency identification device card placed beneath the HFS mannequin’s skin. This device was compared with a laptop placed on a cart, on which the trainee could select and view a labeled video clip from within a case folder. Instructions for building this device are available at http://edus2.com/.
Twenty-five EM trainees were randomized to perform two simulation scenarios using one of the PoCUS simulation devices, followed by a crossover to using the other device for additional two simulation scenarios.
The primary study outcome was trainee and instructor preference for either device. Trainees rated how well the simulation devices addressed the various aspects of resuscitative PoCUS immediately following the session. A qualitative survey weeks later covered which device was preferred.
QUALITY ASSESSMENT
The SGEM HOP team used the Best Evidence in Emergency Medicine (BEEM) tool for appraisal of a randomized controlled trialReference Worster 9 , Reference Worster 10 (RCT) as a reference in our critical appraisal of Olszynski’s trial.
There were several strengths of this study evident from the BEEM RCT appraisal tool. It focused on a relevant population: EM trainees who would normally be performing resuscitative PoCUS in the ED. Participants were recruited consecutively, and participants in both groups were similar, minimizing selection bias. Participants were analysed in the groups to which they were randomized, and were treated equally. Follow-up was complete for both groups, and all important outcomes were considered. Whether the treatment effect was large and precise enough to be significant is debatable.
The study was limited in that randomization was not ideal, as participants were placed into groups based upon the order of arrival to the simulation lab, rather than being randomly allocated. Furthermore, the randomization process was not concealed, and participants were aware of group allocation. In addition, the lead author was involved in the development of the edus2 simulator and explained both simulators to the trial participants, which may have influenced the participants’ preferences.
KEY RESULTS
Both PoCUS simulation devices were found to be an improvement from previous HFS experience in terms of developing resuscitative PoCUS skills. Based on the trainees’ quantitative ratings of how well each device addressed various aspects of resuscitative PoCUS, there was no statistically significant difference between the edus2 and laptop interventions. However, qualitative preference for the edus2 device was unanimous among the 20 trainees who responded to a survey a few weeks after the intervention.
TAKE-TO-WORK POINTS
The integration of a PoCUS simulation device into HFS programs is of value in developing resuscitative PoCUS skills. In this study, trainees unanimously preferred the edus2 simulator device to a bedside laptop.
METHODOLOGY OF THE ANALYSIS OF THE SOCIAL MEDIA RESPONSE
The social media discussion started with the launch of the SGEM blog post and podcastReference Milne 2 on May 8, 2016, and continued for 1 week through May 15, 2016. An invitation to comment on the article was included in the audio of the podcast, the text of the blog post, and on social media communications (Twitter and Facebook). Social media responses written in the SGEM blog’s comment section, the SGEM Facebook page, and on Twitter (directed at @TheSGEM, @socmobem, @OlszynskiP, or using the #SGEMHOP hashtag) were reviewed by the authorship team. A thematic analysis of social media commentary was conducted. PM and CB compiled and reviewed all social media commentary and identified common themes. All authors then participated in a consensus-based analysis to decide which comments represented the general themes of the discussion.
Multiple metrics of dissemination were tracked by the SGEM HOP team:
-
∙ Blog page views were monitored using the Jet Pack plugin by Wordpress.com. 11
-
∙ Facebook “reach” analytics were provided by Facebook and correlate to the number of users who saw the original SGEM Facebook post on their own newsfeed. 12
-
∙ Twitter impressions (the number of users whose newsfeeds contained a tweet featuring the #SGEMHOP hashtag) were tracked using Symplur, a software program that tracks health–care-related twitter conversations.Reference Symplur 13 Tweets not containing the hashtag were not tracked by Symplur. The number of impressions is calculated by taking the number of tweets per Twitter user using the #SGEMHOP hashtag and multiplying them by the number of followers that that participant has.
-
∙ Altmetric scores track the disseminative impact of research articles online by tracking mentions on social media (e.g., Facebook, Twitter) and mentions on blogs, podcasts, and news outlets. 14 Altmetrics of Olszynski et al.’s article were surveyed and compared to articles from the same issue of CJEM, all other articles published in CJEM, research articles from all other outlets, and the articles covered in the first two CJEM-SGEM HOPs.Reference Luckett-Gatopoulos 15 , Reference Purdy, Thoma and Milne 16
RESULTS OF THE SOCIAL MEDIA RESPONSE
Please refer to Table 1 for details of the dissemination and social media reach of this article during the SGEM HOP campaign. Overall, the social media dissemination was broad. During the 1-week period following the blog post, the #SGEMHOP hashtag was used in 116 tweets by 51 unique users. These tweets made 252,172 impressions (the number of tweets per participant multiplied by that participant’s number of followers). A Facebook post with a link to the SGEM blog post reached 1182 newsfeeds. The Altmetric score for Olszynski et al.’s article reached 58, the fifth highest score for articles published in CJEM, placing it within the top 5th percentile of Altmetric scores for all published research (Figure 1).
Figure 1 Screen capture of the altmetrics data retrieved from https://cambridgejournals.altmetric.com/details/4444766 on May 15, 2016.
Table 1 Aggregate analytic data on the online discussion on various social media platforms for the 7 days following SGEM blog posting
ONLINE DISCUSSION SUMMARY
The comments posted directly on the SGEM’s blog provided a detailed, in depth conversation. The high quality of the discussion was noted by the editors of Life in the Fast Lane (http://lifeinthefastlane.com), an EM education portal with more than 2.5 million monthly page views.Reference Lazoff 17 , 18 The website featured the SGEM blog posting in their weekly “LITFL Review,” describing the blog post as being “worth a visit just to read the comment thread.” The comment thread on the SGEM’s blog post provided a unique opportunity for engaged readers and listeners to interact directly with the study’s primary author in a much more responsive manner than the traditional “Letters to the Editor” section of a journal. There were 15 comments on the blog by 9 unique users, with all questions having a response from the primary author, Dr. Olszynski.
Prominent themes identified in the social media discussion included personal experiences with ultrasound simulators, future research directions, and strategies for incorporating ultrasound into existing simulation programs.
Personal experience with ultrasound simulators was highlighted by Rob Woods, the program director of the University of Saskatchewan’s FRCPC Emergency Medicine program. “EDUS2 over a laptop maintains the time sequence reality of image generation. A laptop also takes you away from the patient […], whereas the EDUS2 has you remain in intimate contact with the patient, maintaining fidelity.”
A discussion about future research directions for ultrasound simulators was the second identified theme. Chris Hicks asked, “Now that you know what’s preferred, could you examine what is most effective?” Olszynski replied that work examining this question is already being undertaken, and posted two recently presented abstracts about the clinical integration of ultrasound skills.
Finally, a third common theme in the social media reaction to the SGEM’s podcast was the implementation of ultrasound into existing simulation programs. Both Will Sanderson and Teresa Chan felt that a lesson plan would be beneficial to help guide educators on how to introduce ultrasound simulators into their simulation programs. Olszynski noted that this would be forthcoming.
LIMITATIONS
Although Twitter disseminated Olszynski et al.’s publication widely via the #SGEMHOP hashtag, the Twitter discussion was largely limited to the promotion of the SGEM’s podcast and blog posting. Of the 116 tweets containing the #SGEMHOP hashtag, the vast majority directed the reader to the SGEM blog posting and comments thread and did not provide any commentary on the paper itself. 14 Likewise, posts on the SGEM’s Facebook page did not spark meaningful discussion.
The discussion on the blog may have been limited by sampling bias, with the commenting participants likely to be more representative of a small number of emergency physicians who are engaged on social media and may not be representative of the larger population.
The cited social media analytic scores suffer from limitations as well. The number of twitter impressions merely describes potential views of a hashtag rather than counting actual views. Altmetric scores likewise display activity related to the dissemination of a research article but do not indicate how many times the article itself is read. In a detailed review of Altmetric scores, Trueger et al. highlight that “dissemination scores do not replace the need for readers’ independent ability to critique the primary literature.”Reference Trueger, Thoma and Hsu 19
The social media dissemination of the paper, although impressive, may have been biased and inflated in that the authors of the paper and podcast are quite active on social media and involved in social media broadcasting.
CONCLUSION
The third collaborative knowledge translation effort between CJEM and the SGEM was again a successful endeavor. The original paper suggested that both simulators improve ultrasound education compared to previous experiences, with trainees qualitatively preferring the edus2 simulator over a simpler laptop simulator. The social media distribution of the featured article was widespread, and the article became the fifth highest ranking CJEM article in terms of disseminative impact. Commentary on the SGEM’s post centred on future research directions and strategies for incorporating ultrasound into established simulation programs. The high quality discussion with the author on the SGEM’s blog post was noted by a popular online EM education website with extensive readership, allowing further dissemination of Olszynski et al.’s CJEM article. Overall, the original paper, its critical appraisal in podcast format, and the social media discussion may allow educators and simulation program directors to appropriately select a tool to improve ultrasound education for EM residents.
Competing interests: None declared.
