Introduction
In setting goals for 2020, the Institute of Medicine recognized gaps in translational science related to expanding the clinical research workforce and more comprehensive training approaches [Reference Bonham, Califf, Gallin and Lauer1]. Despite funding efforts, many of those gaps persist resulting in new goals that focus attention on infrastructure and clinical research professional (CRP) workforce development [2]. Unique challenges facing academic medical center (AMC) CRPs include role progression, insufficient salaries, and burdened workloads [Reference Stroo, Asfaw and Deeter3]. Furthermore, there is significant turnover among CRPs in AMCs, which can negatively affect study quality performance, recruitment metrics, study timelines, and costs of training new staff [Reference Stroo, Asfaw and Deeter3]. Moreover, most professional development activities are geared toward new staff training and basic Good Clinical Practice (GCP) education; however, the professional development of more experienced CRP personnel is lacking. Those CRPs with extensive knowledge are a brain trust that are valuable resources for organizational improvement. Learning opportunities geared toward creative problem-solving, leadership development, collaborative team science, and project management would be a desirable higher tiered approach to satisfying the professional learning needs of this group of CRPs.
CRP training has traditionally been “seat of the pants” in nature, or attendance at sessions using passive learning pedagogy [Reference Carter, Jester and Jones4–Reference Speicher, Fromwell and Avery6]. However, opportunities for stimulating professional development of more experienced CRP are generally lacking. Issues of CRP burnout [Reference Gwede, Johnson, Roberts and Cantor7–Reference Spilsbury, Petherick, Cullum, Nelson, Nixon and Mason9] can be mitigated through professional development. This will help to improve job satisfaction, leadership growth, and role progression. Interest in exploring the unique issues of AMC CRP workforce, including issues in training, role progression, and staff turnover, has been discussed widely [Reference Brouwer, Deeter and Hannah10–Reference Fordyce, Roe and Pierre12]. Therefore, we sought to apply an innovative collaborative adult learning approach through the “Un-meeting” concept entitled “Collaborative Conversations: The Critical Need for Professional Workforce Development at Academic Medical Centers.” We targeted experienced CRPs and research administrators located at Clinical and Translational Science (CTSA) program hubs, their partners, and Institutional Development Award Networks for Clinical Translational Research (IDeA-CTR) institutions. This population has a unique perspective on the workforce development needs for CRPs. We originally planned an in-person Un-meeting conference workshop one day prior to the 2020 Association of Clinical Research Professional Global Meeting in Seattle, Washington. However, due to COVID-19 social distancing requirements, we pivoted the “Face-to-Face Un-meeting” format as a series of six consecutive monthly Zoom meetings. Using Zoom as a vehicle for conferences and Un-meetings became a new concept evolving from meeting constraints during the pandemic. Here, we describe our experience.
An “Un-meeting,” sometimes called an “unconference” is a participant-driven collaborative learning experience [Reference Budd, Dinkel and Corpas13]. The Center for Leading Innovation and Collaborations (CLIC), the coordinating center for the National Center for Advancing Translational Sciences (NCATS), located at University of Rochester, has hosted several “Un-meetings” since 2018 to address key translational research topics (Opioid Crisis and Artificial Intelligence). The University of Florida also hosted an Un-meeting addressing rural health equity. Those meetings primarily targeted attendees who were translational scientists, scholars, and trainees. The standard format for such meetings include an initial series of (usually four) lightening presentations followed by a large open-venue brainstorming session, whereby participants jot ideas and concepts on post-it notes in an open common meeting area. Based on the brainstorming, breakout rooms are then quickly organized for smaller topic-oriented group discussions. The outputs from these meetings have included forming collaborations, research proposals, publications, all toward solution finding, and innovations on the key meeting topic.
We structured our Un-meeting to focus on key topics related to AMC CRP workforce issues, namely job titles and job descriptions, onboarding education, continuing education, attrition and retention, and untapped potential workforce. We further aimed to provide a professional development opportunity for experienced CRPs. The social phenomenological relation between one person’s sharing experience and another person’s sharing experience taps into a field of “inter-experience” leading to shared experiences [Reference Neubauer, Witkop and Varpio14]. Moreover, this inter-experience enables participants to meet others working in other CTSA hubs, build networks, and form meaningful collaborations. It further offers positive leadership and mentoring opportunities, contributing to increased role satisfaction.
Needs Assessment
We were motivated from data from an earlier CTSA hub landscape analyses of CRPl training at CTSA Program hubs, performed in 2019 to explore CRP issues more deeply [Reference Kolb, Jones, Shah and Kremidas15]. To confirm the educational need, we conducted a preconference needs assessment through the registration process to measure the extent of issues for CRP workforce development. The survey was categorized as exempt from review by the university institutional review board. Registrants rated challenges for AMC CRP workforce issues related to (a) inadequate salaries and job titles, (b) support for onboarding, continuing education, and (c) opportunities for career advancement and staff turnover (Fig. 1). Participants additionally ranked perceived reasons for each of these challenges to be primarily due to lack of organizational importance, financial constraints, and resource bandwidth [Reference Kolb, Jones, Shah and Kremidas15].
Fig. 1. Challenges of academic medical center clinical research professional workforce stability.
Only 40 of 184 registrants responded to the initial registration needs assessment survey. Since anonymous, we could not correlate the actual number of attendees that completed the needs assessment. The majority of survey respondents were White females. We recognized that there were some technical issues with accessing the survey link during the registration process, which may have contributed to a low response. However, we recognized the value of the content provided from the needs assessment as we finalized planning and launch of the Un-meeting. Additionally, we collected qualitative open-text responses from survey participants outlining their goals and expectations of the meeting and those comments seemed to reach saturation early on. Stated goals from open-ended comments identified “opportunities for networking and collaboration.” Participants wanted to learn new and different approaches, increase knowledge, build awareness, identify projects, and explore implementation strategies. Finally, participants identified several “hopes” for long-term outcomes of the meeting to include creating: 1) new institutional priorities and leadership support; 2) a plan to increase and diversify staff and researchers; 3) plans to increase salaries and funding; 4) competency-based career ladders; 5) standardized competency-based educational programs; and 6) process improvements and solutions. One respondent had hopes that a long-range outcome would be to strengthen research compliance and performance.
Adult Collaborative Participatory Learning
Learning theories centered on best practices for adult learning emphasize approaches that are collaborative and participatory [Reference Brookfield16]. Those adult learning approaches can level the playing field between facilitators and learners. Furthermore, community creation provides an ideal atmosphere for adult learning knowledge creation and common inquiry [Reference Sheridan17]. As such, learning is social, where learners can exchange ideas, share experiences, in an open environment. It allows learners exposure to external ideas, thus expanding their perspectives. Shorter presentations in the large group followed by opportunities to exchange ideas in small-group breakout sessions can enhance participatory learning. Planning participatory learning sessions should include clearly communicating objectives, preparing content materials, breakout questions, and instructions so that everyone can understand session aims [Reference Sheridan17,Reference MacGregor18]. Unlike “seat of the pants” or passive learning modalities, collaborative learning best serves the adult learner seeking to grow in their role.
The CLIC has promoted “Un-meetings” as a method of providing interactive means for adult professionals to “cultivate ideas and make productive connections,” to stimulate collaborations across disciplines and CTSA Program hubs [19]. This cultivation, connection, and collaboration leads to a communal discovery of new knowledge, and deeper learning [Reference Laal and Laal20–Reference Oblinger and Oblinger22]. Deeper learning is a means by which each individual transfers experience and knowledge gained in one circumstance and applies it to new situations and to others in practice or research settings.
Target Learners
Our target learners were experienced CRPs, trainers, and administrators of academic clinical research institutions, primarily the CTSA Program hubs, and their partners or affiliated IDeA CTRs. Experienced CRPs are a brain trust of CTSA hubs, having a vast knowledge base on clinical research operations and management. When experienced, they have extensive understanding of institutional policy, organizational culture, and the broader clinical research enterprise. A brain trust comprises a group of official or unofficial experts from various fields that may be consultants on key matters [Reference Ammer23]. Experienced CRPs are key members of research teams who establish and fine-tune collaborative relationships. They understand the infrastructure of the institution, build bridges, and have innovative ideas for improvements, but lack a platform for engaging in innovations due to the demands of their daily work activities. Most CRPs love the work and are loyal to the institution; however, they are at risk for frustrations and burnout that may motivate them to leave [Reference Stroo, Asfaw and Deeter3,Reference Gwede, Johnson, Roberts and Cantor7,Reference Getz8].
Description of the Educational Activity
Through this collaborative endeavor, we had three key learning objectives for participants:
-
Learn and apply the “Un-meeting” process as a method of team science collaboration and communication.
-
Contribute to key discussions, solution finding, and collaborations for key topics related to CRP workforce development, especially focusing on the AMC workforce.
-
Impact future initiatives and research by becoming a member of new interactive teams, participating in key meeting outputs or generating local outputs.
We purposefully sought broad inclusion across the CTSA network. Our leadership team consisted of individuals from three CTSA Program hubs: Ohio State University, University of Florida, and University of Washington. We added five lightening speakers from five additional hubs. Ultimately 184 participants from 65 CTSA hubs, hub partners, and IDeA-CTRs registered, though not all attended all sessions. Furthermore, we recruited 24 volunteer breakout group facilitators and co-facilitators who were registrants from multiple institutions.
We provided an Un-Meeting Briefing Book to all registrants and presented an overview of the Un-meeting process in our initial kickoff session, accessible at the Collaborative Conversations Un-meeting webpage on the CLIC website. We developed a Scripted Facilitators Guide to ensure breakouts were handled uniformly and discussions in breakouts were participant-driven, not expert- or facilitator-driven. During the breakouts, facilitators uniformly outlined the process, discussed ground rules, formed community through introductions, and provided a safe space for participant continued brainstorming and exchange of ideas about the CRP workforce issues. We solicited a volunteer (non-facilitator) from each breakout group to report a summary of the breakout group’s discussion when the participants returned to the main Zoom room.
On September 16, 2020, we kicked off the monthly 3-hour sessions (n = 109 attendees) with a keynote address by Dr. Michael Kurilla (NCATS) setting the stage on NCATS mission of training the CTSA workforce, including CRPs; explanations of the Un-meeting process and sharing results of the preconference need assessment survey. We emphasized the participant-driven nature of the Un-meeting. The five topical areas of CRP workforce development at AMCs formed the meeting themes:
-
October: Issues in job titles and job descriptions, n = 130 attendees,
-
November: Issues in onboarding training, n = 76 attendees,
-
December: Issues in competency-based continuing education, n = 68 attendees,
-
January: Issues in attrition, retention, and progression, n = 69 attendees,
-
February: Issues for enhancing CRP career pathways, including diversity, equity, and inclusion, n = 51 attendees.
The agenda of each meeting followed the Un-meeting structure (Fig. 2):
1. Welcome, Introductions,
2. Lightening Talk Speaker (one speaker, five slides, 5 min) – introducing the topic,
3. Brainstorming session,
4. Small breakout discussion groups,
5. Return to the main room, report out from group representatives.
Fig. 2. The Un-meeting structure.
Evaluation
Though we had 184 initial registrants, the median number of attendees across sessions after the initial kickoff meeting was 69. We attribute some of the reductions in attendees to busy work schedules, including increased COVID-19 demands or preferred attendance to those meetings with topics of personal interest. Job titles of registrants were wide ranging. Most were advanced CRPs, but we also had higher-level administrators and a vice-chair of research from CTSA hubs. One person worked in human resources.
Prior to breakout sessions, we dedicated brainstorming time for all participants to share their experiences using a Qualtrics (Qualtrics®XM, Provo, UT) survey during the meeting, allowing open-text contributions from all attendees. We also solicited post-meeting feedback through a Qualtrics survey to measure meeting satisfaction and to solicit post-meeting working group volunteers; however, only 34 responded. Participants were very satisfied or satisfied with: a) the format of the Un-meeting format (88%); b) subject matter (91%); c) opportunities to brainstorm (79%); and d) potential for collaboration (71%). Although the post-evaluation survey’s overall response rate was low, the results are comparable with academic online course evaluation rates; although, faculty reminders and providing in-class time to complete course evaluations positively affect rates [Reference Gudar and Malliaris24]. There were frustrations with our method of brainstorming; however, we were constrained by our 3-hour meeting schedule. The planning committee tried using Miro board and had technical issues with connections and user confusion. We decided our meetings would have been challenged with numerous technical issues if we had as many as 69 attendees using a Miro board at one time. Some working from hospital work computers were also limited in accessing Qualtrics links for brainstorming sessions due to strict firewalls. This frustrated those participants. We had excellent initial attendance at the first two sessions; however, our numbers drifted downward during subsequent sessions.
Post-meeting workgroup planning includes further qualitative thematic analyses of the brainstorming Qualtrics results, breakout session, and post-breakout group reports (recorded scripts). These data will inform and stimulate further conversations by the post-meeting workgroups leading to the generation of scholarly and collaborative outputs during 2021–2022. Our first wave of post-meeting volunteers (n = 16) from CTSA Program hubs have formed two initial working groups. Three persons volunteered to do further qualitative analyses; four individuals signed up to lead and co-lead the two working groups. Furthermore, a recent outcome of the meeting is an awarded CLIC Synergy Paper to support these future endeavors and the piloting of co-mentoring circles. We project post-meeting outputs to be in the form of white papers, manuscripts, presentations, policy statements, shared educational programs, and other focused collaborative projects aimed at improving AMC CRP workforce issues. Since the final February session, increasing anecdotal reports of post-COVID-19 CRP workforce losses by AMCs warrant groups to update metrics on those challenges.
To aid future trainers seeking to use a Zoom Un-meeting format, we include seven stages of successful planning and management of a Zoom Un-meeting (Table 1). We were prepared to launch our live Un-meeting in a 4-month period. This planning sheet can be useful for a single Zoom Un-meeting session or several sequential sessions.
Table 1. Seven stages of a Zoom Un-meeting learning event
Conclusion
The Zoom Un-meeting approach is an excellent means for providing adult learner collaborative, deep learning spaces. The Un-meeting approach was especially valuable as an adult learning tool for the experienced CRP giving them a participatory voice and enabled opportunities for group problem-solving, leadership, team science, and scholarship.
Supplemental Materials
The Collaborative Conversations Un-meeting Book can be found at the NCATS Coordinating Center, Center for Leading Innovation and Collaboration (CLIC), University of Rochester: Un-meeting Webpage: https://clic-ctsa.org/sites/default/files/2021-06/Collaborative-Conversations-The-Un-Meeting-Briefing-Book.pdf. Other information about CLIC Un-meetings and our Collaborative Conversations Unmeeting for CRPs can be found at: https://clic-ctsa.org/collaboration/clic-un-meetings
Acknowledgments
The authors wish to acknowledge the participants of the Collaborative Conversations Un-meeting sessions for their valuable input. We also wish to acknowledge Beth Harper, Workforce Innovation Officer at the Association of Clinical Research Professionals who participated in our planning team and provided ACRP continuing education credits for participants. Special appreciation for the breakout session facilitators.
This work was supported by the following NCATS grants: The Ohio State University CCTS #UL1TR002733; University of Washington ITHS #UL1TR00423; University of Florida CTSI #UL1TR001427; and University of Rochester Center for Leading Innovation and Collaboration (CLIC), under Grant U24TR002260. CLIC is the coordinating center for the Clinical and Translational Science Awards (CTSA) Program, funded by the National Center for Advancing Translational Sciences (NCATS) at the National Institutes of Health (NIH). Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the NIH.
Disclosures
The authors have no conflicts of interest to declare.
Open access
