Introduction
Research shows that diverse teams working together and capitalizing on innovative ideas and distinct perspectives outperform homogenous teams [1]. Scientists and trainees from diverse backgrounds and life experiences bring different perspectives, creativity, and individual enterprise to address complex scientific problems [1]. For the National Institutes of Health (NIH)-supported scientific workforce, diversity comes with many tangible benefits, including fostering scientific innovation, enhancing global competitiveness, contributing to robust learning environments, improving the quality of the researchers, advancing the likelihood that underserved or health disparity populations participate in and benefit from health research, and enhancing public trust [1]. However, diversification of the scientific workforce has advanced slowly.
The two primary vehicles for career development in Clinical & Translational Science Awards (CTSAs) at the time of this analysis have been the KL2 Career Development and TL1 training grants (since this analysis, however, National Center for Advancing Translational Science (NCATS) has added K12 and pre-/post-doctoral T32 awards). KL2 awards [2,Reference Sorkness, Scholl, Fair and Umans3] offer formal research training experience for clinical investigators who have recently completed an M.D., Ph.D., or equivalent doctoral degree and who are commencing basic, translational, and/or clinical research. All CTSA Program hubs have a KL2 program, which provides trainees with a rich, mentored career development experience in a multidisciplinary setting. KL2 scholars come from a variety of fields (e.g., medicine, dentistry, nursing, the behavioral sciences, biostatistics, and epidemiology) and typically receive two years of NCATS-funded protected research time and mentored career development support. Many CTSA Program hubs also include TL1 pre- and post-doctoral programs that provide trainees with an introduction to clinical and translational research [4,Reference Sancheznieto, Sorkness and Attia5]. CTSA hubs recruit TL1 candidates for full-time predoctoral training, combined health-professional doctorate-master’s training, and postdoctoral fellows. The goal of the TL1 program is to increase the number of well-trained clinical and translational scientists who can lead the design and oversight of future clinical investigations critical to the overall mission of NCATS and NIH.
Initiated in 2015 for the NIH-funded NCATS program, the Common Metrics Initiative (CMI) [6] has been implemented in over 60 CTSA programs [Reference Daudelin, Peterson and Welch7]. A key component of the Common Metrics is the Careers in Clinical and Translational Research metric, which measures the recruitment and retention of scientists with particular attention to underrepresented persons and women [8]. The metric also supports the NCATS strategic goal of developing and fostering innovative translational training of a highly skilled, creative, and diverse translational science workforce [9]. This national goal recognizes that advancing the field of translational science is through translational science education and training that supports a diverse translational science workforce.
This article describes the methodology employed for engaging CTSAs beyond the CMI process to identify and articulate strategies they were implementing to enhance the diversity of scientists within the field of translational science and will outline the findings from the culminating report disseminated across the CTSA consortium, From Insights to Action. This report summarizes strategies learned from interviewing these academic institutions and is designed to serve as a resource and to guide program implementation and development of strategic institutional plans.
Background
The Center for Leading Innovation and Collaboration (CLIC), the coordinating center for the CTSA Programs, was charged with implementing the CMI across the consortium [10]. The Insights to Inspire (I2I) Program was developed as a way to highlight the innovative and unique strategies implemented by participating CTSA institutions to improve on a given Common Metric [Reference Ruiz, Schwartz and Orlando11]. These insights have been disseminated via a series of blogs, webinars, and webcasts. CLIC dedicated its Insights to Inspire 2020 series to the Careers in Clinical and Translational Research metric [8], with underrepresented persons (URP) and women as the specific measures of interest. We focused on understanding how TL1 and KL2 programs at 18 participating CTSA hubs successfully implemented diversity, equity, and inclusion (DEI) approaches to the recruitment and retention of these future scientists. The approach used qualitative and quantitative methods specifically designed to gain a deeper understanding of implementation within the larger context of what each academic healthcare institution was doing with regard to diversity. For the 2020 I2I program, two separate iterations of qualitative analysis were conducted resulting in different themes and products. In the initial qualitative analysis of the program narratives (Turn-the-Curve Plans), five thematic domains emerged: diversity and inclusion, recruitment, application and screening, mentoring, and follow-up and evaluation. We then extended the analyses of hub successes from Insights to Inspire to create “From Insights to Action," with the goal of providing a sustainable resource of implementable actionable steps for program and institutional leaders on how to recruit and retain a diverse scientific workforce. It is this secondary qualitative analysis of the hub interview scripts that is the focus of this paper.
Methods
Annually, CLIC generates a Common Metrics Initiative Multi-Year Report that represents the Common Metric Initiative data values for each of the CTSA Program hubs in a de-identified and aggregate form. The Careers metric was the focus of the qualitative analysis for the annual report released in 2019, which included data from 2015 to 2018. Two approaches were taken to selecting the hubs with the most quantitative improvement among the 64 CTSAs whom submitted 2018 data. First, we looked at the institutions who were in the top 50% of metric improvement. Then we also looked at the top four institutions with the best results for either KL2 and/or TL1 data. The total number of institutions resulting from this were 30/64 CTSAs. The analysis used the Results Based Accountability Framework [Reference Friedman12] and the Story Behind the Curve sections of each institution’s Turn-the-Curve (TTC) plans. These sections explain both positive and negative factors that affected institutional data values. The process for annual data reporting, hub selection, and interview process is fully described separately[Reference Ruiz, Schwartz and Orlando11].
Hub Interviews: Eighteen unique hubs out of the initial 30 hubs selected participated in 19 interviews (one institution was selected for both their KL2 and TL1 programs). The interviewees consisted of principal investigators, program directors, administrators, and/or evaluators. On average, the interviews were between 30 and 45 minutes long. The interviews were conducted primarily by a CLIC Communications specialist and the CMI team. Table 1 shows the semi-structured interview protocol, designed to allow for participants to steer the direction of the conversation to address what they felt was meaningful. During the interviews, hubs were asked to share the strategies, insights, and processes that they believed led to the improvement of their metric values and were probed on the five themes identified from the TTC qualitative analysis.
Table 1. Semi-structured hub interview protocol
Note: During the interview, the institution was asked about the themes identified from the initial qualitative analysis from the Turn-The-Curve plans. The results of this interview led to these products: 1) blogs and 2) webinars.
CTSA: Clinical &Translational Science Awards
Commonality Analysis
The CMI team conducted a manual and iterative thematic coding process on the interview scripts to identify potential themes for the recruitment and retention of the scientific workforce. An initial round of concept coding examined larger units of text and quotes for general ideas/concepts prevalent in the data. A subsequent round of focused coding categorized the initial concepts as salient potential themes for discussion. These final themes were selected based on categorical overlap and consolidated into more comprehensive themes. The goal was not to provide answers or solutions to other hubs but to assist them in their efforts to identify areas of improvement at their own institutions – solutions that meet the needs of their trainees and scholars.
Results
Initial Analysis: The information obtained in the interviews resulted in a series of blogs and webinars in which selected institutions highlighted effective strategies related within one of the themes (Fig. 1).
Fig. 1. Five themes from interviews translated into blogs and webinars.
The blogs added relatable dimensions to the process, specifically the key messages that could resonate with the CTSA audience, and empowering participants to share their wisdom and expertise among their colleagues and amplify important DEI strategies to a wider audience than the individual hub. The themes published in the blogs and in the From Insights to Action resource were reviewed by all participating institutions prior to dissemination. In qualitative research, this process is referred to as member-checking: when data, analytic categories, interpretations, and conclusions are tested with members of those groups from whom the data were originally obtained [13]. This consensus step involves sharing the work of others (e.g., the survey results, interviews) and attributing quotes to institutions, through which credibility with the community is established [13].
Secondary Analysis: The secondary analysis of 667 minutes of interview recordings resulted in the six themes covering diversity and inclusion efforts at the participating hubs, outlined below. The themes were translated in the From Insights to Action resource into six primary sections, each representative of a strategy identified through qualitative analysis of the hubs interview transcripts. The strategies are not presented in order of importance or merit; readers are encouraged to browse each in the order of their choosing. The sections are composed of brief thematic overviews and a series of discussion questions designed to spark conversation around that theme. The overarching “guiding question” introduces the series of group questions to guide a team conversation. Answers to these questions are not provided as these will be specific to the local context of the program and/or institution. Table 2 provides a summary of the findings along with these conversation-starter questions. Below are the six themes identified from the secondary analysis:
Table 2. Emergent themes / key findings from secondary analysis
-
1. Prioritizing Representation: Diversity and inclusion within the CTSA Program is reflected in many forms: race, ethnicity, gender, area of study, stage of career development, and more. Hubs consistently noted that finding intentional, sustainable, scalable, and public means to represent a wide variety of perspectives in every stage of their programming was a key contributor to the creation of a welcoming and empowering environment. It matters who your scholars and trainees see and hear when they are recruited for, apply for, participate in, and work with your programs. “Matters” refers to the extent to which their race/ethnicity, gender, discipline, etc. are reflected throughout the process.
-
2. Building Partnerships: Career development programs should not stand in isolation. Whether recruiting internally for a K program or across state borders for a T program, hubs held partnerships in high esteem. From engaging in conversations with the leadership of the medical center about shared diversity and inclusion goals or working with a partner institution to enrich the pool of talented recruits, partnerships can take on many forms. Actively seeking out partners on — and off — your campus can further the representation efforts, bolster program recognition, and create meaningful, long-lived connections and collaborations.
-
3. Designing Program Structure: Hubs enhanced the career development of their scholars and trainees through mentorship, curriculum, fellowship, and unique programming and community engagement opportunities. The variety of program structures that work to support the growth of scholars and trainees provide mutually accountable spaces with clearly communicated expectations and benefits. Scholars’ needs may extend beyond the existing programmatic framework, though. Many hubs have acknowledged and supported their scholars as they built structures themselves: exercise groups, affiliation groups, or informal K-clubs, for example. Across hubs, program leaders’ ongoing, iterative work to balance built-in program supports with nurtured space for scholar-driven action continues to advance their capacity to identify and respond to the dynamic needs of their scholars.
-
4. Making It Personal: Scholars and trainees may only be with your program for two years; however, their experiences may serve as catalysts for future careers in clinical and translational research. Camaraderie, collegiality, and care were consistently referenced by hubs when asked how they created environments welcoming to scholars of all backgrounds. Hubs described ways in which — through both words and actions — they celebrated scholars’ pasts, welcomed scholars to their career development community, and helped scholars plan for a fulfilling future.
-
5. Improving through Feedback: Gathering data from scholars, trainees, and mentors can be an exhausting process. While there is no one fix-all solution for optimizing survey response rates, hubs consistently referenced how actions taken during a scholar’s tenure had lasting reverberations for how and why they continued to engage with the program. Hubs discussed feedback as an ongoing, bidirectional practice that manifests itself in formal and informal ways. From the recruitment phase to alumni follow-up, hubs work to find spaces to solicit, provide, and — importantly — act on feedback not restricted to outcomes or scholarly gains.
-
6. Winning Endorsement: Many hubs highlighted their own scholars and trainees as their most valuable recruitment assets. To have program alumni, department heads, and current scholars actively reach out to prospective applicants can be a powerful testament to the work your program has done in creating a welcoming, inclusive environment. Consider who is endorsing your career development program and which voices are missing from the table.
Reporting: The above themes framed the report and resource: From Insights to Action: Enriching the Clinical Research Workforce by Developing Diverse and Inclusive Career Programs [Reference Ruiz, Schwartz and Lackey14]. This resource is designed for institutions seeking insights for developing a synergistic approach for improving their recruitment efforts to diversify the future clinical and translational research workforce. For example, an institution can use the guiding questions in Table 2 as conversation starters for the theme that they wish to improve in their career program. The resource includes all aspects of the lifecycle — from scholar/trainee recruitment through mentorship and follow-up after completion of the program. These insights are based on experiences from the field, from a place of practice.
Fig. 2 visually summarizes the products of both qualitative analyses conducted.
Fig. 2. Products from program narratives and interview scripts qualitative analyses (CTSA: Clinical & Translational Science Awards).
Dissemination: From Insights to Action was disseminated throughout the CTSA consortium via newsletters, the Insights to Inspire webpage [15], on key national stakeholder calls, and social media channels. There were 197 unique website page views for From Insights to Action. A Twitter campaign began in early 2020 to promote From Insights to Action with eight topic-specific tweets to our 849 followers. The number of times (impressions) that these tweets were seen ranged from 163 to 396 across tweets. The Association of American Medical Colleges shared the resource on two of their newsletters on January 2021: Group on Research, Education and Training (GREAT) and Group on Research Advancement and Development (GRAND). Notably, the From Insights to Action resource also served as the blueprint for the “Diversity, Equity and Inclusion, Training” breakout session [16] at the CTSA Program Annual meeting in Fall 2020. There were 133 participants in this breakout.
Discussion
Although the CMI was not perfect, the consortium generated feasible, measurable aggregated metric data from the hubs that served as benchmarks for continued improvement. Through the I2I program, specifically for the Careers metric mixed-methods assessment, we identified themes and strategies used by hubs that were most successful in addressing diversity of their KL2 and TL1 workforce/trainees. These were disseminated to the hubs with several impacts:
-
1. Implemented a consistent approach to measuring trainee diversity at the hub level that allowed for comparisons of median levels at individual hubs and aggregated across the consortium. This addresses NCATS strategic goal of developing a diverse CTS workforce.
-
2. Highlighted the real impact of the Careers CM by getting hubs to examine the percent of graduates who are “actively engaged in research," the main reason they were trained.
-
3. Induced hubs to brainstorm new ways to enhance diversity and to disseminate those techniques to the CTSA consortium.
-
4. Reduced silos by having all hubs participate in the CMI and share best practices (e.g., Insights to Inspire (I2I) Program, From Insights to Action report).
The From Insights to Action resource provides a set of strategies addressing various elements to developing a diverse scientific workforce. The data and insights acquired are based on experiences of hubs within the CTSA consortium that showed significant improvement in, or were highest performers on, the associated quantitative common metrics. This resource was disseminated across the CTSA consortium and more broadly to the medical community, with preliminary measure of uptake indicating positive reach.
The From Insights to Action report included representation from 18 institutions (fewer than 30%) out of about 64 within the CTSA community. While the CTSA program consortium shares many common programs, and all participate in the CMI process, specific hubs may differ in terms of administrative and institutional structures, community engagement, available resources, and other local factors that may impact the potential for implementing a particular strategy offered in the resource. Further, while it is conceivable that the strategies offered may be generalizable beyond the CTSA program, a comparable inquiry process would need to be conducted across other scientific research programs to better assess them as “best practices.”
Going Forward
A combination of ongoing, intentional efforts and commitment is necessary to strengthen inclusivity and diversity in career education training programs. It is important for institutional leadership to view this work as a strategic imperative — not just the right thing to do [Reference Boulware, Corbie and Aguilar-Gaxiola17]. A purposeful devotion of time and resources for these inclusive initiatives will enable shared goals to flourish. Now with the new addition of the NCATS K12 and T32 programs, further research will be needed to identify unique needs and strategies that account for relevant contextual factors within those programs. Although these ideas represent the work within a unique research community, they are translatable to other programs within and outside of the education field that are in search of starting or improving upon their recruitment and retention of a diverse workforce. Finally, it will be important to evaluate the implementation of these strategies to further understand what works across institutions and programs while accounting for local contextual factors.
Acknowledgments
The authors extend their gratitude to all of the CTSA Program institutions who shared their insights that contributed to the success of the Insights to Inspire program and the From Insights to Action report. This collaboration enabled the consortium members to learn from each other and work toward program continuous improvement. CLIC personnel from the Common Metrics, Education team, Website, Communications, and Coordination teams including Ann Schwartz, Theresa Long, Russell Lackey, Melissa Trayhan, Benjamin Tyler, Michaela Butler, Megan Petty, Karen Grabowski, contributed to various aspects of the program.
The project described was funded by the University of Rochester CLIC, under Grant U24TR002260. CLIC is the coordinating center for the CTSA Program, funded by the NCATS at the NIH. This work is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.
Disclosures
The authors have no conflicts of interest to declare.
Open access
