Introduction
In academia, a successful biomedical research career is one in which the investigator is able to design and implement a research project, present and publish the findings, and compete for grant funds in order to carry out basic, clinical, or translational research. Additionally, understanding the academic environment is vital. Programs designed to prepare early-career researchers from underrepresented backgrounds1 (URB) for academia are increasingly common, with a 2012 environmental scan documenting their presence at 29% of 124 MD-degree-granting medical schools.Reference Adanga2 The importance of fostering the careers of fledgling URB biomedical researchers is especially important in light of the national and local scarcity of URB scientists in biomedical research.Reference Milewicz3– 6
The Career Education and Enhancement for Health Care Research Diversity (CEED) program at the University of Pittsburgh Institute for Clinical Research Education (ICRE) is a purposefully designed program of multi-level mentoring, targeted coursework, monthly seminar series, and networking opportunities designed to promote success in academic research for URB biomedical researchers. CEED’s goal is to provide a solid foundation for a successful research career for URB scientists while ensuring a supply of well-qualified clinical and translational investigators in the biomedical research pipeline. CEED aims to achieve this goal by providing mentoring and training in scientific and grant writing, research presentation, and other skills required to apply for competitive career development awards or other types of grant funding appropriate to each Scholar’s level.
While CEED has grown into a competitive, year-long professional and research development program for URB biomedical researchers, its overall effectiveness is unknown. In this paper, we describe the components of the program, quantify program effectiveness on several key outcomes, and discuss future directions.
Description of the CEED Program
CEED was explicitly designed to achieve an increase in the URB biomedical research workforce. The program is housed within the ICRE, which supports the education and training components of the University of Pittsburgh’s Clinical and Translational Science Institute (CTSI). CEED leverages the resources of the ICRE to support CEED Scholars. The program is led by two co-directors (KZA, NEM), with administrative support from a program coordinator. An 11-member Executive Committee reviews and rates applicants for acceptance into the program and provides guidance for future directions. Executive Committee members also serve as advisors to CEED Scholars.
CEED employs a unique, four-component structure that includes a monthly seminar series, multi-level mentoring, targeted coursework, and networking opportunities.
The Monthly Seminar Series is the core of the CEED program and runs from July through April of each program year. CEED Scholars have an opportunity to present their research projects and receive critiques and encouragement in a collegial and friendly environment. The CEED program creates a “safe” environment where issues of race, ethnicity, and gender can be openly discussed. Presentations are complemented by a variety of talks designed to frontload important information and skills needed to succeed in an academic research environment. Topics include Giving Effective Presentations, Curriculum Vitae, Negotiation, Difficult Conversations, Writing Productivity, NIH Training (K) and R Grants, and Letters of Recommendation.
CEED incorporates three levels of mentoring. These levels run concurrently throughout the course of the program to enable the Scholar to integrate the support of each mentor into a cohesive mentoring experience (Fig. 1).
Fig. 1. The Career Education and Enhancement for Health Care Research Diversity (CEED) levels of mentoring.
In addition to the program-specific focus on mentoring, each CEED Scholar is expected to participate in the annual Mentoring Matters Workshop at the ICRE. This interactive, half-day workshop guides participants in developing a mission statement for their careers, finding the best mentor to help achieve their goals, developing expectations for their mentoring relationship, and finding strategies for ensuring success. The workshop provides another unique opportunity for Scholars to work with a diverse and multidisciplinary group of senior faculty members and to network with both clinical and translational researchers from across the University. All ICRE programs require the applicant to have a mentor when enrolled in the ICRE but there is not a formal mentoring program across the entire ICRE.
Courses offered through the ICRE follow a competency-based model and provide CEED Scholars with an opportunity to further their education in areas of research in which they may have had little experience or exposure. The ICRE offers several courses that are particularly beneficial to CEED Scholars (Table 1).
Table 1. Courses beneficial to The Career Education and Enhancement for Health Care Research Diversity (CEED) scholars
Throughout the program year, CEED Scholars are given multiple opportunities to network and connect with senior faculty members. At the beginning of the year, there is a Meet-and-Greet that brings together participants in CEED and CEED II (CEED II is a program for URB medical students and is beyond the scope of this manuscript), CEED program directors, the CEED Executive Committee, the Scholars’ mentors, and CEED alumni. This allows the Scholars to meet with program leaders and converse in an informal setting. At the end of the program year, CEED Scholars present their scholarly work through a poster presentation to all Scholars’ mentors and the CEED Executive Committee. This popular session highlights each Scholar’s research accomplishments for the past year and provides another networking opportunity.
Methods
Study Design
This was a retrospective study design that included all CEED and non-CEED URB trainees who enrolled in the ICRE between the years 2007 and 2017, inclusive. All trainees complete an ICRE-developed survey upon matriculation into any ICRE program. This instrument assesses a wide variety of career success metrics.Reference Robinson7– Reference Primack10 Trainees also are asked to complete it annually while enrolled in any ICRE program, upon graduation, and then every other year as alumni. The number and type of publications and grants were abstracted from submitted CVs while demographic and professional characteristics were self-reported.
Statistical Analysis
Our primary question focused on the effect of the CEED program among URB trainees within the ICRE over 10 program years. Because the CEED and non-CEED trainees were not determined randomly, there were differences between both groups in demographic and other characteristics. Additionally, as the retrospective study design (vs prospective) can add additional bias, we utilized propensity score matching to minimize bias and create a comparable cohort of CEED and non-CEED URB trainees. Logistic regression was used to model the propensity of being in the CEED program as a function of race (White vs non-White), ethnicity (Hispanic vs non-Hispanic), degree (MD vs PhD), and faculty status at the beginning of a trainee’s program year. We created a matched set of non-CEED URB trainees also enrolled in ICRE programs in a 1:1 ratio, after assessing between-group imbalance using standardized mean differences. Descriptive statistics included means, standard deviations, and sample proportions; between-group differences were assessed with two-sample t-test or chi-square tests. Poisson regression was used to compare the rate of publications between CEED and non-CEED URB trainees after adjusting for baseline number of publications. Due to missing publication data, we performed sensitivity analysis by imputing zeros for all missing cases and re-running the Poisson regression on these data.
Statistical analysis was conducted using SAS version 9.3. All statistical testing was done with 5% type I error without adjustment for multiplicity.
Results
Our cohort comprised 45 CEED and 61 non-CEED URB trainees from 2007 to 2017. While similar in terms of gender distribution between groups, there were noticeable differences on race (78% vs 59% non-White; p=0.0072) and ethnicity (33% vs 54% Hispanic/Latino; p=0.0274). Note that URMs may be designated as White and Hispanic/Latino. CEED trainees were more likely to hold PhDs (56% vs 25%; p=0.0018) as well as hold a faculty position at baseline (22% vs 5%; p=0.0177). Finally, the mean number of publications per trainee was higher among the CEED cohort (5.44 vs 2.74; p=0.0007) (left-hand side of Table 2). After using propensity score matching to create a comparable cohort of 24 non-CEED URB trainees, demographics and professional characteristics were similar between groups (right-hand side of Table 2). This was further validated by calculated absolute standardized mean differences less than 0.2 (see supplemental Fig. 1).
Table 2. Characteristics of The Career Education and Enhancement for Health Care Research Diversity (CEED) participants vs Non-CEED participants (2007–2017)
‡ c = Pearson’s chi-square test; f = Fisher’s exact test; w = Wilcoxon rank-rum test.
1. We used the NIH definition of URB: a person who indicates that they belong to at least one of the following groups: Hispanic/Latino, American Indian/Alaskan Native, Black/African American, or Native Hawaiian/Other Pacific Islander. This analysis only includes trainees that started the first non-CEED program in 2007 and beyond.
2. Degree was dichotomized into MD vs PHD, where MD includes MD, MD/PhD, MD/Masters and DO. PHD=all other.
3. Baseline faculty status (when they started the programs) was categorized into three groups: faculty (if positions are “Assistant Professor,” “Instructor (Clinical Educator),” or “Faculty”), non-faculty (if positions are “Clinical Doctorate Student,” “DrPH Student,” “Fellow/Postdoc,” “Medical Student,” “PhD Student,” “Resident/Housestaff,” or “Undergraduate”), and other.
4. Out of 43 CEED alumni and 61 non-CEED underrepresented minority alumni with covariate information, 24 pairs were matched based on propensity model that controls for gender, race, ethnicity, degree, and the baseline faculty status.
Only 44 CEED and 43 non-CEED URB trainees had publication data for analysis. Prior to propensity score matching, CEED trainees had a higher rate of peer-reviewed publications than their non-CEED counterparts (11.02 vs 6.20 publications per person-year; p=0.0005) after adjusting for mean number of publications at baseline. After propensity score matching, this difference was attenuated, yet still significant (10.0 vs 7.69 publications per person-year; p=0.0261). Because only 16 matched pairs had complete publication data, we performed a sensitivity analysis by imputing zero for all missing cases. In this scenario, the results were robust in that CEED trainees continued to have higher publication rates compared to their non-CEED counterparts (8.73 vs 4.69 publications per person-year; p<0.0001) (Table 3).
Table 3. Comparison of number of peer-reviewed publications of The Career Education and Enhancement for Health Care Research Diversity (CEED) graduates vs University of Pittsburgh Institute for Clinical Research Education (ICRE) non-CEED graduates, 2007–2017
After adjusting for matched pairs, CEED graduates were more likely to hold an assistant professor position (54% vs 17%; p=0.0145) and trended toward being in the tenure stream (25% vs 4%; p=0.0971). We found no differences in Career Development Awards (p=0.6085) or Research Project Grants (p=0.4894) (Table 4).
Table 4. Comparison of number and type of academic positions of The Career Education and Enhancement for Health Care Research Diversity (CEED) graduates vs ICRE non-CEED graduates, 2007–2017
‡ c = Pearson’s chi-square test; f = Fisher’s exact test; w = Wilcoxon rank-rum test.
1. PI = Principal Investigator; F/I = Foundation/Institute.
Discussion
CEED was designed to promote career success and help seal the “leaky pipeline” for URB researchers with a purposefully designed program of multi-level mentoring, targeted coursework, monthly seminar series, and networking opportunities. In this study, we aimed to quantify CEED’s effect on several key outcomes by comparing ICRE CEED Scholars to a matched set of ICRE URB trainees not enrolled in CEED using data collected from 2007 to 2017. Our findings indicate participating in CEED can promote career success, thus contributing to retaining URB researchers in the pipeline, and expanding and diversifying the biomedical research workforce. CEED Scholars were significantly more likely to have peer-reviewed publications and hold academic positions at the assistant professor level. There were also more CEED Scholars holding tenure track positions, although this finding did not reach statistical significance. There were also more CEED Scholars with career development awards or research project grants, but these results also did not reach statistical significance. Notably, this success is beginning to be recognized outside the ICRE. CEED leadership is working with other divisions and departments to introduce CEED into the interviewing process for fellowship and junior faculty positions as a potential component of their experience, which may reinforce to URB interviewees the University’s commitment to breaking down the barriers to biomedical research careers.6 CEED has also been recognized with the University of Pittsburgh Chancellor’s Affirmative Action Award.Reference Barlow11
The literature describes other programs designed to promote URB faculty development and recruitment in biomedical research,Reference Adanga2, Reference McGee, Saran and Krulwich4, Reference Beech12– Reference Daley19 and includes at least one program in which developers noted participation was tied to enhanced inclusion, although there was no specific intent to achieve this outcome.Reference Byington20 In those designed specifically for URB participants, most reported on the availability and characteristics of their programs, and evaluated their programs on the basis of the number of grant applications and manuscripts produced or satisfaction with program content.Reference Beech12 Of those we examined, none compared their program participants with a comparison group.
This study’s limitations include the fact that CEED participants self-select into the program (and acceptance is based on a competitive process as described above). However, participants in the ICRE programs also self-select and would self-identify as interested in research. Nevertheless, it is possible that those who elect to participate in a program such as CEED are different in important, unknown ways from those who do not. An additional limitation is the potential confounding effect of mentoring. While mentoring is a key programmatic component of CEED, it is also a part of the larger ICRE (to which CEED belongs). As a result, it is possible that scholars who benefited from CEED also benefited from effective ICRE mentorship. However, since mentoring was implemented across the numerous programs of the ICRE with a higher degree of variability (i.e. degree programs vs career development programs), it is possible that any effect due to the CEED program was underestimated since non-CEED ICRE trainees may have also received effective mentoring. Lastly, the retrospective design of the study allows for biases to impact the overall study results, which ultimately limits our ability to draw any causal conclusions. While propensity score matching helps to mitigate these biases by controlling for known confounding, it cannot prevent unknown confounding.
Our results allow CEED to quantify 10 years of successful advancement of the careers of URB investigators and faculty while expanding the retention of diverse faculty. Because CEED has been successful despite the challenges of a small demographic pool and constrained funding, we believe it is likely other institutions will be able to replicate our results by following our program’s model.
Author ORCIDs
Doris Rubio, https://orcid.org/0000-0001-9540-6174; Colleen Mayowski, https://orcid.org/0000-0003-3752-6455
Acknowledgments
The authors wish to thank the CEED Executive Committee (Sonya Borrero, Paula Davis, Patricia Documet, Willa Marlene Doswell, Michael Fine, Alejandro Hoberman, Chenits Pettigrew, Jeannette South-Paul, and Galen Switzer) and the CEED Monthly Seminar speakers (Esa Davis, Paula Davis, Chyren Hunter, Sandra Murray, Lis Nielsen, Jeannette South-Paul, Natasha Tokowicz, Mary Walsh, and Jennifer Woodward), all of whom have given generously of their time and expertise. The authors also are grateful for the assistance of Joe Weiss, Lindsay Bell, and Yan Huang.
Financial Support
Research reported in this publication was supported by the National Center for Advancing Translational Sciences of the National Institutes of Health under Award Numbers UL1TR000005 and UL1TR001857. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Disclosures
The authors have no conflicts of interest to declare.
Previous presentations
Selected results were included within a poster presented at the 2018 Association for Clinical and Translational Science Annual Meeting.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/cts.2019.364
Open access
