Hostname: page-component-78c5997874-dh8gc Total loading time: 0 Render date: 2024-11-05T04:29:41.059Z Has data issue: false hasContentIssue false

Evaluation of the academic achievements of clinician health services research scientists involved in “pre-K” career development award programs

Published online by Cambridge University Press:  16 April 2021

Erin F. Barreto*
Affiliation:
Department of Pharmacy, Mayo Clinic, Rochester, MN, USA Robert D. and Patricia E. Kern Center for the Science of Health Care Delivery, Mayo Clinic, Rochester, MN, USA
Rozalina G. McCoy
Affiliation:
Robert D. and Patricia E. Kern Center for the Science of Health Care Delivery, Mayo Clinic, Rochester, MN, USA Division of Community Internal Medicine, Department of Medicine, Mayo Clinic, Rochester, MN, USA Division of Health Care Policy & Research, Department of Health Sciences Research, Mayo Clinic, Rochester, MN, USA
Joseph J. Larson
Affiliation:
Division of Biomedical Statistics and Informatics, Mayo Clinic, Rochester, MN, USA
Rahma M. Warsame
Affiliation:
Robert D. and Patricia E. Kern Center for the Science of Health Care Delivery, Mayo Clinic, Rochester, MN, USA Division of Hematology, Department of Medicine, Mayo Clinic, Rochester, MN, USA
Cassie C. Kennedy
Affiliation:
Division of Pulmonary Medicine, Department of Medicine, Mayo Clinic, Rochester, MN, USA
Ashley E. Baker
Affiliation:
Robert D. and Patricia E. Kern Center for the Science of Health Care Delivery, Mayo Clinic, Rochester, MN, USA
Elizabeth S. Hart
Affiliation:
Robert D. and Patricia E. Kern Center for the Science of Health Care Delivery, Mayo Clinic, Rochester, MN, USA
Stephanie M. Pagel
Affiliation:
Robert D. and Patricia E. Kern Center for the Science of Health Care Delivery, Mayo Clinic, Rochester, MN, USA
Samantha A. Whitman
Affiliation:
School for the Future of Innovation in Society, Arizona State University, Tempe, AZ, USA
Kasey R. Boehmer
Affiliation:
Knowledge and Evaluation Research (KER) Unit, Mayo Clinic, Rochester, MN, USA
Felicity T. Enders
Affiliation:
Robert D. and Patricia E. Kern Center for the Science of Health Care Delivery, Mayo Clinic, Rochester, MN, USA Division of Biomedical Statistics and Informatics, Mayo Clinic, Rochester, MN, USA
*
Address for correspondence: E.F. Barreto, PharmD, MS, Department of Pharmacy, Mayo Clinic, 200 1st St SW, Rochester, MN, 55905, USA. Email: [email protected]
Rights & Permissions [Opens in a new window]

Abstract

Introduction:

Research career development awards (CDAs) facilitate development of clinician-scientists. This study compared the academic achievements of individuals in a structured institutional “pre-K” CDA program, the Mayo Clinic Kern Scholars program, with individuals who applied for but were not admitted to the Kern program (“Kern applicants”), and awardees of other unstructured internal CDAs.

Methods:

This was a longitudinal cohort study of clinicians engaged in research at Mayo Clinic between 2010 and 2019. The primary outcome was time to the 15th new peer-reviewed publication after the program start, adjusted for baseline number of publications. Secondarily, we described successful awarding of federal funding by the NIH or VA.

Results:

The median (IQR) number of baseline publications was highest among Kern Scholars compared to Kern Applicants or other CDA awardees [16 (12, 29) vs 5 (1, 11) and 8 (5, 16); P < 0.001]. After adjustment for baseline publications, the time to 15th new publication was significantly shorter for Kern Scholars than for the two comparator groups (P<0.001). Similar findings were observed with total new publications within 5 years (P < 0.001), as well as number of new first-/last-author publications within 5 years (P < 0.001). The overall frequency of K-awards, R-awards (or equivalent), or any funding were similar between groups, with the exception of R03 awards, which were significantly more common among Kern Scholars (P = 0.002).

Conclusion:

The Kern Scholars program is a successful training model for clinician-scientists that demonstrated comparatively greater acceleration of scholarly productivity than other internal CDA programs.

Type
Research Article
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.
Copyright
© The Author(s), 2021. Published by Cambridge University Press on behalf of The Association for Clinical and Translational Science

Introduction

Rapid evolution has occurred in how healthcare is delivered, measured, and funded. There is an increased focus on translational science, quality and value, changes to payment structures, access to large datasets, and the importance of population health and preventative medicine. This has led to the development of health services research (HSR) as an essential field of scientific inquiry. HSR involves evaluation of social factors, economics, technology, organizational context, and personal behaviors that impact healthcare quality, cost, and experience. Clinician-scientists with expertise in HSR (clinician-HSR scientists) are needed to meet these demands for knowledge generation, dissemination, and evaluation [Reference Berwick, Nolan and Whittington1Reference Kluijtmans, de Haan, Akkerman and van Tartwijk4].

Despite the need for clinician-HSR scientists, fewer individuals are pursuing this career path [Reference Roberts, Fischhoff, Sakowski and Feldman5]. A dearth of clinician-HSR scientists is especially prominent among clinicians in surgical subspecialties [Reference Goldstein, Blair and Keswani6], women [Reference Tong, Madhur and Rzeszut7Reference Nikaj, Roychowdhury, Lund, Matthews and Pearson9], nonphysician clinician-scientists [Reference Personett, Hammond, Frazee, Skrupky, Johnson and Schramm10Reference Begley, McCarron, Huntley-Moore, Condell and Higgins12], and underrepresented minorities [Reference Nikaj, Roychowdhury, Lund, Matthews and Pearson9], although these gaps are not unique to HSR. To ensure a sustainable future for the advancement of medicine and its delivery, it is incumbent on the field to identify individual- and organizational-level factors that facilitate the training, persistence, and success of developing clinician-HSR scientists [Reference Barry, De Groot and Baggen3].

One successful strategy to facilitate clinician-HSR scientist development is through mentored career development awards (CDAs). Among the most sought after are the K-series awards from the National Institutes of Health (NIH), which integrate career development and applied research experience [Reference Finney, Amundson and Bi13]. Although K-series and other comparable awards are often designed for early stage investigators, these competitive awards may still be beyond the reach of junior faculty or clinicians-engaged-in-research without an explicit academic trajectory. In these cases, alternative training pathways and funding mechanisms are necessary. Such “pre-K” CDAs may be funded by the institution, a professional society, or an NIH Clinical and Translational Science Award (CTSA) [14]. These “pre-K” programs vary in duration, breadth, and formality, but generally include some core elements of education, mentorship, and applied research experience. There is limited empirical evidence regarding programmatic features of the “pre-K” CDAs that favorably impact the success of clinician-scientists, particularly those in HSR. Previously identified factors that predict a successful transition to research independence for CDA awardees include development of robust networks for collaboration, mentorship, and institutional resources and support [Reference Robinson, Schwartz, Dimeglio, Ahluwalia and Gabrilove15].

Mayo Clinic, an academic integrated healthcare delivery system, has several “pre-K” CDAs available for early career clinical faculty. The Mayo Clinic Kern Scholars Program is one such “pre-K” CDA focused on the development of the clinician-HSR scientist. The Kern Scholars program seeks to develop individuals who could transform health care delivery through rigorous, collaborative, practice-oriented research. The program includes two tracks. The faculty track lasts 2–3 years, provides for up to 40% protected research time, and offers modest research funding (Table S1). The fellow track provides for one year of 90% protected research time and comparable yearly research funding.

The Kern Scholars program includes several key elements to facilitate a candidate’s successful development as a clinician-HSR scientist (Table S2). Peer coaching is based on a cascading mentorship model, where senior scholars provide mentorship to junior scholars [Reference Feldon, Litson and Jeong16Reference Decastro, Sambuco, Ubel, Stewart and Jagsi18]. A physical space within the Kern Center exists to directly facilitate collegiality, engagement with core faculty and resources, and peer coaching. In the 1–3-year-award period, scholars meet weekly for structured “works-in progress” sessions, which include scholar presentations, invited faculty lectures, and career development topics. Each scholar presents their research during this interactive session approximately every six months. As with many elements of the program, works-in-progress sessions are tailored to the needs of the scholars as appropriate for their career stage and ongoing research activities. Examples of topics presented include a research project that encountered obstacles (e.g., slow enrollment, study personnel challenges), a specific aims page or draft of a grant in need of revision, or preliminary data for summary and interpretation. In addition to scholar-led works-in-progress, focused career development training occurs at monthly intervals and during biannual one day retreats. Selected topics transcend the traditional scientific areas in a deliberate attempt to provide a holistic approach to career development. Topics were also chosen to develop clinician-HSR scientists able to build bridges [Reference Barry, De Groot and Baggen3,Reference Kluijtmans, de Haan, Akkerman and van Tartwijk4] across research’s “Valley of Death [Reference Butler19],” i.e., the gap between promising discoveries and bedside treatment (Fig. 1).

Fig. 1. Conceptual framework for developing the bridge between clinician and scientist. Key domains include career development, internal networking and visibility, external networking and visibility, a familiarity with local resources, and expertise in grant writing. The practice and research networks as well as mentors provide necessary infrastructure to support the development of clinician-health services research scientists.

Other internal “pre-K” CDA programs (i.e., benefactor-funded CDAs) exist at Mayo Clinic, which provide up to $50,000 to $100,000 per year over one to three years that may be used to support protected time or other research support (Table S1). Mayo Clinic offers approximately 20 benefactor funded “pre-K” CDAs per year across various disease-state and topical subspecialties. These other internal “pre-K” CDA programs use a traditional cognitive apprentice mentoring model, which is predicated on one individual (generally the principal investigator) mentoring one or more learners [Reference Feldon, Litson and Jeong16Reference Decastro, Sambuco, Ubel, Stewart and Jagsi18]. Each recipient is also separate from other awardees without group activities, so there is less of an opportunity for a cohort effect or peer-to-peer mentorship. Informal interactions with colleagues and mentors not dictated by the award mechanism are expected consistent with usual practice. In addition, these other “pre-K” CDA awardees do not participate in a structured curriculum, though some may choose to utilize a portion of their funding for individualized scientific education (i.e., coursework, conferences, workshops). These distinct “pre-K” training models offer an opportunity for comparison.

In this report, we compare the academic output of Mayo Clinic Kern Scholars to other internal benefactor-funded “pre-K” CDAs. We hypothesized that Kern Scholars would experience greater academic success (measured with peer-reviewed publications and grant funding) than other career development awardees over the same time due to the program’s structured curriculum.

Methods

Setting and Participants

We conducted a longitudinal cohort study that compared scholarly productivity and grant funding among clinicians at three Mayo Clinic campuses (Minnesota, Florida, Arizona) who were classified into three groups between 2010 and 2019: (1) Kern Scholar awardees in the faculty track, (2) Kern Scholar applicants who applied but were not accepted, and (3) recipients of other Mayo Clinic benefactor-funded CDAs. The Mayo Clinic Institutional Review Board approved the protocol as exempt (IRB 19-006187).

Fellows selected for the Kern Scholars program were excluded due to lack of appropriate comparators (N=7). No fellows were included in the other study arms. Individuals selected for both the Kern Scholars program and an internal “pre-K” CDA were handled in two ways. When these two “pre-K” CDAs were granted in the same year, individuals were excluded (N=4) as we were unable to discern the timeline of productivity relative to the award. If the awards were granted in different years (e.g., Kern Scholars acceptance in year 1, benefactor-funded CDA in year 3), individuals were grouped according to the first award received and censored at the start of the second program. The index date was the year of application or entry into the program. Given the variable duration of each program, all individuals were followed until June 2019 (end of the study period). Individuals whose publications and grants could not be characterized through publicly available medical literature databases (i.e., PubMed) or funding agency portals were also excluded (N=6). The Mayo Clinic Institutional Review Board approved the protocol as exempt (IRB 19-006187).

Outcomes

The primary outcome of interest was academic achievement as measured by time to the 15th new peer-reviewed publication (i.e., manuscript published after the index date) [Reference Ehlers, Cornelius and Greenberg-Worisek20] and the number of first author publications, last author publications, or first-/last-author publications between index date and five years of follow-up or the end of study period. Peer-reviewed publications were identified through an institutionally managed database (RE-AIMS) for individuals currently at Mayo Clinic and reconciled with publicly available medical literature databases (i.e., PubMed). Secondarily, we evaluated achievement of federal funding by the CDA awardee as a primary investigator. Federal funding was identified using two publically available databases from the NIH https://projectreporter.nih.gov/reporter.cfm) or the Veterans Administration (https://www.hsrd.research.va.gov/research/cda.cfm). K-, R-, U- and P-awards were counted toward this outcome whereas F32 and F30 awards were not. For the Kern Scholars, we also described all non-institutional (i.e., federal, foundation, or industry) research funding during their time at Mayo Clinic sourced from an internal data repository. This information (foundation and industry awards) was not available for the other two groups. Exploratory analyses in women, underrepresented minorities, and those affiliated with surgical subspecialties were performed to determine differential success within these subgroups. The H-index [Reference Schreiber and Giustini21] was also retrospectively calculated for all individuals at the index date and at the date of last follow-up (June 2019). The H-index integrates publication and citation counts to generate a summary measure of scientific achievement for an individual researcher [Reference Schreiber and Giustini21]. The intra-individual change in H-index from index date to the end of the study period was described.

Analyses

Descriptive comparisons between Kern Scholars, Kern applicants (not selected), and other CDA awardees used means with standard deviations (SD) or medians with interquartile ranges (IQR), and frequencies with percentages, as appropriate. Time to 15th new publication after the index date was examined using Kaplan Meier survival analysis. Comparisons across the three groups were made with the log-rank test. Univariate Cox-models were fit for time to 15th new publication, first author publication, last author publication, or first-/last-author publication. Models were then adjusted for the number of publications prior to the index date. To assess pairwise differences between the Kern Scholars and other comparator groups in the number of new publications within five years of the index date, quasi-Poisson regressions models were fit. The quasi-Poisson model was chosen to address overdispersion due to the proportion of individuals with zero publications within the five years of index date. Univariate analyses were performed as well as analyses adjusted for number of publications prior to the index date. The presence or absence of funding at the conclusion of the study timeframe was treated as a binary variable and analyzed with the Chi-square or Fisher’s Exact test. A P-value <0.05 was considered statistically significant. All analyses were conducted using the R (v3.6.2, Vienna, Austria) statistical package.

Results

During the study period, there were 46 Kern applicants (not selected), 24 Kern Scholars, and 129 other CDA awardees included in the analytic cohort (Fig. 2). Demographic characteristics of individuals in each group are summarized in Table 1. At baseline, the median (IQR) total number of peer-reviewed publications at the time of program enrollment/application was higher for Kern Scholars than Kern applicants [16 (12, 29) vs 5 (1, 11); P < 0.001] and for Kern Scholars than other CDA awardees [16 (12, 29) vs 8 (5, 16); P < 0.001]. Numbers of first author publications and first-/last-author publications were different across groups (Table 1). The H-index was able to be calculated for 191 (96%) individuals. Kern Applicants had the lowest H-index at baseline at 6 (3.5, 14.0), compared to 10 (7.4, 13.8) in Kern Scholars, and 13 (8.3, 20.0) in other CDA awardees (P < 0.001).

Fig. 2. Study flow diagram

CDA, Career development award.

Table 1. Characteristics of the study cohort

CDA, Career development award.

a Refers to individuals who applied to the Kern Scholars program and were not selected.

b Data were available for N = 44 Kern Scholars, N = 24 Kern Applicants, and N = 124 other CDA awardees.

c Data were available for N = 46 Kern Scholars, N = 24 Kern Applicants, and N = 126 other CDA awardees.

d Data were available for N = 46 Kern Scholars, N = 24 Kern Applicants, and N = 125 other CDA awardees.

e No Kern Scholars were included from 2019 as their program would have begun in July which post-dates the study follow-up period.

f Data represented medians with interquartile ranges prior to the index date. Two-way comparisons indicated statistically more total publications prior to the index date among Kern Scholars compared to other CDA awardees (P < 0.001). No statistical difference existed between Kern Scholars and other CDA awardees in number of first author (P = 0.30), last-author (P = 0.69), or first-/last-author publications (P = 0.56) at baseline. Kern Scholars had more baseline publications than Kern applicants in each of the prior publication categories with the exception of last author publications (P = 0.10).

Endpoints

Time to 15th new publication was significantly shorter for Kern Scholars than the two comparator groups (P < 0.001) (Fig. 3, upper panel). The 15th new publication occurred at a median of 2.4 (95% CI: 1.5, 3.9) years from the index date for Kern Scholars, compared to more than 8.5 years for the other two groups. By eight years, 90% of Kern Scholars had achieved 15 new publications compared to 6% and 35% in Kern Applicants and other CDA awardees, respectively. Similar findings were observed for time to 15th new first-/last-author publication (P < 0.001) (Fig. 3, lower panel). After adjustment for prior publications at the index date, Kern Scholars had a significantly faster time to 15th new publication and 15th new first-/last-author publication compared to Kern applicants and other CDA awardees, though no difference was noted when first authorship and last authorship were evaluated separately (Table 2). The median (IQR) number of total new publications within five years was significantly different across groups [Kern Scholars 27 (17, 42) vs Kern applicants 4 (1, 7) vs other CDA awardees 4 (1, 8); P <0.001]. Similarly, the median number of first author publications was significantly different [Kern Scholars 7 (3, 10) vs Kern applicants 1 (0, 3) vs other CDA awardees 1 (0, 2); P <0.001]. After adjustment for publications prior to the index date, the total number of publications within five years was higher among Kern Scholars than the other two groups (Table S3).

Fig. 3. Time to 15th new publication across groups. Upper panel: Time to 15th new publication between Kern Scholars and Kern applicants or other CDA awardees. Lower panel: Time to 15th new first or last author publication between Kern Scholars and Kern applicants or other CDA awardees. Kern Scholars achieved a more rapid time to 15th new publication and 15th new first-/last-author publication than comparator groups (P < 0.001 in both cases). CDA: Career development award.

Table 2. Cox proportional hazards models for predicting time to 15th new publication based on various definitions, adjusted for publications prior to the index date

CDA, Career development award; HR, Hazard ratio.

a Unable to be calculated as no individuals in the Kern applicants group reached 15 new first author publications.

Overall, 48 (31%) individuals successfully secured extramural funding as a principal investigator from the NIH during the follow-up period (Kern Scholars 33% vs other CDA awardees 31%; P = 0.82). No individuals were identified with funding through the VA. No Kern applicants received this type of funding during the study timeframe. Twenty six (17%) individuals were awarded K-type grants and 35 (23%) received R-, P- or U-awards. There was no significant difference between Kern Scholars and other CDA awardees in the frequency of any funding, receipt of a K-award, or receipt of an R-, U-, or P-award as PI (Table 3). Kern Scholars were significantly more likely to receive an R03 than other CDA awardees (21% vs 4%, respectively; P = 0.002). For the 24 Kern Scholars, data on all types of noninstitutional funding were available for 18 (75%) individuals. During the study time frame, the median total funding from external sources per person was $563,874 ($63,114, $2,626,112). Median funding secured by Kern Scholars as principal investigator was $298,386 ($19,459, $1,050,655) and as coinvestigator was $56,250 ($625, $864,110).

Table 3. National Institutes of Health Funding as Principal Investigator

CDA, Career development award.

a Values expressed as frequencies with percentages.

b No Kern applicants were awarded NIH funding during the study timeframe; thus, they are omitted from the denominator.

c Considers K-, R-, P-, and U-series awards; F awards were not included in the analysis.

Exploratory Analyses

In stratified analyses according to sex (male vs female), race (white vs nonwhite), and specialty (surgical vs nonsurgical), the primary outcome of time to 15th new publication appeared generally similar to the overall group, but statistical inferences were limited due to small sample sizes in some subgroups (Figure S1S3). The change in H-index per year from baseline over the follow-up period was similar across groups (P = 0.21).

Discussion

Training the next generation of clinician-HSR scientists, particularly those directly embedded in the practice, is a central priority for the medical community, health systems, and society. We describe the programmatic elements and outcomes of the Mayo Clinic Kern Scholars program, a mentored institutional “pre-K” CDA, which effectively trained 24 clinician-HSR scientists over the past ten years. We demonstrated considerable success of this program in terms of academic productivity and successful transition to extramural funding. Compared to other “pre-K” intramural CDA awardees, who received a similar amount of protected time and financial resources, Kern Scholars demonstrated a 10-fold faster time to 15th new publication and a 4-fold faster time to 15th new first-/last-author publication. Similar findings were observed among women, underrepresented minorities, and individuals in the surgical subspecialties. One-third of Kern Scholars and other CDA awardees secured extramural funding as a principal investigator from the NIH during the follow-up period. The comparative frequency of successful extramural funding was similar across award subtypes, with the exception of R03s, which were more common among Kern Scholars. When considering all extramural funding sources (federal, foundation, and industry), Kern Scholars received a median of $563,874 in grant funding per person.

Evidence suggests that formal training in clinical translational research promotes scholarly output. In a study of 70 clinical fellows who completed formal postdoctoral clinical translational research training, matched to fellows who did not, the number of peer-reviewed manuscripts, including first-/last-author manuscripts, and the H-index were higher among those who completed formal training [Reference Ehlers, Cornelius and Greenberg-Worisek20]. While these data suggest a benefit of enhanced research training, this study did not assess whether specific elements of a training program enhance the probability of long-term success. It is expected that all individuals in the present study were clinicians interested in an academic path, given their application to these programs. The comparison between Kern Scholars and other CDA awardees allowed us to probe the influence of structured programmatic elements on candidate success. Research time and funding amounts were likely less influential on the observed outcomes as these factors were similar between groups. Using an estimated salary based on the NIH cap of approximately $200,000, Kern Scholars received $61,000 per year for three years (salary, fringe, and prorated research funds over the life of the award) compared to $50–$100,000 per year for other CDA awardees (Table S1). Detailed individual award amounts were not available. We hypothesize that one reason for greater probability of academic success among the Kern Scholars was the frequent and facilitated interactions among and between the Scholars and the program leadership, which complemented the training and mentoring from mentors. While not directly measured in this study, we surmise that weekly interactions may enhance skill building, feedback, modeling, confidence, and emotional support.

The Kern Scholars Program approach differs from the traditional dyadic model of one protégé/mentee and one senior faculty mentor, common in PhD and other research training programs [Reference Feldon, Litson and Jeong16Reference Decastro, Sambuco, Ubel, Stewart and Jagsi18]. In an integrated model such as the Kern Scholars program, emphasis is instead placed on cascading mentorship wherein junior scholars are trained by senior scholars. This horizontal mentorship or mentor network model [Reference Decastro, Sambuco, Ubel, Stewart and Jagsi18,Reference Byington, Keenan and Phillips22] may be more effective in cultivating clinician-HSR scientists, as it mimics the familiar layered learning model used in medicine (i.e., hierarchical team model of medical student, resident, fellow, staff) [Reference Loy, Yang, Moss, Kemp and Brown23] and the composition of multiprofessional collaborative clinical teams. This is aligned with data from a qualitative study of 40 former KL2 or K12 Scholars, which suggested that research networks and mentorship are key factors associated with successfully securing independent funding at the conclusion of the CDA. One respondent in this evaluation described several types of mentors necessary for academic success one of which was a peer mentor to serve as a role model and guide [Reference Robinson, Schwartz, Dimeglio, Ahluwalia and Gabrilove15].

Internal “pre-K” CDAs are one mechanism to foster the development of clinician-HSR scientists. These awards are often designed as a bridge toward external funding via a K-type award (i.e., K08 Mentored Clinical Scientist Research CDA or K23 Mentored Patient-Oriented Research CDA) or an R-series award. In this study, 31% of individuals were awarded federal funding as PI during the follow-up interval. Seventeen percent of individuals were awarded a K-series grant. It is unknown what percentage of people in these programs submitted NIH grant applications during the follow-up period and were not funded. Estimates indicate that success rates for K08 or K23 applications for the general applicant pool range from 35 to 45% [Reference Yin, Gabrilove, Jackson, Sweeney, Fair and Toto17]. Other than federal support, possible sources of funding include from industry and foundation grants. Our data demonstrates that 71% of Kern Scholars were awarded noninstitutional funding (federal, foundation, or industry) during or after program completion. This suggests that these other sources of funding play a considerable role in the academic careers of clinician-scientists. Future studies and training programs should include industry and foundation funding as metrics for success alongside federal awards.

While these data are promising, the study has several limitations. Internal awards require substantial resources from the institution, divisions/departments, and/or benefactors and as such are heavily influenced by the environment. Evidence from one survey of academic emergency departments indicated that 27% of centers provided institutional research funds to support extramural grant applications for junior researchers [Reference Mumma, Chang, Kea and Ranney24]. Mayo Clinic is a large academic integrated health care delivery system with an extensive research program; thus, these results may not be generalizable. However, Mayo Clinic is also not a part of a larger university and as such its research training programs are directly embedded in the practice, demonstrating the possibility of successful early career research training outside of a strictly academic setting. Another consideration is that Kern Scholars may have been more academically accomplished at baseline than the other groups rendering these findings self-fulfilling. Indeed at baseline, Kern Scholars had more peer-reviewed publications than the other two groups, but the H-indices between Kern Scholars and other CDA awardees were similar at baseline. Still differences between groups in the academic trajectory were realized even after adjustment for baseline productivity. It is also possible that there is bias associated with allocation to the various groups under study. The justification for selection of a given individual for participation in any one of these programs was unobserved. Another limitation is that the time horizon during which one should measure academic success and the metrics to be used remain a subject of debate. Among other CDA awardees (not exclusively HSR), data collection for biologic/mechanistic work may require more time to complete thereby delaying publications relative to HSR scholars. In this study, we attempted to minimize this effect by extending follow-up out to at least five years. We acknowledge that not all peer-reviewed publications carry the same clinical impact, methodological rigor, or potential to elevate an early investigator’s career. To address this, we also examined first and last author publications, which reflect more active involvement from the investigator. Finally, while we performed descriptive analyses that suggest the robustness of our findings among women, underrepresented minorities, and individuals from surgical subspecialties, the small sample size in these groups limited inferences that could be made.

While our work has identified marked success for the Kern Scholars program when compared to another “pre-K” CDA training pathway, further research will be needed to determine the programmatic elements to which the success can be attributed. It is not clear whether the observed success is a function of the structured academic environment, candidate selection, added mentorship by program leaders experienced in HSR, and/or the cascading peer mentorship philosophy. We do not capture the frequency of papers coauthored by multiple Scholars/awardees, but this could signal enhanced collaboration and be a target for future investigation. It is also not clear whether some trainees may need assistance in different arenas, which could further enrich their careers. For instance, scholars are located at three sites (Minnesota, Florida, Arizona). It is likely that cascading peer mentorship requires a critical mass of individuals, but it is not clear whether it is more important for those individuals to have a common geographic location, similar research, or personal features. Ongoing qualitative assessments will address these questions. We would also like to see future research assess which productivity components are most predictive of future leadership roles. While the traditional research outcomes we assessed are critical for developing a research program, ongoing sustainability and practice integration of HSR research depend upon elevation of clinician-HSR scientists as practice leaders.

The job of the clinician-HSR scientist has simultaneously become more vital and more challenging in recent years. The pace of academic medicine is accelerating, and the breadth and scope of clinical data is increasing exponentially. The primary focus of published literature has shifted from preclinical and mechanistic studies toward public health, quality, epidemiological research [Reference Druss and Marcus25]. Increasing clinical responsibilities and patient complexity tax the demands of already busy clinicians, diminishing motivation for a career in research [Reference Mumma, Chang, Kea and Ranney24]. With more knowledge to review, appraise, synthesize, and apply, the gap between academic research and meaningful changes to clinical practice remains broad. Development of clinician-HSR scientists is essential to address these challenges and improve patient care quality and efficiency.

Conclusions

The approach to and impact of “pre-K’ training pathways for clinician-HSR scientists have not been extensively studied, but are critical to the development and preservation of a robust clinician-HSR scientist workforce. We found the Kern Scholars program was a successful training model for clinician-HSR scientists that demonstrated comparatively greater scholarly productivity than other internal CDA programs. Future studies will explore the first-person experience of Kern Scholars through in-depth qualitative interviews. These data will elucidate perceived strengths and limitations of the current model and opportunities for future iteration. Additional study is needed to further characterize the impact of CDA programs on other metrics of academic success including promotion and leadership roles.

Supplementary material

To view supplementary material for this article, please visit https://doi.org/10.1017/cts.2021.780.

Acknowledgments

The authors would like to thank Ms. Marla Battey for her assistance with calculating the H indices for this study.

This project was supported in part by the National Institute of Allergy and Infectious Diseases of the National Institutes of Health under award number K23AI143882 (PI; EFB), National Institute of Diabetes and Digestive and Kidney Diseases award number K23DK114497 (PI RGM), and the National Heart Lung and Blood Institute under award number K23HL128859 (PI; CCK). The funding sources had no role in study design; data collection, analysis, or interpretation; writing the report; or the decision to submit the report for publication. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the NIH.

Disclosures

The authors have no conflicts of interest to declare.

References

Berwick, DM, Nolan, TW, Whittington, J. The triple aim: care, health, and cost. Health Affairs 2008; 27: 759769. doi: 10.1377/hlthaff.27.3.759 CrossRefGoogle ScholarPubMed
Frogner, BK. Update on the Stock and Supply of Health Services Researchers in the United States. Health Services Research 2018; 53: 39453966. doi: 10.1111/1475-6773.12988 CrossRefGoogle ScholarPubMed
Barry, M, De Groot, E, Baggen, Y, et al. Understanding the broker role of clinician-scientists: a realist review on how they link research and practice. Academic Medicine 2019; 94: 15891598. doi: 10.1097/ACM.0000000000002819 CrossRefGoogle ScholarPubMed
Kluijtmans, M, de Haan, E, Akkerman, S, van Tartwijk, J. Professional identity in clinician-scientists: brokers between care and science. Medical Education 2017; 51: 645655. doi: 10.1111/medu.13241 CrossRefGoogle ScholarPubMed
Roberts, SF, Fischhoff, MA, Sakowski, SA, Feldman, EL. Perspective: Transforming science into medicine: How clinician-scientists can build bridges across research’s “valley of Death”. Academic Medicine 2012; 87: 266270. doi: 10.1097/ACM.0b013e3182446fa3 CrossRefGoogle ScholarPubMed
Goldstein, AM, Blair, AB, Keswani, SG, et al. A Roadmap for Aspiring Surgeon-Scientists in Today’s Healthcare Environment. Annals of Surgery 2019; 269: 6672. doi: 10.1097/SLA.0000000000002840 CrossRefGoogle ScholarPubMed
Tong, CW, Madhur, MS, Rzeszut, AK, et al. Status of early-career academic cardiology: a global perspective. Journal of the American College of Cardiology 2017; 70: 22902303. doi: 10.1016/j.jacc.2017.09.030 CrossRefGoogle ScholarPubMed
Reed, DA, Enders, F, Lindor, R, McClees, M, Lindor, KD. Gender differences in academic productivity and leadership appointments of physicians throughout academic careers. Academic Medicine 2011; 86: 4347. doi: 10.1097/ACM.0b013e3181ff9ff2 CrossRefGoogle ScholarPubMed
Nikaj, S, Roychowdhury, D, Lund, PK, Matthews, M, Pearson, K. Examining trends in the diversity of the U.S. National Institutes of Health participating and funded workforce. The FASEB Journal 2018; 32: 64106422. doi: 10.1096/fj.201800639 CrossRefGoogle Scholar
Personett, HA, Hammond, DA, Frazee, EN, Skrupky, LP, Johnson, TJ, Schramm, GE. Road map for research training in the residency learning experience. Journal of Pharmacy Practice 2018; 31: 489496. doi: 10.1177/0897190017727382 CrossRefGoogle ScholarPubMed
Parker, RB, Ellingrod, V, Dipiro, JT, Bauman, JL, Blouin, RA, Welage, LS. Preparing clinical pharmacy scientists for careers in clinical/ translational research: Can we meet the challenge?: ACCP research affairs committee commentary. Pharmacotherapy 2013; 33. doi: 10.1002/phar.1348 CrossRefGoogle ScholarPubMed
Begley, C, McCarron, M, Huntley-Moore, S, Condell, S, Higgins, A. Successful research capacity building in academic nursing and midwifery in Ireland: An exemplar. Nurse Education Today 2014; 34: 754760. doi: 10.1016/j.nedt.2013.09.016 CrossRefGoogle Scholar
Finney, JW, Amundson, EO, Bi, X, et al. Evaluating the productivity of VA, NIH, and AHRQ health services research career development awardees. Academic Medicine 2016; 91: 563569. doi: 10.1097/ACM.0000000000000982 CrossRefGoogle ScholarPubMed
National Cen ter for Advancing Translational Sciences. Clinical and Translational Science Awards (CTSA) Program [Internet], 2020 [cited Sept 21 2020]. (https://ncats.nih.gov/ctsa) Google Scholar
Robinson, GFWB, Schwartz, LS, Dimeglio, LA, Ahluwalia, JS, Gabrilove, JL. Understanding career success and its contributing factors for clinical and translational investigators. Academic Medicine 2016; 91: 570582. doi: 10.1097/ACM.0000000000000979 CrossRefGoogle ScholarPubMed
Feldon, DF, Litson, K, Jeong, S, et al. Postdocs’ lab engagement predicts trajectories of PhD students’ skill development. Proceedings of the National Academy of Sciences of the United States of America 2019; 116: 2091020916. doi: 10.1073/pnas.1912488116 CrossRefGoogle ScholarPubMed
Yin, HL, Gabrilove, J, Jackson, R, Sweeney, C, Fair, AM, Toto, R. Sustaining the clinical and translational research workforce. Academic Medicine 2015; 90: 861865. doi: 10.1097/ACM.0000000000000758 CrossRefGoogle ScholarPubMed
Decastro, R, Sambuco, D, Ubel, PA, Stewart, A, Jagsi, R. Mentor networks in academic medicine: Moving beyond a dyadic conception of mentoring for junior faculty researchers. Academic Medicine 2013; 88: 488496. doi: 10.1097/ACM.0b013e318285d302 CrossRefGoogle ScholarPubMed
Butler, D. Translational research: Crossing the valley of death. Nature 2008; 453: 840842. doi: 10.1038/453840a CrossRefGoogle ScholarPubMed
Ehlers, SL, Cornelius, KE, Greenberg-Worisek, AJ, et al. A matched cohort examination of publication rates among clinical subspecialty fellows enrolled in a translational science training program. Journal of Clinical and Translational Science 2018; 2: 327333. doi: 10.1017/cts.2018.336 CrossRefGoogle Scholar
Schreiber, WE, Giustini, DM. Measuring scientific impact with the h-index. American Journal of Clinical Pathology 2019; 151: 286291. doi: 10.1093/ajcp/aqy137 CrossRefGoogle ScholarPubMed
Byington, CL, Keenan, H, Phillips, JD, et al. A matrix mentoring model that effectively supports clinical and translational scientists and increases inclusion in biomedical research. Academic Medicine 2016; 91: 497502. doi: 10.1097/ACM.0000000000001021 CrossRefGoogle ScholarPubMed
Loy, BM, Yang, S, Moss, JM, Kemp, DW, Brown, JN. Application of the layered learning practice model in an academic medical center. Hospital Pharmacy 2017; 52: 266272. doi: 10.1310/hpj5204-266 CrossRefGoogle Scholar
Mumma, BE, Chang, AM, Kea, B, Ranney, ML. Career development awards in emergency medicine: resources and challenges. Hauswald M, editor. Academic Emergency Medicine 2017; 24: 855863. doi: 10.1111/acem.13189 CrossRefGoogle Scholar
Druss, BG, Marcus, SC. Growth and decentralization of the medical literature: implications for evidence-based medicine. Journal of the Medical Library Association 2005; 93: 499501.Google ScholarPubMed
Figure 0

Fig. 1. Conceptual framework for developing the bridge between clinician and scientist. Key domains include career development, internal networking and visibility, external networking and visibility, a familiarity with local resources, and expertise in grant writing. The practice and research networks as well as mentors provide necessary infrastructure to support the development of clinician-health services research scientists.

Figure 1

Fig. 2. Study flow diagramCDA, Career development award.

Figure 2

Table 1. Characteristics of the study cohort

Figure 3

Fig. 3. Time to 15th new publication across groups. Upper panel: Time to 15th new publication between Kern Scholars and Kern applicants or other CDA awardees. Lower panel: Time to 15th new first or last author publication between Kern Scholars and Kern applicants or other CDA awardees. Kern Scholars achieved a more rapid time to 15th new publication and 15th new first-/last-author publication than comparator groups (P < 0.001 in both cases). CDA: Career development award.

Figure 4

Table 2. Cox proportional hazards models for predicting time to 15th new publication based on various definitions, adjusted for publications prior to the index date

Figure 5

Table 3. National Institutes of Health Funding as Principal Investigator

Supplementary material: File

Barreto et al. supplementary material

Barreto et al. supplementary material

Download Barreto et al. supplementary material(File)
File 221.6 KB