Grants, writers, and reviewers

In 2021, NCATS released the new UM1 solicitation along with five companion funding opportunities (K12, R25, T32-Pre, T32-Post, and RC2). Of the five companion applications, the required K12 Clinical Scientist Institutional Career Development Program Award application is mandatory for the UM1 review. A successfully funded seven-year UM1 application dictates the funding of all companion applications (with a five-year funding cycle), including the companion K12. Any unsuccessful application can be resubmitted following a successful UM1 application. The K12 calls for applicants to propose innovative institutional research career development programs designed to prepare late-stage postdoctoral fellows and junior faculty scholars in clinical and translational science. The NIH Research Education Program (R25), another UM1 companion, supports short-term research educational activities with a primary focus on research experiences related to clinical and translational research stages: preclinical (T1), clinical (T2), clinical implementation (T3), and public health (T4). The T32-Pre and T32-Post Research Training Grants are two independent programs focused on enhancing the research training of individuals seeking a doctoral degree (PhD) or postdoctoral experience while contributing to a heterogeneous pipeline of clinical and translational scientists. Finally, the High Impact Specialized Innovation Program (RC2) supports nonclinical trial initiatives that develop unique hub capabilities and resources to address critical gaps and/or roadblocks in clinical and translational science at an awarded UM1 CTSA hub. We submitted two RC2 applications: Program for Opioid Management Implementation for Patients in Resource-limited Settings (PROMISE) and Model for Outpatient Delivery of Excellence in Leukemia Treatment (MODEL-T). The principal investigator (PI) of each application was the lead and primary writer of the proposal. For the UM1, the CTSA contact PI designated the Element leaders who led the UM1 Elements/Modules writing task. There were two steps involved in the review process: internal and external reviews. During the internal review, the proposal drafts were exchanged between the writers (proposal PIs) who reviewed each other’s work. Next, the revised proposals were assigned to nine contributors, including members from our External Advisory Board (EAB) and PIs from other CTSAs, who were requested to serve as external reviewers.

Administration (Admin) team

The CCTS Admin team included twelve members: seven Program Managers, two Administrative Assistants, and three Support Staff. All Program Managers worked under the CCTS Executive Director, who liaised between the PIs, writers, reviewers, and the institutional UTHealth Sponsored Projects Administration (SPA) specialists. Each Program Manager was designated to assist one PI. The Program Manager shared the writing instructions, checklists, and timelines with the PI and was responsible for document accuracy, completeness, and adherence to the submission timeline. The scientific documents included the research plan, specific aims page, project narrative, and abstract. The nonscientific documents included the biosketches, letters of intent, letters of support, data management/sharing plans, and other accessory documents as required by the FOA. The Program Managers were also delegated to work on (i) formatting tables, figures, and references; (ii) working with the CCTS Financial Director on the budget; and (iii) assisting the CCTS Executive Director in planning and scheduling, communicating with PIs, and setting deadlines. The Admin Assistants and Support Staff were tasked with scheduling and/or facilitating virtual and/or in-person meetings, printing materials, and providing logistical support during the two-day writing retreat and in-person final review.

Partner institutions

The leaders from our hub’s six partner institutions were critically involved in the proposal development process by becoming actively engaged in proposal writing and/or internal review. At the EAB meeting, the institutional leaders shared “Progress-to-Date” and “Going-Forward” activities, outlining their ongoing and prospective initiatives. Additionally, their insight into other events, including our Grant Writing Retreat, led to the successful integration of the six Key Components: Community and Stakeholder Engagement (C&SE), Dissemination and Implementation (D&I), Diversity, Equity, Inclusion, and Accessibility (DEI-A), Workforce Development (WD), Health Informatics (HI), and Evaluation (EV) into all proposals.

Approach

The new 2021 FOA consisted of separate UM1, K12, T32-Pre, T32-Post, R25, and RC2 applications and required programmatic structural and functional reorganization of the UM1. To accomplish a single “voice” when many writing teams were in play, we: (i) built writing teams using previously established working groups; (ii) set deadlines that were put out by the CTSA contact PI, followed by specific instructions from the CCTS executive director; and (iii) defined roles and responsibilities for all team members. The simultaneous submission of seven proposals culminated in logistical advantages, including: (i) the creation of supporting document templates that were “tailored” to each proposal; (ii) the effective and time-saving delivery of instructions and communications during meetings or via e-mail; and (iii) the successful tracking of all activities by a master timeline. The magnitude of our approach was highlighted when we estimated ∼ 38,616 hours (∼4.4 years!) time was devoted by all participants toward this ambitious project. Using the standard 2,000 hours per year for a full-time employee, the total hours are equivalent to 19 person-years. A diagram of the hub team members involved in this endeavor is shown in Figure 1.

Figure 1. Organization of the CCTS (CTSA hub) team members involved in the simultaneous submission of a UM1 along with 6 companion grants. The CTSA contact PI, Executive Director, and Financial Director planned and directed the development and submission processes. For each grant, a Program Manager was designated to exclusively assist the companion grant PI who served as the team leader. For the UM1, the CTSA contact PI designated the Element leaders who worked closely with the Components (Elements C, D, and E) and/or partner institution leaders (Elements A and B).

Goals

To address the new FOA, we identified three key goals: (i) To successfully transition from a 16 Components organizational structure to five operational units defined as Elements (A–E) and their subunits or Modules; (ii) To integrate our D&I capabilities into all initiatives which required us to redefine D&I as a Key Component; and (iii) To create a strong sense of community and teamwork within our hub where the PIs, writers, reviewers, Admin team, and SPA specialists felt motivated to actively contribute to a joint effort with a focus on collaborative teamwork, leading to the simultaneous submission of seven CTSA proposals. The specific approach used to accomplish these three key goals is shown in Table 1.

First, to transition from Components to Elements, the UM1 FOA was “dissected” to match the existing 16 Components with the newly proposed Elements A-E and their Modules and to build working groups with designated leaders (Table 2). The Elements/Modules leaders were designated by the CTSA contact PI with approval from the existing Component Directors. The working groups received several supporting materials from checklists to specific writing instructions and general formatting guidelines. Similar materials were created for the companion K12, T32-Pre, T32-Post, R25, and RC2 proposals (Table 3). All supporting materials were developed by the CCTS Admin team for the benefit of PIs, writers, and reviewers in an effort to facilitate consistency during writing and review.

Second, prior to this submission, D&I was a part of our C&SE Component. To promote its new centralized role, the existing D&I leader met with the Component Directors to provide individual feedback and promote the integration of D&I. The new FOA initiated us to redefine D&I as an individual Key Component along with DEI-A, C&SE, WD, HI, and EV, which were weaved into all seven grants. This strategy was further sustained by having the D&I, C&SE and HI leaders as MPIs of the UM1 proposal. To promote a collaborative effort and disseminate individual Key Component Directors input, we conducted a two-day in-person writing retreat. The retreat brought together the leaders of all partner institutions, components, and companion grants to discuss potential opportunities to enrich and integrate the six identified key components into current and future efforts.

Finally, the writing retreat also facilitated accomplishing our third goal: to approach the seven-proposal submission as a team effort. Additionally, our EAB meeting, held seven months prior to submission, created a perfect opportunity to provide the time and space needed for autoevaluation, redirection, and integration as a cohesive team. Transitioning from “less siloed to more bonded” was our EAB’s recommendation, which we adapted as the motto going forward. Pursuing the simultaneous submission of seven proposals motivated the leaders of each partner institution, Component, and companion grant to synthesize innovative ways to integrate all our programs and work as a team from start to finish.

Table 1. Goals, challenges, and approaches

BERD = Biostatistics, Epidemiology, and Research Design; C&SE = Community and Stakeholder Engagement; CRUs = Clinical Research Units; D&I = Dissemination and Implementation; DEI-A = Diversity, Equity, Inclusion, and Accessibility; EV = Evaluation; HI = Health Informatics; HLT = Hub Liaison Team; TIN = Trial Innovation Network; ISP = Integrating Special Populations; RKS = Regulatory Knowledge & Support; WD = Workforce Development.

Table 2. Writing teams

BERD = Biostatistics, Epidemiology and Research Design; C&SE = Community and Stakeholder Engagement; CRUs = Clinical Research Units; D&I = Dissemination and Implementation; DEI-A = Diversity, Equity, Inclusion and Accessibility; EV = Evaluation; HI = Health Informatics; LHC = Learning Healthcare; TIN = Trial Innovation Network; ISP = Integrating Special Populations; RKS = Regulatory Knowledge & Support; WD = Workforce Development.

Table 3. Supporting materials (proposal)

Submission timeline & key meetings

The entire grant writing effort was coordinated using a master timeline developed by the Admin team seven months before submission. Figure 2 shows a summarized version of the master submission timeline. The original document was designed using a calendar format. All activities included tasks that were planned on a bi-weekly basis. The tasks were classified as: (i) proposal writing, (ii) draft review, (iii) preparation of required nonscientific documents, and (iv) proposal submission. Preceding the start of each task, the CTSA contact PI and the CCTS Executive Director met to discuss and elaborate on a detailed strategic plan, including specific subtasks and associated deadlines. This was shared with all hub members via a presentation during our weekly mandatory Monday CCTS meeting. This hybrid meeting allowed for the active engagement of all hub members, including PIs, members from our partner institutions in Northeast, South, and West Texas, Component Directors, and the Admin team. Occasional absentees were provided with a video recording. Seven months pre-submission, the well-established CCTS weekly meeting became pivotal as it served as an effective channel for open discussion, feedback, updates, and logistical communication.

Figure 2. Submission timeline. Simplified version of the master timeline used for the simultaneous submission of a UM1 along with 6 companion proposals.

Additionally, the Admin team met virtually with the CCTS Executive Director every two weeks for four months, weekly for three months, and daily for two weeks pre-submission. The Admin meetings were used to share regular updates and obtain feedback to resolve emerging problems and discuss strategies to provide PIs with the best support. This meeting was held in person the week before submission to discuss grant document receipt and grant submission status. The CTSA contact PI met in person with the CCTS Executive Director two times a week during the last four months and weekly with the financial director for two months pre-submission.

During the whole process, the Admin team was committed to answering any questions and providing prompt support. Further, the CCTS Executive Director was available around the clock to meet with PIs, Admin team members, and the institutional SPA specialists with whom regular communication was established for guidance, input, and feedback related to proposal submission.

Evaluation

To collect feedback and improve the future submission process, following the seven-grant submission endeavor, the CCTS Admin and Evaluation teams created an anonymous post-submission CTSA grant writing survey (Qualtrics software, Provo, UT). Participants were asked to select among five defined roles: PI, writer, reviewer, Admin team, and SPA specialist. The five-to-ten-minute survey included about 20 questions/role and was distributed via e-mail (hyperlink and QR code) by the CCTS Executive Director to all 56 participants involved in grant writing, review, and/or submission processes.

Key survey results

1. a. Thirty-four out of fifty-six participants completed the online survey (response rate of 61%). Fifteen percent of the respondents identified themselves as PIs, 32% as writers, 23% as reviewers, and 30% as administrators (Admin team and SPA specialists). The distribution of the survey respondents by grant demonstrated that 22% worked on UM1, 12% on K12, 13% on T32-Pre, 14% on T32-Post, 15% on R25, 12% on RC2-PROMISE, and 11% on RC2-MODEL-T, showing an even distribution across all grants.

2. b. All participants were asked to share three words that reflected their grant preparation experience. The words provided by the survey participants were pooled by role, and the percentage of words with a positive connotation was obtained. The data reflected an overall positive experience in 55% of PIs, 58% of writers, 76% of reviewers, and 47% of administrators.

3. c. Forty-seven percent of the respondents indicated involvement in the previous grant renewal in 2018, indicating that over half (53%) of the participants were new to the current proposal development process.

4. d. When asked for feedback, the survey respondents identified 4 ± 1 as the appropriate number of drafts for all proposals. This was in agreement with the UM1 and K12 writers delivering five drafts: initial, post-solo-external, post-internal, post-external, and final (indicated as drafts 1–5 in Figure 2); and the T32-Pre, T32-Post, R25, and RC2 proposals having four drafts: initial, post-internal, post-external, and final (indicated as drafts 1–4 in Figure 2).

5. e. The time allocated to address the reviewer’s comments ranged from 1 week (fourth and fifth drafts) to 2 weeks (first to third draft). Eighty-four percent of the combined group of writers and reviewers agreed that the time between drafts was appropriate.

6. f. For preferred communication practices, 38% of the writers chose virtual meetings, 32% selected e-mail, 16% chose in-person meetings, and 14% selected phone.

7. g. The writers chose the biosketch template and samples (87%), checklists with deadlines (87%), and the submission timeline (73%) as the top-three most helpful support materials for nonscientific documents.

8. h. As for the scientific document’s supporting materials, 79% of the writers found the reviewer’s feedback most beneficial, followed by the follow-up emails with deadlines (73%), the writing instructions, templates and links (67%), and the use of Google Drive as a draft repository and tracking (50%).

9. i. The final review of all proposals was conducted in person, where printed documents were used to receive edits and comments. Forty percent of the reviewers endorsed this approach as effective, and 60% considered it somewhat effective.

10. j. Seventy-four percent of the PIs, 77% of the writers, and 100% of the reviewers reported that the overall Admin support was very effective.

11. k. Fifty percent of the PIs reported being very satisfied regarding their overall experience, while 33% were satisfied, and 17% were neither dissatisfied nor satisfied.

Lessons learned & recommendations

1. a. Early start. Although our EAB meeting marked the formal beginning of the seven-proposal development process (seven months pre-submission), most planning meetings were initiated about ten months before submission. All survey respondent PIs agreed that the starting time was appropriate.

2. b. Pre-established working groups. Most of the PIs have been working together since the inception of the hub in 2006. This well-established existing infrastructure provided the foundation required for the simultaneous preparation and submission of seven CTSA proposals.

3. c. Weekly CCTS and Admin meetings. The weekly CCTS and Admin meetings served as the reference point to “keep moving forward.” The existent weekly CCTS meeting was used as regular assembly time to receive weekly updates from all leaders including those from our partner institutions. Steered by the hub contact PI and attended by all seven grant PIs, writers, and the Admin team, its goal was to provide strategic direction by sharing plans and establishing deadlines. The Admin meeting was led by the CCTS executive director to discuss logistics and execute scheduled tasks in a timely manner via coordination with the seven Program Managers who were working side-by-side with PIs, writers, and internal reviewers. Both meetings served as an excellent opportunity to keep the teams motivated. Acknowledging the effort of each team member and providing the necessary support was critical to maintaining an upbeat morale, constant engagement, and high productivity during a lengthy, draining, and often frustrating process. Additionally, these meetings allowed for open group discussions, with significant advantages over one-on-one meetings, as they saved time and the whole group benefited from the shared information and knowledge.

4. d. Early engagement of the institutional Grants Administration office (SPA). Engaging with SPA specialists seven months pre-submission to inform them about the multiple proposal submission and build a SPA-inclusive timeline was critical to establish a collaborative-cohesive team. This effort was paramount to dealing with a 48-hour submission delay caused by a system glitch on our institutional submission portal. Additionally, having an internal deadline twelve business days prior to the application due date allowed for time to fix the technical glitch and still submit the applications before the NCATS application due date.

5. e. New budget guidelines. For the UM1, PAR-21-293 requires the budget to be organized by elements/modules. Discussing the new budget structure and thus ensuring that the PIs are fully aware of the change early in the process will save time and effort. From our experience, we learned that facilitating meetings between the Element leaders and the CCTS Financial Director well before beginning the writing process is crucial. Importantly, securing institutional funding could be a significant challenge, especially for new applicants. Therefore, addressing these conversations earlier will allow for a time-efficient budget preparation process.

6. f. Individual point-of-contact Admins. A designated Program Manager working with each element leader/PI aided the Admin team to execute assigned tasks simultaneously while tracking proposal progression. Our survey demonstrated that 50% of the respondent PIs approved this approach.

7. g. One Admin accountable for a few tasks. Designating one Program Manager to a specific proposal allowed for seamless proposal development progress until submission. As previously eluted, the Admin meetings evaluated progress and addressed difficulties. Each Admin team member was accountable for the tasks assigned. Our experience provides validation that only one Admin member should upload documents onto the institutional submission portal. This process allows for consistency with file names, avoids wrong versions and/or duplications of uploaded documents, and provides a single point-of-contact for the contact PIs and SPA specialists. Ideally, a second Admin member who is well aware of all proceedings should be designated as a backup.

8. h. Internal review. All Component Directors of the UM1 and PIs of the companion proposals were invited to serve as internal reviewers. However, this approach resulted in a variable number of reviewers per draft and led to coordination challenges owing to varying schedules around a single deadline. Based on this experience, we recommend limiting one to two internal reviewers per draft.

9. i. Internal submission deadline. As mentioned earlier, having twelve extra business days makes a difference between a chaotic and a less chaotic submission process, especially during a multiple-proposal submission endeavor. In our case, the twelve-day internal deadline allowed for enough time to maneuver a last-minute technical glitch while still keeping the submission prior to the application due date.

10. j. Survey. To avoid overwhelming the survey respondents, questions in our survey did not require reasoning behind the respondent’s choice. Nonetheless, this hampered our ability to capture valuable information for further improvement.