Introduction
Journal clubs organized by either departments or training grants are commonplace in academic medical centers. Most of these journal clubs focus on a single paper per session, and the topic is often related to the focus of the training grant, mentor, or the department and usually goes into great detail to dissect the strengths and weaknesses of the study design. The format of these journal clubs is fairly standard and explores the paper section by section with a critique and discussion of each section with an overall wrap up. While the importance of the paper may be discussed, it is usually within a very narrow focus and does not span consideration of the translation of the discovery all the way through to a therapeutic agent. Rarely is there a succession of presentations with a single theme.
The mission of the National Center for Advancing Translational Science (NCATS) is to catalyze the generation of innovative methods and technologies that will enhance the development, testing, and implementation of diagnostics and therapeutics across a wide range of human diseases and conditions. Reference Fort1 NCATS funds T32 training grants through the Clinical and Translational Science Award Program titled TL1 Clinical Research Training Awards. Our institution is the recipient of one of these grants focused on training predoctoral trainees. The grant funds 10 trainees yearly, including both PhD and MD/PhD trainees in their graduate years. Given the mission of NCATS, we decided to develop a novel Translational Journal Club that would educate TL1 trainees on the process of translation from discovery to dissemination and implementation. Since another focus of NCATS is the promotion of team science, our approach would also promulgate a team-based approach. The trainees were divided into groups of three and tasked with working as a team to choose three papers that would exemplify translational research from T0 to T3/4. Thus, the Translational Journal Club focused on the translation of fundamental discoveries and also working as a team.
Methods
The schedule for the Translational Journal Club is outlined in Table 1. Trainees were divided into teams of three, with one team having four trainees. The teams were purposefully chosen to provide the greatest diversity of the members’ scientific backgrounds. For example, trainees came from the departments of biochemistry and molecular biology, neuroscience, regenerative medicine and cell biology, and biostatistics and epidemiology. All were in a PhD or MD/PhD training program. The teams were charged with working together and selecting the papers for presentation; the rationale for the choice of papers was solely the team/trainees’ responsibility. The three papers to be discussed were distributed to all of the trainees prior to the team’s three presentations. The program directors did not participate in the selection process but approved the selected publications, attended all the presentations, and provided comments when appropriate. In addition, the trainees’ mentors were strongly encouraged to attend their individual mentee’s presentation and provide additional feedback. There were two sessions on how to present scientific/research data to prepare them for presenting to their peers and at national meetings, including the Association for Clinical and Translational Sciences annual meeting.
Table 1. Schedule for the Translational Journal Club. The first five sessions were presentations given by faculty. Weekly presentations were made by a different member of the team
One member of the team presented a publication in successive weeks: (1) a basic science publication that ultimately led to a clinical trial, (2) a corresponding clinical trial publication, and (3) a large-scale clinical trial publication or a publication that provided guidance on the use of the therapeutic clinically (dissemination and implementation). All trainees were encouraged to participate in discussion about the presentations. After all of the team’s presentations, the trainees were given a series of questions pertaining to the three papers and they had 1 week to answer them (Fig. 1). Their answers were graded and returned with comments; the trainees were graded honors/pass/no pass for their answers to the questions and also as their final grade for the Translational Journal Club.
Fig. 1. Translational Journal Club articles evaluation form. The article evaluation form was formatted using REDcap. The trainees were given the following instructions: Attached are five questions that pertain to the series of journal articles that each team presented. For each of the first three questions, there is a yes or no answer. Once you have chosen yes or no, provide 3–4 sentences backing up your choice. You will then have to answer either question 4 or 5. You will have 1 week to turn in your answers. Your final grade for the Translational Journal Club will be based on your answers.
To assess the impact of the Translational Journal Club and the perceived value of participation, the trainees were given an electronic survey to complete (Table 2) at the end of the semester and after they received their final grade for the Translational Journal Club. Survey questions were developed by a multidisciplinary team for the purpose of program evaluation and journal club improvement. Questions were evaluated by the team for clarity, content, and comprehensiveness related to journal club objectives. Survey questions asked the respondents their level of agreement using a Likert Scale (Strongly Disagree = 1; Disagree = 2; Neutral = 3; Agree = 4; Strongly Agree = 5). Students were also given the Net Promoter Score (NPS) survey item, which is commonly used to assess customer satisfactionReference Reichheld2 by assessing the respondent’s willingness to recommend the journal club to a friend/colleague (0–10, where 0 is not at all likely and 10 is extremely likely). In addition, four open-ended items were included to collect more detailed information on: (1) How the journal club differed from the student’s prior journal club experiences?; (2) The student’s primary reason for scoring the NPS “likelihood to recommend” question; (3) The team process for selecting papers; and (4) How the journal club can be improved for next year?. The study was classified as program evaluation/quality improvement by the Institutional Review Board.
Table 2. Translational Journal Club Evaluation. The survey was administered confidentially. Rating Scale: Strongly Disagree = 1; Disagree = 2; Neutral = 3; Agree = 4; Strongly Agree = 5
Results
Our goal was not only to enhance the trainees’ understanding of the processes from T0 to T3/4, but also to challenge them to work outside of their comfort zones. At the same time, we wanted the trainees to appreciate the power of working as a team to explore an area for which they were not familiar and learn from each other. They collaborated on topic identification and paper selection. The trainees chose the topic based on several different rationales. For example, one team chose papers associated with Rituximab because one of the members saw a patient receiving the drug in a clinic that she attended as a requirement for the TL1 program. In another situation, the papers were chosen because of interest in the disease (hepatitis C) due to familiarity with a patient with the disease. A third was chosen for the novelty of the therapeutic approach (siRNA). The trainees often supplemented their presentations with information from sources other than the publications, demonstrating that they embraced the idea of developing an in-depth understanding of both the data in the paper and additional background information. This was a significant factor for the presentations since the trainees had, at best, limited prior knowledge concerning the basic science that ultimately led to the therapeutic. Similarly, they did not have significant clinical knowledge but were able to present the clinical papers in a cogent manner.
To ensure that the trainees comprehended the concepts behind the papers, they were given a series of five questions to answer after each team’s presentation of the three papers. The trainees were able to provide cogent answers to defend their choice of a yes or no answer for each question. However, as noted in the survey (Table 2) (score 3.7), the trainees felt that the questions were not very challenging. Clearly, a more challenging set of questions will need to be developed.
A REDCap survey was administered at the end of the semester, after receipt of the final grade, to assess the trainees’ opinion of the structure and perceived value of the Translational Journal Club. The quantitative results of the survey are shown in Table 2. The questions that dealt with increasing their understanding of team science, dissemination and implementation, and translational science were all rated 4.2 or higher out of 5. Of particular relevance and rated at 5 was that the format promoted discussion.
Certain common anecdotal themes/comments emerged from the survey. When asked if they would recommend the Translational Journal Club to their colleagues on a scale from 0 to 10, with 0 being least likely and 10 being most likely, the average score was 8.7 ± 0.9 (mean ± SD). The trainees all stated that this was a very unique journal club format and they had not experienced a similar format before. Some descriptions of the Translational Journal Club were, “fun, truly enjoyed attending, looked forward to attending, engaging, entertaining, informative and provided a unique opportunity to interact with peers.” The trainees stated that they liked the continuity of the presentations and the discussions; in particular, getting the “big picture” was an important component. They also liked not dissecting the paper as is often done in the typical journal clubs, although there were critiques and discussions. They all gained more appreciation for how a basic science discovery proceeds through clinical trials (translational process) and ultimately becomes a drug, and the time that it takes to develop a new therapeutic. Interestingly, there were no negative comments. There were constructive recommendations about including a clinician’s perspective, exploring topics outside of drug discovery, and revising the final questionnaire.
In addition to the survey, the program director and associate program director met with the trainees at the end of the semester to solicit their thoughts on the Translational Journal Club. A presentation that the trainees found most interesting was the development of the therapeutics for cystic fibrosis, given by one of our clinical faculty whose expertise is in cystic fibrosis and who has participated in many clinical trials. Seeing the development of a basic science discovery into a new drug used in the clinic was most illuminating for them; in particular, they noted the length of time that it took from a fundamental discovery to an approved drug. A consensus suggestion was for a clinician with expertise in the disease area being discussed to join the discussion, so that the trainees could get a better appreciation of the disease that was being treated and the established therapeutic approaches. They also found it interesting to discover how practicing physicians get information about new drugs. A topic suggested for future journal clubs to further enhance their understanding of the T0–T3/T4 process was a lecture about the FDA approval process for new drugs.
Discussion
This paper describes a team-based translational science approach journal club that to our knowledge is unique and previously unreported. Deenadayalan et al.Reference Deenadayalan3 described the “characteristics of successful journal clubs included regular and anticipated meetings, mandatory attendance, clear long- and short-term purpose, appropriate meeting timing and incentives, a trained journal club leader to choose papers and lead discussion, circulating papers prior to the meeting, using the internet for wider dissemination and data storage, using established critical appraisal processes and summarizing journal club findings.” Our Translational Journal Club implemented many of those characteristics. One significant difference was that we used the team approach for choosing the papers rather than a trained journal club leader. In this regard, we feel that the trainees benefited greatly from the Translational Journal Club because they had to demonstrate that the chosen papers would stand up to a critical/rigorous assessment and questions from classmates, the program directors, and/or mentors. We used a questionnaire at the end of the team presentations to assess the summarization and comprehension of the three papers.
We conducted a Pubmed search looking for descriptions of journal clubs that utilized a similar approach. We found several that had an implementation-based approach but none that emphasized a T0–T3/4 approach. One paperReference Austin and Richter4 described a journal club that created a liaison between practicing physical therapy clinicians and academic physicians to impart an evidence-based approach to physical therapy procedures. It utilized a three-step approach of evaluating randomized clinical trials, systematic reviews, diagnostic papers, and clinical guidelines. It appeared that most of the discussions were led by the academic physicians. Another journal club focused on a multidisciplinary approach to burn therapies.Reference Carta5
Overall, the Translational Journal Club was viewed positively by the trainees. They noted gaining a greater appreciation of translational research, the challenges associated with it, and the significant length of time it takes from discovery to approval and implementation of a new therapeutic. The trainees also learned to work together as a team to choose the papers for discussion. Based on the feedback that we received from the trainees, one addition to be made in future years is to enlist participation of a clinician to be part of the team.
In summary, we have devised a novel team-based Translational Research Journal Club that can be easily implemented by other training programs or departments.
Acknowledgments
This project was supported in part by the NCATS of the National Institutes of Health under Grant Numbers TL1 TR001451 and UL1 TR001450. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Disclosures
The authors have no conflicts of interest to disclose.
Open access
