Patient and stakeholder involvement enhances the conduct and applicability of comparative effectiveness research (CER). However, examples of engagement practices for CER leveraging real-world data (i.e., data from routine clinical practice) are scarce. Notably, these studies differ from traditional clinical trials in their technical complexity and minimal prospective data collection, posing unique challenges for stakeholder involvement. This paper describes patient and stakeholder engagement in a CER study of type 2 diabetes (T2D) medications using real-world data from a large administrative claims database.

A Patient and Stakeholder Advisory Group (PSAG) was formed to guide study design, conduct, and dissemination. The PSAG (n = 12) included individuals with T2D, clinicians, health systems leaders, professional society representatives, and a payer representative. Members were surveyed post-study initiation to assess their participation goals and experiences to date.

PSAG members influenced key design and methodological decisions, including cohort selection and adding an aim focused on patient preference elicitation. Survey results indicated high satisfaction with engagement processes and a desire for ongoing involvement. Most PSAG members cited their main goals as impacting the lives of people with T2D and ensuring the research’s relevance to clinicians.

Best practices for engaging stakeholders in CER using real-world data are underdeveloped. Our experience suggests that an inclusive, stakeholder-engaged approach enriches the research process and ensures diverse perspectives are integrated into study design and conduct. Ongoing efforts will focus on assessing long-term engagement outcomes and PSAG member satisfaction.

As posthumous data use policy within the broader scope of navigating postmortem data privacy is a procedurally complex landscape, our study addresses this by exploring patterns in individuals’ willingness to donate data with health researchers after death and developing practical recommendations.

An electronic survey was conducted in April 2021 among adults (≥18 years of age) registered in ResearchMatch (www.researchmatch.org), a national health research registry. Descriptive statistics were used to observe trends in, and multinomial logistic regression analyses were conducted at a 95% confidence interval to determine the association between, willingness to donate some, all, or no data after death with researchers based on the participants’ demographics (education level, age range, duration of using online medical websites, and annual frequency of getting ill).

Of 399 responses, most participants were willing to donate health data (electronic medical record data [67%], prescription history data [63%], genetic data [54%], and fitness tracker data [53%]) after death. Among 397 respondents, we identified that individuals were more likely to donate some data after death (vs. no data) if they had longer duration of using online medical websites (adjusted relative risk ratio = 1.22, p= 0.04, 95% CI: 1.01 to 1.48). No additional significant findings were observed between willingness to donate all, some, or none of their data after death and other demographic factors.

Engaging patients in online medical websites may be one potential mechanism to encourage or inspire individuals to participate in posthumous data donation for health research purposes.

Researchers and policymakers recognize that leveraging data routinely collected in clinical practice can support improved research and patient care. Embedding elements of clinical trials, such as patient identification and trial data acquisition, into clinical practice can enable research access and increase efficiencies by reducing duplication of trial and care activities. Yet, cultural, administrative, and data barriers exist. The Clinical Trials Transformation Initiative (CTTI) developed evidenced-based, multi-partner recommendations to facilitate embedding interventional, randomized trials into clinical practice.

We conducted in-depth interviews (IDIs) with trial designers and implementers to describe their motivations for embedding interventional, randomized trials into clinical practice. Additionally, we aimed to identify barriers and potential solutions to implementing such trials. Interviews were audio-recorded and analyzed using applied thematic analysis.

We conducted 16 IDIs with 18 trial designers and implementers. Motivations for embedding trials into clinical practice included the desire to implement a learning health system and evaluate trials in real-world settings. Barriers to trial implementation focused on limited staff time and availability, the lack of buy-in, and difficulties using electronic health record data. Solutions included minimizing healthcare settings and patient burden, having a sufficient data and research infrastructure in place, and creating a culture change.

The results informed CTTI recommendations to facilitate the design and operation of embedded trials. These recommendations emphasize areas where sponsors and investigators can rethink the design and conduct of clinical trials to ultimately realize an aligned system of research and care.

The expansion of electronic health record (EHR) data networks over the last two decades has significantly improved the accessibility and processes around data sharing. However, there lies a gap in meeting the needs of Clinical and Translational Science Award (CTSA) hubs, particularly related to real-world data (RWD) and real-world evidence (RWE).

We adopted a mixed-methods approach to construct a comprehensive needs assessment that included: (1) A Landscape Context analysis to understand the competitive environment; and (2) Customer Discovery to identify stakeholders and the value proposition related to EHR data networks. Methods included surveys, interviews, and a focus group.

Thirty-two CTSA institutions contributed data for analysis. Fifty-four interviews and one focus group were conducted. The synthesis of our findings pivots around five emergent themes: (1) CTSA segmentation needs vary according to resources; (2) Team science is key for success; (3) Quality of data generates trust in the network; (4) Capacity building is defined differently by researcher career stage and CTSA existing resources; and (5) Researchers’ unmet needs.

Based on the results, EHR data networks like ENACT that would like to meet the expectations of academic research centers within the CTSA consortium need to consider filling the gaps identified by our study: foster team science, improve workforce capacity, achieve data governance trust and efficiency of operation, and aid Learning Health Systems with validating, applying, and scaling the evidence to support quality improvement and high-value care. These findings align with the NIH NCATS Strategic Plan for Data Science.

Type 2 diabetes (T2DM) poses a significant public health challenge, with pronounced disparities in control and outcomes. Social determinants of health (SDoH) significantly contribute to these disparities, affecting healthcare access, neighborhood environments, and social context. We discuss the design, development, and use of an innovative web-based application integrating real-world data (electronic health record and geospatial files), to enhance comprehension of the impact of SDoH on T2 DM health disparities.

We identified a patient cohort with diabetes from the institutional Diabetes Registry (N = 67,699) within the Duke University Health System. Patient-level information (demographics, comorbidities, service utilization, laboratory results, and medications) was extracted to Tableau. Neighborhood-level socioeconomic status was assessed via the Area Deprivation Index (ADI), and geospatial files incorporated additional data related to points of interest (i.e., parks/green space). Interactive Tableau dashboards were developed to understand risk and contextual factors affecting diabetes management at the individual, group, neighborhood, and population levels.

The Tableau-powered digital health tool offers dynamic visualizations, identifying T2DM-related disparities. The dashboard allows for the exploration of contextual factors affecting diabetes management (e.g., food insecurity, built environment) and possesses capabilities to generate targeted patient lists for personalized diabetes care planning.

As part of a broader health equity initiative, this application meets the needs of a diverse range of users. The interactive dashboard, incorporating clinical, sociodemographic, and environmental factors, enhances understanding at various levels and facilitates targeted interventions to address disparities in diabetes care and outcomes. Ultimately, this transformative approach aims to manage SDoH and improve patient care.

In France, decisions for pricing and reimbursement for medicinal products are based on appraisals performed by the National authority for health (Haute Autorité de Santé (HAS)). During the appraisal process, additional real-world evidence can be requested as “Post-Registration Studies” (PRS) when there are uncertainties in evidence that could be resolved by additional data collection. To facilitate PRS planning, a retrospective exploratory analysis was conducted to identify the characteristics of medicinal products associated with a PRS request.

This analysis encompassed all appraisals finalized between January 1, 2016 and December 31, 2021 and compared products for which the appraisal led to a PRS request with those that did not.

Six hundred positive opinions for reimbursement were identified, with a PRS request present in 17 percent (n = 103) of cases. The independent characteristics associated with a PRS request were a mild or moderate clinical benefit score, a major to moderate or minor clinical added value score, previous availability under an early access program, and certain therapeutic areas (neurology, pulmonology, and endocrinology). These findings suggest two different profiles of PRS requests: (i) products for which there is uncertainty in the size of the clinical benefit and (ii) innovative products for which a substantial benefit is expected but uncertainties persist.

These results will assist health technology developers to better anticipate data generation to promptly address uncertainties identified by HAS. It may also help HAS and other assessment agencies to work together to improve postlaunch evidence generation according to the characteristics of the medicinal products.

Spinal muscular atrophy (SMA) is a progressive genetic disorder characterized by muscle weakness ultimately leading to pulmonary impairments that can be fatal. The recent approval of nusinersen, a disease-modifying therapy, substantially changed the prognosis for patients, particularly in children. However, real-world evidence about its long-term effectiveness in adults remains limited. This study aimed to document longitudinal data on motor function, pulmonary function and patient-reported outcome measures of Canadian adults with SMA type 2 and 3 treated with nusinersen.

Outcomes from 17 patients were collected at the Institut de réadaptation en déficience physique de Québec during routine clinical visits over 36 months post nusinersen treatment, using the Hammersmith Functional Motor Scale Expanded for SMA (HFMSE), Revised Upper Limb Module (RULM), 6-Minute Walk Test (6MWT), Children’s Hospital of Philadelphia Adult Test of Neuromuscular Disorders (CHOP-ATEND), SMA functional rating scale (SMAFRS), pulmonary function testing and subjective changes reported by patients.

After 36 months, 9 patients showed motor function improvement. Changes beyond the minimal clinically important difference were seen for four patients on the HFMSE, four patients on the RULM and five patients on the 6MWT. Pulmonary function remained stable for most subjects. Subjective positive changes were reported in 88% of patients and five patients showed improvement in the SMAFRS.

This real-world study demonstrates the positive effects of nusinersen in adults with SMA types 2 and 3. Although stabilizing the patient’s condition is considered therapeutic success, this study shows an improvement in motor function and subjective gains in several patients.

Despite the availability of effective therapies, many patients with major depressive disorder (MDD) develop treatment-resistant depression (TRD).

To evaluate and compare prescribing patterns, contact with specialist services and treatment outcomes in patients with MDD and TRD.

This was a retrospective analysis of linked primary and secondary care National Health Service data in the north-west London Discover-NOW data-set. Eligible patients were adults who had diagnostic codes for depression and had been prescribed at least one antidepressant between 2015 and 2020.

A total of 110 406 patients were included, comprising 101 333 (92%) with MDD and 9073 (8%) with TRD. Patients with TRD had significantly higher risks of suicidal behaviour and comorbidities such as anxiety, asthma, and alcohol or substance misuse (all P < 0.0001). Citalopram, sertraline, fluoxetine and mirtazapine accounted for 83% of MDD and 71% of TRD prescriptions. Use of antidepressant switching (1% MDD, 7% TRD) and combination therapy (1%, 5%) was rare, whereas augmentation occurred more frequently in the TRD group (4%, 35%). Remission was recorded in 42 348 (42%) patients with MDD and 1188 (13%) with TRD (P < 0.0001), whereas relapse was seen in 20 970 (21%) and 4923 (54%), respectively (P < 0.0001). Mean times from diagnosis to first contact with mental health services were 38.9 (s.d. 33.6) months for MDD and 41.5 (s.d. 32.0) months for TRD (P < 0.0001).

There appears to be a considerable difference between treatment guidelines for depression and TRD and the reality of clinical practice. Long-term treatment with single antidepressants, poor remission, and high relapse rates among patients in primary care highlight the need to optimise treatment pathways and access to newer therapies.

The Health Technology Assessment (HTA) process aims to optimize health system funding of technologies. In recent years there has been an increase in what is known as Real-World Evidence (RWE) as a complement to clinical trials. The objective of Health Technology Assessment International’s Latin American Policy Forum 2022 was to explore the utility of incorporating RWE into HTA and decision-making processes in the region.

This article is based on a background document, survey, and the deliberative work of the country representatives who participated in the Forum.

There is a growing interest in the use of Real-World Data / Real-World Evidence in HTA processes in Latin America, although currently there are no specific local guidelines for RWE use by HTA agencies. At present, its use is limited to certain areas such as adding context to HTA reports, the evaluation of adverse events, or cost estimation.

Potential future uses of RWE were identified, including the creation of risk-sharing agreements, the assessment of technology performance in routine practice, providing information on outcomes that are not so easily evaluated in clinical trials (e.g., the identification of specific subpopulations or quality of life), and the estimation of input parameters for economic evaluations.

The participants agreed that there are several areas presenting significant potential to expand the application of RWD/RWE and that the development of normative frameworks for its use could be helpful.

The United States Congress passed the 21st Century Cures Act mandating the development of Food and Drug Administration guidance on regulatory use of real-world evidence. The Forum on the Integration of Observational and Randomized Data conducted a meeting with various stakeholder groups to build consensus around best practices for the use of real-world data (RWD) to support regulatory science. Our companion paper describes in detail the context and discussion of the meeting, which includes a recommendation to use a causal roadmap for study designs using RWD. This article discusses one step of the roadmap: the specification of a sensitivity analysis for testing robustness to violations of causal model assumptions.

We present an example of a sensitivity analysis from a RWD study on the effectiveness of Nifurtimox in treating Chagas disease, and an overview of various methods, emphasizing practical considerations on their use for regulatory purposes.

Sensitivity analyses must be accompanied by careful design of other aspects of the causal roadmap. Their prespecification is crucial to avoid wrong conclusions due to researcher degrees of freedom. Sensitivity analysis methods require auxiliary information to produce meaningful conclusions; it is important that they have at least two properties: the validity of the conclusions does not rely on unverifiable assumptions, and the auxiliary information required by the method is learnable from the corpus of current scientific knowledge.

Prespecified and assumption-lean sensitivity analyses are a crucial tool that can strengthen the validity and trustworthiness of effectiveness conclusions for regulatory science.

Real-world data, such as administrative claims and electronic health records, are increasingly used for safety monitoring and to help guide regulatory decision-making. In these settings, it is important to document analytic decisions transparently and objectively to assess and ensure that analyses meet their intended goals.

The Causal Roadmap is an established framework that can guide and document analytic decisions through each step of the analytic pipeline, which will help investigators generate high-quality real-world evidence.

In this paper, we illustrate the utility of the Causal Roadmap using two case studies previously led by workgroups sponsored by the Sentinel Initiative – a program for actively monitoring the safety of regulated medical products. Each case example focuses on different aspects of the analytic pipeline for drug safety monitoring. The first case study shows how the Causal Roadmap encourages transparency, reproducibility, and objective decision-making for causal analyses. The second case study highlights how this framework can guide analytic decisions beyond inference on causal parameters, improving outcome ascertainment in clinical phenotyping.

These examples provide a structured framework for implementing the Causal Roadmap in safety surveillance and guide transparent, reproducible, and objective analysis.

Ideally, real-world data (RWD) collected to generate real-world evidence (RWE) should lead to impact on the care and health of real-world patients. Deriving from care in which clinicians and patients try various treatments to inform therapeutic decisions, N-of-1 trials bring scientific methods to real-world practice.

These single-patient crossover trials generate RWD and RWE by giving individual patients various treatments in a double-blinded way in sequential periods to determine the most effective treatment for a given patient.

This approach is most often used for patients with chronic, relatively stable conditions that provide the opportunity to make comparisons over multiple treatment periods, termed Type 1 N-of-1 trials. These are most helpful when there is heterogeneity of treatment effects among patients and no a priori best option. N-of-1 trials also can be done for patients with rare diseases, potentially testing only one treatment, to generate evidence for personalized treatment decisions, designated as Type 2 N-of-1 trials. With both types, in addition to informing individual’s treatments, when uniform protocols are used for multiple patients with the same condition, the data collected in the individual N-of-1 trials can be aggregated to provide RWD/RWE to inform more general use of the treatments. Thereby, N-of-1 trials can provide RWE for the care of individuals and for populations.

To fulfill this potential, we believe N-of-1 trials should be built into our current healthcare ecosystem. To this end, we are building the needed infrastructure and engaging the stakeholders who should receive value from this approach.

Clinical trials provide the “gold standard” evidence for advancing the practice of medicine, even as they evolve to integrate real-world data sources. Modern clinical trials are increasingly incorporating real-world data sources – data not intended for research and often collected in free-living contexts. We refer to trials that incorporate real-world data sources as real-world trials. Such trials may have the potential to enhance the generalizability of findings, facilitate pragmatic study designs, and evaluate real-world effectiveness. However, key differences in the design, conduct, and implementation of real-world vs traditional trials have ramifications in data management that can threaten their desired rigor.

Three examples of real-world trials that leverage different types of data sources – wearables, medical devices, and electronic health records are described. Key insights applicable to all three trials in their relationship to Data and Safety Monitoring Boards (DSMBs) are derived.

Insight and recommendations are given on four topic areas: A. Charge of the DSMB; B. Composition of the DSMB; C. Pre-launch Activities; and D. Post-launch Activities. We recommend stronger and additional focus on data integrity.

Clinical trials can benefit from incorporating real-world data sources, potentially increasing the generalizability of findings and overall trial scale and efficiency. The data, however, present a level of informatic complexity that relies heavily on a robust data science infrastructure. The nature of monitoring the data and safety must evolve to adapt to new trial scenarios to protect the rigor of clinical trials.