Today's healthcare decision makers, including payers, patients, and doctors, face an increasingly large choice of new drug therapies and a greater need to understand the comparative safety, effectiveness and costs of available treatment options (1;Reference Mestre-Ferrandiz, Sussex and Towse2). Meanwhile, manufacturers are faced with a disconnect between the evidence requirements of regulatory agencies and payers, making it difficult to meet all demands. In the United States, drug efficacy trials for U.S. Food and Drug Administration (FDA) approval are usually not representative of real-world patient populations, often lacking appropriate comparators and assessing outcomes of limited relevance to payers (Reference Oderda and Sifford-Wilson3–Reference Chokshi, Avorn and Kesselheim6). FDA trials can offer some insights into the relative efficacy of different drugs studied, however, regulations do not require comparative effectiveness research (CER) for new drugs (Reference Edwards7), and the utility of results to decision makers remains unclear for studies that do compare alternative treatments (Reference Chokshi, Avorn and Kesselheim6). Evidence requirements, including appropriate comparators, are also inconsistent across European regulators, payers, and health technology assessment (HTA) bodies (Reference Schoonveld8–Reference Kleijnen, George, Goulden and d'Andon11). While European HTA agencies still regard randomized controlled trials (RCTs) as the gold standard, many recognize some use for non-RCT data in HTA (Reference Wilsdon and Serota12). As evidence requirements shift, the cost of bringing a new drug to market continues to rise (Reference Mestre-Ferrandiz, Sussex and Towse2). Simply adding CER/relative effectiveness (RE) studies on top of regulatory evidence is most likely unsustainable, and many worry the escalating costs of drug development may soon limit innovation (1). According to Schneeweiss et al. (Reference Schneeweiss, Gagne, Glynn, Ruhl and Rassen5), the increasing demand from payers for CER evidence may slow the process of bringing new drugs to market, unless manufacturers can build in modifications to anticipate CER scenarios in the future. Given economic pressure on payers and providers to allocate health resources effectively and a growing demand for stakeholder-driven evidence, CER in the United States and RE research in Europe may play a critical role for future drug development to ensure coverage and competitive pricing of new drugs (1;Reference Oderda and Sifford-Wilson3–Reference Schneeweiss, Gagne, Glynn, Ruhl and Rassen5;Reference Nelleson, Birnbaum and Greenberg13).
We refer to CER and RE as defined by the Institute of Medicine (Reference Sox and Greenfield14) and High Level Pharmaceutical Forum (15), respectively, but use these terms interchangeably as CER/RE throughout this study. We also consider the role of patient-centered outcomes research (PCOR), tradeoffs between different CER/RE designs, and the reliability of existing methods for adjusting for bias in observational research, which remains a major challenge to its acceptability by payers (Reference Schneeweiss, Gagne, Glynn, Ruhl and Rassen5;Reference Watkins16–Reference Leung, Halpern and West18).
Our aim was to gather perspectives from a diverse group of payers on the rapidly evolving fields of CER/RE. In particular, we focused on the acceptability and use of CER/RE evidence for access decisions on new drugs in the future healthcare environments of the United States and Europe. We considered study designs and methods used or potentially useful for CER/RE, the extent to which they are used within the current paradigm of coverage decision making for new drugs, and how current trends might lead to greater (or less) acceptability by payers in 2020. We grounded our questions in hypothetical reference cases of products we expect payers to encounter frequently in the future to help identify conditions that might modify payers' evidentiary requirements.
BACKGROUND
Payers may often desire head-to-head comparisons and distrust of pharmaceutical sponsored registries, observational cohort studies, or modeling to generate relevant real-world evidence for decision making (Reference Wang, Halbert, Baerwaldt and Nordyke17–Reference Dean, Ko and Graff19). Furthermore, the relative value and weight of different evidence used by payers in decision making varies and is not always transparent. Costs and evidence certainty can be “more likely than efficacy and safety data to influence level of formulary access” (Reference Dean, Ko and Graff19). Manufacturers must struggle with the inconsistent demands from different payers, as well as the widening disconnect between regulatory and payer evidentiary requirements.
Evidence requirements of European regulators are often inconsistent with those of payers and health technology assessment (HTA) bodies. The European Medicines Agency (EMA) may accept placebo-controlled or non-inferiority trials to demonstrate the quality, safety and clinical efficacy of their new medicines for marketing authorization applications (Reference Schoonveld8;Reference Garattini and Bertele'9). Payers, on the other hand, generally demand evidence that the medicine under evaluation is more effective than alternative treatment options. Regulatory agencies and payers can also express differences in opinion about endpoints, with the former accepting surrogate endpoints as valid markers and the latter preferring clinical outcomes and accepting surrogate outcomes only if clinically relevant and/or validated (Reference Shah, Mestre-Ferrandiz, Towse and Smyth10;Reference Kleijnen, George, Goulden and d'Andon11;Reference Levy20). Evidence suggests that many HTA bodies and payers in Europe prefer head-to-head randomized trials (Reference Levy20;Reference Sorensen21). However, it also recognized that there is a place for other, non-RCT data sources (Reference Wilsdon and Serota12).
In a recent survey regarding pragmatic clinical trials (PCTs), U.S. payers demonstrated limited use and varying knowledge of PCTs; however, they were interested in the real-world evidence benefits and anticipated increased acceptance of PCTs in future (Reference Ratner22). Given payer preferences to see trial data validated in their broader, real-world patient populations, PCTs (which can be thought of as “real-world” RCTs) may be appropriate to address questions relevant to payers while still retaining high internal validity.
The evolution of Accountable Care Organizations (ACOs) may also have major implications for CER evidence demands, with a variety of new care delivery models appearing that align payments with benefits and introduce measures focused on quality of care and the reduction of overall costs. Many anticipate that payers will want to see evidence of economic value in addition to clinical value, as compared to existing alternatives (Reference Nelleson, Birnbaum and Greenberg13;Reference Brixner and Watkins23). Although costs are excluded from the Patient Protection and Affordable Care Act of 2010, the establishment of the Center for Medicare and Medicaid Innovation placed a clear focus on improving quality and lowering costs by evaluating innovative payment and care delivery models.
Despite existing payer studies and advances in CER/RE methods, the extent to which payers anticipate how they will use CER/RE for future access decisions remains unclear. Our study attempts to address this unknown by grounding our discussion in hypothetical cases and examines specific methodologies often associated with CER/RE (e.g., indirect comparisons, cluster RCTs) to obtain perspectives about their usefulness. Importantly, however, we look to the near-future, anticipating how the environments in the United States and Europe are likely to change between now and 2020, how changes will influence payer information needs, and how CER/RE evidence may be accepted to fulfill those information needs.
METHODS
Recruitment and Selection of Key Informants
Key informants were identified from existing professional contacts and online executive/organizational leadership rosters and recruited by means of email to participate in the project. We sought senior officials representing large payers, pharmacy benefits managers (PBMs), HTA groups, and pricing and reimbursement bodies in the United States (as this represents a sizeable portion of current pharmaceutical company revenue) and Europe (in recognition of the global environment in which companies operate), who played a role in coverage and/or pricing decisions or policies in their organizations. Our initial set of European countries included the five major markets (France, Italy, Germany, Spain, and the United Kingdom) and two additional countries (the Netherlands and Sweden) where we found evidence of systematic re-evaluations based on additional data collected while a drug is used in practice (Reference Sorensen21). In the United Kingdom, two HTA bodies (the National Institute for Health and Care Excellence and the Scottish Medicines Consortium) operating in different jurisdictions were considered in the study as both consider cost effectiveness data in their decision making. We also recruited U.S. key informants for integrated health systems and ACOs.
Online Survey and Key Informant Interviews
To guide the development of our survey and interview plan, we reviewed key literature in CER/RE methods (Table 1). Using the Qualtrix© online survey platform, we developed a brief survey instrument for our interviewees to complete before their interviews, assessing current use of CER/RE evidence and potential trends that might influence its use for decision making by 2020. A glossary of key terms accompanied the survey as a resource for key informants. We developed a semi-structured interview guide that could be tailored to individuals' survey responses. For both the United States and Europe, interviews were conducted by a pair of researchers (one interviewer and one note taker). Interviews were also recorded and transcribed.
Table 1. Literature Review
Pilot
The survey was piloted for duration, ease of use, interpretability and appropriateness of questions, among three professional contacts in the United States and Canada and two in Europe who are experts in HTA, CER/RE and coverage decision making. Pilot participants then completed 60- to 90-minute cognitive interviews to share personal interpretations of the survey and semi-structured interview questions and help identify contextual elements of the hypothetical references cases (Table 2) that might create conflict or challenges for payers. Cases were revised and became the center of the semi-structured interview based on this feedback. Survey items were also modified, separated out by study designs, methods and sources of information in a series of matched “current” and “future” questions.
Table 2. Description of Hypothetical Reference Cases
Analysis
Survey responses were tabulated, however individual-level changes in paired “current” and “future” questions were documented and incorporated into the semi-structured key informant interviews. We conducted a systematic content review, similar to a directed content analysis (Reference Hsieh and Shannon24). Instead of in-depth coding of interviews, we sought to extract explicit content in response to the questions raised during the key informant interviews. A single reviewer extracted relevant information specific to an existing framework which followed the construct of the semi-structured interview guide. The team reviewed the completed framework, discussed and resolved any disagreements.
RESULTS
Fourteen key informants from five countries participated (Table 3); of these, two completed only the interview, although their perspectives on the survey questions were gathered in the interview itself. Survey responses are available online (Supplementary Tables 1–3).
Table 3. Characteristics of Payer Key Informants, United States and Europe
Overall, RCTs were payers' preferred source of CER/RE evidence, both generally and in response to the hypothetical reference cases. When asked to define CER in their own words, U.S. informants emphasized randomization and the use of appropriate comparators as essential aspects of CER. Efficacy RCTs were considered a potential type of CER study, depending on their usefulness, such as active-comparator trials with head-to-head comparisons of relevant clinical alternatives, or placebo-controlled trials that could be combined with other data, for example, in indirect comparisons. In general, a broad range of observational and experimental study types fit under “CER.” No one found CER and PCOR to conflict, but suggested CER could be conducted without being patient-centered (e.g., comparing the effectiveness of treatment alternatives for glycemic control may be important to physicians and payers, whereas diabetes patients may be more concerned with quality of life, vascular conditions, or mortality). When asked to define RE in their own words, European key informants described similar meanings: in general, how a technology performs in practice compared to an appropriate comparator. Some pointed out that RE is the “incremental effectiveness” part of an HTA.
In the survey, payers favored direct head-to-head comparisons as “extremely relevant” to their decision making (Supplementary Table 1). The inclusion of meaningful endpoints and study populations representative of “real world” patients were “very” or “extremely” relevant to U.S. informants, and all European informants indicated that a study population being representative of real-world patients was “very” or “extremely” relevant. Table 4 summarizes high-level take-away points from the key informant interviews in response to the open-ended questions regarding study designs, methods and data sources which preceded the hypothetical reference cases. By 2020, RCTs will still be preferred for initial market access in the United States and Europe, despite improvements in observational research methods and a growing data-rich environment. There may be a larger role for observational study designs following initial approval, to support decisions about formulary placement and premium pricing in competitive markets. European informants also anticipated the continuing growth of adaptive trials and use of study data from registries, and considered patient reported outcomes (PROs) to be only one component of RE.
Table 4. High Level Findings from Non–Case-Specific Interview Questions
aFor purposes of the project, we defined “big data” as a repository of a voluminous amount of unstructured or semi-structured data on diverse care settings that can be speedily accessed and the use of predictive analytics and data mining to undercover hidden patterns and unknown correlations.
Key informants expected drugs resembling Case 1 (Table 2) to be more common by 2020. For this case, they most often asked for an RCT or PCT in a broader population comparing outcomes of patients without a biomarker to other alternatives. Indirect comparisons, retrospective analyses of payer data, registries, or rigorous prospective observational studies were also discussed. Among U.S. key informants, the majority expressed concern about the potential for indication creep (or diffusion of the use of the new drug into broader, off-label populations) and would attempt to use available policy tools to control uptake and diffusion. European key informants focused on appropriate outcomes, that is, the acceptance of surrogate outcomes may depend on the drug's place in therapy. Consensus was that manufacturers should be responsible for generating post-launch RE evidence in Europe, although there was recognition that in some cases there are insufficient incentives to do so. Coordination with healthcare services was also mentioned. Half of the U.S. key informants would expect strong evidence to come from manufacturers, three would analyze their own data, and one would consider a partnership akin to coverage with evidence development.
Regarding Case 2 (Table 2), public and private U.S. informants emphasized that convenience was not considered medically necessary. (According to Thompson v. Western States Medical Center, 535 U.S. 357, 2002 and Section 1862(a)(1)(A) of the Social Security Act (the Act), 42 U.S.C. 1395y(a): the Secretary may only pay for items and services that are “reasonable and necessary for the diagnosis or treatment of illness or injury or to improve the functioning of a malformed body member.”) Conventional, less costly alternatives would likely be preferred, without robust evidence that linked the new drug to improved clinical outcomes or (in some cases) adherence as a surrogate. This view was echoed amongst European respondents. Three U.S. and two European informants might accept registry data or other prospective observational studies instead of randomized trials in therapeutic areas where adherence was more clinically important for disease management, where disadvantages of existing alternatives (such as tolerability or patient burden) were well documented, or where the price differential between new and existing drugs was not as large. However, a key concern was that important differences would exist between the patients on different alternatives at the time of product launch, restricting the ability to compare in observational data, because the data may reside with different entities. Two U.S. key informants would look at their own claims to generate the appropriate evidence. European key informants may also ask for cost effectiveness analyses, stated preference studies, and RCTs that sought to capture adherence.
Key informants were familiar with products resembling Case 3 (Table 2) and typically stated that underpowered post hoc subgroup analyses were not convincing. Drug price, availability of generics, and unmet need were important considerations. Key informants would prefer RCT data with prospectively identified subgroups that were sufficiently powered to consider premium pricing. However, several key informants expressed a willingness to consider peer-reviewed, prospective observational cohort studies, under certain circumstances. Other key informants expressed concerns about manufacturers providing the evidence, publication bias and underpowered subgroup analyses.
Case 4 (Table 2) was considered rare, and perspectives varied, due in part to the ethical and design challenges raised by a standard of care (surgery) of questionable effectiveness. Although two European and four U.S. key informants “ideally” wanted to see a head-to-head trial, most informants recognized that the availability of evidence on the surgical procedure would influence their demands for randomization, and that an effectiveness study of the surgery might be needed first (which could be publicly funded in Europe). One U.S. informant suggested modeling or meta-analyses, and one European informant suggested identifying patient subgroups where the new treatment would be more appropriate than a surgical procedure.
Although responses to hypothetical reference cases were similar to U.S. informants, European informants expect to use more national registry data, more data collected post-launch/HTA decision (to reduce uncertainty of pre-launch findings) and potentially from other jurisdictions. Several barriers may influence the availability of RE evidence, including: lack of funding for RE from healthcare providers, lack of incentives for industry to collect RE evidence (in many countries the access decision is made at launch), and comparators being inappropriate and different across countries. Case 1 (companion diagnostic) and Case 3 (post hoc subgroup analysis) sparked several observations about the future of personalized medicine. More targeted therapies were expected by 2020. U.S. payers in particular anticipate a broader picture of personalized medicine enabled by the availability of clinically rich individual level payer data, analyzed in prospective real-world studies to identify observable factors that can guide decision making.
DISCUSSION
We heard from payers that RCTs will still be expected for establishing causality and providing evidence of therapeutic effect for initial market access in both the United States and Europe, even with improved standards and data quality. Payer preferences for appropriate comparators and information on real-world patients support greater use of pragmatic trials before market release in future drug development. However, whether or not PCTs can become an efficient design by leveraging existing data is somewhat uncertain. Many U.S. payers anticipate a significant growth in the availability of electronic data for CER evidence generation, with implications for multi-stakeholder sharing of the burden of CER/RE evidence development. However, European payers expressed concerns that poor health system infrastructure may be a barrier to electronic health records reaching their full potential. On the other hand, investments by payers, health care systems and countries in electronic health data and registries may facilitate delayed design studies and retrospective claims analyses to examine use in covered populations and refine formularies (for example, delayed design studies might resemble randomized or staggered implementation of policies within a covered population to compare drug usage and health outcomes). We might also anticipate a growing role of observational study designs following initial approval to support decisions about formulary placement and premium pricing in competitive markets. As evidenced by reactions to the hypothetical reference cases, although payers/HTA bodies may remain circumspect about observational studies or data mining conducted by the pharmaceutical industry, such studies may be acceptable for certain products, under certain contexts. Like Wang et al. (Reference Wang, Halbert, Baerwaldt and Nordyke17), we found that (large) U.S. payers express comfort with their own claims data, which are increasingly being enriched with clinical and laboratory data to provide good quality data for observational research. The use of novel designs may also reduce concerns about bias in peer-reviewed observational studies on newly marketed drugs (Reference Schneeweiss, Gagne, Glynn, Ruhl and Rassen5). For example, inception cohorts or new user designs may help avoid biased comparisons of new users versus prevalent users of a drug, who may differ in several important ways (Reference Schneeweiss25).
Manufacturers may benefit from investing in infrastructure for post-market follow-up that anticipates greater competition over time. A panel of nine public and private U.S. payers involved in formulary decision making preferred long term cost and outcome data and acknowledged the appropriateness of well-designed observational studies to complement RCT findings in broader populations (Reference Oderda and Sifford-Wilson3). These same payers also recommended that manufacturers validate trial findings in real world studies to show payers that outcomes in their plans' membership reflect RCT findings (Reference Oderda and Sifford-Wilson3). Epstein suggests more collaboration in analyzing findings, and shared access to raw data including those from clinical studies (Reference Epstein4). Schneeweiss et al. (Reference Schneeweiss, Gagne, Glynn, Ruhl and Rassen5) suggest a multifaceted approach given the lack of early observational data and limitations of electronic health records (EHRs), registries, and claims data. They propose sequential cohort designs, the extension of phase III and IV trials (i.e., patients are followed for longer, perhaps using an observational design), indirect comparisons of efficacy, and the use of modeling and trial simulation. Like several of our key informants, they see an accelerated trend toward third-party generation of CER/RE evidence that leverages longitudinal claims and EHR data from payers and healthcare systems.
In a future healthcare environment most conducive to CER/RE, where PCT enrollment is facilitated by EHR enriched environments, access to patient registries and manufacturer partnerships with advocacy groups or healthcare systems, the drug development paradigm might rely heavily on PCTs with extended observational follow-up, including sequential cohort studies and registries. However, a less integrated scenario where electronic data offer some but not comprehensive benefits, may be more likely. Still, new platforms for research offer partnership opportunities for manufacturers seeking more efficient ways to conduct RCTs. Health systems investments and population-based registries offer a basis for efficiently identifying trial populations and collecting clinical trial data using routine data sources. The implications for evidence collection during drug development may depend heavily on whether a product is being marketed broadly in a crowded market but seeking special status, versus being marketed to a targeted population, but with the potential to benefit other groups who are not on label. A major challenge for companies in the competitive “me too” scenario (Case Reference Epstein 4 ) is the market competition that cannot be predicted at start of a product portfolio: companies often end up in a crowded market, but not by design. Improving the efficiency of evidence generation while maintaining validity is key, but a product portfolio may need to allow a certain amount of flexibility to adapt to emerging competitors (e.g., add study arms or other outcomes), whereas historically many manufacturers might have scrambled toward the end of a product's evidence development to deal with this competition. Companies may also want to consider the future of exploratory research and how the decision point regarding a product's continuance or discontinuance within a company's development portfolio might shift.
The capabilities of European countries and the United States currently differ; how each healthcare environment evolves between now and 2020 has implications for healthcare systems capabilities and data availability, which may in turn influence the ability to generate CER/RE evidence in different regions. For example, Europeans healthcare systems have longitudinal data on patients over the course of care, whereas the U.S. healthcare system is disconnected, but a wealth of electronic data from EHRs and other databases exist. This may explain the greater emphasis on registries among our European informants in contrast to the demand for randomized studies from our U.S. informants.
Surprisingly, there was little mention by key informants about an increased role of coverage with evidence development, except in the pilot discussions. However, we heard repeatedly from the U.S. payers we interviewed that they will increasingly attempt to restrict access through prior authorization, step-therapy, or other policy levers. As data accumulate or classes become crowded, they will re-evaluate outcomes in their own data, a re-iterative process aimed at refining formularies. Some European informants thought the evidence received at launch would depend on whether the EMA changes its approval process. The studies payers accept may depend on additional evidence the EMA is seeking post-launch as part of an adaptive licensing process. Should a product get early marketing authorization, there may be a political pressure to provide the product within that country's health system; this may require greater use of post-launch studies, perhaps linked to some form of conditional reimbursement.
Limitations
Our study is limited by a small sample size. Only six of fourteen European key informants agreed to participate. Perhaps the inclusion of more advisors to decision makers rather than officials representing the selected decision-making bodies could have increased the response rate. In addition, some European countries included in the final sample are over-represented (two experts from Sweden and three from the United Kingdom, although in the latter informants were from different jurisdictions). Those countries rely primarily on cost effectiveness assessments to make coverage decisions which is not the case in other countries (i.e., Germany and Italy, which not included in the study due to non-response). Further studies including broader spectrum of European countries might provide additional insights on the future use of RE.
CONCLUSION
Our study offers insights into the evidence demands manufacturers might anticipate for market access in future healthcare environments. In the most likely future environment, manufacturers will still need to go distinctly for regulatory approval. However, more pragmatic Phase III trial designs may be increasingly feasible and affordable. Next steps could involve engaging payers, manufacturers and regulatory agencies in discussions regarding key methodological tradeoffs. From a manufacturer perspective, greater emphasis on post launch investments may first require more clarity on what returns on investment they can expect. Payers may contribute by communicating back to manufacturers clear ways in which evidence will have implications for decisions.
SUPPLEMENTARY MATERIAL
Supplementary Tables 1–3 http://dx.doi.org/10.1017/S0266462315000203
CONFLICTS OF INTEREST
The authors would like to acknowledge funding support for this project from the following sponsors: Amgen, Eli Lilly & Company, GlaxoSmithKline, Novartis, and Sanofi-Aventis. The authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed.
Open access
