Objectives: The evaluation of telemedicine from the patient's point of view has focused on the patient pathway and patient satisfaction. However, research in this field does not provide us with the means to assess a patient's perception of the procedure if their reasoning ability is impaired. In this study, we use direct observation of a patient's behavior and mood to assess their perception of an oral teleconsultation procedure.

Methods: This study has been conducted in the context of a pilot project using an asynchronous teleconsultation to improve access to dental care for the dependent elderly, disabled people, and prisoners, some of whom may be cognitively impaired. We use a direct observation form consisting of five behavioral variables and eight affect variables to reflect the patient's experience of the oral teleconsultation procedure.

Results: A total of 135 patients were evaluated; 10 refused the procedure. Psychotic patients (n = 33) had a somewhat negative experience during the oral teleconsultation procedure. Patients who were not psychotic had a positive experience; this decreased as we moved from the autonomous to the semi-autonomous and then to the dependent sub-group. Some gender differences were also noted.

Conclusions: Improving evidence on evaluating the acceptance of the cognitively impaired is required to improve the technology development process so that it can be translated into an improved patient experience and adherence. Although the study was specifically focused on teledentistry, the approach described in this study could be adapted to other forms of teleconsultation.

Objectives: The headroom method was introduced for the very early evaluation of the potential value of new technologies. It allows for establishing a ceiling price for technologies to still be cost-effective by combining the maximum effect a technology might yield, the maximum willingness-to-pay (WTP) for this effect, and potential downstream expenses and savings. Although the headroom method is QALY-based, not all innovations are expected to result in QALY gain.

Methods: This study explores the feasibility and usefulness of the headroom method in the evaluation of technologies that are unlikely to result in QALY gain. This will be illustrated with the diagnostic trajectory of complex pediatric neurology (CPN).

Results: Our headroom analysis showed a large room for improvement in the current diagnostic trajectory of CPN in terms of diagnostic yield. Combining this with a maximum WTP value for an additional diagnosis and the potential downstream expenses and savings, resulted in a total headroom of €15,028. This indicates that a new technology in this particular diagnostic trajectory, might be cost-effective as long as its costs do not exceed €15,028.

Conclusions: The headroom method seems a useful tool in the very early evaluation of medical technologies, also in cases when immediate QALY gain is unlikely. It allows for allocating healthcare resources to those technologies that are most promising. It should be kept in mind, however, that the headroom assumes an optimistic scenario, and for that reason cannot guarantee future cost-effectiveness. It might be most useful for ruling out those technologies that are unlikely to be cost-effective.

Objectives: The aim of this study was to review 5 years of activity from a new system devised by the National Institute for Health and Care Excellence (NICE), for assessing medical devices and diagnostics aimed at identifying and speeding adoption of technologies with clinical and cost advantages, compared with current practice in the United Kingdom healthcare system.

Methods: All eligible notified technologies were classified using the Food and Drug Administration and Global Medical Device Nomenclature nomenclatures. Decisions about selecting technologies for full assessment to produce NICE recommendations were reviewed, along with the reasons given to companies for not selecting products.

Results: Between 2009 and 2014, 186 technologies were notified (46 percent therapeutic and 54 percent diagnostic). Thirty-nine were judged ineligible (no regulatory approval), and 147 were considered by an independent committee. Of these, eighty (54 percent) were not selected for full assessment, most commonly because of insufficient evidence (86 percent): there were uncertainties specifically about benefits to the health service (54 percent), to patients (39 percent), and about cost (24 percent). The remaining 67 were selected and assessed for Medical Technology guidance (52 percent) (noninferior and/or lower cost consequences than current practice), for Diagnostics guidance (43 percent) or other NICE recommendations about adoption and use. Classifying technologies by two different systems showed no selection bias for any technology type or disease area.

Conclusions: Identifying new or under-used devices and diagnostics with potential benefits and promoting their adoption is important to health services in the United Kingdom and worldwide. This new system offers a means of fostering both uptake and further research. Lack of research data on new products is a major obstacle to evaluation.

Objectives: The evaluation of public health interventions poses some challenges. As a consequence, health technology assessment (HTA) methods for public health interventions (PHI) have to be adapted. This study aimed to summarize the available guidance on methods for HTA of PHI.

Methods: We systematically searched for methodological guidance on HTA of PHIs. Our focus was on research synthesis methods to evaluate effectiveness. Relevant information was synthesized narratively in a standardized way.

Results: Only four guidance documents were identified specifically for HTAs of PHI. The approaches used for HTAs of PHIs are broader and more flexible than those for medical interventions. For this reason, there is a tendency to identify the intervention components and context factors that influence the effectiveness and transferability of an intervention rather than to assess its effectiveness in general. The details in the guidance vary without justification. Unjustified heterogeneity between the different guidance approaches is most pronounced for quality assessment, assessment of applicability, and methods to integrate qualitative and quantitative evidence. Descriptions for the assessment of integrity, heterogeneity, sustainability, context factors, and applicability are often vague.

Conclusions: The heterogeneity in approaches indicates that there is currently no consensus on methods to deal with the challenges of the PHI evaluations. A possible explanation for this may be that the methods are not sufficiently developed, and advantages and disadvantages of a certain method in relation to the research question (e.g., broad/focused) have not yet been sufficiently evaluated.

Objectives: Zika virus (ZikaV) is currently one of the most important emerging viruses in the world which has caused outbreaks and epidemics and has also been associated with severe clinical manifestations and congenital malformations. Traditional approaches to combat the ZikaV outbreak are not effective for detection and control. The aim of this study is to propose a cloud-based system to prevent and control the spread of Zika virus disease using integration of mobile phones and Internet of Things (IoT).

Methods: A Naive Bayesian Network (NBN) is used to diagnose the possibly infected users, and Google Maps Web service is used to provide the geographic positioning system (GPS)-based risk assessment to prevent the outbreak. It is used to represent each ZikaV infected user, mosquito-dense sites, and breeding sites on the Google map that helps the government healthcare authorities to control such risk-prone areas effectively and efficiently.

Results: The performance and accuracy of the proposed system are evaluated using dataset for 2 million users. Our system provides high accuracy for initial diagnosis of different users according to their symptoms and appropriate GPS-based risk assessment.

Conclusions: The cloud-based proposed system contributed to the accurate NBN-based classification of infected users and accurate identification of risk-prone areas using Google Maps.

Objectives: Traditional economic evaluations for most health technology assessments (HTAs) have previously not included environmental outcomes. With the growing interest in reducing the environmental impact of human activities, the need to consider how to include environmental outcomes into HTAs has increased. We present a simple method of doing so.

Methods: We adapted an existing clinical-economic model to include environmental outcomes (carbon dioxide [CO2] emissions) to predict the consequences of adding insulin to an oral antidiabetic (OAD) regimen for patients with type 2 diabetes mellitus (T2DM) over 30 years, from the United Kingdom payer perspective. Epidemiological, efficacy, healthcare costs, utility, and carbon emissions data were derived from published literature. A scenario analysis was performed to explore the impact of parameter uncertainty.

Results: The addition of insulin to an OAD regimen increases costs by 2,668 British pounds per patient and is associated with 0.36 additional quality-adjusted life-years per patient. The insulin-OAD combination regimen generates more treatment and disease management-related CO2 emissions per patient (1,686 kg) than the OAD-only regimen (310 kg), but generates fewer emissions associated with treating complications (3,019 kg versus 3,337 kg). Overall, adding insulin to OAD therapy generates an extra 1,057 kg of CO2 emissions per patient over 30 years.

Conclusions: The model offers a simple approach for incorporating environmental outcomes into health economic analyses, to support a decision-maker's objective of reducing the environmental impact of health care. Further work is required to improve the accuracy of the approach; in particular, the generation of resource-specific environmental impacts.

Objectives: To investigate the use, challenges and opportunities associated with using patient-reported outcomes (PROs) in studies with patients with rare lysosomal storage diseases (LSDs), we conducted interviews with researchers and health technology assessment (HTA) experts, and developed the methods for a systematic review of the literature. The purpose of the review is to identify the psychometrically sound generic and disease-specific PROs used in studies with patients with five LSDs of interest: Fabry, Gaucher (Type I), Niemann-Pick (Type B) and Pompe diseases, and mucopolysaccharidosis (Types I and II).

Methods: Researchers and HTA experts who responded to an email invitation participated in a telephone interview. We used qualitative content analysis to analyze the anonymized transcripts. We conducted a comprehensive literature search for studies that used PROs to investigate burden of disease or to assess the impact of interventions across the five LSDs of interest.

Results: Interviews with seven researchers and six HTA experts representing eight countries revealed five themes. These were: (i) the importance of using psychometrically sound PROs in studies with rare diseases, (ii) the paucity of disease-specific PROs, (iii) the importance of having PRO data for economic analyses, (iv) practical and psychometric limitations of existing PROs, and (v) suggestions for new PROs. The systematic review has been completed.

Conclusions: The interviews highlight current challenges and opportunities experienced by researchers and HTA experts involved in work with rare LSDs. The ongoing systematic review will highlight the experience, opportunities, and limitations of PROs in LSDs and provide suggestions for future research.

Objectives: Although consideration of ethical issues is recognized as a crucial part of health technology assessment, ethics analysis for HTA is generally perceived as methodologically underdeveloped in comparison to other HTA domains. The aim of our study is (i) to verify existing tools for quality assessment of ethics analyses for HTA, (ii) to consider some arguments for and against the need for quality assessment tools for ethics analyses for HTA, and (iii) to propose a preliminary set of criteria that could be used for assessing the quality of ethics analyses for HTA.

Methods: We systematically reviewed the literature, reviewed HTA organizations’ Web sites, and solicited views from thirty-two experts in the field of ethics for HTA.

Results: The database and HTA agency Web site searches yielded 420 references (413 from databases, seven from HTA Web sites). No formal instruments for assessing the quality of ethics analyses for HTA purposes were identified. Thirty-two experts in the field of ethics for HTA from ten countries, who were brought together at two workshops held in Edmonton (Canada) and Cologne (Germany) confirmed the findings from the literature.

Conclusions: Generating a quality assessment tool for ethics analyses in HTA would confer considerable benefits, including methodological alignment with other areas of HTA, increase in transparency and transferability of ethics analyses, and provision of common language between the various participants in the HTA process. We propose key characteristics of quality assessment tools for this purpose, which can be applied to ethics analyses for HTA purposes.

Background: Cardiovascular disease is one of the most common causes of death worldwide, with many individuals having experienced acute coronary syndrome (ACS). How patients with a history of ACS value aspects of their medical treatment have been evaluated rarely. The aim of this study was to determine patient priorities for long-term drug therapy after experiencing ACS.

Methods: To identify patient-relevant treatment characteristics, a systematic literature review and qualitative patient interviews were conducted. A questionnaire was developed to elicit patient's priorities for different characteristics of ACS treatment using Analytic Hierarchy Process (AHP). To evaluate the patient-relevant outcomes, the eigenvector method was applied.

Results: Six-hundred twenty-three patients participated in the computer-assisted personal interviews and were included in the final analysis. Patients showed a clear priority for the attribute “reduction of mortality risk” (weight: 0.402). The second most preferred attribute was the “prevention of a new myocardial infarction” (weight: 0.272), followed by “side effect: dyspnea” (weight: 0.165) and “side effect: bleeding” (weight: 0.117). The “frequency of intake” was the least important attribute (weight: 0.044).

Conclusion: In conclusion, this study shows that patients strongly value a reduction of the mortality risk in post-ACS treatment. Formal consideration of patient preferences and priorities can help to inform a patient-centered approach, clinical practice, development of future effective therapies, and health policy for decision makers that best represents the needs and goals of the patient.

Objectives: Rapid reviews can be conducted in a narrower time frame, as compared to systematic reviews, by featuring restrictions. To estimate the validity of the results, assessment of methodological quality is required. Our aim was to analyze the methodological restrictions of rapid reviews compared with systematic reviews using the AMSTAR checklist and assess its feasibility for rapid reviews.

Methods: A systematic search for literature on rapid reviews of surgical interventions was conducted in three databases: Medline, Embase, and the Cochrane library. Additionally, health technology assessment (HTA) databases were searched. We analyzed reviews using AMSTAR and additionally compared the results with those of an overview of reviews on the same topic.

Results: Items found more frequently in rapid reviews were search for gray literature (65 percent versus 33 percent), listing of excluded studies (59 percent versus 37 percent), and provision of study characteristics (77 percent versus 44 percent), whereas consideration of study quality in formulating conclusions, conduct of meta-analysis, and statement of conflicts of interest were less frequent. Median time between search and publication was 8 months, with a range between 1 and 27.

Conclusions: With some adjustments, AMSTAR can be used as a checklist for rapid reviews to describe methodological restrictions in comparison to systematic reviews and to roughly estimate the validity of the results. Strikingly, only 14.3 percent of rapid reviews were published within 3 months.

Objectives: Rapid reviews are characterized as an accelerated evidence synthesis approach with no universally accepted methodology or definition. This modified Delphi consensus study aimed to develop a comprehensive set of defining characteristics for rapid reviews that may be used as a functional definition.

Methods: Expert panelists with knowledge in rapid reviews and evidence synthesis were identified. In the first round, panelists were asked to answer a seventeen-item survey addressing a variety of rapid review topics. Results led to the development of statements describing the characteristics of rapid reviews that were circulated to experts for agreement in a second survey round and further revised in a third round. Consensus was reached if ≥70 percent of experts agreed and there was stability in free-text comments.

Results: A panel of sixty-six experts participated. Consensus was reached on ten of eleven statements describing the characteristics of rapid reviews. According to the panel, rapid reviews aim to meet the requirements and timelines of a decision maker and should be conducted in less time than a systematic review. They use a variety of approaches to accelerate the evidence synthesis process, tailor the methods conventionally used to carry out systematic reviews, and use the most rigorous methods that the delivery time frame will allow.

Conclusions: This study achieved consensus on ten statements describing the defining characteristics of rapid reviews based on the opinion of a panel of knowledgeable experts. Areas of disagreement were also highlighted. Findings emphasize the role of the decision maker and stress the importance of transparent reporting.

Objectives: To review and compare current Health Technology Assessment (HTA) activities for medical devices (MDs) across European HTA institutions.

Methods: A comprehensive approach was adopted to identify institutions involved in HTA in European countries. We systematically searched institutional Web sites and other online sources by using a structured tool to extract information on the role and link to decision making, structure, scope, process, methodological approach, and available HTA reports for each included institution.

Results: Information was obtained from eighty-four institutions, forty-seven of which were analyzed. Fifty-four methodological documents from twenty-three agencies in eighteen countries were identified. Only five agencies had separate documents for the assessment of MDs. A few agencies made separate provisions for the assessment of MDs in their general methods. The amount of publicly available HTA reports on MDs varied by device category and agency remit.

Conclusions: Despite growing consensus on their importance and international initiatives, such as the EUnetHTA Core Model®, specific tools for the assessment of MDs are rarely developed and implemented at the national level. Separate additional signposts incorporated in existing general methods guides may be sufficient for the evaluation of MDs.

Objective: As health technology assessment (HTA) organizations in Canada and around the world seek to involve the public and patients in their activities, frameworks to guide decisions about whom to involve, through which mechanisms, and at what stages of the HTA process have been lacking. The aim of this study was to describe the development and outputs of a comprehensive framework for involving the public and patients in a government agency's HTA process.

Methods: The framework was informed by a synthesis of international practice and published literature, a dialogue with local, national and international stakeholders, and the deliberations of a government agency's public engagement subcommittee in Ontario, Canada.

Results: The practice and literature synthesis failed to identify a single, optimal approach to involving the public and patients in HTA. Choice of methods should be considered in the context of each HTA stage, goals for incorporating societal and/or patient perspectives into the process, and relevant societal and/or patient values at stake. The resulting framework is structured around four actionable elements: (i) guiding principles and goals for public and patient involvement (PPI) in HTA, (ii) the establishment of a common language to support PPI efforts, (iii) a flexible array of PPI approaches, and (iv) on-going evaluation of PPI to inform adjustments over time.

Conclusions: A public and patient involvement framework has been developed for implementation in a government agency’s HTA process. Core elements of this framework may apply to other organizations responsible for HTA and health system quality improvement.

Objectives: Treatment switching refers to the situation in a randomized controlled trial where patients switch from their randomly assigned treatment onto an alternative. Often, switching is from the control group onto the experimental treatment. In this instance, a standard intention-to-treat analysis does not identify the true comparative effectiveness of the treatments under investigation. We aim to describe statistical methods for adjusting for treatment switching in a comprehensible way for nonstatisticians, and to summarize views on these methods expressed by stakeholders at the 2014 Adelaide International Workshop on Treatment Switching in Clinical Trials.

Methods: We describe three statistical methods used to adjust for treatment switching: marginal structural models, two-stage adjustment, and rank preserving structural failure time models. We draw upon discussion heard at the Adelaide International Workshop to explore the views of stakeholders on the acceptability of these methods.

Results: Stakeholders noted that adjustment methods are based on assumptions, the validity of which may often be questionable. There was disagreement on the acceptability of adjustment methods, but consensus that when these are used, they should be justified rigorously. The utility of adjustment methods depends upon the decision being made and the processes used by the decision-maker.

Conclusions: Treatment switching makes estimating the true comparative effect of a new treatment challenging. However, many decision-makers have reservations with adjustment methods. These, and how they affect the utility of adjustment methods, require further exploration. Further technical work is required to develop adjustment methods to meet real world needs, to enhance their acceptability to decision-makers.

Background: The adoption of new medical devices and diagnostics is often hampered by lack of published evidence which makes conventional health technology assessment (HTA) difficult. We now have 5 years’ experience of the Medical Technologies Advisory Committee of the National Institute for Health and Care Excellence (NICE) in the United Kingdom, addressing this problem. This committee assesses devices and diagnostics against claims of advantage, to produce guidance on adoption for the health service.

Methods: We have reflected on the practical, technical, and intellectual processes we have used in developing guidance for the health service.

Results: When scientific and clinical evidence is sparse, promise and plausibility play an increased part in decision-making. Drivers of promise include a clear design and mechanism of action, the possibility of radical improvement in care and/or resource use, and improving health outcomes for large numbers of patients. Plausibility relates to judgements about the whether the promise is likely to be delivered in a “real world” setting. Promise and plausibility need to be balanced against the amount of evidence available. We examine the influence they may have on decision-making compared with other factors such as risk and cost.

Conclusions: Decisions about adoption of new devices and diagnostics with little evidence are influenced by judgements of their promise and the plausibility of claims that they will provide benefits in a real-world setting. This kind of decision making needs to be transparent and this article explains how these influences can be balanced against the use of more familiar criteria.

Objectives: A patient reported outcome (PRO) is “any report of the status of a patient's health condition that comes directly from the patient without interpretation of the patient's response by a clinician or anyone else” (USFDA 2009). PROs are discussed widely, and many regard the patients’ perspective on treatment benefit as very valuable. Although many PROs have shown satisfactory measurement properties including reliability, validity, and responsiveness, there is great concern about risk of bias, that is, in clinical trials.

Methods: Differences in perspectives of PRO measurement in rare diseases are given arising from methodology, clinical, HTA, and patient advocacy views.

Results: PROs are playing an important role in dealing with treatment benefit especially in small sample size as occurring often in rare diseases. Challenges remain especially regarding lack of responsiveness of generic measures, limited capture of all patient relevant aspects, study design and high risk of bias.

Conclusions: PROs seem a valuable instrument to detect patient relevant aspects in rare diseases. They should be seen in addition to other approved assessment methods as randomized controlled trials but not as their substitute.

Objectives: The aim of this project was to identify the ten most important research questions for attention deficit/hyperactivity disorder (ADHD) treatment as identified by people with ADHD together with personnel involved in the treatment of ADHD in school, health, and correction services.

Methods: A working group consisting of consumers and personnel was established. The method for prioritization was primarily based on James Lind Alliance's guidebook, consisting of an interim priority setting exercise and a workshop.

Results: The top ten list includes the risk of drug dependency later in life when treated with methylphenidate as a child, teacher support, multimodal therapy, comparisons between atomoxetine and methylphenidate, methylphenidate treatment in substance abusers, parental support programmes, supported conversation, computer-aided working memory training, psychoeducative treatment, and melatonin.

Conclusions: We have shown that consumers and personnel can reach consensus on research priorities for treatments for ADHD. We encourage researchers and funders to consider the list for future studies.

Objectives: The aim of this study was to develop criteria for the prioritization of topics for health technology assessment (HTA) in the healthcare system of Kazakhstan.

Methods: Initial proposals for criteria were suggested through consultation with Ministry of Health (MoH) policy areas. These were refined through a workshop attended by HTA department staff, persons from medical universities and research institutes, and MoH policy makers. The workshop included discussion on methods used in international HTA practice. Opinions of participants on selection of criteria from those specified in a review of prioritization processes were used to define a list for inclusion in an instrument for routine use. A scoring system was established in later discussion.

Results: Selected criteria for HTA prioritization were burden of disease, availability of alternative technology, clinical effectiveness, economic efficiency, budget impact, and ethical, legal, and/or psychosocial aspects. For each criterion, a health technology under consideration is given a score from 3 (High) to 1 (Low). The total score determines whether the technology is of high to medium priority or of low priority. Determination of priorities for assessment, using the instrument, should be carried out by an expert group appointed by the MoH. The process was applied in 2014 to a selection of topics, and three health technologies were chosen for full assessments.

Conclusions: Criteria for prioritization have evolved with development of the HTA program in Kazakhstan. A method for HTA prioritization has been developed that is easy to apply, requires comparatively few resources, and is compatible with processes required by the MoH.

Objectives: External experts can be consulted at different stages of an HTA. When using vague information sources, it is particularly important to plan, analyze, and report the information processing in a standardized and transparent way. Our objective was to search and analyze recommendations regarding where and how to include expert data in HTA.

Methods: We performed a systematic database search and screened the Internet pages of seventy-seven HTA organizations for guidelines, recommendations, and methods papers that address the inclusion of experts in HTA. Relevant documents were downloaded, and information was extracted in a standard form. Results were merged in tables and narrative evidence synthesis.

Results: From twenty-two HTA organizations, we included forty-two documents that consider the use of expert opinion in HTA. Nearly all documents mention experts in the step of preparation of the evidence report. Six documents address their role for priority setting of topics, fifteen for scoping, twelve for the appraisal of evidence and results, another twelve documents mention experts when considering the dissemination of HTA results.

During the assessment step, experts are most often asked to amend the literature search or to provide expertise for special data analyses. Another issue for external experts is to appraise the HTA results and refer them back to a clinical and social context. Little is reported on methods of expert elicitation when their input substitutes study data.

Conclusions: Despite existing recommendations on the use of expert opinion in HTA, common standards for elicitation are scarce in HTA guidelines.

Objectives: We describe a new evidence-based method for screening and evaluating emerging medical technologies. Washington State agencies, under legislative direction, have granted authority to its agency Medical Directors and policy leaders to make coverage decisions on medical technologies using a “dossier” process. The dossier process is employed when technology advocates or manufacturers request Washington State healthcare purchasers to pay for new and emerging technologies. This offers the advocate an opportunity to submit scientific evidence and information classically associated with a more formal health technology assessment.

Methods: The submitted information is independently reviewed and summarized for Washington State's public healthcare purchasers allowing a more standardized coverage decision for all public purchasers in Washington State.

Results: This process has allowed Washington State to make twelve evidence-based coverage decisions at a fraction of the cost of classic technology assessment. To date, of twelve reviews over 6 years, one health technology was approved for coverage, ten were not covered and one did not require a coverage decision.

Conclusions: This evidence-based dossier process has yielded high-value coverage decisions of new and emerging medical technologies for public healthcare purchasers in Washington State.