Medical assistance in dying for mental illness as a sole underlying medical condition (MAiD MI-SUMC) is a controversial and complex policy in terms of psychosocial and ethical medical practice implications. We discuss the status of MAiD MI-SUMC in Canada and argue for the use of the UK Medical Research Council's framework on complex interventions in programme evaluations of MAiD MI-SUMC. It is imperative to carefully and rigorously evaluate the implementation of MAiD MI-SUMC to ensure an understanding of the multiple facets of implementation in contexts permeated by unique social, economic, cultural and historical influences, with a correspondingly diverse array of outcomes. This requires a complexity-informed programme evaluation focused on context-dependent mechanisms and stakeholder experiences, including patients, service providers and other people affected by the policy. It is also important to consider the economic impact on health and social welfare systems. Such evaluations can provide the data needed to guide evidence-informed decision-making that can contribute to safer implementation and refinement of MAiD MI-SUMC.

Conventional approaches to evidence that prioritise randomised controlled trials appear increasingly inadequate for the evaluation of complex mental health interventions. By focusing on causal mechanisms and understanding the complex interactions between interventions, patients and contexts, realist approaches offer a productive alternative. Although the approaches might be combined, substantial barriers remain.

Objectives: The aim of this study was to describe an initial exploration by CADTH, Canada's pan-Canadian health technology assessment (HTA) agency, in using the INTEGRATE-HTA guidance in the production of an HTA that examined the use of both in-center and in-home dialysis modalities for the treatment of end-stage kidney disease in adults in Canada.

Methods and Results: We outline CADTH's standard HTA production process and context and then describe the experience of the assessment team in using the INTEGRATE-HTA guidance, specifically to help structure and guide the use of a logic model, the identification of implementation issues, and the identification and examination of ethical issues. For each of the aspects, we describe and reflect on how the assessment team used the guidance, challenges that were encountered in its use, and whether and how we might address these challenges when using the INTEGRATE-HTA guidance in the future.

Conclusions: INTEGRATE-HTA provided detailed and helpful guidance for truly integrating wide-ranging aspects of HTA. Our agency was challenged by a steep learning curve for assessment team members, tight project timelines, and a misalignment of current HTA processes with those required to implement the guidance. Nevertheless, using the guidance initiated a dialogue about what might be needed to assess complex interventions and the potential process changes that could facilitate conducting more integrated assessments.

Animal health planning activities are not always providing a satisfactory positive impact on herd health and welfare. Moreover, evaluating the impact of advisory programmes is complex due to multiple interacting elements that influence its outcome. Therefore, measuring solely health outcomes is not sufficient: the whole process of the implementation and use of such programmes should be evaluated. In order to evaluate the impact of an intervention with a Herd Health and Production Management (HHPM) programme a process evaluation framework was designed and used. The intervention involved 20 organic dairy cattle farmers and their advisors, in both France and Sweden. In both countries 20 organic dairy farms were selected as control herds. The evaluation of the HHPM programme was based on: (a) the compliance to the programme; (b) the programme’s functions influencing herd health management practices and stimulating dialogue between farmers and advisors; (c) its effectiveness in terms of improving herd health compared with control farms. Complete compliance to the programme was fulfilled by 21 out of 40 farmers–advisors. Results from a questionnaire showed that the programme functioned as intended (e.g. by allowing early identification of herd health problems), stimulated change in farmers’ herd health management practices and farmer–advisor dialogue. Even though the majority of the users perceived that the programme contributed to herd health improvements, no significant differences in health outcomes were found when compared with control farms 12 months after the start of the intervention. The programme allowed creating an environment promoting the exchange of information between farmers and advisors, necessary to define pertinent advice in a farm-specific situation. Future research should aim at improving methods for the evaluation of the effect of advisory programmes, by identifying early indicators for effective advice and developing methods to evaluate the quality of advisory situations without interfering with them.

Objectives: The INTEGRATE-HTA project provided methodology to evaluate complex technologies. This study provides guidance on how to retrieve and critically appraise available evidence on moderators and predictors of treatment effects and on patient preferences for treatment outcomes as a source of complexity.

Methods: Search filters for PubMed were developed by hand-searching a large volume of articles reporting on relevant aspects. Search terms were retrieved from selected papers and algorithmically combined to find the optimal combination of search terms. For the development of the appraisal checklists literature was searched in PubMed and Google Scholar together with citation chasing. For the CHecklist for the Appraisal of Moderators and Predictors (CHAMP) a Delphi procedure was used to value a set of eligible appraisal criteria retrieved from the literature.

Results: Search filters were developed optimized for different accuracy measures. The final version of CHAMP consists of a seventeen questions covering the design, analysis, results and transferability of results of moderator and predictor analysis. The final checklist for appraisal of literature on patient preferences for treatment outcomes consist of six questions meant to help the user to identify relevant quality issues together with a guidance toward existing tools concerning the appraisal of specific preference elicitation methods.

Conclusions: Incorporating knowledge on subgroups for whom a specific treatment will produce more benefit holds the promise of better targeting and, ultimately, enhancing overall effectiveness and efficiency of healthcare technology. Finally, incorporating information on preferences for treatment outcomes will foster health technology assessment that addresses outcomes that are important to patients.

Objectives: Current health technology assessment (HTA) is not well equipped to assess complex technologies as insufficient attention is being paid to the diversity in patient characteristics and preferences, context, and implementation. Strategies to integrate these and several other aspects, such as ethical considerations, in a comprehensive assessment are missing. The aim of the European research project INTEGRATE-HTA was to develop a model for an integrated HTA of complex technologies.

Methods: A multi-method, four-stage approach guided the development of the INTEGRATE-HTA Model: (i) definition of the different dimensions of information to be integrated, (ii) literature review of existing methods for integration, (iii) adjustment of concepts and methods for assessing distinct aspects of complex technologies in the frame of an integrated process, and (iv) application of the model in a case study and subsequent revisions.

Results: The INTEGRATE-HTA Model consists of five steps, each involving stakeholders: (i) definition of the technology and the objective of the HTA; (ii) development of a logic model to provide a structured overview of the technology and the system in which it is embedded; (iii) evidence assessment on effectiveness, economic, ethical, legal, and socio-cultural aspects, taking variability of participants, context, implementation issues, and their interactions into account; (iv) populating the logic model with the data generated in step 3; (v) structured process of decision-making.

Conclusions: The INTEGRATE-HTA Model provides a structured process for integrated HTAs of complex technologies. Stakeholder involvement in all steps is essential as a means of ensuring relevance and meaningful interpretation of the evidence.