It is not clear yet whether new digital health interventions can and should be assessed by using ‘conventional’ health technology assessment (HTA) methodology. In response to the question about how much and which type of evidence is needed for decisions on new digital health interventions, this presentation discusses complimentary evidence as generated through a critical-realist review and a qualitative meta-synthesis. This work follows from earlier work by AG Ekeland, AH Hansen and TS Bergmo.

A realist review addresses complex social interventions investigated in real life settings. The review was conducted with the purpose of generating knowledge on what works, for whom and under which circumstances. The aim was to enable decision-makers to reach a deeper understanding of how the intervention can be made to work most effectively. A critical review goes beyond mere description of identified articles and includes analysis and conceptual innovation. An effective critical review synthesizes material from diverse sources, provides an opportunity to ‘take stock’ and evaluate what is of value related to a previous body of work.

User patterns of clinical videoconferencing turned out to be dependent on contextual factors like clinical condition, motivation, technological skills, professional and organizational arrangements, trust and the perceived value they add compared with “services as usual”. The pattern of what works, for whom and under which circumstances was heterogeneous and dynamic. The review types helped identify and conceptualize new user categories, and understand the complex patterns of use.

The in-depth accounts of different clinical use resulting from such a review provide decision makers with a highly practical understanding of complex social interventions which is likely to be of use when planning and implementing programs at a national, regional or local level. A critical-realist review of digital services can complement controlled studies and evidence summaries in HTA.