In the past decade, health technology assessment (HTA) has narrowed its scope to the analysis of mainly clinical and economic benefits. However, twenty-first century technology challenges require the need for more holistic assessments to obtain accurate recommendations for decision-making, as it was in HTA's foundations. VALues In Doing Assessments of health TEchnologies (VALIDATE) methodology approaches complex technologies holistically to provide a deeper understanding of the problem through analysis of the heterogeneity of stakeholders’ views, allowing for more comprehensive HTAs. This study aimed to assess a pharmaceutical clinical decision support system (CDSS) using VALIDATE.

A systematic review of the empirical evidence on CDSS was conducted according to PRISMA guidelines. PubMed, the Cochrane Library, and Web of Science databases were searched for literature published between 2000 and 2020. Additionally, a review of grey literature and semi-structured interviews with different hospital stakeholders (pharmacists, physicians, computer engineers, etc.) were conducted. Content analysis was used for data integration.

Preliminary literature results indicated consensus regarding the effectiveness of CDSS. Nevertheless, when including multistakeholder views, CDSS appeared to not be fully accepted in clinical practice. The main reasons for this appeared to be alert fatigue and disruption of workflow. Preliminary results based on information from the literature were contrasted with stakeholder interview responses.

Incorporation of facts and stakeholder values into the problem definition and scoping for a health technology is essential to properly conduct HTAs. The lack of an inclusive multistakeholder scoping can lead to inaccurate information, and in this particular case to suboptimal CDSS implementation concerning decision-making for the technology being evaluated.