In the past decade, health technology assessment (HTA) has narrowed its scope to analyses of mainly clinical and economic benefits. Technology challenges in the 21st century emphasize the need for holistic assessments to obtain accurate recommendations for decision-making, as in HTA’s foundations. Using the VALues In Doing Assessments of health TEchnologies (VALIDATE) methodology for complex technologies provides a deeper understanding of problems through analysis of stakeholders’ views, allowing for more comprehensive HTAs. This study aimed to assess a pharmaceutical clinical decision support system (CDSS) using VALIDATE.

Semi-structured interviews with different stakeholders were conducted in the following domains: problem definition (medication error [ME] occurrence and prevention); judgement of solution (existing preventive methods and previous experiences of the CDSS); background theories (future impact and personal beliefs); and barriers to and facilitators of implementation. The following individuals were interviewed: medical informatic specialists (n=3), pharmacists (n=2), nurses (n=2), physicians (n=2), CDSS company representatives (n=1), electronic health record developer (n=1), and health consultancy firm representatives (n=1). Content analysis was used to integrate and analyze the data.

The multistakeholder interviews identified various barriers to the acceptance and implementation of a pharmaceutical CDSS that were different from those reported in the literature. These included: (i) occurrence of ME (no traceability of medication taken or poor patient medication empowerment); (ii) perception of current level of MEs (huge improvement from ten years ago); (iii) perception of technology as a tool to prevent ME (not enough if only implemented at one point of care); (iv) previous experiences with a CDSS (low rates of development of CDSSs are due to medication prescriptions being digitalized last in hospitals); (v) CDSS metrics (input data should be measured to control CDSS performance); and (vi) other barriers.

Including facts and stakeholders’ values in problem definition and the scoping of health technologies is essential for the proper conduct of HTAs. Incorporating views from multiple stakeholders when scoping the assessment of health technologies brings additional values to literature findings, resulting in a more holistic evaluation. The lack of multistakeholder scoping can lead to inaccurate information and result in wrong decisions about if, when, and how to adopt a CDSS.