The Agency for Health Quality and Assessment of Catalonia (AQuAS) is developing an evaluation tool for mobile health (mHealth) solutions to be used by health technology assessment (HTA) agencies and evaluation experts. In order to have a practical and comprehensive tool taking into account the particularities and challenges of mobile interventions, we considered the views and opinions of key stakeholders. The objective was to present the final selection of general aspects (dimensions) to be assessed in the evaluation, as well as the specific items (criteria) to be included in each of these topics, as a result of different co-design approaches with health professionals, developers, hospital managers, HTA agencies and patients.

A list of criteria used for health apps evaluation were drawn from a literature review. The initial list included eighty-nine criteria items grouped in nine domains. Those criteria and domains were discussed during four focus groups (FG). The importance of the criteria that were not considered as mandatory were later rated through a Delphi online sub-study, in a scale from one to six points, taking as consensus value when median value (median 6, Interquartile range, 0–1) was reached.

FG reduced domains and criteria from nine to seven and from eighty-nine to thirty-three, respectively. Most mandatory criteria were related with security, user experience, and clinical effectiveness. Fifty-seven individuals (53.7% of 106 invited to participate) were registered in the online platform (50.1% women, 68.4% 35–64years old and 42.1% from HTA agencies). From fifty non-mandatory criteria under consensus, ten criteria reached consensus (most from solution's content and health problem covered domains) concluding with a 43/7 criteria/domain tool.

Insights from main stakeholders on the content of the tool for mHealth assessment were considered through the FG and Delphi technique. The dimensions of security and privacy, clinical effectiveness, solutions’ content, technological aspects, users’ experience and costs were considered mandatory. The dimension related to the impact on the organization was appraised as a secondary domain for evaluation. A workshop with AQuAS research team and HTA external researchers will help to define: the assessment methods (type of instrument, dichotomous responses and/or Likert scales) for the evaluation and the format and dimension's weights of the final design of the tool.