The methodological guides for the assessment of new/emerging non-pharmacological technologies differ from the traditional health technology assessment (HTA) guidelines developed by the Spanish Network of Agencies for Assessing National Health System Technologies and Performance (RedETS). The aim of this study is to identify the special features and challenges of carrying out HTA on new/emergent non-pharmacological technologies.

The application of traditional and new/emergent HTA guidelines is compared along the consecutive evaluation phases in four practical cases carried out at the Agency for Health Quality and Assessment of Catalonia (AQuAS) in 2017-2018.

Main learning and outstanding challenges: (i) Instead of following a defined protocol, the evaluations are carried out from a preliminary short report which generates a lack of justification and delimitation of its scope. (ii) References’ identification and data extraction are often limited due to lack of studies, and sometimes require the use of grey literature or other sources less informative, for example, trial registries. It can be challenging to exclude references related to other indications. (iii) The assessment of resource use and costs of running the technology is complicated due to the lack of public prices information and specific impacts of use. (iv) The evidence considered during the assessment usually does not meet high quality requirements (risk of bias) because of indirect evidence, lack of comparator or no having clearly defined outcomes, among others. (v) It's difficult to draw conclusions and, consequently, recommendations due to abovementioned aspects and especially for the usual evidence gap that faces this type of technology in early stages of diffusion and/or in a competition situation of manufacturer companies.

The most recent innovation in non-pharmacological technologies merits a differentiated assessment approach. However, there is need to reconsider the methodology applied in order to overcome the challenges and limitations identified.