The Scottish Health Technologies Group (SHTG) set out to assess the impact of HTA products. Two questions were posed: Does advice from SHTG have influence? How is SHTG advice used?

SHTG adapted a tool developed by the International Network of Agencies for Health Technology Assessment (INAHTA). The INAHTA framework investigates indications of impact and categorizes outputs into levels of impact. Over three years, potential users of SHTG advice were contacted six to twelve months after advice was published and asked how the advice had been used. HTA outputs were categorized into the four levels of influence they achieved: ‘major influence’, ‘some influence’, ‘some consideration’ and ‘no known influence’.

HTA products were found to have been used in four main ways: ‘informed discussion’, ‘referenced’, ‘informed policy’ or ‘directly informed practice’. Levels of influence had steadily increased over the three years assessed. The findings were well received by internal audiences, with particular interest in the various ways HTA recommendations had been used. There was also feedback about ‘marking our own homework’. These results have informed a new SHTG strategy and supported clear messaging around the value of HTA.

SHTG has found a pragmatic, resource-light way to explore the impact of HTA outputs, which has proved valuable for driving strategy and messaging.