One of the challenges of large scale Health Technology Assessment (HTA) projects is managing the large volume of studies retrieved by the requisite comprehensive literature searches. At the scoping stage of the project, a pragmatic judgement needs to be made as to how sensitive the search strategy should be in order to find all the relevant papers without returning an overwhelming volume of irrelevant studies.

For this HTA (evaluating prognostic and predictive markers in rheumatoid arthritis), the research team already had prior knowledge of several key markers of interest, but wanted to ensure that no others had been missed. Advice from practising clinicians was obtained, but for additional validation, a broad scoping search was conducted for ‘rheumatoid arthritis’ using the sensitive Haynes filters for prognostic (1) and clinical prediction (2) studies. Unsurprisingly, this initial search retrieved too many studies for them all to be admitted to the full review; but once those dealing with known markers had been removed, a sample of the remaining records was loaded into a software visualization tool (3) to display “heat maps” of frequently occurring terms and phrases.

On this occasion, no additional markers were identified, however this provided reassurance that the advice obtained from clinicians was comprehensive, enabling the HTA team to proceed confidently with its evaluation of the selected markers.

Visualization offers an alternative means of exploring and interrogating large text archives, and has the potential to complement the role of traditional search methods in identifying literature for systematic reviews and health technology assessments. As processing power increases and more and more full-text papers become available open access, it may provide a solution to some of the limitations associated with comprehensive searching.