The reliability of health technology assessment (HTA) is built on accessing evidence systematically to inform conclusions and recommendations; however, the availability of primary evidence is a source of bias which can undermine an HTA. This omission is often because attempts to generate primary evidence have not been completely successful. Where partial evidence exists, ignoring it constitutes avoidable bias. Taking the Hip Op trial as an example (a study of developmental dysplasia of the hip (DDH)) we consider how despite lack of quantitative outcomes data, rich information was obtained that should inform HTA in this area.

The Hip Op trial was an open label trial comparing early against late surgery in the management of DDH. In parallel, a qualitative study attempted to explore the experience of parents of children with DDH.

The trial protocol called for recruitment of 636 children, but due to changes in clinician equipoise and service configuration only 29 could be recruited. The trial was stopped early. While baseline data for the 29 children was available, no estimate of effect was attempted due to a lack of outcome data; however, the qualitative data was rich, representing the biggest qualitative sample worldwide on this topic. It reflected the patient experience, and shows a clear preference towards early intervention, despite the absence of quantitative evidence.

The qualitative work here gives a clear indication that parents have a strong preference. This is data which would not be captured in traditional HTA reports, which tend to focus on quantitative data and meta-analysis. This is, however, information that is important to patients, and should inform clinicians and payers. We discuss how HTA do-ers should make efforts to find this data from ‘failed’ primary research and incorporate it into their reports, and how HTA do-ers could be alert to this situation.