Methods for selecting the best evidence to inform a NICE technology appraisal on selective internal radiation therapies for hepatocellular carcinoma

Background Systematic reviews of medical devices are particularly challenging as the quality of evidence tends to be more limited than evidence on pharmaceutical products. This article describes the methods used to identify, select and critically appraise the best available evidence on selective internal radiation therapy devices for treating hepatocellular carcinoma, to inform a technology appraisal for the National Institute for Health and Care Excellence. Methods A comprehensive search of ten medical databases and six grey literature sources was undertaken to identify studies of three devices (TheraSphere®, SIR-Spheres® and QuiremSpheres®) for treating hepatocellular carcinoma. The large evidence base was scoped before deciding what level of evidence to include for data extraction and critical appraisal. The methodological quality of the included studies was assessed using criteria relevant to each study design. Results Electronic searches identified 4755 records; over 1000 met eligibility criteria after screening titles and abstracts. A hierarchical process was used to scope these records, prioritising comparative studies over non-comparative studies, where available. One hundred ninety-four full papers were ordered; 64 met the eligibility criteria. For each intervention, studies were prioritised by study design and applicability to current UK practice, resulting in 20 studies subjected to critical appraisal and data extraction. Only two trials had a low overall risk of bias. In view of the poor quality of the research evidence, our technology appraisal focused on the two higher quality trials, including a thorough critique of their reliability and generalisability to current UK practice. The 18 poorer quality studies were briefly summarised; many were very small and results were often contradictory. No definitive conclusions could be drawn from the poorer quality research evidence available. Conclusions A systematic, pragmatic process was used to select and critically appraise the vast quantity of research evidence available in order to present the most reliable evidence on which to develop recommendations. Systematic review registration PROSPERO CRD42019128383.