Ultrasound Liver Imaging Reporting and Data System (US LI-RADS) Visualization Score: a reliability analysis on inter-reader agreement

Background & Aim The American College of Radiology Ultrasound Liver Imaging Reporting and Data System (ACR US LI-RADS) Visualization Score conveys the expected level of sensitivity of screening and surveillance ultrasound exams in patients at risk for hepatocellular carcinoma (HCC). We sought to determine inter-reader agreement of the Visualization Score which is currently unknown. Methods Consecutive 6998 ultrasound HCC screening and surveillance studies in 3115 patients from 2017 to 2020 were retrospectively retrieved. Of these, 6154 (87.9%) studies were Visualization A (No or minimal limitations), 709 (10.1%) were Visualization B (Moderate limitations), and 135 (1.9%) were Visualization C (Severe limitations). Randomly sampled 90 studies, with 30 studies in each Visualization category, were included for analysis. Nine radiologists (3 senior attendings, 3 junior attendings and 3 body imaging fellows) blinded to the original categorization independently reviewed each study and assigned a Visualization Score. Intraclass correlation coefficient (ICC) was used to quantify inter-reader agreement. Results ICC among all 9 radiologists was 0.70 (95% CI 0.63–0.77). ICCs among senior attendings, junior attendings and body imaging fellows were 0.68 (CI 0.58–0.76), 0.72 (CI 0.62–0.80) and 0.76 (CI 0.68–0.83), respectively. Subgroup analysis by liver parenchyma was further performed. ICC was highest in the patient group with normal liver parenchyma (0.69, CI 0.56–0.81), followed by steatosis (0.66, CI 0.54–0.79) and cirrhosis (0.58, CI 0.43–0.73), respectively. Conclusions US LI-RADS Visualization Score is a reliable tool with good inter-reader agreement that can be used to indicate the expected level of sensitivity of a screening and surveillance ultrasound examination for detecting focal liver observations.