External Validation of the IDENTIFY Risk Calculator for Predicting Urinary Tract Cancer in Patients with Haematuria

Abstract Aim Risk-stratification for patients referred with haematuria is important in diagnosing cancer early and for health resource efficiency. A recent prediction model developed from the IDENTIFY study created a cancer risk calculator that aimed to help clinicians with their decision-making in the investigation of patients with haematuria. We aimed to externally validate this model in a separate cohort from the development population. Method Data were collected on patient characteristics used for the risk calculator, and the cancer outcome recorded. Patients with a previous or existing urological malignancy were excluded. Development and validation cohort demographics were compared. The area under the receiver operating characteristic curve (AUC), calibration slope, calibration-in-the-large (CITL), and Brier’s score were used to assess external validation. Results There were 3483 patients from 111 hospitals across 27 countries. 12 countries were new countries not used in the development cohort. The development and validation cohorts were well matched. When comparing the observed vs predicted outcomes of the prediction model on the validation cohort, the calibration slope was 0.97 and CITL was 0.33. AUC was 0.79. Brier’s score was 0.15. Conclusions External validation of the IDENTIFY risk calculator shows good accuracy and discrimination for predicting urinary tract cancer in patients referred with haematuria. The model underestimates the risk slightly and this can be recalibrated by adjusting the intercept of the regression equation. The recalibrated risk calculator can be introduced into diagnostic haematuria pathways to improve risk stratification and health resource allocation.