Abstract 13393: Equity Model Evaluation of a Predictive Cardiac Positron-Emission Tomography (PET) Risk Score for 90-Day and One-Year Major Adverse Cardiac Events and Revascularization

Introduction: Artificial Intelligence (AI) algorithms should be evaluated to discern differences in prediction amongst population groups. The Intermountain data science group has developed a software package iEAI, which performs a model evaluation (ME) analysis to detect such meaningful prediction differences. Here we apply this software package to a recently developed cardiac PET/CT stress test-based risk score that predicts 90-day and one-year major adverse cardiac events (including death and myocardial infarction) and revascularization (MACE-Revasc). The results of these analyses will reveal any discrepancies in prediction and help with equity. Methods: 5049 patients who had a clinically indicated PET/CT study from January 1, 2018, to December 31, 2018, were considered. The ME analyses were done for gender, age, and race. Precision (or positive predictive value) and recall (or sensitivity) were measured and meaningful differences were those that were statistically significant and had a ratio < 0.85 or >1.25 in magnitude than the reference category (e.g., the largest category). Results: The ME analysis of the PET/CT risk score for MACE-Revasc are shown in the table. For the 90-day PET/CT risk score a recall bias existed for females compared to males (0.49 vs 0.62). There was also bias with improved recall for Native-Hawaiian/ Pacific Islanders compared to Whites (0.87 vs 0.58). For the one-year PET/CT risk score, recall (0.38 vs 0.52) and precision (0.34 vs 0.47) biases were found for females compared to males, and recall biases were found for those aged 50-59 compared to 60-69 (0.34 vs 0.50). Conclusion: Meaningful biases were found in both the PET/CT risk scores for 90-day and One-year MACE-Revasc. As the application of AI in healthcare grows, evaluations like the above should be carried out. Then any detected inequalities could be addressed by refining or redeveloping the risk score models to eliminate biases.