An Artificial Intelligence Algorithm for ADPKD: Are We Close to Successful Clinical Implementation?

Artificial intelligence (AI) has the potential to be transformative for clinical medicine and for radiology specifically.1Obermeyer Z. Emanuel E.J. Predicting the future—big data, machine learning, and clinical medicine.N Engl J Med. 2016; 375: 1216-1219Crossref PubMed Scopus (1491) Google Scholar The fast development and increasing availability of AI algorithms that support clinical medicine have resulted in growing interest and investigations into their potential clinical implementation. However, to date, most efforts have focused on algorithm discovery, development, and initial validation, usually in a controlled setting different from clinical workflows and real-time patient care, whereas large-scale clinical implementation has not yet been achieved. Coordinated interdisciplinary efforts aimed at integrating AI-based algorithms into clinical workflows are necessary for them to achieve their full potential. There are, however, several barriers to AI-based algorithm implementation in real-life clinical practice.2Recht M.P. Dewey M. Dreyer K. et al.Integrating artificial intelligence into the clinical practice of radiology: challenges and recommendations.Eur Radiol. 2020; 30: 3576-3584Crossref PubMed Scopus (62) Google Scholar The principal barrier is the integration of an AI-based algorithm into an already complex clinical workflow, which involves multiple intersecting systems and stakeholders, with the attendant exposure to a plethora of unpredictable variables; exposure to such variables makes the clinical implementation environment different from the controlled developmental setting. Other challenges include proper validation of AI-based algorithms, regulatory compliance and ethical issues related to the use of AI in health care, and cybersecurity. In this issue of Mayo Clinic Proceedings, Potretzke et al3Potretzke T. Korfiatis P. Blezek D. et al.Clinical implementation of an artificial intelligence algorithm for MR-derived measurement of total kidney volume.Mayo Clin Proc. 2023; 98: 689-700Abstract Full Text Full Text PDF PubMed Scopus (1) Google Scholar take a step forward with AI application to real-time clinical care in the specific context of patients with autosomal dominant polycystic kidney disease (ADPKD). The most common genetic cause of chronic kidney disease, ADPKD is characterized by massive kidney enlargement due to the formation and growth of multiple fluid-filled cysts that crowd and damage the surrounding, normal renal parenchyma; this leads to a progressive decline in kidney function and, ultimately, end-stage kidney disease.4Torres V.E. Harris P.C. Pirson Y. Autosomal dominant polycystic kidney disease.Lancet. 2007; 369: 1287-1301Abstract Full Text Full Text PDF PubMed Scopus (1048) Google Scholar Total kidney volume (TKV) in ADPKD5Grantham J.J. Torres V.E. The importance of total kidney volume in evaluating progression of polycystic kidney disease.Nat Rev Nephrol. 2016; 12: 667-677Crossref PubMed Scopus (78) Google Scholar is recognized by EU and US regulatory agencies as a prognostic biomarker for enriching patient populations for clinical trials of disease-specific therapies for ADPKD and is an acceptable surrogate end point. Here, Potretzke et al3Potretzke T. Korfiatis P. Blezek D. et al.Clinical implementation of an artificial intelligence algorithm for MR-derived measurement of total kidney volume.Mayo Clin Proc. 2023; 98: 689-700Abstract Full Text Full Text PDF PubMed Scopus (1) Google Scholar have demonstrated the potential for successful clinical implementation of an internally developed and previously validated AI algorithm for magnetic resonance (MR)–derived measurement of TKV in ADPKD in complex radiology practice. Applying their AI tool to 170 cases in 161 unique ADPKD patients at 3 Mayo Clinic sites in the United States, they found that the AI algorithm for TKV measurement proved to be noninferior to the non–AI-assisted procedure in use, despite needing manual editing in half of all cases to meet standards. However, these were minor edits, with a mean percentage difference between pre- and post-correction TKV (3%) in the same order as previously determined interrater difference. Moreover, the agreement on disease classification between AI-based and manually edited segmentation was high, with only a few cases being assigned a different diagnostic severity class6Irazabal M.V. Rangel L.J. Bergstralh E.J. et al.Imaging classification of autosomal dominant polycystic kidney disease: a simple model for selecting patients for clinical trials.J Am Soc Nephrol. 2015; 26: 160-172Crossref PubMed Scopus (348) Google Scholar and no changes greater than 1 class. The AI tool also led to huge time savings, reducing the time needed to a few minutes, even in cases that required manual editing to meet manual standards, instead of the 60 to 90 minutes required for the usual manual processing. Notably, until now, TKV computation had not been feasible in clinical practice as manual segmentation was too time-intensive, and simplified techniques such as the ellipsoid method could provide only a TKV estimate.7Sharma K. Caroli A. Quach L.V. et al.Kidney volume measurement methods for clinical studies on autosomal dominant polycystic kidney disease.PloS One. 2017; 12e0178488Crossref Scopus (21) Google Scholar The implementation of an AI-based algorithm for TKV measurement makes it possible to provide reliable volumetric measurements within required turnaround times for ordering clinicians, making the TKV biomarker compatible with clinical practice to assign disease severity class, to predict future estimated glomerular filtration rate decline, and to determine eligibility for specific therapies. The net result is a significant improvement in clinical workflow efficiency. The results of Potretzke et al3Potretzke T. Korfiatis P. Blezek D. et al.Clinical implementation of an artificial intelligence algorithm for MR-derived measurement of total kidney volume.Mayo Clin Proc. 2023; 98: 689-700Abstract Full Text Full Text PDF PubMed Scopus (1) Google Scholar highlight a few specific issues that are worth discussing. Concomitant polycystic liver disease, and especially severe polycystic liver disease, may make a big difference in terms of the need to manually edit AI-based kidney segmentations. Indeed, polycystic liver can often cause problems with, for example, assigning adjacent cysts to the right kidney or liver on MR images, even in manual segmentation. Surprisingly, women were more likely to require manual editing than men. Quantitative assessment of liver volumes could help to clarify this apparent sex difference, potentially biasing the AI-based algorithm’s results. Regarding the successful implementation of an AI-based algorithm in real-life clinical practice, special attention should be paid to generalizability and to the application of the tool under various conditions. In fact, because of its natural dependence on the training data set and development environment, an AI-based algorithm that performs well in an implementation environment closely matching development conditions may completely fail under different conditions. The performance of any AI-based segmentation algorithm is also highly dependent on MR image acquisition; in addition, to ensure adequate performance, it is important to use similar acquisition protocols on which the AI-based algorithms were trained. In this regard, the standardization of image acquisition plays a key role in successful clinical implementation of AI-based algorithms for imaging biomarker quantitation.8deSouza N.M. van der Lugt A. Deroose C.M. et al.Standardised lesion segmentation for imaging biomarker quantitation: a consensus recommendation from ESR and EORTC.Insights Imaging. 2022; 13: 159Crossref PubMed Scopus (4) Google Scholar Moreover, to encourage the wide clinical adoption of the AI-based algorithm, the required image sequence should be widely available. The issue of the generalizability of their study findings has been addressed properly by Potretzke et al,3Potretzke T. Korfiatis P. Blezek D. et al.Clinical implementation of an artificial intelligence algorithm for MR-derived measurement of total kidney volume.Mayo Clin Proc. 2023; 98: 689-700Abstract Full Text Full Text PDF PubMed Scopus (1) Google Scholar who implemented the AI-based algorithm clinically for ADPKD at 3 different Mayo Clinic sites, under various conditions and practice patterns. Moreover, the MR imaging sequence that is relevant for TKV calculation, a routine single-shot fast spin echo sequence commonly used in ADPKD clinical trials and clinical practice, was first standardized, and technical requirements for algorithm implementation and reporting were deployed; this resulted in the preservation of algorithm performance across sites. Setting aside these finer details, the principal insight gained from this elegant study is that to ensure that there are real-world patient benefits to the implementation of an AI-based algorithm, communication with and education of all stakeholders involved in clinical implementation are critical to its success. Indeed, the need for education and training on AI has been underlined by most stakeholders in the context of a recent literature search.9Yang L. Ene I.C. Arabi Belaghi R. Koff D. Stein N. Santaguida P.L. Stakeholders’ perspectives on the future of artificial intelligence in radiology: a scoping review.Eur Radiol. 2022; 32: 1477-1495Crossref PubMed Scopus (11) Google Scholar In the case of the clinical implementation of the TKV computation algorithm, all participants involved in the clinical workflow, including nephrologists, radiologists, MR technologists, and medical image analysts, were properly educated through role-specific educational material, learning modules, and the transfer of information. Educational efforts emphasize the real-world patient benefits of implementing the algorithm. In the case of TKV, this measurement is not only an accepted prognostic biomarker for ADPKD patients but also essential for patient care, for instance, in counseling and drug eligibility—and reassures stakeholders that despite the challenge that inevitably accompanies workflow change, implementing it will not be too onerous for them. Finally, in general, the clinical implementation of AI-based algorithms comes with major ethical concerns,10Schönberger D. Artificial intelligence in healthcare: a critical analysis of the legal and ethical implications.Int J Law Inf Technol. 2019; 27: 171-203Google Scholar including fairness, accountability, transparency, data protection, and privacy. While implementing AI solutions, it is necessary to recognize possible biases that come from the use of data sets that may overrepresent, underrepresent, or entirely miss relevant characteristics. Moreover, the clinical use of AI involves personal health information, raising concerns about data protection and privacy. The rights and interests of people who may have been excluded from the data sets used to train AI algorithms, usually marginalized or vulnerable populations, should also be given adequate consideration as the AI-based algorithm could possibly further exacerbate health inequalities. Possible benefits and harms resulting from the clinical implementation of the AI-based algorithm should be assessed as deeply and as comprehensively as possible beforehand.2Recht M.P. Dewey M. Dreyer K. et al.Integrating artificial intelligence into the clinical practice of radiology: challenges and recommendations.Eur Radiol. 2020; 30: 3576-3584Crossref PubMed Scopus (62) Google Scholar To conclude, the study by Potretzke et al3Potretzke T. Korfiatis P. Blezek D. et al.Clinical implementation of an artificial intelligence algorithm for MR-derived measurement of total kidney volume.Mayo Clin Proc. 2023; 98: 689-700Abstract Full Text Full Text PDF PubMed Scopus (1) Google Scholar provides a successful example of the clinical implementation of an AI-based algorithm. It could open the way for the wider adoption of AI, eventually leading to a significant improvement in clinical workflow efficiency, and could be transformative for clinical routine. This study also clearly highlights the challenges of clinical implementation of AI-based algorithms, beyond technological hurdles. Good performance, noninferiority, and the benefits of the AI-based algorithm compared with "gold standard" methods, for instance, in terms of saving time, should first be proved, and the impact of the AI-based algorithm on intended clinical questions should be analyzed carefully. Moreover, the generalizability of the algorithm’s performance and the education of interdisciplinary stakeholders are crucial steps toward the successful implementation of an AI-based algorithm in real-life clinical practice and with a view to its being adopted widely, not only in the setting of ADPKD. The authors report no competing interests. Clinical Implementation of an Artificial Intelligence Algorithm for Magnetic Resonance–Derived Measurement of Total Kidney VolumeMayo Clinic ProceedingsVol. 98Issue 5PreviewTo evaluate the performance of an internally developed and previously validated artificial intelligence (AI) algorithm for magnetic resonance (MR)–derived total kidney volume (TKV) in autosomal dominant polycystic kidney disease (ADPKD) when implemented in clinical practice. Full-Text PDF Open Access