Diagnostic accuracy of artificial intelligence-assisted radiology assessment of cancer: a systematic review

Objective: Perform a systematic review and meta-analysis of studies using multi-reader multi-case (MRMC) study designs for cancer diagnosis with artificial intelligence (AI). Review diagnostic accuracy, study design and reporting. Methods: A search of several databases between January 1, 2014 and February 28, 2024 was performed. Diagnostic accuracy studies that compared radiologists with and without AI-assistance in cancer diagnostic tasks over all imaging modalities were included. Meta-analysis using Summary Receiver Operating Characteristics (SROC) curves were plotted for pooled sensitivity and specificity. Risk of bias was assessed by using the Quality Assessment of Diagnostic Accuracy Studies-Comparative (QUADAS-C) and the Checklist for Artificial intelligence in Medical Imaging (CLAIM). Results: Thirty-four studies were included of which 23 were included in meta-analysis. Eight identified cancers on Chest X-rays, 17 on CT, 9 on MRI. Pooled sensitivity and specificity were 0.67 (95%CI 0.58-0.74) and 0.82 (95%CI 0.75-0.88), respectively, for clinicians and 0.79 (95%CI 0.71-0.88) and 0.87 (95%CI 0.82-0.91) for AI-assistance. 17 of 34 studies (50%) had concern of bias with QUADAS-C. CLAIM assessment highlighted reporting issues in several domains of methodology in a proportion of studies. Conclusion: Artificial intelligence assistance tools may benefit clinician diagnostic performance in cancer diagnosis. Updated reporting guidelines may help to overcome potential methodological limitations to clarify AI’s value in healthcare. Advances in knowledge: Previous reviews compare AI accuracy alone against a clinician. We focus on MRMC study designs to ass AI use in a clinical environment.