Cookies on this website
We use cookies to ensure that we give you the best experience on our website. If you click 'Continue' we'll assume that you are happy to receive all cookies and you won't see this message again. Click 'Find out more' for information on how to change your cookie settings.

OBJECTIVE: Receiver operating characteristic (ROC) curves show how well a risk prediction model discriminates between patients with and without a condition. We aim to investigate how ROC curves are presented in the literature and discuss and illustrate their potential limitations. STUDY DESIGN AND SETTING: We conducted a pragmatic literature review of contemporary publications that externally validated clinical prediction models. We illustrated limitations of ROC curves using a testicular cancer case study and simulated data. RESULTS: Of 86 identified prediction modelling papers, 52 (60%) presented ROC curves without thresholds and 1 (1%) presented a ROC curve with only a few thresholds. We illustrate that ROC curves in their standard form withhold threshold information, have an unstable shape even for the same area under the curve (AUC), and are problematic for comparing model performance conditional on threshold. We compare ROC curves with classification plots, which show sensitivity and specificity conditional on risk threshold. CONCLUSION: ROC curves do not offer more information than the AUC to indicate discriminative ability. To assess the model's performance for decision-making, results should be provided conditional on risk threshold. Therefore, if discriminatory ability must be visualized, classification plots are attractive.

Original publication

DOI

10.1016/j.jclinepi.2020.01.028

Type

Journal article

Journal

J Clin Epidemiol

Publication Date

23/07/2020

Keywords

Receiver-operating-characteristics curve, classification plots, risk prediction models, risk threshold