The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Classifiers on Imbalanced Datasets
Binary classifiers are routinely evaluated with performance measures such as sensitivity and specificity, and performance is frequently illustrated with Receiver Operating Characteristics (ROC) plots. Alternative measures such as positive predictive value (PPV) and the associated Precision/Recall (PRC) plots are used less frequently. Many bioinformatics studies develop and evaluate classifiers that are to be applied to strongly imbalanced datasets in which the number of negatives outweighs the number of positives significantly. While ROC plots are visually appealing and provide an overview of a classifier's performance across a wide range of specificities, one can ask whether ROC plots could be misleading when applied in imbalanced classification scenarios. We show here that the visual interpretability of ROC plots in the context of imbalanced datasets can be deceptive with respect to conclusions about the reliability of classification performance, owing to an intuitive but wrong interpretation of specificity. PRC plots, on the other hand, can provide the viewer with an accurate prediction of future classification performance due to the fact that they evaluate the fraction of true positives among positive predictions. Our findings have potential implications for the interpretation of a large number of studies that use ROC plots on imbalanced datasets.
this paper
works it cites
works citing it
node size = global citations · hover for the full title
Citation edge evidence
- The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy i…→The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Cl…Legacy edge — source detail unavailable
- Classification assessment methods→The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Cl…Legacy edge — source detail unavailable
- The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Cl…→SMOTE: Synthetic Minority Over-sampling TechniqueLegacy edge — source detail unavailable
- The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Cl…→An introduction to ROC analysisLegacy edge — source detail unavailable
- The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Cl…→A training algorithm for optimal margin classifiersLegacy edge — source detail unavailable
- The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Cl…→R: A Language for Data Analysis and GraphicsLegacy edge — source detail unavailable
- The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Cl…→Learning from Imbalanced DataLegacy edge — source detail unavailable
- The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Cl…→The relationship between Precision-Recall and ROC curvesLegacy edge — source detail unavailable
- The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Cl…→ROCR: visualizing classifier performance in RLegacy edge — source detail unavailable
- The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Cl…→The class imbalance problem: A systematic study1Legacy edge — source detail unavailable
- The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Cl…→EditorialLegacy edge — source detail unavailable
- The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Cl…→The foundations of cost-sensitive learningLegacy edge — source detail unavailable
- The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Cl…→Cost-sensitive boosting for classification of imbalanced dataLegacy edge — source detail unavailable
- The class imbalance problem: A systematic study1→The foundations of cost-sensitive learningLegacy edge — source detail unavailable
- Editorial→The foundations of cost-sensitive learningLegacy edge — source detail unavailable
- Editorial→The class imbalance problem: A systematic study1Legacy edge — source detail unavailable
- Editorial→SMOTE: Synthetic Minority Over-sampling TechniqueLegacy edge — source detail unavailable
- Cost-sensitive boosting for classification of imbalanced data→The foundations of cost-sensitive learningLegacy edge — source detail unavailable
- Cost-sensitive boosting for classification of imbalanced data→The class imbalance problem: A systematic study1Legacy edge — source detail unavailable
- Cost-sensitive boosting for classification of imbalanced data→EditorialLegacy edge — source detail unavailable
- Cost-sensitive boosting for classification of imbalanced data→SMOTE: Synthetic Minority Over-sampling TechniqueLegacy edge — source detail unavailable
- Learning from Imbalanced Data→The foundations of cost-sensitive learningLegacy edge — source detail unavailable
- Learning from Imbalanced Data→The class imbalance problem: A systematic study1Legacy edge — source detail unavailable
- Learning from Imbalanced Data→The relationship between Precision-Recall and ROC curvesLegacy edge — source detail unavailable
- Learning from Imbalanced Data→EditorialLegacy edge — source detail unavailable
- Learning from Imbalanced Data→Cost-sensitive boosting for classification of imbalanced dataLegacy edge — source detail unavailable
- Learning from Imbalanced Data→SMOTE: Synthetic Minority Over-sampling TechniqueLegacy edge — source detail unavailable
- Learning from Imbalanced Data→An introduction to ROC analysisLegacy edge — source detail unavailable
- Classification assessment methods→The relationship between Precision-Recall and ROC curvesLegacy edge — source detail unavailable
- Classification assessment methods→Learning from Imbalanced DataLegacy edge — source detail unavailable
- Classification assessment methods→An introduction to ROC analysisLegacy edge — source detail unavailable
- The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy i…→An introduction to ROC analysisLegacy edge — source detail unavailable
- The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy i…→Classification assessment methodsLegacy edge — source detail unavailable
What this paper cites, inside the corpus
| Paper | Year | Cited |
|---|---|---|
| SMOTE: Synthetic Minority Over-sampling Technique | 2002 | 32,402 |
| An introduction to ROC analysis | 2005 | 21,837 |
| A training algorithm for optimal margin classifiers | 1992 | 11,664 |
| R: A Language for Data Analysis and Graphics | 1996 | 10,486 |
| Learning from Imbalanced Data | 2009 | 10,209 |
| The relationship between Precision-Recall and ROC curves | 2006 | 6,427 |
| ROCR: visualizing classifier performance in R | 2005 | 3,367 |
| The class imbalance problem: A systematic study1 | 2002 | 3,353 |
| Editorial | 2004 | 2,081 |
| The foundations of cost-sensitive learning | 2001 | 1,837 |
| Cost-sensitive boosting for classification of imbalanced data | 2007 | 1,438 |
What cites it, inside the corpus
| Paper | Year | Cited |
|---|---|---|
| The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy i… | 2020 | 6,132 |
| Classification assessment methods | 2018 | 2,377 |
Links
Download BibTeX · Download RIS · DOI · Open access copy · OpenAlex record
Topics
| Imbalanced Data Classification Techniques | Computer Science |
| Financial Distress and Bankruptcy Prediction | Business, Management and Accounting |
| Data Mining Algorithms and Applications | Computer Science |
Is this record sound?
complete
Nothing in this record contradicts itself and no field we check is missing.
- supports2 author record(s) attached.
- supports57 reference(s) recorded.
- neutralThe DOI carries no year to check against.
- supportsA title is present.
Provenance
sha256 648f8491aaceaa34…