Caveats and pitfalls of ROC analysis in clinical microarray research (and how to avoid them)

No abstract is available for this paper.

• Publication year

  2012

• Venue

  Briefings Bioinform.

• Publication date

  Unknown publication date

• Fields of study

  Biology, Medicine, Computer Science

• Identifiers
  DOI 10.1093/bib/bbr008PMID 21422066
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar, PubMed

• No claims are published for this paper.

• No concepts are published for this paper.

• R: A language and environment for statistical computing.

  2014cited by this paper
• Diagnostic accuracy and receiver-operating characteristics curve analysis in surgical research and decision making.

  2011cited by this paper
• Encyclopedia of Machine Learning

  2011cited by this paper
• Small-sample precision of ROC-related estimates

  2010cited by this paper
• A CROC stronger than ROC: measuring, visualizing and optimizing early retrieval

  2010cited by this paper
• Deriving chemosensitivity from cell lines: Forensic bioinformatics and reproducible research in high-throughput biology

  2009cited by this paper
• Measuring classifier performance: a coherent alternative to the area under the ROC curve

  2009influential reference
• The WEKA data mining software: an update

  2009cited by this paper
• Statistical Issues in Translational Cancer Research

  2008cited by this paper
• Managing bias in ROC curves

  2008cited by this paper
• A genomic approach to colon cancer risk stratification yields biologic insights into therapeutic opportunities

  2008cited by this paper
• Receiver-operating characteristic (ROC) curve analysis in diagnostic, prognostic and predictive biomarker research

  2008influential reference
• Run batch effects potentially compromise the usefulness of genomic signatures for ovarian cancer.

  2008cited by this paper
• AUC: a misleading measure of the performance of predictive distribution models

  2008cited by this paper
• An Integrated Approach to the Prediction of Chemotherapeutic Response in Patients with Breast Cancer

  2008cited by this paper
• StAR: a simple tool for the statistical comparison of ROC curves

  2008cited by this paper
• A critical analysis of variants of the AUC

  2008cited by this paper
• ROC analysis: applications to the classification of biological sequences and 3D structures

  2008cited by this paper
• Gene expression variation to predict 10-year survival in lymph-node-negative breast cancer

  2008cited by this paper
• Pharmacogenomic strategies provide a rational approach to the treatment of cisplatin-resistant patients with advanced cancer.

  2007cited by this paper
• Use of 70-gene signature to predict prognosis of patients with node-negative breast cancer: a prospective community-based feasibility study (RASTER).

  2007cited by this paper
• Validation of gene signatures that predict the response of breast cancer to neoadjuvant chemotherapy: a substudy of the EORTC 10994/BIG 00-01 clinical trial.

  2007cited by this paper
• An integrated genomic-based approach to individualized treatment of patients with advanced-stage ovarian cancer.

  2007influential reference
• The prognostic role of a gene signature from tumorigenic breast-cancer cells.

  2007cited by this paper
• ROC Graphs: Notes and Practical Considerations for Researchers

  2007cited by this paper
• Gene expression profiling in breast cancer: understanding the molecular basis of histologic grade to improve prognosis.

  2006cited by this paper
• Cost curves: An improved method for visualizing classifier performance

  2006cited by this paper
• Classifier Technology and the Illusion of Progress

  2006cited by this paper
• Gene Expression Programs in Response to Hypoxia: Cell Type Specificity and Prognostic Significance in Human Cancers

  2006cited by this paper
• Improving Text Classifier Performance based on AUC

  2006cited by this paper
• Genomic signatures to guide the use of chemotherapeutics

  2006cited by this paper
• Pharmacogenomic predictor of sensitivity to preoperative chemotherapy with paclitaxel and fluorouracil, doxorubicin, and cyclophosphamide in breast cancer.

  2006cited by this paper
• Molecular classification of tamoxifen-resistant breast carcinomas by gene expression profiling.

  2005cited by this paper
• ROCR: visualizing classifier performance in R

  2005cited by this paper
• Robustness, scalability, and integration of a wound-response gene expression signature in predicting breast cancer survival.

  2005cited by this paper
• Microarrays and breast cancer clinical studies: forgetting what we have not yet learnt

  2005cited by this paper
• Roadmap for developing and validating therapeutically relevant genomic classifiers.

  2005cited by this paper
• Gene-expression profiles to predict distant metastasis of lymph-node-negative primary breast cancer.

  2005cited by this paper
• Bias as a threat to the validity of cancer molecular-marker research

  2005cited by this paper
• What properties should an overall measure of test performance possess?

  2005cited by this paper
• Prediction of cancer outcome with microarrays: a multiple random validation strategy.

  2005cited by this paper
• Prediction of docetaxel response in human breast cancer by gene expression profiling.

  2005cited by this paper
• ROC curves in clinical chemistry: uses, misuses, and possible solutions.

  2004cited by this paper
• Confidence Intervals for the Area Under the ROC Curve

  2004influential reference
• Genomics, prior probability, and statistical tests of multiple hypotheses.

  2004cited by this paper
• Gene expression profiles predict complete pathologic response to neoadjuvant paclitaxel and fluorouracil, doxorubicin, and cyclophosphamide chemotherapy in breast cancer.

  2004cited by this paper
• AUC: a Statistically Consistent and more Discriminating Measure than Accuracy

  2003cited by this paper
• Gene expression profiling for the prediction of therapeutic response to docetaxel in patients with breast cancer.

  2003cited by this paper
• Supervised analysis when the number of candidate features (p) greatly exceeds the number of cases (n)

  2003cited by this paper
• Gene expression profiling predicts clinical outcome of breast cancer

  2002cited by this paper
• Low P-Values or Narrow Confidence Intervals: Which Are More Durable?

  2001cited by this paper
• Gene expression patterns of breast carcinomas distinguish tumor subclasses with clinical implications

  2001cited by this paper
• Comparing classifiers when the misallocation costs are uncertain

  1999cited by this paper
• Writing for Epidemiology

  1998cited by this paper
• Regret graphs, diagnostic uncertainty and Youden's Index.

  1996cited by this paper
• The Area under the ROC Curve and Its Competitors

  1991influential reference
• Analyzing a Portion of the ROC Curve

  1989cited by this paper
• Better Bootstrap Confidence Intervals

  1987cited by this paper
• Confidence intervals rather than P values: estimation rather than hypothesis testing.

  1986cited by this paper
• The meaning and use of the area under a receiver operating characteristic (ROC) curve.

  1982cited by this paper
• A Simple Sequentially Rejective Multiple Test Procedure

  1979cited by this paper
• A show of confidence.

  1978cited by this paper

• Using workload capacity indicators to evaluate rule-based early warning tools and their relationship to escalation events

  2026cites this paper
• Review of the psychometric properties and measurement invariance of the Adult Self-Report Scale for ADHD in a sample of employees in Puerto Rico.

  2026influential citation
• Metabolic and Immune Adaptations in Preterm Neonates at Early Postnatal Period: Integrated Analysis of Key Metabolites and Pathways

  2025influential citation
• A Problem-Oriented Taxonomy of Evaluation Metrics for Time Series Anomaly Detection

  2025cites this paper
• Stress induces behavioral disorders by promoting microglial phagocytosis via decreasing neuronal Dkk3

  2025cites this paper
• Network pharmacology combined with experimental verification for exploring the potential antidepressant mechanism of Traditional Chinese Medicine Buyang Huanwu Decoction in lipopolysaccharide-induced depressed mouse model.

  2025cites this paper
• Sensitivity Analysis of the Soil Criterion in Forest Road Planning

  2025cites this paper
• Assessment of Salivary Biomarkers of Gastric Ulcer in Horses from a Clinical Perspective

  2025cites this paper
• EEG microstate analysis in children with prolonged disorders of consciousness

  2025cites this paper
• Supervised Machine Learning Thyroid Carcinoma Diagnosis Using Wide-Field SHG Microscopy

  2025cites this paper
• Pressure Injury Prediction in Intensive Care Units Using Artificial Intelligence: A Scoping Review

  2025cites this paper
• Early identification of bovine pregnancy status and embryonic mortality

  2025cites this paper
• International external validation of the SORG machine learning algorithm for predicting sustained postoperative opioid prescription after anterior cervical discectomy and fusion using a Taiwanese cohort of 1,037 patients.

  2025cites this paper
• Evaluation of performance measures in predictive artificial intelligence models to support medical decisions: overview and guidance.

  2025cites this paper
• LOCC: a novel visualization and scoring of cutoffs for continuous variables with hepatocellular carcinoma prognosis as an example

  2024cites this paper
• Direct detection of 4-dimensions of SARS-CoV-2: infection (vRNA), infectivity (antigen), binding antibody, and functional neutralizing antibody in saliva

  2024cites this paper
• External validation of the German Registry for Acute Aortic Dissection Type A score in patients undergoing surgery for acute type A aortic dissection

  2024cites this paper
• Anomaly Detection in Spacecraft Telemetry: Forecasting vs. Classification

  2024cites this paper
• Machine learning-based predictive models for perioperative major adverse cardiovascular events in patients with stable coronary artery disease undergoing noncardiac surgery

  2024cites this paper
• Prediction Model of Delayed Hemothorax in Patients with Traumatic Occult Hemothorax Using a Novel Nomogram

  2024cites this paper
• Construction of a Risk Prediction Model for Postoperative Urinary Tract Infection in Women Undergoing Transvaginal Gynecologic Urinary Surgery

  2024influential citation
• Area under the ROC Curve has the most consistent evaluation for binary classification

  2024cites this paper
• A Model-Agnostic Interpretability Approach to Predicting Customer Churn in the Telecommunications Industry

  2024cites this paper
• Machine learning model with output correction: Towards reliable bradycardia detection in neonates

  2024cites this paper
• External Validation of the ARCH Score in Patients Undergoing Aortic Arch Reconstruction Under Circulatory Arrest.

  2024cites this paper
• Towards optimal model evaluation: enhancing active testing with actively improved estimators

  2024cites this paper
• Non-obese lipedema patients show a distinctly altered quantitative sensory testing profile with high diagnostic potential

  2024cites this paper
• Multiplex Serology for Sensitive and Specific Flavivirus IgG Detection: Addition of Envelope Protein Domain III to NS1 Increases Sensitivity for Tick-Borne Encephalitis Virus IgG Detection

  2024cites this paper
• Improving the Prediction of 1-Year Right Ventricular Failure After Left Ventricular Assist Device Implantation

  2024cites this paper
• The Matthews correlation coefficient (MCC) should replace the ROC AUC as the standard metric for assessing binary classification

  2023cites this paper
• Direct Detection of 4-Dimensions of SARS-CoV-2: Infection (vRNA), Infectivity (Antigen), Binding Antibody, and Functional Neutralizing Antibody in Saliva

  2023cites this paper
• Novel nomogram for predicting paradoxical chest wall movement in patients with flail segment of traumatic rib fracture: a retrospective cohort study

  2023cites this paper
• DeepGAMI: deep biologically guided auxiliary learning for multimodal integration and imputation to improve genotype–phenotype prediction

  2023cites this paper
• A critical review of the hemodynamics in assessment of volume responsiveness by using passive leg raising (PLR)

  2023cites this paper
• The Role of the Multi-Inflammatory Index as a Novel Predictor of Hospital Mortality in Acute Ischemic Stroke

  2023cites this paper
• More accurate simulation for insurance data based on a modified SVM polynomial method

  2023cites this paper
• Changes in serum amyloid A, plasma high-density lipoprotein cholesterol and apolipoprotein A-I as useful biomarkers for Mycobacterium tuberculosis infection.

  2023cites this paper
• Lipedema patients show a distinctly altered Quantitative Sensory Testing (QST) profile

  2023cites this paper
• LOCC: a novel visualization and scoring of cutoffs for continuous variables

  2023cites this paper
• Navigating the metric maze: a taxonomy of evaluation metrics for anomaly detection in time series

  2023cites this paper
• Three-dimensional hierarchical plasmonic nano-architecture based label-free surface-enhanced Raman spectroscopy detection of urinary exosomal miRNA for clinical diagnosis of prostate cancer.

  2022cites this paper
• Structure-based discovery and bio-evaluation of a cyclopenta[4,5]thieno[2,3-d]pyrimidin-4-one as a phosphodiesterase 10A inhibitor

  2022cites this paper
• Construct validity of the Single Arm Military Press (SAMP) test for upper limb function in patients with neck pain.

  2022cites this paper
• Chronic stress causes striatal disinhibition mediated by SOM-interneurons in male mice

  2022cites this paper
• PToPI: A Comprehensive Review, Analysis, and Knowledge Representation of Binary Classification Performance Measures/Metrics

  2022cites this paper
• Identifying patients at risk of unplanned re-hospitalisation using statewide electronic health records

  2022cites this paper
• Explainable machine learning for real-time deterioration alert prediction to guide pre-emptive treatment

  2022cites this paper
• N 6 -Methyladenosine-Related Long Non-Coding RNAs Are Identified as a Potential Prognostic Biomarker for Lung Squamous Cell Carcinoma and Validated by Real-Time PCR

  2022cites this paper
• Performance metric curve analysis framework to assess impact of the decision variable threshold, disease prevalence, and dataset variability in two-class classification

  2022cites this paper
• A Comparison between Accelerated Failure Time Models in Analyzing the Survival of Breast Cancer Patients

  2022cites this paper
• A Comparative Analysis of Novel Biomarkers in Sepsis and Cardiovascular Disease

  2022cites this paper
• Binary disease prediction using tail quantiles of the distribution of continuous biomarkers

  2021cites this paper
• Immune variations throughout the course of tuberculosis treatment and its relationship with adrenal hormone changes in HIV-1 patients co-infected with Mycobacterium tuberculosis.

  2021cites this paper
• Identification of 14-3-3 epsilon as a regulator of the neural apoptotic pathway for chronic-stress-induced depression

  2021cites this paper
• Usefulness of postural sway spectral analysis in the diagnostic route and clinical integration of cervicogenic and vestibular sources of dizziness: A cross-sectional preliminary study

  2021cites this paper
• Pathogenic potential assessment of the Shiga toxin–producing Escherichia coli by a source attribution–considered machine learning model

  2021cites this paper
• A real use case of semi-supervised learning for mammogram classification in a local clinic of Costa Rica

  2021cites this paper
• Acute myocardial injury secondary to severe acute liver failure: A retrospective analysis supported by animal data

  2021cites this paper
• Causal reasoning over knowledge graphs leveraging drug-perturbed and disease-specific transcriptomic signatures for drug discovery

  2021cites this paper
• Chemical imaging and machine learning for sub‐classification of oesophageal tissue histology

  2021cites this paper
• Development and Characterization of a Multiplex Assay to Quantify Complement-Fixing Antibodies against Dengue Virus

  2021cites this paper
• Identifying diverse metal oxide nanomaterials with lethal effects on embryonic zebrafish using machine learning

  2021cites this paper
• Using predictive machine learning models for drug response simulation by calibrating patient-specific pathway signatures

  2020cites this paper
• Deep learning as a predictive tool for fetal heart pregnancy following time-lapse incubation and blastocyst transfer.

  2020cites this paper
• A Comparison of T2 Relaxation-Based MRI Stroke Timing Methods in Hyperacute Ischemic Stroke Patients: A Pilot Study

  2020cites this paper
• Pediatric contact allergy: A comparative study with adults

  2020cites this paper
• A Somatic Coliphage Threshold Approach To Improve the Management of Activated Sludge Wastewater Treatment Plant Effluents in Resource-Limited Regions

  2020cites this paper
• Drug2ways: Reasoning over causal paths in biological networks for drug discovery

  2020cites this paper
• Limitations of ROC on Imbalanced Data: Evaluation of LVAD Mortality Risk Scores

  2020cites this paper
• Effects of Study Population, Labeling and Training on Glaucoma Detection Using Deep Learning Algorithms

  2020cites this paper
• Redox-sensitive trace metals as paleoredox proxies: A review and analysis of data from modern sediments

  2020cites this paper
• A Practitioner Methodology for Mitigating Electronic Data Risk A Practitioner Methodology for Mitigating Electronic Data Risk Associated with Human Error Associated with Human Error

  2020influential citation
• In vitro functional characterization and in silico prediction of rare genetic variation in the bile acid and drug transporter, Na+-taurocholate co-transporting polypeptide (NTCP, SLC10A1).

  2020cites this paper
• Evaluation of an optimal cutoff of parathyroid venous sampling gradient for localizing primary hyperparathyroidism

  2020cites this paper
• Chronic Stress Induces Activity, Synaptic, and Transcriptional Remodeling of the Lateral Habenula Associated with Deficits in Motivated Behaviors.

  2019cites this paper
• Deep Learning-Based Damage Mapping With InSAR Coherence Time Series

  2019cites this paper
• Sighting acute myocardial infarction through platelet gene expression

  2019cites this paper
• Rethinking Drug Repositioning and Development with Artificial Intelligence, Machine Learning, and Omics

  2019cites this paper
• Evaluation of diagnostic and treatment strategies for the management of Helicobacter pylori infection

  2019cites this paper
• MicroRNAs in porcine uterus and serum are affected by zearalenone and represent a new target for mycotoxin biomarker discovery

  2019cites this paper
• ROC Curves for the Statistical Analysis of Microarray Data.

  2019cites this paper
• Performance Evaluation in Machine Learning: The Good, the Bad, the Ugly, and the Way Forward

  2019cites this paper
• Performance Measures for Binary Classification

  2019cites this paper
• Patient Clustering Improves Efficiency of Federated Machine Learning to predict mortality and hospital stay time using distributed Electronic Medical Records

  2019cites this paper
• A urinary microRNA panel that is an early predictive biomarker of delayed graft function following kidney transplantation

  2019cites this paper
• Bayes' Theorem and Naive Bayes Classifier

  2019cites this paper
• Chronic stress induces activity, synaptic and transcriptional remodeling of the lateral habenula associated with deficits in motivated behaviors

  2019cites this paper
• Rainfall Predictive Approach for La Trinidad, Benguet using Machine Learning Classification

  2019influential citation
• What (not) to expect when classifying rare events

  2018cites this paper
• Partner‐specific prediction of RNA‐binding residues in proteins: A critical assessment

  2018cites this paper
• Discovery of natural product inhibitors of phosphodiesterase 10A as novel therapeutic drug for schizophrenia using a multistep virtual screening

  2018cites this paper
• Circulating levels of adiponectin and leptin in patients with prostate cancer

  2018cites this paper
• Model Evaluation Guidelines for Geomagnetic Index Predictions

  2018cites this paper
• Screening for intellectual disabilities: a validation of the Hayes Ability Screening Index for in-patients with substance use disorder

  2018cites this paper
• Combination of conventional culture, vial culture, and broad-range PCR of sonication fluid for the diagnosis of prosthetic joint infection.

  2018cites this paper
• EER Calculation and DET Approximation in a Multi-Threshold Biometric System

  2018cites this paper
• Diagnostic Accuracy of Transient Ischemic Attack from Physician Claims

  2017cites this paper
• Improving Performance Prediction on Education Data with Noise and Class Imbalance

  2017cites this paper
• The receiver operating characteristic (ROC) curve

  2017cites this paper
• Enhancing prediction on imbalance data by thresholding technique with noise filtering

  2017cites this paper