Effects of sample size on robustness and prediction accuracy of a prognostic gene signature

BackgroundFew overlap between independently developed gene signatures and poor inter-study applicability of gene signatures are two of major concerns raised in the development of microarray-based prognostic gene signatures. One recent study suggested that thousands of samples are needed to generate a robust prognostic gene signature.ResultsA data set of 1,372 samples was generated by combining eight breast cancer gene expression data sets produced using the same microarray platform and, using the data set, effects of varying samples sizes on a few performances of a prognostic gene signature were investigated. The overlap between independently developed gene signatures was increased linearly with more samples, attaining an average overlap of 16.56% with 600 samples. The concordance between predicted outcomes by different gene signatures also was increased with more samples up to 94.61% with 300 samples. The accuracy of outcome prediction also increased with more samples. Finally, analysis using only Estrogen Receptor-positive (ER+) patients attained higher prediction accuracy than using both patients, suggesting that sub-type specific analysis can lead to the development of better prognostic gene signaturesConclusionIncreasing sample sizes generated a gene signature with better stability, better concordance in outcome prediction, and better prediction accuracy. However, the degree of performance improvement by the increased sample size was different between the degree of overlap and the degree of concordance in outcome prediction, suggesting that the sample size required for a study should be determined according to the specific aims of the study.

• Publication year

  2009

• Venue

  BMC Bioinformatics

• Publication date

  2009-05-16

• Fields of study

  Biology, Medicine, Computer Science

• Identifiers
  DOI 10.1186/1471-2105-10-147PMID 19445687PMCID 2689196
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar, PubMed

• No claims are published for this paper.

• No concepts are published for this paper.

• How Large a Training Set is Needed to Develop a Classifier for Microarray Data?

  2008cited by this paper
• [''R"--project for statistical computing].

  2008cited by this paper
• Clinical application of the 70-gene profile: the MINDACT trial.

  2008cited by this paper
• Pooling breast cancer datasets has a synergetic effect on classification performance and improves signature stability

  2008cited by this paper
• Predicting prognosis using molecular profiling in estrogen receptor-positive breast cancer treated with tamoxifen

  2008cited by this paper
• The humoral immune system has a key prognostic impact in node-negative breast cancer.

  2008influential reference
• The removal of multiplicative, systematic bias allows integration of breast cancer gene expression datasets – improving meta-analysis and prediction of prognosis

  2008cited by this paper
• Strong Time Dependence of the 76-Gene Prognostic Signature for Node-Negative Breast Cancer Patients in the TRANSBIG Multicenter Independent Validation Series

  2007cited by this paper
• An immune response gene expression module identifies a good prognosis subtype in estrogen receptor negative breast cancer

  2007cited by this paper
• Is molecular profiling ready for use in clinical decision making?

  2007cited by this paper
• Python Programming Language

  2007cited by this paper
• Individualization of therapy using Mammaprint: from development to the MINDACT Trial.

  2007cited by this paper
• Definition of clinically distinct molecular subtypes in estrogen receptor-positive breast carcinomas through genomic grade.

  2007influential reference
• Enhanced NFκB and AP-1 transcriptional activity associated with antiestrogen resistant breast cancer

  2007cited by this paper
• Adjusting batch effects in microarray expression data using empirical Bayes methods.

  2007cited by this paper
• Genomic and transcriptional aberrations linked to breast cancer pathophysiologies.

  2006cited by this paper
• NCBI GEO: mining tens of millions of expression profiles—database and tools update

  2006influential reference
• EGFR Activation and Ultraviolet Light-Induced Skin Carcinogenesis

  2006cited by this paper
• Thousands of samples are needed to generate a robust gene list for predicting outcome in cancer.

  2006cited by this paper
• Concordance among gene-expression-based predictors for breast cancer.

  2006cited by this paper
• ArrayExpress—a public database of microarray experiments and gene expression profiles

  2006cited by this paper
• A consensus prognostic gene expression classifier for ER positive breast cancer

  2006cited by this paper
• Gene signature evaluation as a prognostic tool: challenges in the design of the MINDACT trial

  2006cited by this paper
• Estrogen-regulated genes predict survival in hormone receptor-positive breast cancers.

  2006cited by this paper
• Bias as a threat to the validity of cancer molecular-marker research

  2005cited by this paper
• Genes that mediate breast cancer metastasis to lung

  2005cited by this paper
• Evaluating technologies for classification and prediction in medicine

  2005cited by this paper
• Establishment and characterization of two breast cancer xenografts in immunodeficient mice for studies on hormone-dependent and hormone-independent tumor growth, progression and invasion

  2005cited by this paper
• Gene expression profiling spares early breast cancer patients from adjuvant therapy: derived and validated in two population-based cohorts

  2005cited by this paper
• Prediction of cancer outcome with microarrays: a multiple random validation strategy.

  2005influential reference
• Gene-expression profiles to predict distant metastasis of lymph-node-negative primary breast cancer.

  2005cited by this paper
• An expression signature for p53 status in human breast cancer predicts mutation status, transcriptional effects, and patient survival.

  2005cited by this paper
• Limitations of the odds ratio in gauging the performance of a diagnostic, prognostic, or screening marker.

  2004cited by this paper
• The challenges in translating present knowledge of the molecular biology of breast cancer into clinical use

  2004cited by this paper
• A multigene assay to predict recurrence of tamoxifen-treated, node-negative breast cancer.

  2004cited by this paper
• Adjustment of systematic microarray data biases

  2004cited by this paper
• Repeated observation of breast tumor subtypes in independent gene expression data sets

  2003cited by this paper
• A gene-expression signature as a predictor of survival in breast cancer.

  2002cited by this paper
• Gene expression profiling predicts clinical outcome of breast cancer

  2002cited by this paper
• Comparison of Discrimination Methods for the Classification of Tumors Using Gene Expression Data

  2002cited by this paper
• Gene expression patterns of breast carcinomas distinguish tumor subclasses with clinical implications

  2001cited by this paper
• Molecular portraits of human breast tumours

  2000cited by this paper

• Prediction of recovery grade in Bell’s palsy using machine learning models

  2026cites this paper
• Validation of a Machine-Learning Clinical Decision Aid for the Differential Diagnosis of Transient Loss of Consciousness

  2025cites this paper
• Evaluating Leaf Water Potential of Maize Through Multi-Cultivar Dehydration Experiments and Segmentation Thresholding

  2025cites this paper
• Droplet Digital PCR Improves Detection of BRCA1/2 Copy Number Variants in Advanced Prostate Cancer

  2025cites this paper
• Comparing three neural networks to predict depression treatment outcomes in psychological therapies.

  2025cites this paper
• A feature selection method for multimodal multispectral LiDAR sensing

  2024cites this paper
• An approach for developing a blood-based screening panel for lung cancer based on clonal hematopoietic mutations

  2024cites this paper
• Strategy for Identifying a Robust Metabolomic Signature Reveals the Altered Lipid Metabolism in Pituitary Adenoma.

  2024cites this paper
• Severity Identification of Graves Orbitopathy via Random Forest Algorithm

  2023cites this paper
• Fault diagnosis for rotor based on multi-sensor information and progressive strategies

  2023cites this paper
• Immersive capability and spatial presence in virtual reality photo-based tours: implications for distance education

  2023cites this paper
• A Hybrid Machine Learning Approach to Screen Optimal Predictors for the Classification of Primary Breast Tumors from Gene Expression Microarray Data

  2023cites this paper
• Correlation Between Statin Use and Symptomatic Venous Thromboembolism Incidence in Patients With Ankle Fracture: A Machine Learning Approach

  2023cites this paper
• Re-evaluation of publicly available gene-expression databases using machine-learning yields a maximum prognostic power in breast cancer

  2023cites this paper
• Predicting dropout from psychological treatment using different machine learning algorithms, resampling methods, and sample sizes

  2023cites this paper
• A Systematic Review of Applications of Machine Learning and Other Soft Computing Techniques for the Diagnosis of Tropical Diseases

  2022cites this paper
• Convolutional neural networks for expediting the determination of minimum volume requirements for studies of microstructurally small cracks, Part I: Model implementation and predictions

  2022cites this paper
• Predicting the functional outcomes of anti‐LGI1 encephalitis using a random forest model

  2022cites this paper
• Developing a new real-time traffic safety management framework for urban expressways utilizing reinforcement learning tree.

  2022cites this paper
• Improving Machine-Learning Diagnostics with Model-Based Data Augmentation Showcased for a Transformer Fault

  2021cites this paper
• Model-based data augmentation to improve the performance of machine-learning diagnostic systems

  2021cites this paper
• A Comparison of Three Airborne Laser Scanner Types for Species Identification of Individual Trees

  2021cites this paper
• Surfing the Big Data Wave: Omics Data Challenges in Transplantation

  2021cites this paper
• Limitations of Explainability for Established Prognostic Biomarkers of Prostate Cancer

  2021cites this paper
• Identifying Subgroups of Patients With Autism by Gene Expression Profiles Using Machine Learning Algorithms

  2021cites this paper
• Simple Baseline Machine Learning Text Classifiers for Small Datasets

  2021cites this paper
• WingBank: A Wing Image Database of Mosquitoes

  2021cites this paper
• Improving prediction for medical institution with limited patient data: Leveraging hospital-specific data based on multicenter collaborative research network

  2021cites this paper
• Prognostic gene expression signatures of breast cancer are lacking a sensible biological meaning

  2021cites this paper
• Determining representative sample size for validation of continuous, large continental remote sensing data

  2021cites this paper
• KPCA and AE Based Local-Global Feature Extraction Method for Vibration Signals of Rotating Machinery

  2020cites this paper
• Deep‐Learning‐Based Neural Tissue Segmentation of MRI in Multiple Sclerosis: Effect of Training Set Size

  2020cites this paper
• Effect of sample size on habitat suitability estimation using random forests: a case of bluegill, Lepomis macrochirus

  2020cites this paper
• Improvements in the Large p, Small n Classification Issue

  2020cites this paper
• A multicenter random forest model for effective prognosis prediction in collaborative clinical research network

  2020cites this paper
• Divide and Conquer: Clustering Patients With Autism by Gene Expression Pro les Using Machine Learning Algorithms

  2020cites this paper
• Classification models using circulating neutrophil transcripts can detect unruptured intracranial aneurysm

  2020cites this paper
• Predicting outcome in patients with moderate to severe traumatic brain injury using electroencephalography

  2019cites this paper
• Methods for predicting vaccine immunogenicity and reactogenicity

  2019cites this paper
• A framework for testing different imputation methods for tabular datasets

  2019cites this paper
• Feature extraction of rotor fault based on EEMD and curve code

  2019cites this paper
• The effects of sample size on omics study: from the perspective of robustness and diagnostic accuracy

  2019cites this paper
• Integrating ensemble systems biology feature selection and bimodal deep neural network for breast cancer prognosis prediction

  2019cites this paper
• A Hybrid Ensemble Approach for Identifying Robust Differentially Methylated Loci in Pan-Cancers

  2019cites this paper
• Effect of Subset Size on the Stability of Feature Selection for Gene Expression Data

  2019cites this paper
• Novel application of approaches to predicting medication adherence using medical claims data.

  2019cites this paper
• Quantile normalization for combining gene-expression datasets

  2018cites this paper
• Genome-wide expression profiling of glioblastoma using a large combined cohort

  2018cites this paper
• A data-driven interactome of synergistic genes improves network-based cancer outcome prediction

  2018cites this paper
• Experimental vaccination for onchocerciasis and the identification of early markers of protective immunity

  2018cites this paper
• Estimating the Sample Size for Training Intrusion Detection Systems

  2017cites this paper
• Opportunities and Challenges of Multiplex Assays: A Machine Learning Perspective.

  2017cites this paper
• Multiplex Biomarker Techniques: Methods and Applications

  2017cites this paper
• The Utility of Multiplex Assays for Identification of Proteomic Signatures in Psychiatry.

  2017cites this paper
• Machine learning in healthcare : an investigation into model stability

  2017cites this paper
• An experimental study of the intrinsic stability of random forest variable importance measures

  2016cites this paper
• Stabilizing l1-norm prediction models by supervised feature grouping

  2016cites this paper
• Low Concordance between Gene Expression Signatures in ER Positive HER2 Negative Breast Carcinoma Could Impair Their Clinical Application

  2016cites this paper
• Network-Assisted Disease Classification and Biomarker Discovery.

  2016cites this paper
• A Critical Perspective On Microarray Breast Cancer Gene Expression Profiling

  2015cites this paper
• Evaluation of drug-targetable genes by defining modes of abnormality in gene expression

  2015cites this paper
• iRDA: a new filter towards predictive, stable, and enriched candidate genes

  2015cites this paper
• A new vision of evaluating gene expression signatures

  2015cites this paper
• Essentially all models are wrong, but some are useful

  2015cites this paper
• Gene Expression Profiling in Human Lung Development: An Abundant Resource for Lung Adenocarcinoma Prognosis

  2014cites this paper
• ROBUST FEATURE SELECTION FROM MICRO ARRAY DATA USING LINEAR KERNEL SVM-RFE ALLIED WITH BOOTSTRAPPING

  2014cites this paper
• The prognostic value of BRCA1 promoter methylation in early stage triple negative breast cancer

  2014cites this paper
• Robust Signature Discovery for Affymetrix GeneChip® Cancer Classification

  2014cites this paper
• A Framework for High-throughput Gene Signatures with Microarray-based Brain Cancer Gene Expression Profiling Data

  2014cites this paper
• Statistical Methods for the Modelling of Label-Free Shotgun Proteomic Data in Cell Line Biomarker Discovery

  2014cites this paper
• Exploiting dependencies of pairwise comparison outcomes to predict patterns of gene response

  2014cites this paper
• Biomarkers of tolerance

  2013cites this paper
• hsa-miR-9 and drug control over the P38 network as driving disease outcome in GBM patients

  2013cites this paper
• Predicting membrane protein types by incorporating protein topology, domains, signal peptides, and physicochemical properties into the general form of Chou's pseudo amino acid composition.

  2013cites this paper
• Gene Selection Guided by Feature Interdependence

  2013cites this paper
• Sample size planning for survival prediction with focus on high‐dimensional data

  2013influential citation
• Bioinformatics and Biomedical Informatics

  2013cites this paper
• Effects of Sample Size on Differential Gene Expression, Rank Order and Prediction Accuracy of a Gene Signature

  2013cites this paper
• Biomarker robustness reveals the PDGF network as driving disease outcome in ovarian cancer patients in multiple studies

  2012cites this paper
• A Need for Biomarkers of Operational Tolerance in Liver and Kidney Transplantation

  2012cites this paper
• Gene Expression Profiles for Predicting Metastasis in Breast Cancer: A Cross-Study Comparison of Classification Methods

  2012cites this paper
• Reproducibility and Concordance of Differential DNA Methylation and Gene Expression in Cancer

  2012cites this paper
• The Interaction of Entropy-Based Discretization and Sample Size: An Empirical Study

  2012cites this paper
• Comparison of different normalization assumptions for analyses of DNA methylation data from the cancer genome.

  2012cites this paper
• A prognostic model for lymph node-negative breast cancer patients based on the integration of proliferation and immunity

  2012cites this paper
• Predicting sample size required for classification performance

  2012cites this paper
• Effect of Size and Heterogeneity of Samples on Biomarker Discovery: Synthetic and Real Data Assessment

  2012cites this paper
• A Machine Learning Approach for Identifying Novel Cell Type–Specific Transcriptional Regulators of Myogenesis

  2012cites this paper
• Biomarker lists stability in genomic studies: analysis and improvement by prior biological knowledge integration into the learning process

  2012cites this paper
• A review of the stability of feature selection techniques for bioinformatics data

  2012cites this paper
• A gene expression signature identifies two prognostic subgroups of basal breast cancer

  2011cites this paper
• Do Two Machine-Learning Based Prognostic Signatures for Breast Cancer Capture the Same Biological Processes?

  2011cites this paper
• Gene expression and network-based analysis reveals a novel role for hsa-miR-9 and drug control over the p38 network in glioblastoma multiforme progression

  2011cites this paper
• Lack of sufficiently strong informative features limits the potential of gene expression analysis as predictive tool for many clinical classification problems

  2011cites this paper
• An Evaluation Protocol for Subtype-Specific Breast Cancer Event Prediction

  2011cites this paper
• Gene and sample selection for cancer identification.

  2011cites this paper
• Hunting for robust gene signature from cancer profiling data: sources of variability, different interpretations, and recent methodological developments.

  2010cites this paper
• Stable Feature Selection for Biomarker Discovery

  2010cites this paper
• Multi-platform Data Integration in Microarray Analysis

  2009cites this paper
• A review of transcriptomics in cutaneous chemical exposure

  2009cites this paper