Higher criticism thresholding: Optimal feature selection when useful features are rare and weak

No abstract is available for this paper.

• Publication year

  2008

• Venue

  Proceedings of the National Academy of Sciences of the United States of America

• Publication date

  2008-09-30

• Fields of study

  Biology, Mathematics, Computer Science, Medicine

• Identifiers
  DOI 10.1073/pnas.0807471105PMID 18815365PMCID PMC2553037
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar, PubMed

• No claims are published for this paper.

• No concepts are published for this paper.

• Impossibility of successful classification when useful features are rare and weak

  2009cited by this paper
• Behavior

  2009cited by this paper
• Theoretical measures of relative performance of classifiers for high dimensional data with small sample sizes

  2008cited by this paper
• 15 Years of Reproducible Research in Computational Harmonic Analysis

  2008cited by this paper
• High Dimensional Classification Using Features Annealed Independence Rules.

  2007cited by this paper
• Adapting to unknown sparsity by controlling the false discovery rate

  2005cited by this paper
• Expression Data

  2004influential reference
• Higher criticism for detecting sparse heterogeneous mixtures

  2004influential reference
• Large-Scale Simultaneous Hypothesis Testing

  2004cited by this paper
• An Introduction to Multivariate Statistical Analysis (3rd ed.) (Book)

  2004cited by this paper
• Some theory for Fisher''s linear discriminant function

  2004cited by this paper
• Detecting and estimating sparse mixtures

  2003cited by this paper
• Boosting for Tumor Classification with Gene Expression Data

  2003cited by this paper
• Diagnosis of multiple cancer types by shrunken centroids of gene expression

  2002influential reference
• Gene expression correlates of clinical prostate cancer behavior.

  2002cited by this paper
• Random Forests

  2001cited by this paper
• Empirical Bayes Analysis of a Microarray Experiment

  2001cited by this paper
• Gene-expression profiles in hereditary breast cancer.

  2001cited by this paper
• Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays.

  1999cited by this paper
• Molecular classification of cancer: class discovery and class prediction by gene expression monitoring.

  1999cited by this paper
• A Tutorial on Support Vector Machines for Pattern Recognition

  1998cited by this paper
• BEXA: A Covering Algorithm for Learning Propositional Concept Descriptions

  1996cited by this paper
• Controlling the false discovery rate: a practical and powerful approach to multiple testing

  1995cited by this paper
• Wavelets and Statistics

  1995cited by this paper
• In Wavelets and Statistics

  1995cited by this paper
• Minimax Risk over l p-Balls for l q-error

  1994cited by this paper
• Minimax risk overlp-balls forlp-error

  1994cited by this paper
• Maximum Entropy and the Nearly Black Object

  1992cited by this paper
• An Introduction to Multivariate Statistical Analysis

  1986cited by this paper
• Acknowledgments.-The authors would like to

  1982cited by this paper
• An Introduction to Multivariate Statistical Analysis

  1959cited by this paper

• Beyond agroecological delineation: A functional understanding of productivity drivers across row crop agroecosystems in the United States

  2026cites this paper
• ReliefF guided variable spiral tuna swarm optimization algorithm with somersault foraging for feature selection

  2025cites this paper
• A Comparison of DeepSeek and other LLMs

  2025influential citation
• DILC-ESAE: Data-Info envelope stacked autoencoder on correlation among samples rather than themselves

  2025cites this paper
• A Chi-Square Side-Channel Energy Leakage Assessment Method Based on HC Test

  2025cites this paper
• Optimal Network-Guided Covariate Selection for High-Dimensional Data Integration

  2025influential citation
• Can we spot a fake?

  2024influential citation
• Detection of Sparse Mixtures With Differential Privacy

  2024cites this paper
• Detecting adversarial attacks on random samples

  2024cites this paper
• Higher-criticism for sparse multi-stream change-point detection

  2024cites this paper
• Neighboring Envelope Embedded Stacked Autoencoder for Deep Learning on Hierarchically Structured Samples

  2024cites this paper
• Benign Overfitting under Learning Rate Conditions for $\alpha$ Sub-exponential Input

  2024cites this paper
• Critical biblical studies via word frequency analysis: Unveiling text authorship

  2024cites this paper
• Discovering Candidate Genes Regulated by GWAS Signals in Cis and Trans

  2024cites this paper
• Higher criticism for rare and weak non-proportional hazard deviations in survival analysis

  2023influential citation
• Differentially Private Detection of Sparse Gaussian Mixtures

  2023cites this paper
• Improving clinical decision support through interpretable machine learning and error handling in electronic health records

  2023cites this paper
• Overparameterized Maximum Likelihood Tests for Detection of Sparse Vectors

  2023cites this paper
• An Information-Theoretic Method for Detecting Edits in AI-Generated Text

  2023cites this paper
• The impact of sources of heterogeneity on accuracy of genomic prediction

  2023cites this paper
• Metabolic profiling of in vivo right ventricular function and exercise performance in pulmonary arterial hypertension

  2023cites this paper
• QDA classification of high-dimensional data with rare and weak signals

  2023cites this paper
• A new clustering framework

  2023cites this paper
• Bayesian screening for feature selection

  2022influential citation
• Accurate Tumor Subtype Detection with Raman Spectroscopy via Variational Autoencoder and Machine Learning

  2022cites this paper
• Machine Learning Algorithms Used in PSE Environments: A Didactic Approach and Critical Perspective

  2022cites this paper
• The Lifecycle of a Statistical Model: Model Failure Detection, Identification, and Refitting

  2022cites this paper
• Comparison of Two Automated Targeted Metabolomics Programs to Manual Profiling by an Experienced Spectroscopist for 1H-NMR Spectra

  2022cites this paper
• A new Stack Autoencoder: Neighbouring Sample Envelope Embedded Stack Autoencoder Ensemble Model

  2022cites this paper
• MAP and Bayes tests in sparse vectors detection

  2022cites this paper
• The general goodness-of-fit tests for correlated data

  2022cites this paper
• Learning Gaussian Mixtures with Generalised Linear Models: Precise Asymptotics in High-dimensions

  2021influential citation
• [Psychosis speech recognition algorithm based on deep embedded sparse stacked autoencoder and manifold ensemble].

  2021cites this paper
• Detecting anomalies in heterogeneous population–scale VAT networks

  2021cites this paper
• Hybrid Feature Embedded Sparse Stacked Autoencoder and Manifold Dimensionality Reduction Ensemble for Mental Health Speech Recognition

  2021cites this paper
• Supplementary information for Learning Gaussian Mixtures with Generalised Linear Models: Precise Asymptotics in High-dimensions

  2021cites this paper
• Focused goodness of fit tests for gene set analyses

  2021cites this paper
• Automated Feature Selection and Classi�cation for High-Dimensional Biomedical Data

  2021cites this paper
• ChunkedHCs algorithm for authorship verification problems: Reddit case study

  2021cites this paper
• An adaptive decorrelation procedure for signal detection

  2021cites this paper
• The impossibility region for detecting sparse mixtures using the higher criticism

  2021cites this paper
• On the Inability of the Higher Criticism to Detect Rare/Weak Departures

  2021cites this paper
• Supervised dimensionality reduction for big data

  2021cites this paper
• Coefficient tree regression: fast, accurate and interpretable predictive modeling

  2021influential citation
• Performance of approaches relying on multidimensional intermediary data to decipher causal relationships between the exposome and health: A simulation study under various causal structures.

  2021cites this paper
• Embedded stacked group sparse autoencoder ensemble with L1 regularization and manifold reduction

  2021cites this paper
• Risk Bounds for Over-parameterized Maximum Margin Classification on Sub-Gaussian Mixtures

  2021cites this paper
• High Dimensional Quadratic Discriminant Analysis: Optimality and Phase Transitions

  2021influential citation
• Modeling the dynamics of drought resilience in South-Central United States using a Bayesian Network

  2020cites this paper
• Robust tests of the equality of two high-dimensional covariance matrices

  2020cites this paper
• Hybrid Embedded Deep Stacked Sparse Autoencoder with w_LPPD SVM Ensemble

  2020cites this paper
• Finite-sample analysis of interpolating linear classifiers in the overparameterized regime

  2020cites this paper
• Two-sample Testing for Large, Sparse High-Dimensional Multinomials under Rare/Weak Perturbations

  2020cites this paper
• Higher criticism to compare two large frequency tables, with sensitivity to possible rare and weak differences

  2020cites this paper
• Beyond HC: More sensitive tests for rare/weak alternatives

  2020cites this paper
• Relying on repeated biospecimens to reduce the effects of classical-type exposure measurement error in studies linking the exposome to health.

  2020cites this paper
• Distributions and Power of Optimal Signal-Detection Statistics in Finite Case

  2020cites this paper
• A Comprehensive Analysis of Nature-Inspired Meta-Heuristic Techniques for Feature Selection Problem

  2020cites this paper
• A weak‐signal‐assisted procedure for variable selection and statistical inference with an informative subsample

  2020cites this paper
• Higher Criticism for Discriminating Word-Frequency Tables and Testing Authorship

  2019cites this paper
• Higher criticism for discriminating word-frequency tables and authorship attribution

  2019cites this paper
• Signal detection in extracellular neural ensemble recordings using higher criticism

  2019cites this paper
• Diagonally Dominant Principal Component Analysis

  2019cites this paper
• The Univariate Flagging Algorithm (UFA): An interpretable approach for predictive modeling

  2019influential citation
• A Strongly Consistent Sparse k-means Clustering with Direct l1 Penalization on Variable Weights

  2019cites this paper
• 8-31-2018 p-Value Histograms : Inference and Diagnostics

  2019cites this paper
• Efficient Nonprofiling 2nd-Order Power Analysis on Masked Devices Utilizing Multiple Leakage Points

  2019cites this paper
• On the asymptotic distribution of the scan statistic for empirical distributions

  2019cites this paper
• Cost-sensitive selection of variables by ensemble of model sequences

  2019cites this paper
• Emerging disparities in community resilience to drought hazard in south-central United States

  2019cites this paper
• Recurrent Neural Network Based Feature Selection for High Dimensional and Low Sample Size Micro-array Data

  2019cites this paper
• Variational discriminant analysis with variable selection

  2018cites this paper
• ALTERNATIVE TO HIGHER CRITICISM FOR HIGH-DIMENSIONAL MEANS UNDER SPARSITY AND COLUMN-WISE DEPENDENCE By

  2018cites this paper
• Generalized Goodness-Of-Fit Tests for Correlated Data

  2018cites this paper
• Sketching Method for Large Scale Combinatorial Inference

  2018cites this paper
• Beyond smartphones and sensors: choosing appropriate statistical methods for the analysis of longitudinal data

  2018cites this paper
• The impact of different sources of heterogeneity on loss of accuracy from genomic prediction models

  2018influential citation
• Detection of sparse positive dependence

  2018cites this paper
• p-Value Histograms: Inference and Diagnostics

  2018cites this paper
• Optimal Covariance Estimation for Condition Number Loss in the Spiked model

  2018cites this paper
• Sufficient Dimension Reduction for Classification

  2018cites this paper
• Provable Variable Selection for Streaming Features

  2018cites this paper
• Diagonal Discriminant Analysis With Feature Selection for High-Dimensional Data

  2018cites this paper
• Reduced-Rank Modeling for High-Dimensional Model-Based Clustering

  2018cites this paper
• Feature screening in large scale cluster analysis

  2017cites this paper
• Sparse Gaussian mixture detection: Low complexity, high performance tests via quantization

  2017cites this paper
• Optimizing parameters and algorithms of multivariate pattern classification for hypothesis testing in high-density EEG

  2017cites this paper
• Distributions and Statistical Power of Optimal Signal-Detection Methods In Finite Cases

  2017cites this paper
• Statistics in Side Channel Analysis -Modeling, Metric, Leakage Detection Testing

  2017cites this paper
• The power of tests for signal detection in high-dimensional data

  2017cites this paper
• 50 Years of Data Science

  2017cites this paper
• Geometric Dimensionality Reduction for Subsequent Classification

  2017cites this paper
• Large scale inference under sparse and weak alternatives: non-asymptotic phase diagram for CsCsHM statistics

  2017cites this paper
• Adaptive False Discovery Rate for Wavelet Denoising of Pavement Continuous Deflection Measurements

  2017cites this paper
• Towards Sound and Optimal Leakage Detection Procedure

  2017cites this paper
• ( A ) Obtain Samples ( B ) Learn Projection ( C ) Project Samples ( D ) Learn Classifier

  2017cites this paper
• Signal localization: a new approach in signal discovery

  2017cites this paper
• Study design in high-dimensional classification analysis.

  2016cites this paper
• Detecting Rare and Weak Spikes in Large Covariance Matrices

  2016cites this paper
• Signal identication in ERP data by decorrelated Higher

  2016cites this paper