Stochastic Stepwise Ensembles for Variable Selection

In this article, we advocate the ensemble approach for variable selection. We point out that the stochastic mechanism used to generate the variable-selection ensemble (VSE) must be picked with care. We construct a VSE using a stochastic stepwise algorithm and compare its performance with numerous state-of-the-art algorithms. Supplemental materials for the article are available online.

• Publication year

  2010

• Venue

  Journal of Computational and Graphical Statistics

• Publication date

  2010-03-30

• Fields of study

  Mathematics, Computer Science

• Identifiers
  DOI 10.1080/10618600.2012.679223arXiv 1003.5930
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• 50 years of biometric research: Accomplishments, challenges, and opportunities

  2016cited by this paper
• Better subset regression

  2012influential reference
• RANDOM LASSO.

  2011influential reference
• Discussion of "Stability selection" by N. Meinshausen and P. Buhlmann

  2010cited by this paper
• Statistical Learning from a Regression Perspective

  2009cited by this paper
• Discussion of "Sure independence screening for ultra-high dimensional feature space" by Fan and Lv.

  2008cited by this paper
• Stability selection

  2008cited by this paper
• Variable Inclusion and Shrinkage Algorithms

  2008influential reference
• Sure independence screening for ultrahigh dimensional feature space Discussion

  2008cited by this paper
• Relaxed Lasso

  2007influential reference
• Darwinian Evolution in Parallel Universes: A Parallel Genetic Algorithm for Variable Selection

  2006influential reference
• The Adaptive Lasso and Its Oracle Properties

  2006cited by this paper
• Sure independence screening for ultrahigh dimensional feature space

  2006influential reference
• PREDICTION/ESTIMATION WITH SIMPLE LINEAR MODELS: IS IT REALLY THAT SIMPLE?

  2006cited by this paper
• Can the Strengths of AIC and BIC Be Shared

  2005cited by this paper
• [Least Angle Regression]: Discussion

  2004cited by this paper
• Computational approaches to motor learning by imitation.

  2003cited by this paper
• Importance Sampled Learning Ensembles

  2003cited by this paper
• Genetic Algorithms

  2002influential reference
• Monte Carlo Strategies in Scientific Computing

  2002cited by this paper
• Principal Component Analysis

  2002cited by this paper
• Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties

  2001influential reference
• Greedy function approximation: A gradient boosting machine.

  2001cited by this paper
• Monte Carlo strategies in scientific computing

  2001cited by this paper
• Random Forests

  2001influential reference
• Special Invited Paper-Additive logistic regression: A statistical view of boosting

  2000cited by this paper
• Additive Logistic Regression : a Statistical

  1998cited by this paper
• The Random Subspace Method for Constructing Decision Forests

  1998cited by this paper
• Experiments with a New Boosting Algorithm

  1996cited by this paper
• Regression Shrinkage and Selection via the Lasso

  1996influential reference
• Bagging Predictors

  1996influential reference
• Better subset regression using the nonnegative garrote

  1995influential reference
• De-noising by soft-thresholding

  1995influential reference
• Genetic Algorithm for Feature Selection for Parallel Classifiers

  1993cited by this paper
• On the Distribution of the Sample Correlation Coefficient

  1992cited by this paper
• Subset Selection in Regression

  1992cited by this paper
• Subset Selection in Regression

  1991influential reference
• Genetic Algorithms in Search Optimization and Machine Learning

  1988influential reference
• Information Theory and an Extension of the Maximum Likelihood Principle

  1973cited by this paper
• © Institute of Mathematical Statistics, 2010 To Explain or to Predict?

  year unknowncited by this paper
• © Institute of Mathematical Statistics, 2004 LEAST ANGLE REGRESSION

  year unknowninfluential reference

• A comparative study of the diabetes progression prediction techniques

  2025cites this paper
• A penalized variable selection ensemble algorithm for high-dimensional group-structured data

  2024cites this paper
• Research on Estimation Model of Carbon Stock Based on Airborne LiDAR and Feature Screening

  2024cites this paper
• XAIRE: An ensemble-based methodology for determining the relative importance of variables in regression tasks. Application to a hospital emergency department

  2023cites this paper
• StableMate: a new statistical method to select stable predictors in omics data

  2023cites this paper
• StableMate: a statistical method to select stable predictors in omics data

  2023influential citation
• Ensemble variable selection using genetic algorithm

  2022cites this paper
• Flexible variable selection in the presence of missing data

  2022cites this paper
• Using machine learning for the personalised prediction of revision endoscopic sinus surgery

  2022cites this paper
• Particle swarm stepwise (PaSS) algorithm for information criteria-based variable selections

  2021influential citation
• Ensemble Approach to Variable Selection

  2021cites this paper
• A novel heterogeneous ensemble approach to variable selection for gas-liquid two-phase CO2 flow metering

  2021cites this paper
• Ensemble binary segmentation for irregularly spaced data with change-points

  2020cites this paper
• On selective learning in stochastic stepwise ensembles

  2019cites this paper
• Developing and validating models for predicting nursing home admission using only RAI-HC instrument data

  2019cites this paper
• An improved polynomial-based nonlinear variable importance measure and its application to degradation assessment for high-voltage transformer under imbalance data

  2019cites this paper
• An association-based evolutionary ensemble method of variable selection

  2019cites this paper
• A novel bagging approach for variable ranking and selection via a mixed importance measure

  2018influential citation
• Early stopping aggregation in selective variable selection ensembles for high-dimensional linear regression models

  2018cites this paper
• Pruning variable selection ensembles

  2017influential citation
• Development and validation of classifiers and variable subsets for predicting nursing home admission

  2017cites this paper
• Ensembling Variable Selectors by Stability Selection for the Cox Model

  2017cites this paper
• A ranking-based strategy to prune variable selection ensembles

  2017cites this paper
• Generalizations and Applications of the Stochastic Block Model to Basketball Games and Variable Selection Problems

  2017cites this paper
• A novel technique to prune variable selection ensembles

  2017influential citation
• Stochastic correlation coefficient ensembles for variable selection

  2017influential citation
• Two-Step Constrained Nonlinear Spectral Mixture Analysis Method for Mitigating the Collinearity Effect

  2016cites this paper
• An Ensemble EM Algorithm for Bayesian Variable Selection

  2016cites this paper
• Randomizing outputs to increase variable selection accuracy

  2016influential citation
• PBoostGA: pseudo-boosting genetic algorithm for variable ranking and selection

  2016influential citation
• A Novel Bagging Ensemble Approach for Variable Ranking and Selection for Linear Regression Models

  2015cites this paper
• RandGA: injecting randomness into parallel genetic algorithm for variable selection

  2015influential citation
• Use of majority votes in statistical learning

  2015influential citation
• Opinion versus practice regarding the use of rehabilitation services in home care: an investigation using machine learning algorithms

  2015cites this paper
• A VIF-based optimization model to alleviate collinearity problems in multiple linear regression

  2014cites this paper
• Boosting variable selection algorithm for linear regression models

  2014influential citation
• Using Machine Learning to Plan Rehabilitation for Home Care Clients: Beyond "Black-Box" Predictions

  2014cites this paper
• Functional data analysis in phonetics

  2013cites this paper
• Analysis of presence-only data via semi-supervised learning approaches

  2013cites this paper
• Machine Learning Methods for Feature Selection and Rule Extraction in Genome-wide Association Studies (GWASs).

  2013cites this paper
• Variable Ranking by Solution-path Algorithms

  2012influential citation
• of Computational and Graphical Statistics

  2012cites this paper
• University of Warwick institutional repository: http://go

  2008cites this paper