Sparse Modeling of Categorial Explanatory Variables

Shrinking methods in regression analysis are usually designed for metric predictors. If independent variables are categorial some modifications are necessary. In this article two L1-penalty based methods for factor selection and clustering of categories are presented and investigated. The first approach is designed for nominal scale levels, the second one for ordinal predictors. All methods are illustrated and compared in simulation studies, and applied to real world data from the Munich rent standard. The paper is a preprint of an article published in The Annals of Applied Statistics. Please use the journal version for citation.

• Publication year

  2011

• Venue

  The Annals of Applied Statistics

• Publication date

  2011-01-07

• Fields of study

  Mathematics, Economics

• Identifiers
  DOI 10.1214/10-AOAS355arXiv 1101.1421
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Simultaneous Factor Selection and Collapsing Levels in ANOVA

  2009influential reference
• Selection of Ordinally Scaled Independent Variables

  2009cited by this paper
• Penalized Regression with Ordinal Predictors

  2009cited by this paper
• The group lasso for logistic regression

  2008cited by this paper
• Relaxed Lasso

  2007influential reference
• The Adaptive Lasso and Its Oracle Properties

  2006influential reference
• Model selection and estimation in regression with grouped variables

  2006influential reference
• Sparsity and smoothness via the fused lasso

  2005influential reference
• The Dantzig selector: Statistical estimation when P is much larger than n

  2005influential reference
• kernlab - An S4 Package for Kernel Methods in R

  2004influential reference
• Boosting With the L2 Loss

  2003cited by this paper
• Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties

  2001cited by this paper
• Boosting with the L_2-Loss: Regression and Classification

  2001cited by this paper
• Regression Shrinkage and Selection via the Lasso

  1996cited by this paper
• Coding ordinal independent variables in multiple regression analyses.

  1987cited by this paper
• © Institute of Mathematical Statistics, 2004 LEAST ANGLE REGRESSION

  year unknowninfluential reference

• R2VF: A Two-Step Regularization Algorithm to Cluster Categories in GLMs

  2025cites this paper
• Nash: Neural Adaptive Shrinkage for Structured High-Dimensional Regression

  2025cites this paper
• Collapsing Categories for Regression with Mixed Predictors

  2025influential citation
• A powerful penalized multinomial logistic regression approach

  2025cites this paper
• A high‐dimensional multinomial logit model

  2024cites this paper
• Explaining Central Government’s Tax Revenue Categories through the Bradley-Terry Regression Trunk Model

  2024cites this paper
• The Fragility of Sparsity

  2023cites this paper
• Handling categorical features with many levels using a product partition model

  2023cites this paper
• Grouped rank centrality: Ranking and grouping from pairwise comparisons simultaneously

  2023cites this paper
• Spatial Shrinkage Prior: A Probabilistic Approach to Model for Categorical Variables with Many Levels

  2023cites this paper
• Combining eye-tracking and verbal reports in vocabulary research: Benefits and challenges

  2023cites this paper
• On clustering levels of a hierarchical categorical risk factor

  2023cites this paper
• Regularization approaches in clinical biostatistics: A review of methods and their applications

  2022cites this paper
• Pattern recovery and signal denoising by SLOPE when the design matrix is orthogonal

  2022cites this paper
• Simultaneous factors selection and fusion of their levels in penalized logistic regression

  2022influential citation
• Improving Group Lasso for High-Dimensional Categorical Data

  2022cites this paper
• Improving the External Validity of Conjoint Analysis: The Essential Role of Profile Distribution

  2021cites this paper
• Group Lasso merger for sparse prediction with high-dimensional categorical data

  2021influential citation
• Sparse regression for large data sets with outliers

  2021cites this paper
• Learning sparse log-ratios for high-throughput sequencing data

  2021cites this paper
• Modelling Heterogeneity Using Bayesian Structured Sparsity

  2021cites this paper
• Multiple imputation and selection of ordinal level 2 predictors in multilevel models: An analysis of the relationship between student ratings and teacher practices and attitudes

  2020influential citation
• Statistics Corner: Reporting Descriptive Statistics

  2020cites this paper
• Automatically Identifying Relevant Variables for Linear Regression with the Lasso Method: A Methodological Primer for its Application with R and a Performance Contrast Simulation with Alternative Selection Strategies

  2020cites this paper
• Modelling high‐dimensional categorical data using nonconvex fusion penalties

  2020cites this paper
• Multiple imputation and selection of ordinal level 2 predictors in multilevel models. An analysis of the relationship between student ratings and teacher beliefs and practices

  2019influential citation
• A regularized hidden Markov model for analyzing the ‘hot shoe’ in football

  2019cites this paper
• Modelling Heterogeneity Using Complex Sparsity

  2019cites this paper
• Real Estate Dictionaries Across Space and Time

  2019cites this paper
• LASSO-type penalization in the framework of generalized additive models for location, scale and shape

  2019influential citation
• A high-dimensional multinomial choice model

  2019cites this paper
• Determining factors affecting congestion pricing acceptability

  2019cites this paper
• Finding Success in Clinical Trial Recruitment: A Trial Registry Analysis

  2019cites this paper
• Motor insurance claim modelling with factor collapsing and Bayesian model averaging

  2018cites this paper
• Sparse regression with Multi-type Regularized Feature modeling

  2018influential citation
• The Gains from Dimensionality

  2018cites this paper
• Empirical Study on High-Dimensional Variable Selection and Prediction Under Competing Risks

  2018cites this paper
• The sparse Luce model

  2018cites this paper
• Variable Selection for a Categorical Varying-Coefficient Model with Identifications for Determinants of Body Mass Index

  2017influential citation
• Effect fusion using model-based clustering

  2017cites this paper
• Structure estimation of binary graphical models on stratified data: Application to the description of injury tables for victims of road accidents

  2017cites this paper
• Modelling Clustered Heterogeneity: Fixed Effects, Random Effects and Mixtures

  2017cites this paper
• High‐dimensional variable selection and prediction under competing risks with application to SEER‐Medicare linked data

  2017cites this paper
• Homogeneity detection for the high-dimensional generalized linear model

  2017cites this paper
• Motor Insurance Accidental Damage Claims Modeling with Factor Collapsing and Bayesian Model Averaging

  2017cites this paper
• Subject-specific modelling of paired comparison data: A lasso-type penalty approach

  2017cites this paper
• Tree-structured modelling of categorical predictors in generalized additive regression

  2017cites this paper
• Modellierung der Spielstärke von Fußballmannschaften mit ordinalen Paarvergleichsmodellen

  2016cites this paper
• Structured fusion lasso penalized multi‐state models

  2016cites this paper
• Discussion of ‘regularized regression for categorical data’

  2016cites this paper
• Risk of motor vehicle collisions associated with medical conditions and medications: rationale and study protocol

  2016cites this paper
• Regularized regression for categorical data

  2016influential citation
• Frailty Models and Heterogeneity

  2016cites this paper
• Discussion: Bayesian regularization and effect smoothing for categorical predictors

  2016cites this paper
• Rejoinder: Regularized regression for categorical data

  2016cites this paper
• Predictive Accuracy of Linear Models with Ordinal Regressors

  2016cites this paper
• Estimation and model selection for dynamic biomedical images

  2015cites this paper
• Tree-Structured Clustering in Fixed Effects Models

  2015cites this paper
• Delete or merge regressors for linear model selection

  2015cites this paper
• Selection and Fusion of Categorical Predictors with L 0 -Type Penalties

  2015cites this paper
• Supervised dimension reduction for ordinal predictors

  2015influential citation
• Penalized regression for discrete structures

  2015influential citation
• Modelling Heterogeneity in Paired Comparison Data - an L1 Penalty Approach with an Application to Party Preference Data

  2015cites this paper
• Extended ordered paired comparison models with application to football data from German Bundesliga

  2015cites this paper
• A uniform framework for the combination of penalties in generalized structured models

  2015cites this paper
• Boosting in Nonlinear Regression Models with an Application to DCE-MRI Data

  2015cites this paper
• On the theory and practice of variable selection for functional data

  2015cites this paper
• Discussion of “The Evolution of Boosting Algorithms” and “Extending Statistical Boosting”

  2014cites this paper
• Extending Statistical Boosting

  2014cites this paper
• Spatially regularized estimation for the analysis of dynamic contrast‐enhanced magnetic resonance imaging data

  2014cites this paper
• Regularization and model selection with categorical predictors and effect modifiers in generalized linear models

  2014cites this paper
• A comparative study of variable selection methods in the context of developing psychiatric screening instruments

  2014cites this paper
• Extended ordered paired comparison models with application to football data from German Bundesliga

  2014cites this paper
• Fused least absolute shrinkage and selection operator for credit scoring

  2014cites this paper
• Tree-Structured Modelling of Categorical Predictors in Regression

  2014cites this paper
• High-Dimensional Sparse Matched Case-Control and Case-Crossover Data: A Review of Recent Works, Description of an R Tool and an Illustration of the Use in Epidemiological Studies

  2013cites this paper
• Sparse least trimmed squares regression for analyzing high-dimensional large data sets

  2013cites this paper
• Regularization and Model Selection with Categorical Covariates

  2013cites this paper
• Rating Scales as Predictors—The Old Question of Scale Level and Some Answers

  2013cites this paper
• A General Family of Penalties for Combining Differing Types of Penalties in Generalized Structured Models

  2013cites this paper
• Proportional Odds Models with High‐Dimensional Data Structure

  2013cites this paper
• Factor Selection and Structural Identification in the Interaction ANOVA Model

  2013cites this paper
• Group variable selection in cardiopulmonary cerebral resuscitation data for veterinary patients

  2012cites this paper
• The ranking lasso and its application to sport tournaments

  2012influential citation
• Sparse Least Trimmed Squares Regression

  2011cites this paper
• Regularization approaches for generalized linear models and single index models

  2011cites this paper
• Pairwise Fused Lasso

  2011cites this paper
• Selection of ordinally scaled independent variables with applications to international classification of functioning core sets

  2011cites this paper
• Sparsity with sign-coherent groups of variables via the cooperative-Lasso

  2011cites this paper
• Testing linearity and relevance of ordinal predictors

  2011cites this paper
• Regression for Categorical Data: List of Examples

  2011cites this paper
• Relaxed Exponential Kernels for Unsupervised Learning

  2011cites this paper
• Regularization and Model Selection with Categorial Effect Modifiers

  2010cites this paper
• Selection of Ordinally Scaled Independent Variables

  2009cites this paper
• Interpretable House Price Prediction Using a Collection of Local Machine Learning Models

  year unknowncites this paper
• Feature Extraction in Regression and Classification with Structured Predictors

  year unknowncites this paper