SPARSITY IN MULTIPLE KERNEL LEARNING

The problem of multiple kernel learning based on penalized empirical risk minimization is discussed. The complexity penalty is determined jointly by the empirical L2 norms and the reproducing kernel Hilbert space (RKHS) norms induced by the kernels with a data-driven choice of regularization parameters. The main focus is on the case when the total number of kernels is large, but only a relatively small number of them is needed to represent the target function, so that the problem is sparse. The goal is to establish oracle inequalities for the excess risk of the resulting prediction rule showing that the method is adaptive both to the unknown design distribution and to the sparsity of the problem.

• Publication year

  2010

• Venue

  Annals of Statistics

• Publication date

  2010-12-01

• Fields of study

  Mathematics, Computer Science

• Identifiers
  DOI 10.1214/10-AOS825arXiv 1211.2998
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Oracle inequalities in empirical risk minimization and sparse recovery problems

  2011cited by this paper
• Sparse recovery in convex hulls via entropy penalization

  2009cited by this paper
• Lower bounds on minimax rates for nonparametric regression with additive sparsity and smoothness

  2009cited by this paper
• The Dantzig selector and sparsity oracle inequalities

  2009cited by this paper
• Sparsity in penalized empirical risk minimization

  2009cited by this paper
• Statistical performance of support vector machines

  2008cited by this paper
• Introduction to Nonparametric Estimation

  2008cited by this paper
• SIMULTANEOUS ANALYSIS OF LASSO AND DANTZIG SELECTOR

  2008cited by this paper
• Sparse Recovery in Large Ensembles of Kernel Machines On-Line Learning and Bandits

  2008influential reference
• High-dimensional additive modeling

  2008cited by this paper
• SIMULTANEOUS ANALYSIS OF LASSO AND DANTZIG

  2007cited by this paper
• Consistency of the group Lasso and multiple kernel learning

  2007cited by this paper
• SpAM: Sparse Additive Models

  2007cited by this paper
• Learning Bounds for Support Vector Machines with Learned Kernels

  2006cited by this paper
• Component selection and smoothing in multivariate nonparametric regression

  2006cited by this paper
• The Dantzig selector: Statistical estimation when P is much larger than n

  2005cited by this paper
• Learning the Kernel Function via Regularization

  2005influential reference
• Geometric Parameters of Kernel Machines

  2002influential reference
• A Bennett concentration inequality and its application to suprema of empirical processes

  2002cited by this paper
• On the Complexity of Learning the Kernel Matrix

  2002influential reference
• Kernel Design Using Boosting

  2002cited by this paper
• Learning the Kernel Matrix with Semidefinite Programming

  2002influential reference
• New concentration inequalities in product spaces

  1996cited by this paper
• Weak Convergence and Empirical Processes: With Applications to Statistics

  1996cited by this paper
• Probability in Banach spaces

  1991cited by this paper
• Probability on Banach spaces

  1978cited by this paper
• Theory of Reproducing Kernels.

  1950cited by this paper

• Representation Retrieval Learning for Heterogeneous Data Integration

  2025influential citation
• Estimation and uniform inference in sparse high-dimensional additive models

  2025cites this paper
• Kernel-Based Sparse Additive Nonlinear Model Structure Detection through a Linearization Approach

  2025cites this paper
• Estimation of High-Dimensional Nonlinear Vector Autoregressive Models

  2025cites this paper
• Debiased Maximum Likelihood Estimators of Hazard Ratios Under Kernel-Based Machine Learning Adjustment

  2025cites this paper
• Sparse Multiple Kernel Learning: Alternating Best Response and Semidefinite Relaxations

  2025cites this paper
• Multi-Layer Kernel Machines: Fast and Optimal Nonparametric Regression with Uncertainty Quantification

  2024cites this paper
• Estimating Double Sparse Structures Over ℓu(ℓq) -Balls: Minimax Rates and Phase Transition

  2024cites this paper
• Graph-based Square-Root Estimation for Sparse Linear Regression

  2024cites this paper
• Variable Selection for Max-Affine Regression via Sparse Gradient Descent

  2024cites this paper
• Kernel Multigrid: Accelerate Back-fitting via Sparse Gaussian Process Regression

  2024cites this paper
• Universality in Block Dependent Linear Models With Applications to Nonlinear Regression

  2024cites this paper
• Asymptotically faster estimation of high‐dimensional additive models using subspace learning

  2024cites this paper
• Sparse multiple kernel learning: Minimax rates with random projection

  2024cites this paper
• Locally adaptive sparse additive quantile regression model with TV penalty

  2024cites this paper
• Sparse additive support vector machines in bounded variation space

  2024cites this paper
• Benzodiazepine-related dementia risks and protopathic biases revealed by multiple-kernel learning with electronic medical records

  2023cites this paper
• Universality in block dependent linear models with applications to nonparametric regression

  2023cites this paper
• Spectral Deconfounding for High-Dimensional Sparse Additive Models

  2023cites this paper
• Transfer Learning for Kernel-based Regression

  2023cites this paper
• Towards a Unified Analysis of Kernel-based Methods Under Covariate Shift

  2023cites this paper
• Decentralized learning over a network with Nyström approximation using SGD

  2023cites this paper
• Minimax Signal Detection in Sparse Additive Models

  2023cites this paper
• Deep neural networks for nonparametric interaction models with diverging dimension

  2023cites this paper
• Properties of Standard and Sketched Kernel Fisher Discriminant

  2023cites this paper
• The Teaching Dimension of Regularized Kernel Learners

  2022cites this paper
• Block-wise primal-dual algorithms for large-scale doubly penalized ANOVA modeling

  2022cites this paper
• Distributed Feature Selection for High-dimensional Additive Models

  2022cites this paper
• Supplementary Material for “Improved Learning Rates of a Functional Lasso-type SVM with Sparse Multi-Kernel Representation”

  2022cites this paper
• Classification by sparse generalized additive models

  2022cites this paper
• DDAC-SpAM: A Distributed Algorithm for Fitting High-dimensional Sparse Additive Models with Feature Division and Decorrelation

  2022cites this paper
• Improved Estimation of High-dimensional Additive Models Using Subspace Learning

  2022cites this paper
• How many moments does MMD compare?

  2021cites this paper
• Error Bound of Mode-Based Additive Models

  2021cites this paper
• Post-Regularization Confidence Bands for Ordinary Differential Equations

  2021cites this paper
• Grouped Variable Selection with Discrete Optimization: Computational and Statistical Perspectives

  2021cites this paper
• Inference for high-dimensional varying-coefficient quantile regression

  2021influential citation
• Guaranteed Functional Tensor Singular Value Decomposition

  2021cites this paper
• Kernel Knockoffs Selection for Nonparametric Additive Models

  2021cites this paper
• Sketched quantile additive functional regression

  2021cites this paper
• Optimal prediction for high-dimensional functional quantile regression in reproducing kernel Hilbert spaces

  2021cites this paper
• The Internet of Federated Things (IoFT)

  2021cites this paper
• Extreme eigenvalues of nonlinear correlation matrices with applications to additive models

  2021cites this paper
• Approximate nonparametric quantile regression in reproducing kernel Hilbert spaces via random projection

  2021cites this paper
• Predicting Census Survey Response Rates via Interpretable Nonparametric Additive Models with Structured Interactions

  2021cites this paper
• A semiparametric model for matrix regression

  2020cites this paper
• Functional Linear Regression with Mixed Predictors

  2020influential citation
• Information based complexity for high dimensional sparse functions

  2020cites this paper
• Adaptive Estimation In High-Dimensional Additive Models With Multi-Resolution Group Lasso

  2020influential citation
• Randomized sketches for sparse additive models

  2020cites this paper
• A reproducing kernel Hilbert space approach to high dimensional partially varying coefficient model

  2020cites this paper
• Risk upper bounds for RKHS ridge group sparse estimator in the regression model with non-Gaussian and non-bounded error

  2020influential citation
• Estimates on Learning Rates for Multi-Penalty Distribution Regression

  2020cites this paper
• Kernel Ordinary Differential Equations

  2020cites this paper
• Learning rates for partially linear support vector machine in high dimensions

  2020cites this paper
• Randomized sketches for kernel CCA

  2020cites this paper
• Sparse RKHS estimation via globally convex optimization and its application in LPV-IO identification

  2020cites this paper
• Post-selection inference on high-dimensional varying-coefficient quantile regression model

  2020cites this paper
• Nonlinear Variable Selection via Deep Neural Networks

  2020cites this paper
• Functional Autoregressive Processes in Reproducing Kernel Hilbert Spaces

  2020influential citation
• Local Inference in Additive Models with Decorrelated Local Linear Estimator

  2019influential citation
• Hierarchical Total Variations and Doubly Penalized ANOVA Modeling for Multivariate Nonparametric Regression

  2019cites this paper
• ST ] 2 6 A pr 2 01 9 A Theoretical Framework for Bayesian Nonparametric Regression

  2019cites this paper
• Minimax Lower Bounds

  2019cites this paper
• Extreme Nonlinear Correlation for Multiple Random Variables and Stochastic Processes with Applications to Additive Models

  2019influential citation
• High-Dimensional Statistics

  2019cites this paper
• Learning General Sparse Additive Models from Point Queries in High Dimensions

  2019cites this paper
• Official Journal of the Bernoulli Society for Mathematical Statistics and Probability

  2019cites this paper
• Decorrelated Local Linear Estimator: Inference for Non-linear Effects in High-dimensional Additive Models

  2019cites this paper
• Doubly penalized estimation in additive regression with high-dimensional data

  2019influential citation
• Sparse Learning for Variable Selection with Structures and Nonlinearities

  2019cites this paper
• Generalized Sparse Additive Models

  2019influential citation
• Metric entropy and its uses

  2019cites this paper
• Rates of contraction with respect to $L_{2}$-distance for Bayesian nonparametric regression

  2019cites this paper
• Sparse linear models in high dimensions

  2019cites this paper
• Basic tail and concentration bounds

  2019cites this paper
• Structured nonlinear variable selection

  2018cites this paper
• Nonparametric variable screening for multivariate additive models

  2018cites this paper
• Large-scale Nonlinear Variable Selection via Kernel Random Features

  2018cites this paper
• Pathway aggregation for survival prediction via multiple kernel learning

  2018cites this paper
• Detecting Nonlinear Causality in Multivariate Time Series with Sparse Additive Models

  2018cites this paper
• Oracle inequalities for sparse additive quantile regression in reproducing kernel Hilbert space

  2018influential citation
• Backfitting algorithms for total‐variation and empirical‐norm penalized additive modelling with high‐dimensional data

  2018influential citation
• Learning General Sparse Additive Models from Point Queries in High Dimensions

  2018cites this paper
• Back tting algorithms for total-variation and empirical-norm penalized additive modeling with high-dimensional data

  2018cites this paper
• Non-Parametric Sparse Additive Auto-Regressive Network Models

  2018cites this paper
• Multiple Kernel Learningの学習理論

  2018cites this paper
• Feature weighting for multinomial kernel logistic regression and application to action recognition

  2018cites this paper
• Learning non-smooth sparse additive models from point queries in high dimensions

  2018cites this paper
• A Theoretical Framework for Bayesian Nonparametric Regression: Orthonormal Random Series and Rates of Contraction

  2017cites this paper
• Metamodel Construction for Sensitivity Analysis

  2017influential citation
• Additive models with trend filtering

  2017influential citation
• Multi-Penalty Regularization for Detecting Relevant Variables

  2017cites this paper
• A Theoretical Framework for Bayesian Nonparametric Regression

  2017cites this paper
• Learning Rates for Classification with Gaussian Kernels

  2017cites this paper
• A unified penalized method for sparse additive quantile models: an RKHS approach

  2017influential citation
• Penalized Estimation in Additive Regression with High-Dimensional Data

  2017influential citation
• A ug 2 01 7 Learning with Optimal Interpolation Norms ∗

  2017cites this paper
• Learning with Optimal Interpolation Norms: Properties and Algorithms

  2016cites this paper
• Learning Sparse Additive Models with Interactions in High Dimensions

  2016cites this paper