Error Analysis of Regularized Trigonometric Linear Regression With Unbounded Sampling: A Statistical Learning Viewpoint

The effectiveness of non-parametric, kernel-based methods for function estimation comes at the price of high computational complexity, which hinders their applicability in adaptive, model-based control. Motivated by approximation techniques based on sparse spectrum Gaussian processes, we focus on models given by regularized trigonometric linear regression. This letter provides an analysis of the performance of such an estimation set-up within the statistical learning framework. In particular, we derive a novel bound for the sample error in finite-dimensional spaces, accounting for noise with potentially unbounded support. Next, we study the approximation error and discuss the bias-variance trade-off as a function of the regularization parameter by combining the two bounds.

• Publication year

  2023

• Venue

  IEEE Control Systems Letters

• Publication date

  2023-03-16

• Fields of study

  Mathematics, Computer Science, Engineering

• Identifiers
  DOI 10.1109/LCSYS.2023.3291690arXiv 2303.09206
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Bayesian Multi-Task Learning MPC for Robotic Mobile Manipulation

  2022cited by this paper
• Bayesian multi-task learning using finite-dimensional models: A comparative study

  2021cited by this paper
• Learning-Based Model Predictive Control: Toward Safe Learning in Control

  2020cited by this paper
• Projected Stochastic Primal-Dual Method for Constrained Online Learning With Kernels

  2019cited by this paper
• Representer Theorem

  2019cited by this paper
• Bio-inspired synthesis of xishacorenes A, B, and C, and a new congener from fuscol

  2019cited by this paper
• When Gaussian Process Meets Big Data: A Review of Scalable GPs

  2018cited by this paper
• Learning-Based Model Predictive Control for Safe Exploration

  2018cited by this paper
• Appendix for the Paper : Prediction under Uncertainty in Sparse Spectrum Gaussian Processes with Applications to Filtering and Control

  2017cited by this paper
• On Asymptotic Properties of Hyperparameter Estimators for Kernel-based Regularization Methods

  2017cited by this paper
• Generalization Properties of Learning with Random Features

  2016cited by this paper
• Distributed Learning with Regularized Least Squares

  2016cited by this paper
• Tuning complexity in regularized kernel-based regression and linear system identification: The robustness of the marginal likelihood estimator

  2015cited by this paper
• Kernel methods in system identification, machine learning and function estimation: A survey

  2014cited by this paper
• Learning without Concentration

  2014cited by this paper
• Concentration Inequalities: A Nonasymptotic Theory of Independence

  2013influential reference
• I and J

  2012cited by this paper
• Concentration estimates for learning with unbounded sampling

  2011cited by this paper
• Optimal learning rates for least squares regularized regression with unbounded sampling

  2011cited by this paper
• Sparse Spectrum Gaussian Process Regression

  2010cited by this paper
• Regularization in kernel learning

  2010cited by this paper
• Support vector machines

  2008cited by this paper
• Introduction to Nonparametric Estimation

  2008cited by this paper
• Concentration Inequalities

  2008influential reference
• Optimal Rates for the Regularized Least-Squares Algorithm

  2007cited by this paper
• Learning Theory Estimates via Integral Operators and Their Approximations

  2007influential reference
• Learning Theory: An Approximation Theory Viewpoint: The framework of learning

  2007cited by this paper
• Learning Theory: An Approximation Theory Viewpoint: Index

  2007cited by this paper
• Nonparametric Regression, Confidence Regions and Regularization

  2007cited by this paper
• Learning Rates of Least-Square Regularized Regression

  2006cited by this paper
• Learning Bounds for Kernel Regression Using Effective Data Dimensionality

  2005cited by this paper
• Shannon sampling II: Connections to learning theory

  2005cited by this paper
• Shannon sampling and function reconstruction from point values

  2004cited by this paper
• Best choices for regularization parameters in learning theory

  2003cited by this paper
• Capacity of reproducing kernel spaces in learning theory

  2003cited by this paper
• On the mathematical foundations of learning

  2001cited by this paper
• Ridge Regression: Biased Estimation for Nonorthogonal Problems

  2000cited by this paper
• Generalization bounds for function approximation from scattered noisy data

  1999cited by this paper
• Markov chain Monte Carlo in Practice

  1996cited by this paper
• Local polynomial fitting: A standard for nonparametric regression

  1993cited by this paper
• An introduction to hilbert space

  1990cited by this paper
• Empirical Bayes Methods with Applications

  1989cited by this paper
• Oscillation matrices with spline smoothing

  1975cited by this paper
• Resolution

  1969cited by this paper
• Theory of linear operators in Hilbert space

  1961cited by this paper
• Theory of Reproducing Kernels.

  1950cited by this paper

• Gaussian processes for dynamics learning in model predictive control

  2025cites this paper
• Small noise analysis for Tikhonov and RKHS regularizations

  2023cites this paper