Bayesian and regularization approaches to multivariable linear system identification: The role of rank penalties

Recent developments in linear system identification have proposed the use of non-parameteric methods, relying on regularization strategies, to handle the so-called bias/variance trade-off. This paper introduces an impulse response estimator which relies on an ℓ2-type regularization including a rank-penalty derived using the log-det heuristic as a smooth approximation to the rank function. This allows to account for different properties of the estimated impulse response (e.g. smoothness and stability) while also penalizing high-complexity models. This also allows to account and enforce coupling between different input-output channels in MIMO systems. According to the Bayesian paradigm, the parameters defining the relative weight of the two regularization terms as well as the structure of the rank penalty are estimated optimizing the marginal likelihood. Once these hyperameters have been estimated, the impulse response estimate is available in closed form. Experiments show that the proposed method is superior to the estimator relying on the “classic” ℓ2-regularization alone as well as those based in atomic and nuclear norm.

• Publication year

  2014

• Venue

  IEEE Conference on Decision and Control

• Publication date

  2014-09-29

• Fields of study

  Mathematics, Computer Science, Engineering

• Identifiers
  DOI 10.1109/CDC.2014.7039610arXiv 1409.8327
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Tuning complexity in kernel-based linear system identification: The robustness of the marginal likelihood estimator

  2014cited by this paper
• System Identification Via Sparse Multiple Kernel-Based Regularization Using Sequential Convex Optimization Techniques

  2014cited by this paper
• Parsimonious model identification via atomic norm minimization

  2014cited by this paper
• Kernel methods in system identification, machine learning and function estimation: A survey

  2014cited by this paper
• Regularization strategies for nonparametric system identification

  2013cited by this paper
• System Identification: Theory for the User, 2nd Edition (Ljung, L.; 1999) [On the Shelf]

  2012cited by this paper
• Subspace system identification via weighted nuclear norm optimization

  2012cited by this paper
• On the estimation of transfer functions, regularizations and Gaussian processes - Revisited

  2012cited by this paper
• Non-Convex Rank Minimization via an Empirical Bayesian Approach

  2012cited by this paper
• Linear system identification via atomic norm regularization

  2012cited by this paper
• On the Estimation of Hyperparameters for Empirical Bayes Estimators: Maximum Marginal Likelihood vs Minimum MSE

  2012cited by this paper
• System Identification이란 무엇인가

  2011cited by this paper
• Prediction error identification of linear systems: A nonparametric Gaussian regression approach

  2011cited by this paper
• Iterative reweighted least squares for matrix rank minimization

  2010cited by this paper
• A new kernel-based approach for linear system identification

  2010cited by this paper
• MODEL SELECTION AND INFERENCE: FACTS AND FICTION

  2005cited by this paper
• A rank minimization heuristic with application to minimum order system approximation

  2001cited by this paper
• The statistical theory of linear systems

  1989influential reference
• DIMENSIONAL LINEAR SYSTEMS

  1981cited by this paper
• Solutions of ill-posed problems

  1977cited by this paper

• Efficient low-order system identification from low-quality step response data with rank-constrained optimization

  2021cites this paper
• On model order priors for Bayesian identification of SISO linear systems

  2019cites this paper
• S Y ] 4 N ov 2 01 5 Regularization and Bayesian Learning in Dynamical Systems : Past , Present and Future

  2018influential citation
• An empirical Bayes approach to identification of modules in dynamic networks

  2017cites this paper
• Non-Parametric Bayesian Methods for Linear System Identification

  2017cites this paper
• Identification of modules in dynamic networks: An empirical Bayes approach

  2016cites this paper
• Nuclear norm-enhanced recursive subspace identification: Closed-loop estimation of rapid variations in system dynamics

  2016cites this paper
• On risk-coherent input design and Bayesian methods for nonlinear system identification

  2016cites this paper
• A Bayesian approach to sparse plus low rank network identification

  2015cites this paper
• Maximum Entropy vector kernels for MIMO system identification

  2015cites this paper
• Regularized linear system identification using atomic, nuclear and kernel-based norms: The role of the stability constraint

  2015cites this paper
• Regularization and Bayesian learning in dynamical systems: Past, present and future

  2015influential citation
• Model reduction for linear bayesian system identification

  2015influential citation
• The role of rank penalties in linear system identification

  2015cites this paper