Finite-dimensional Gaussian approximation with linear inequality constraints

Introducing inequality constraints in Gaussian process (GP) models can lead to more realistic uncertainties in learning a great variety of real-world problems. We consider the finite-dimensional Gaussian approach from Maatouk and Bay (2017) which can satisfy inequality conditions everywhere (either boundedness, monotonicity or convexity). Our contributions are threefold. First, we extend their approach in order to deal with general sets of linear inequalities. Second, we explore several Markov Chain Monte Carlo (MCMC) techniques to approximate the posterior distribution. Third, we investigate theoretical and numerical properties of the constrained likelihood for covariance parameter estimation. According to experiments on both artificial and real data, our full framework together with a Hamiltonian Monte Carlo-based sampler provides efficient results on both data fitting and uncertainty quantification.

• Publication year

  2017

• Venue

  SIAM/ASA J. Uncertain. Quantification

• Publication date

  2017-10-20

• Fields of study

  Mathematics, Computer Science

• Identifiers
  DOI 10.1137/17M1153157arXiv 1710.07453
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Approximation

  2020cited by this paper
• Finite-dimensional approximation of Gaussian processes with inequality constraints

  2017cited by this paper
• A supermartingale approach to Gaussian process based sequential design of experiments

  2016cited by this paper
• A Practical Sequential Stopping Rule for High-Dimensional Markov Chain Monte Carlo

  2016influential reference
• Gaussian Process Emulators for Computer Experiments with Inequality Constraints

  2016influential reference
• Kriging of financial term-structures

  2016cited by this paper
• Generalization of the Kimeldorf-Wahba correspondence for constrained interpolation

  2016cited by this paper
• The normal law under linear restrictions: simulation and estimation via minimax tilting

  2016influential reference
• Multivariate output analysis for Markov chain Monte Carlo

  2015influential reference
• Cross-Validation Estimations of Hyper-Parameters of Gaussian Processes with Inequality Constraints☆

  2015cited by this paper
• DiceDesign and DiceEval: Two R Packages for Design and Analysis of Computer Experiments

  2015cited by this paper
• Modelling and Control of Dynamic Systems Using Gaussian Process Models

  2015cited by this paper
• Introduction to Continuity

  2015cited by this paper
• Sampling Constrained Probability Distributions Using Spherical Augmentation

  2015influential reference
• A New Rejection Sampling Method for Truncated Multivariate Gaussian Random Variables Restricted to Convex Sets

  2014influential reference
• Monotone Emulation of Computer Experiments

  2013influential reference
• Cross Validation and Maximum Likelihood estimations of hyper-parameters of Gaussian processes with model misspecification

  2013cited by this paper
• Exact Hamiltonian Monte Carlo for Truncated Multivariate Gaussians

  2012influential reference
• Gaussian process modeling with inequality constraints

  2012influential reference
• Handbook of Markov Chain Monte Carlo

  2011influential reference
• Gaussian processes with monotonicity information

  2010cited by this paper
• Fixed-domain asymptotic properties of tapered maximum likelihood estimators

  2009cited by this paper
• Level Sets and Extrema of Random Processes and Fields

  2009cited by this paper
• Approximations for Binary Gaussian Process Classification

  2008cited by this paper
• Sparse Log Gaussian Processes via MCMC for Spatial Epidemiology

  2007cited by this paper
• Fixed-domain asymptotics for a subclass of Matern-type Gaussian random fields

  2005cited by this paper
• THE MORET 4.B MONTE CARLO CODE New features to treat complex criticality systems

  2005cited by this paper
• Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning)

  2005influential reference
• Sparse Gaussian Processes using Pseudo-inputs

  2005cited by this paper
• Inconsistent Estimation and Asymptotically Equal Interpolations in Model-Based Geostatistics

  2004influential reference
• Nonstationary Covariance Functions for Gaussian Process Regression

  2003influential reference
• A CLASS OF GLOBALLY CONVERGENT OPTIMIZATION METHODS BASED ON CONSERVATIVE CONVEX SEPARABLE APPROXIMATIONS∗

  2002cited by this paper
• Interpolation of spatial data: some theory for kriging

  2001cited by this paper
• Estimating structured correlation matrices in smooth Gaussian random field models

  2000cited by this paper
• Matrix Analysis and Applied Linear Algebra

  2000cited by this paper
• Interpolation of Spatial Data: Some Theory for Kriging

  2000influential reference
• Design and analysis of computer experiments

  1998cited by this paper
• Simulating Normalizing Constants: From Importance Sampling to Bridge Sampling to Path Sampling

  1998cited by this paper
• Bayesian learning for neural networks

  1995influential reference
• Maximum likelihood estimation of parameters under a spatial sampling scheme

  1993cited by this paper
• Numerical Computation of Multivariate Normal Probabilities

  1992influential reference
• Practical Markov Chain Monte Carlo

  1992cited by this paper
• An introduction to continuity, extrema, and related topics for general Gaussian processes

  1990cited by this paper
• Hybrid Monte Carlo

  1988influential reference
• SPECTRAL THEORY OF RANDOM FIELDS (Translation Series in Mathematics and Engineering)

  1984cited by this paper
• Spectral theory of random fields

  1983cited by this paper
• Gaussian Random Processes

  1978cited by this paper

• Efficient monotonic Gaussian processes via Randomize-then-Optimize

  2026cites this paper
• Blending physics with data using an efficient Gaussian process regression with soft inequality and monotonicity constraints

  2025cites this paper
• Fast Gaussian Processes under Monotonicity Constraints

  2025cites this paper
• Tracking Control and Experiment for Propeller‐Driven Wall‐Climbing Robot Considering Actuator Dynamics and Saturation

  2025cites this paper
• A Bayesian computational technique for learning the airflow resistance of acoustical fibrous materials at high temperatures

  2025influential citation
• Physics-informed Gaussian Processes as Linear Model Predictive Controller

  2025cites this paper
• Bayesian preference elicitation for decision support in multiobjective optimization

  2025cites this paper
• Doubly Truncated Mode Kriging

  2025cites this paper
• Nonlinear response modelling of material systems using constrained Gaussian processes

  2024influential citation
• Gaussian Process Regression with Soft Inequality and Monotonicity Constraints

  2024cites this paper
• Knowledge-based modeling of simulation behavior for Bayesian optimization

  2024cites this paper
• A review of physics-based learning for system health management

  2024cites this paper
• Linearly Constrained Gaussian Processes are SkewGPs: application to Monotonic Preference Learning and Desirability

  2024influential citation
• Gaussian process with dissolution spline kernel for in vitro dissolution testing

  2024cites this paper
• Large-scale constrained Gaussian processes for shape-restricted function estimation

  2024cites this paper
• Integrating small data and shape prior knowledge with gradient-enhanced Kriging through adaptive knowledge sampling

  2024cites this paper
• Block-Additive Gaussian Processes under Monotonicity Constraints

  2024cites this paper
• Error Bounds for a Kernel-Based Constrained Optimal Smoothing Approximation

  2024cites this paper
• Equivalence between constrained optimal smoothing and Bayesian estimation

  2024cites this paper
• Nonparametric Bayesian approach for quantifying the conditional uncertainty of input parameters in chained numerical models

  2023cites this paper
• Physics-based Penalization for Hyperparameter Estimation in Gaussian Process Regression

  2023cites this paper
• Coastal Flood at Gâvres (Brittany, France): A Simulated Dataset to Support Risk Management and Metamodels Development

  2023cites this paper
• High-dimensional additive Gaussian processes under monotonicity constraints

  2022influential citation
• Beyond Surrogate Modeling: Learning the Local Volatility Via Shape Constraints

  2022cites this paper
• Non-parametric synergy modeling of chemical compounds with Gaussian processes

  2022cites this paper
• Finite-dimensional approximation of Gaussian processes with linear inequality constraints and noisy observations

  2022influential citation
• Constraining Gaussian processes for physics-informed acoustic emission mapping

  2022cites this paper
• Physics-informed machine learning for reliability and systems safety applications: State of the art and challenges

  2022cites this paper
• Relaxed Gaussian process interpolation: a goal-oriented approach to Bayesian optimization

  2022influential citation
• Sensing Cox Processes via Posterior Sampling and Positive Bases

  2021cites this paper
• Constrained Gaussian process regression: an adaptive approach for the estimation of hyperparameters and the verification of constraints with high probability

  2021cites this paper
• Convex optimization of nonlinear inequality with higher order derivatives

  2021cites this paper
• The Probabilistic Deconvolution of the Distribution of Relaxation Times with Finite Gaussian Processes

  2021cites this paper
• Non-parametric synergy modeling with Gaussian processes

  2021cites this paper
• Information fusion and machine learning for sensitivity analysis using physics knowledge and experimental data

  2021cites this paper
• No-regret Algorithms for Capturing Events in Poisson Point Processes

  2021cites this paper
• Spectrum Gaussian Processes Based On Tunable Basis Functions

  2021cites this paper
• Posterior contraction rates for constrained deep Gaussian processes in density estimation and classification

  2021cites this paper
• Infrastructure recovery curve estimation using Gaussian process regression on expert elicited data

  2020cites this paper
• Sequential Construction and Dimension Reduction of Gaussian Processes Under Inequality Constraints

  2020influential citation
• Asymptotic Analysis of Maximum Likelihood Estimation of Covariance Parameters for Gaussian Processes: An Introduction with Proofs

  2020cites this paper
• Non-colliding Gaussian Process Regressions

  2020influential citation
• Sequential construction and dimension reduction of Gaussian processes under constraints

  2020influential citation
• OF MACHINE LEARNING FOR MODELING AND COMPUTING

  2020cites this paper
• Fixed-domain asymptotic properties of maximum composite likelihood estimators for Gaussian processes

  2020cites this paper
• Nonnegativity-Enforced Gaussian Process Regression

  2020cites this paper
• Gaussian process optimization with failures: classification and convergence proof

  2020cites this paper
• Some Statistical Issues in Climate Science

  2020cites this paper
• Gaussian process metamodeling of functional-input code for coastal flood hazard assessment

  2020cites this paper
• Gaussian process regression with linear inequality constraints

  2020cites this paper
• Increasing the discriminatory power of bounding models using problem-specific knowledge when viewing design as a sequential decision process

  2020cites this paper
• Sparse Gaussian Process Based On Hat Basis Functions

  2020influential citation
• A Survey of Constrained Gaussian Process Regression: Approaches and Implementation Challenges

  2020cites this paper
• Multi-Output Gaussian Processes with Functional Data: A Study on Coastal Flood Hazard Assessment

  2020cites this paper
• Exploratory-Phase-Free Estimation of GP Hyperparameters in Sequential Design Methods—At the Example of Bayesian Inverse Problems

  2020cites this paper
• Augmented Gaussian random field: Theory and computation

  2020cites this paper
• A method to efficiently localize non-dominated regions using surrogate modeling with multi-fidelity data from a sequential decision process

  2019influential citation
• Gaussian processes with linear operator inequality constraints

  2019influential citation
• A Role for Symmetry in the Bayesian Solution of Differential Equations

  2019influential citation
• Adaptive design for Gaussian process regression under censoring

  2019cites this paper
• An overview of kriging for researchers

  2019cites this paper
• Statistical Molecule Counting in Super-Resolution Fluorescence Microscopy: Towards Quantitative Nanoscopy

  2019cites this paper
• Gaussian Process Modulated Cox Processes under Linear Inequality Constraints

  2019cites this paper
• Gaussian process optimization with simulation failures

  2019cites this paper
• Tensor Basis Gaussian Process Models of Hyperelastic Materials

  2019cites this paper
• Integrals over Gaussians under Linear Domain Constraints

  2019cites this paper
• Finite-dimensional Gaussian approximation with linear inequality constraints

  2019cites this paper
• Contributions aux processus gaussiens, à la quantification d'incertitudes et à l’inférence post-sélection de modèle

  2018cites this paper
• Approximating Gaussian Process Emulators with Linear Inequality Constraints and Noisy Observations via MC and MCMC

  2018influential citation
• Physically-inspired Gaussian processes for transcriptional regulation in Drosophila melanogaster

  2018influential citation
• Physically-Inspired Gaussian Process Models for Post-Transcriptional Regulation in Drosophila

  2018cites this paper
• On the Bayesian Solution of Differential Equations

  2018cites this paper
• Composite likelihood estimation for a Gaussian process under fixed domain asymptotics

  2018cites this paper
• Gaussian process regression models under linear inequality conditions

  2017influential citation
• Efficient constrained Gaussian process approximation using elliptical slice sampling

  year unknowncites this paper