Kernels for Vector-Valued Functions: a Review

Kernel methods are among the most popular techniques in machine learning. From a regularization perspective they play a central role in regularization theory as they provide a natural choice for the hypotheses space and the regularization functional through the notion of reproducing kernel Hilbert spaces. From a probabilistic perspective they are the key in the context of Gaussian processes, where the kernel function is known as the covariance function. Traditionally, kernel methods have been used in supervised learning problems with scalar outputs and indeed there has been a considerable amount of work devoted to designing and learning kernels. More recently there has been an increasing interest in methods that deal with multiple outputs, motivated partially by frameworks like multitask learning. In this monograph, we review different methods to design or learn valid kernel functions for multiple outputs, paying particular attention to the connection between probabilistic and functional methods.

• Publication year

  2011

• Venue

  Found. Trends Mach. Learn.

• Publication date

  2011-06-30

• Fields of study

  Mathematics, Computer Science

• Identifiers
  DOI 10.1561/2200000036arXiv 1106.6251
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Non-Stationary Spatial Modeling

  2022cited by this paper
• Statistical Learning Theory

  2021influential reference
• Multi-output learning via spectral filtering

  2012cited by this paper
• Bayesian Multitask Classification With Gaussian Process Priors

  2011cited by this paper
• Learning sparse inverse covariance matrices in the presence of confounders

  2011cited by this paper
• Probabilistic uncertainty analysis of an FRF of a structure using a Gaussian process emulator

  2011cited by this paper
• Convolved Gaussian process priors for multivariate regression with applications to dynamical systems

  2011cited by this paper
• A Survey on Transfer Learning

  2010cited by this paper
• Bayesian emulation of complex multi-output and dynamic computer models

  2010influential reference
• Gaussian Process Preference Elicitation

  2010cited by this paper
• Adaptive and Learning Systems for Signal Processing, Communication, and Control

  2010cited by this paper
• Joint covariate selection and joint subspace selection for multiple classification problems

  2010cited by this paper
• Latent Force Models

  2009cited by this paper
• Predicting Vehicle Crashworthiness: Validation of Computer Models for Functional and Hierarchical Data

  2009cited by this paper
• Generalization Errors and Learning Curves for Regression with Multi-task Gaussian Processes

  2009cited by this paper
• Computer Model Calibration Using High-Dimensional Output

  2008cited by this paper
• Gaussian Process Models for Computer Experiments With Qualitative and Quantitative Factors

  2008cited by this paper
• Efficient Sampling for Gaussian Process Inference using Control Variables

  2008cited by this paper
• Calibration and Uncertainty Analysis for Computer Simulations with Multivariate Output

  2008influential reference
• Towards Real-Time Information Processing of Sensor Network Data Using Computationally Efficient Multi-output Gaussian Processes

  2008cited by this paper
• Universal Multi-Task Kernels

  2008cited by this paper
• Graphical Multi-Task Learning

  2008cited by this paper
• Sparse Convolved Gaussian Processes for Multi-output Regression

  2008influential reference
• An Algorithm for Transfer Learning in a Heterogeneous Environment

  2008cited by this paper
• Clustered Multi-Task Learning: A Convex Formulation

  2008cited by this paper
• Multi-task Gaussian Process Learning of Robot Inverse Dynamics

  2008cited by this paper
• Gaussian process modelling of latent chemical species: applications to inferring transcription factor activities

  2008cited by this paper
• Efficient Emulators for Multivariate Deterministic Functions

  2008influential reference
• A dynamic process convolution approach to modeling ambient particulate matter concentrations

  2008cited by this paper
• Convex multi-task feature learning

  2008cited by this paper
• Multi-Task Learning for Classification with Dirichlet Process Priors

  2007cited by this paper
• This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE Gaussian Process Dynamical Model

  2007cited by this paper
• Refined error estimates for matrix-valued radial basis functions

  2007cited by this paper
• Gaussian Processes for Prediction

  2007cited by this paper
• Multivariate Spatial Modeling for Geostatistical Data Using Convolved Covariance Functions

  2007cited by this paper
• Gaussian Processes for Regression and Optimisation

  2007influential reference
• Multi-task Gaussian Process Prediction

  2007influential reference
• Gaussian Processes for Prediction Technical Report PARG-0701

  2007influential reference
• Nonparametric Bayesian Kernel Models

  2007cited by this paper
• Maximum‐likelihood estimation for multivariate spatial linear coregionalization models

  2007cited by this paper
• Some topics in convolution-based spatial modeling

  2007cited by this paper
• VECTOR VALUED REPRODUCING KERNEL HILBERT SPACES OF INTEGRABLE FUNCTIONS AND MERCER THEOREM

  2006cited by this paper
• Bayesian analysis of computer code outputs: A tutorial

  2006cited by this paper
• Pattern Recognition and Machine Learning (Information Science and Statistics)

  2006cited by this paper
• Pattern Recognition and Machine Learning

  2006cited by this paper
• Modelling transcriptional regulation using Gaussian Processes

  2006cited by this paper
• Adaptive and Learning Systems for Signal Processing, Communications, and Control

  2006influential reference
• Semiparametric latent factor models

  2005cited by this paper
• A density theorem for matrix-valued radial basis functions

  2005cited by this paper
• Learning Gaussian processes from multiple tasks

  2005cited by this paper
• A Unifying View of Sparse Approximate Gaussian Process Regression

  2005cited by this paper
• Learning Multiple Tasks with Kernel Methods

  2005cited by this paper
• Learning with large data sets using filtered {G}aussian Process priors

  2005cited by this paper
• Sparse Gaussian Process Classification With Multiple Classes

  2004cited by this paper
• Nonstationary multivariate process modeling through spatially varying coregionalization

  2004influential reference
• Fitting the Linear Model of Coregionalization by Generalized Least Squares

  2004cited by this paper
• Some Properties of Regularized Kernel Methods

  2004cited by this paper
• Learning to learn with the informative vector machine

  2004cited by this paper
• Kernels for Multi--task Learning

  2004cited by this paper
• Kernel Methods for Pattern Analysis

  2004cited by this paper
• Transformations of Gaussian Process Priors

  2004cited by this paper
• Multivariate Geostatistics

  2004influential reference
• Flexible Spatial Models for Kriging and Cokriging Using Moving Averages and the Fast Fourier Transform (FFT)

  2004cited by this paper
• Maximal Margin Labeling for Multi-Topic Text Categorization

  2004cited by this paper
• Dependent Gaussian Processes

  2004influential reference
• On Learning Vector − Valued Functions

  2003cited by this paper
• The Elements of Statistical Learning

  2003cited by this paper
• Analysis of Some Methods for Reduced Rank Gaussian Process Regression

  2003cited by this paper
• Hierarchical Bayesian Models for Predicting The Spread of Ecological Processes

  2003cited by this paper
• Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond

  2003cited by this paper
• Nonstationary Covariance Functions for Gaussian Process Regression

  2003cited by this paper
• Fitting a linear model of coregionalization for soil properties using simulated annealing

  2002cited by this paper
• Spectral methods for nonstationary spatial processes

  2002cited by this paper
• Space and Space-Time Modeling using Process Convolutions

  2002cited by this paper
• A kernel-based spectral model for non-Gaussian spatio-temporal processes

  2002cited by this paper
• Fast Sparse Gaussian Process Methods: The Informative Vector Machine

  2002cited by this paper
• Coregionalization by Linear Combination of Nonorthogonal Components

  2002influential reference
• Interpolation of nonstationary air pollution processes: a spatial spectral approach

  2002cited by this paper
• A Generalized Representer Theorem

  2001cited by this paper
• Analogies and correspondences between variograms and covariance functions

  2001cited by this paper
• On the mathematical foundations of learning

  2001cited by this paper
• Geostatistics for Natural Resources Evaluation

  2000cited by this paper
• The Nature of Statistical Learning Theory

  2000cited by this paper
• A process-convolution approach to modelling temperatures in the North Atlantic Ocean

  1998cited by this paper
• Vegetation-climate feedbacks in a greenhouse world

  1998cited by this paper
• Design and analysis of computer experiments

  1998cited by this paper
• Bayesian Classification With Gaussian Processes

  1998cited by this paper
• Constructing and fitting models for cokriging and multivariable spatial prediction

  1998cited by this paper
• Hierarchical Bayesian space-time models

  1998cited by this paper
• Multitask Learning

  1997cited by this paper
• Generalized cross-covariances and their estimation

  1997cited by this paper
• Blackbox Kriging: Spatial Prediction Without Specifying Variogram Models

  1996cited by this paper
• Is Learning The n-th Thing Any Easier Than Learning The First?

  1995cited by this paper
• Generalized Hermite interpolation via matrix-valued conditionally positive definite functions

  1994cited by this paper
• Universal cokriging under intrinsic coregionalization

  1994cited by this paper
• 5. Statistics for Spatial Data

  1993cited by this paper
• On the pseudo cross-variogram

  1993cited by this paper
• Linear coregionalization model: Tools for estimation and choice of cross-variogram matrix

  1992cited by this paper
• on Knowledge and Data Engineering,

  1990cited by this paper
• Networks for approximation and learning

  1990cited by this paper

• Intrinsic Estimation of Frenet Curvatures for 3D Trajectory Analysis

  2026cites this paper
• Goal-Driven Adaptive Sampling Strategies for Machine Learning Models Predicting Fields

  2026cites this paper
• Aerodynamic Coefficient Extrapolation via Multi-Fidelity Gaussian Processes

  2026cites this paper
• Laplacian Kernelized Bandit

  2026cites this paper
• Defensive Generation

  2026cites this paper
• Multi-task Modeling for Engineering Applications with Sparse Data

  2026cites this paper
• Deep Gaussian process-based cost-aware batch Bayesian optimization for complex materials design campaigns

  2026cites this paper
• MAST: A Multi-fidelity Augmented Surrogate model via Spatial Trust-weighting

  2026cites this paper
• Uncertainty Quantification in Coupled Multiphysics Systems via Gaussian Process Surrogates: Application to Fuel Assembly Bow

  2026cites this paper
• Vector-valued ensemble regression model based on additive Gaussian process and sufficient dimension reduction

  2026cites this paper
• Gaussian Process-Based Extended Kalman Filter for Trajectory Estimation in sUAV Traffic Management

  2026cites this paper
• Multi-Output Gaussian Process Regression for Rapid Multi-Nutrient Prediction in Soil Using Near-Infrared Spectroscopy

  2026cites this paper
• Transfer learning in surrogate modeling with emphasis on aircraft design

  2026cites this paper
• Computationally Efficient Estimation of Localized Treatment Effects in High-Dimensional Design Spaces using Gaussian Process Regression

  2026cites this paper
• Data-driven identification of macroscopic dynamics with implicit equation-free sampling and Gaussian process regression: for the example of an integrate-and-fire neural network

  2026cites this paper
• Strong Linear Baselines Strike Back: Closed-Form Linear Models as Gaussian Process Conditional Density Estimators for TSAD

  2026cites this paper
• Exchangeable Gaussian Processes for Staggered-Adoption Policy Evaluation

  2026cites this paper
• Scalable multitask Gaussian processes for complex mechanical systems with functional covariates

  2026cites this paper
• Multivariate Gaussian process regression for characterization of geo-data spatial variability from limited and non-co-located measurements

  2026cites this paper
• Probabilistic Interpolation of Sagittarius A*’s Multiwavelength Light Curves Using Diffusion Models

  2026influential citation
• Wireless Dataset Similarity: Measuring Distances in Supervised and Unsupervised Machine Learning

  2026cites this paper
• Physics-informed machine learning for the structural health monitoring and early warning of a long highway viaduct with displacement transducers

  2026cites this paper
• A Digital Twin Framework With Deep Feature Extraction and Gaussian Process for Multi-Objective Optimization in Semiconductor Manufacturing

  2026cites this paper
• SMOG: Scalable Meta-Learning for Multi-Objective Bayesian Optimization

  2026cites this paper
• Deep Learning and Machine Learning for Differentiation Between Contrast Extravasation and Hemorrhagic Transformation in Post-Thrombectomy Stroke CT.

  2026cites this paper
• Epistemic and aleatoric uncertainty in optical vegetation trait retrieval: Concepts, Methods, and Outlook

  2026cites this paper
• From Local Interactions to Global Operators: Scalable Gaussian Process Operator for Physical Systems

  2026cites this paper
• On Solutions to Discrete-Time Inverse Problems via Sparse Superpositions of Decaying Shifted Heaviside Functions: A Hardy Space Approach

  2026cites this paper
• Universal Kernel Models for Iterated Completely Positive Maps

  2025cites this paper
• Learning Paths for Dynamic Measure Transport: A Control Perspective

  2025cites this paper
• Physics-informed Bayesian optimization of expensive-to-evaluate black-box functions

  2025cites this paper
• Multi-Output Robust and Conjugate Gaussian Processes

  2025influential citation
• Accelerating phase diagram construction through activity coefficient prediction.

  2025cites this paper
• Distribution Grid Voltage Estimation by Correlated Gaussian Processes

  2025cites this paper
• Identification of contractive Lur'e-type systems via kernel-based Lipschitz design

  2025cites this paper
• Bayesian Nonparametric Dynamical Clustering of Time Series

  2025cites this paper
• BALLAST: Bayesian Active Learning with Look-ahead Amendment for Sea-drifter Trajectories under Spatio-Temporal Vector Fields

  2025cites this paper
• Split Conformal Classification with Unsupervised Calibration

  2025cites this paper
• Learning orbitally stable dynamics via transverse contraction criteria for modeling periodic tasks

  2025cites this paper
• Unified Adaptive and Cooperative Planning Using Multi-Task Coregionalized Gaussian Processes

  2025cites this paper
• A multi-output Gaussian process regression with negative transfer mitigation for generating boundary test scenarios of multi-UAV systems

  2025cites this paper
• Kernel Regression for Predicting Magnetic Field Distributions in Railway Catenary Systems

  2025cites this paper
• Bayesian Basis Function Approximation for Scalable Gaussian Process Priors in Deep Generative Models

  2025influential citation
• Semantic-Aware Gaussian Process Calibration with Structured Layerwise Kernels for Deep Neural Networks

  2025influential citation
• Adaptive Vector-Valued Splines for the Resolution of Inverse Problems

  2025cites this paper
• Adaptive Parameter Optimization in Gaussian Processes: A Comprehensive Study of Uncertainty Quantification and Dimensional Scaling

  2025influential citation
• Improving Speaker Verification Robustness With Multilingual Phonetic Information and Feature Decorrelation

  2025cites this paper
• Unbiased Stochastic Optimization for Gaussian Processes on Finite Dimensional RKHS

  2025cites this paper
• High-Fidelity Machine Learning Framework for Fracture Energy Prediction in Fiber-Reinforced Concrete

  2025cites this paper
• Kernel Model Validation: How To Do It, And Why You Should Care

  2025cites this paper
• Bayesian Transfer Operators in Reproducing Kernel Hilbert Spaces

  2025influential citation
• On the continuity and differentiability of matrix-valued covariance kernels depending on Euclidean or spherical distances

  2025cites this paper
• Learning Passive Continuous-Time Dynamics with Multistep Port-Hamiltonian Gaussian Processes

  2025cites this paper
• Multi-task Neural Diffusion Processes

  2025cites this paper
• Prediction of Blade Vibration Response with Uncertainty Propagation Using Physics-Informed Neural Networks and Physics-Informed Gaussian Process Regression

  2025cites this paper
• Progressive multi-fidelity learning with neural networks for physical system predictions

  2025cites this paper
• A joint kriging model with application to constrained classification

  2025influential citation
• ExTrEMO: Transfer Evolutionary Multiobjective Optimization With Proof of Faster Convergence

  2025cites this paper
• A Factor Graph Approach to Variational Sparse Gaussian Processes

  2025influential citation
• Hybrid multi-output regression with residual correction for smart agriculture: a scalable and interpretable approach

  2025cites this paper
• From Local Interactions to Global Operators: Scalable Gaussian Process Operator for Physical Systems

  2025cites this paper
• Bayesian Quality-Diversity optimization for conditional search-space problems

  2025cites this paper
• On Efficient Bayesian Exploration in Model-Based Reinforcement Learning

  2025cites this paper
• Spatio-Temporal Prediction via Operator-Valued RKHS and Koopman Approximation

  2025cites this paper
• Generating global high-resolution current and future weather timeseries, and the implications for India

  2025cites this paper
• A Multi-output Gaussian Process Regression with Negative Transfer Mitigation for Generating Boundary Test Scenarios of Multi-UAV Systems

  2025cites this paper
• Towards Quantum Operator-Valued Kernels

  2025cites this paper
• Physics-informed multi-output Gaussian process for dynamical system modeling

  2025cites this paper
• Low-rank computation of the posterior mean in Multi-Output Gaussian Processes

  2025cites this paper
• Transfer Learning Across Fixed-Income Product Classes

  2025cites this paper
• Distributional encoding for Gaussian process regression with qualitative inputs

  2025cites this paper
• Novel Pivoted Cholesky Decompositions for Efficient Gaussian Process Inference

  2025cites this paper
• Meta-learning Population-based Methods for Reinforcement Learning

  2025cites this paper
• Multivariate Gaussian Process Regression for 3D site characterization from CPT and categorical borehole data

  2025cites this paper
• Learning to Solve Related Linear Systems

  2025influential citation
• The Hardness of Validating Observational Studies with Experimental Data

  2025cites this paper
• EEG Multi-Mode Oscillatory Brain State Allocation Using Switching Spectral Gaussian Processes

  2025cites this paper
• Effective Data-Driven Joint Friction Modeling and Compensation With Physical Consistency

  2025cites this paper
• Cross-disciplinary perspectives on the potential for artificial intelligence across chemistry

  2025cites this paper
• A Dictionary of Closed-Form Kernel Mean Embeddings

  2025cites this paper
• A Gaussian Process surrogate approach for analyzing parameter uncertainty in mechanics-based structural finite element models

  2025cites this paper
• A Survey on Uncertainty Quantification in Dynamical Systems

  2025cites this paper
• Fast, Robust and Global Optimisation for Antenna Design using Meta Modelling

  2025cites this paper
• Multi-Output Gaussian Processes for Graph-Structured Data

  2025influential citation
• MoCA: Multi-modal Cross-masked Autoencoder for Digital Health Measurements

  2025cites this paper
• Enhanced Detection of Demagnetization in Direct-Drive Wind Turbine Generators Using eMIST and Machine Learning

  2025cites this paper
• Strain monitoring-based diagnosis and prognosis for hidden cracks in metallic lap-joint structures

  2025cites this paper
• Accurate and uncertainty-aware multi-task prediction of HEA properties using prior-guided deep Gaussian processes

  2025cites this paper
• A Parametric, Data-Driven, Non-Intrusive Reduced-Order Model Framework for Crystal Plasticity Simulations of Voids

  2025cites this paper
• Multi-Output Machine-Learning Prediction of Volatile Organic Compounds (VOCs): Learning from Co-Emitted VOCs

  2025cites this paper
• A Review on Applications of Gaussian Processes for Complex Control Systems

  2025cites this paper
• Thompson Sampling in Function Spaces via Neural Operators

  2025cites this paper
• A probabilistic machine learning framework for stiffness tensor estimation of carbon composite laminate

  2025cites this paper
• SABR-informed multitask Gaussian process: A synthetic-to-real framework for implied volatility surface construction

  2025cites this paper
• Holistic Bioprocess Development Across Scales Using Multi-Fidelity Batch Bayesian Optimization

  2025cites this paper
• Nonparametric learning of stochastic differential equations from sparse and noisy data

  2025cites this paper
• Accelerated controller tuning using human feedback and Multi-Task Preferential Bayesian Optimization

  2025cites this paper
• Learning Spatio-Temporal Dynamics via Operator-Valued RKHS and Kernel Koopman Methods

  2025cites this paper
• Multifidelity Bayesian Sequential Optimization and Reliability Assessment for Aerospace Systems Design

  2025cites this paper
• Physics-Informed Multifidelity Gaussian Process: Modeling the Effect of Water and Temperature on the Viscosity of a Deep Eutectic Solvent

  2025cites this paper