Warm starting Bayesian optimization

We develop a framework for warm-starting Bayesian optimization, that reduces the solution time required to solve an optimization problem that is one in a sequence of related problems. This is useful when optimizing the output of a stochastic simulator that fails to provide derivative information, for which Bayesian optimization methods are well-suited. Solving sequences of related optimization problems arises when making several business decisions using one optimization model and input data collected over different time periods or markets. While many gradient-based methods can be warm started by initiating optimization at the solution to the previous problem, this warm start approach does not apply to Bayesian optimization methods, which carry a full metamodel of the objective function from iteration to iteration. Our approach builds a joint statistical model of the entire collection of related objective functions, and uses a value of information calculation to recommend points to evaluate.

• Publication year

  2016

• Venue

  Online World Conference on Soft Computing in Industrial Applications

• Publication date

  2016-08-11

• Fields of study

  Mathematics, Business, Computer Science

• Identifiers
  DOI 10.1109/WSC.2016.7822140arXiv 1608.03585
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Multi-Information Source Optimization with General Model Discrepancies

  2016cited by this paper
• Kernel Interpolation for Scalable Structured Gaussian Processes (KISS-GP)

  2015cited by this paper
• Fast Kronecker Inference in Gaussian Processes with non-Gaussian Likelihoods

  2015cited by this paper
• Bayesian optimization for materials design

  2015cited by this paper
• Random projections for Bayesian regression

  2015cited by this paper
• Using Meta-Learning to Initialize Bayesian Optimization of Hyperparameters

  2014cited by this paper
• Simulation-optimization via Kriging and bootstrapping: a survey

  2014cited by this paper
• Multi-Task Bayesian Optimization

  2013influential reference
• The Correlated Knowledge Gradient for Simulation Optimization of Continuous Parameters using Gaussian Process Regression

  2011cited by this paper
• Value of information methods for pairwise sampling with correlations

  2011cited by this paper
• Decision‐Theoretic Foundations of Simulation Optimization

  2011cited by this paper
• Efficient Gaussian process regression for large datasets.

  2011cited by this paper
• A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning

  2010cited by this paper
• The Knowledge-Gradient Policy for Correlated Normal Beliefs

  2009cited by this paper
• Implementation of warm-start strategies in interior-point methods for linear programming in fixed dimension

  2008cited by this paper
• Author Biographies

  2006cited by this paper
• Discrete Optimization via Simulation Using COMPASS

  2006cited by this paper
• Interior Point Methods: A Survey, Short Survey of Applications to Power Systems, and Research Opportunities

  2004cited by this paper
• Variable Fidelity Optimization Using a Kriging Based Scaling Function

  2004cited by this paper
• Warm-Start Strategies in Interior-Point Methods for Linear Programming

  2002cited by this paper
• Bayesian methods for simulation

  2000cited by this paper
• Efficient Global Optimization of Expensive Black-Box Functions

  1998cited by this paper
• Comment on "An innovations approach to least-squares estimation, part I: Linear filtering in additive white noise"

  1970cited by this paper
• An innovations approach to least-squares estimation--Part II: Linear smoothing in additive white noise

  1968cited by this paper

• Dynamic meta-learning acquisition function method for Bayesian optimization with early stopping criteria for hyperparameter optimization

  2026cites this paper
• An Empirical Study on Ensemble-Based Transfer Learning Bayesian Optimisation with Mixed Variable Types

  2026cites this paper
• Self-driving laboratories with artificial intelligence: An overview of process systems engineering perspective

  2025cites this paper
• Provable Accelerated Bayesian Optimization with Knowledge Transfer

  2025cites this paper
• Why risk matters for protein binder design

  2025cites this paper
• On-Chip Implementation of Neural Network-Based Classifier Models for E-Nose With Chemometric Analysis

  2025cites this paper
• Bayesian Methods for Magnetic and Mechanical Optimization of Superconducting Magnets for Fusion

  2025cites this paper
• OptiCarVis: Improving Automated Vehicle Functionality Visualizations Using Bayesian Optimization to Enhance User Experience

  2025cites this paper
• None To Optima in Few Shots: Bayesian Optimization with MDP Priors

  2025cites this paper
• Language-Based Bayesian Optimization Research Assistant (BORA)

  2025cites this paper
• ADMIRE-BayesOpt: Accelerated Data MIxture RE-weighting for Language Models with Bayesian Optimization

  2025cites this paper
• From Problem to Solution: A General Pipeline to Solve Optimisation Problems on Quantum Hardware

  2024cites this paper
• Sequential Large Language Model-Based Hyper-Parameter Optimization

  2024cites this paper
• Robust Transfer Learning for Active Level Set Estimation with Locally Adaptive Gaussian Process Prior

  2024cites this paper
• Efficient inverse design optimization through multi-fidelity simulations, machine learning, and boundary refinement strategies

  2024cites this paper
• Accelerating Digital Twin Calibration with Warm-Start Bayesian Optimization

  2024cites this paper
• Fast maximum likelihood estimation for general hierarchical models

  2024cites this paper
• Improving Hyperparameter Optimization with Checkpointed Model Weights

  2024cites this paper
• Reinforced In-Context Black-Box Optimization

  2024cites this paper
• Large Language Models to Enhance Bayesian Optimization

  2024cites this paper
• Data-driven AI system for learning how to run transcript assemblers

  2024cites this paper
• Bayesian Optimization Based Neural Architecture Search for Classification of Gases/Odors Mixtures

  2024cites this paper
• A hybrid quantum-classical approach to warm-starting optimization

  2023cites this paper
• Stochastic Approximation for Multi-period Simulation Optimization with Streaming Input Data

  2023cites this paper
• Warm-Starting and Quantum Computing: A Systematic Mapping Study

  2023cites this paper
• Transfer Learning for Bayesian Optimization: A Survey

  2023cites this paper
• Bad priors, their threats to Bayesian optimization and some remedies via prior learning

  2022cites this paper
• On Provably Robust Meta-Bayesian Optimization

  2022cites this paper
• Towards Learning Universal Hyperparameter Optimizers with Transformers

  2022cites this paper
• GPTuneBand: Multi-task and Multi-fidelity Autotuning for Large-scale High Performance Computing Applications

  2022cites this paper
• Pre-training helps Bayesian optimization too

  2022influential citation
• Selection and Optimization of Hyperparameters in Warm-Started Quantum Optimization for the MaxCut Problem

  2022cites this paper
• Pre-trained Gaussian Processes for Bayesian Optimization

  2021influential citation
• Indirectly Supervised Anomaly Detection of Clinically Meaningful Health Events from Smart Home Data

  2021cites this paper
• JUMBO: Scalable Multi-task Bayesian Optimization using Offline Data

  2021cites this paper
• Amortized Auto-Tuning: Cost-Efficient Transfer Optimization for Hyperparameter Recommendation

  2021cites this paper
• Automatic prior selection for meta Bayesian optimization with a case study on tuning deep neural network optimizers

  2021cites this paper
• Incorporating Expert Prior Knowledge into Experimental Design via Posterior Sampling

  2020cites this paper
• Warm Starting CMA-ES for Hyperparameter Optimization

  2020cites this paper
• ConBO: Conditional Bayesian Optimization

  2020cites this paper
• Practical Bayesian Optimization of Objectives with Conditioning Variables.

  2020cites this paper
• Scalable Bayesian Optimization with Sparse Gaussian Process Models

  2020cites this paper
• Amazon SageMaker Automatic Model Tuning: Scalable Black-box Optimization

  2020cites this paper
• A Quantile-based Approach for Hyperparameter Transfer Learning

  2020cites this paper
• Multi-task Bayesian Optimization via Gaussian Process Upper Confidence Bound

  2020cites this paper
• Automatic Tuning of Stochastic Gradient Descent with Bayesian Optimisation

  2020cites this paper
• Neural Architecture Search using Bayesian Optimisation with Weisfeiler-Lehman Kernel

  2020cites this paper
• Neural Architecture Generator Optimization

  2020cites this paper
• On Hyper-parameter Tuning for Stochastic Optimization Algorithms

  2020cites this paper
• Learning search spaces for Bayesian optimization: Another view of hyperparameter transfer learning

  2019cites this paper
• Bayesian Optimization for Policy Search via Online-Offline Experimentation

  2019cites this paper
• Coordinated Traffic Signal Control Via Bayesian Optimization for Hierarchical Conditional Spaces

  2019cites this paper
• A Copula approach for hyperparameter transfer learning

  2019cites this paper
• Efficient Transfer Bayesian Optimization with Auxiliary Information

  2019cites this paper
• Scalable Meta-Learning for Bayesian Optimization

  2018cites this paper
• Regret bounds for meta Bayesian optimization with an unknown Gaussian process prior

  2018cites this paper
• Hyperparameter Learning via Distributional Transfer

  2018influential citation
• Scalable Meta-Learning for Bayesian Optimization using Ranking-Weighted Gaussian Process Ensembles

  2018cites this paper
• Continuous multi-task Bayesian Optimisation with correlation

  2018cites this paper
• BOHB: Robust and Efficient Hyperparameter Optimization at Scale

  2018cites this paper
• Practical Transfer Learning for Bayesian Optimization

  2018cites this paper
• ON PARALLELIZING MULTI-TASK BAYESIAN OPTIMIZATION

  2018cites this paper
• Scalable Hyperparameter Transfer Learning

  2018cites this paper
• Towards a unifying theory of generalization

  2017cites this paper
• Efficient expected improvement estimation for continuous multiple ranking and selection

  2017cites this paper
• Bayesian Optimization with Parallel Function Evaluations and Multiple Information Sources: Methodology with Applications in Biochemistry, Aerospace Engineering, and Machine Learning

  2017cites this paper
• Learning to Warm-Start Bayesian Hyperparameter Optimization

  2017cites this paper
• Multi-Information Source Optimization

  2016cites this paper
• Sequential design of computer experiments

  2015cites this paper