A Knowledge-Gradient Policy for Sequential Information Collection

In a sequential Bayesian ranking and selection problem with independent normal populations and common known variance, we study a previously introduced measurement policy which we refer to as the knowledge-gradient policy. This policy myopically maximizes the expected increment in the value of information in each time period, where the value is measured according to the terminal utility function. We show that the knowledge-gradient policy is optimal both when the horizon is a single time period and in the limit as the horizon extends to infinity. We show furthermore that, in some special cases, the knowledge-gradient policy is optimal regardless of the length of any given fixed total sampling horizon. We bound the knowledge-gradient policy's suboptimality in the remaining cases, and show through simulations that it performs competitively with or significantly better than other policies.

• Publication year

  2008

• Venue

  SIAM Journal of Control and Optimization

• Publication date

  2008-01-01

• Fields of study

  Mathematics, Computer Science

• Identifiers
  DOI 10.1137/070693424
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Simulation Allocation for Determining the Best Design in the Presence of Correlated Sampling

  2007cited by this paper
• Opportunity Cost and OCBA Selection Procedures in Ordinal Optimization for a Fixed Number of Alternative Systems

  2007cited by this paper
• Selecting the Best System: Simulation

  2006cited by this paper
• On the Asymptotic Validity of Fully Sequential Selection Procedures for Steady-State Simulation

  2006cited by this paper
• New developments in ranking and selection: an empirical comparison of the three main approaches

  2005cited by this paper
• Optimal computing budget allocation for Monte Carlo simulation with application to product design

  2003cited by this paper
• New Procedures to Select the Best Simulated System Using Common Random Numbers

  2001cited by this paper
• New Two-Stage and Sequential Procedures for Selecting the Best Simulated System

  2001influential reference
• Simple Procedures for Selecting the Best Simulated System When the Number of Alternatives is Large

  2001cited by this paper
• A fully sequential procedure for indifference-zone selection in simulation

  2001cited by this paper
• Computing efforts allocation for ordinal optimization and discrete event simulation

  2000cited by this paper
• Simulation Budget Allocation for Further Enhancing the Efficiency of Ordinal Optimization

  2000cited by this paper
• Convergence Results for Single-Step On-Policy Reinforcement-Learning Algorithms

  2000cited by this paper
• New Development Of Optimal Computing Budget Allocation For Discrete Event Simulation

  1997influential reference
• A gradient approach for smartly allocating computing budget for discrete event simulation

  1996cited by this paper
• Bayesian look ahead one-stage sampling allocations for selection of the best population

  1996influential reference
• An effective approach to smartly allocate computing budget for discrete event simulation

  1995cited by this paper
• Design and Analysis of Experiments for Statistical Selection, Screening, and Multiple Comparisons

  1995cited by this paper
• Bayesian look ahead one stage sampling allocations for selecting the largest normal mean

  1994cited by this paper
• Sequential procedures for selecting the best one of k Koopman–Darmois populations

  1994cited by this paper
• Learning in embedded systems

  1993cited by this paper
• An improvement on paulson's procedure for selecting the poprlation with the largest mean from k normal populations with a common unknown variance

  1991cited by this paper
• On two-stage selection procedures and related probability-inequalities

  1978cited by this paper
• A Sequential Procedure for Selecting the Population with the Largest Mean from $k$ Normal Populations

  1964cited by this paper
• The Greatest of a Finite Set of Random Variables

  1961cited by this paper

• A Bayesian Approach to Multiobjective Aerodynamic Shape Optimization

  2026cites this paper
• Knowledge Gradient for Preference Learning

  2026cites this paper
• PyPRS: A Python Software Package for Parallel Ranking and Selection Procedures

  2026cites this paper
• Adaptive Gaussian Process Surrogate Modeling; Application to Flutter Boundary Uncertainty Quantification

  2026cites this paper
• Wasserstein-enabled characterization of designs and myopic decisions in Bayesian Optimization

  2026cites this paper
• Goal-Oriented Influence-Maximizing Data Acquisition for Learning and Optimization

  2026cites this paper
• Explore Then Confirm: Investment Portfolios for New Drug Therapies

  2025cites this paper
• A composite Bayesian optimisation framework for material and structural design

  2025cites this paper
• Bayesian Optimisation for Sensor Scheduling and Tracking with Different Acquisition Functions

  2025cites this paper
• A Sequential Optimal Learning Approach to Automated Prompt Engineering in Large Language Models

  2025cites this paper
• Huxley-Gödel Machine: Human-Level Coding Agent Development by an Approximation of the Optimal Self-Improving Machine

  2025cites this paper
• Informing Acquisition Functions via Foundation Models for Molecular Discovery

  2025cites this paper
• Efficient Active Learning Strategies for Computer Experiments

  2025cites this paper
• Learning Personalized Treatment Strategies with Predictive and Prognostic Covariates in Adaptive Clinical Trials

  2025cites this paper
• Dual-Directed Algorithm Design for Efficient Pure Exploration

  2025cites this paper
• Multi-Fidelity Bayesian Optimization for Nash Equilibria with Black-Box Utilities

  2025cites this paper
• Efficient Gaussian Process-based Motor Hotspot Hunting with Concurrent Optimization of TMS Coil Location and Orientation

  2025cites this paper
• Fast Bayesian Optimization of Function Networks with Partial Evaluations

  2025cites this paper
• One Set to Rule Them All: How to Obtain General Chemical Conditions via Bayesian Optimization over Curried Functions

  2025cites this paper
• Bayesian Optimization in Bioprocess Engineering—Where Do We Stand Today?

  2025cites this paper
• On Some Tunable Multi-fidelity Bayesian Optimization Frameworks

  2025cites this paper
• A Unified Framework for Entropy Search and Expected Improvement in Bayesian Optimization

  2025cites this paper
• Adaptive Elicitation of Latent Information Using Natural Language

  2025cites this paper
• Multiobjective aerodynamic design optimization of the NASA common research model

  2025cites this paper
• Accelerating multimetallic catalyst discovery with robotics and agentic AI

  2025cites this paper
• Spatial-adaptive active learning identifies ultra-durable and highly active catalysts for acidic oxygen evolution reaction.

  2025cites this paper
• NeST-BO: Fast Local Bayesian Optimization via Newton-Step Targeting of Gradient and Hessian Information

  2025cites this paper
• Enhanced Upper Confidence Bound Procedure for Large-Scale Ranking and Selection

  2025cites this paper
• Bayesian optimization based on Expected P-box Improvement and application in HEG Shock Tunnel Cylindrical Model Experiment

  2025cites this paper
• Data‐Driven Ranking and Selection With Simultaneous Input Data Collection and Simulation

  2025cites this paper
• Enhancing Batch Diversity in Surrogate Optimization: A Determinantal Point Processes Approach

  2025cites this paper
• None To Optima in Few Shots: Bayesian Optimization with MDP Priors

  2025cites this paper
• Efficient simulation budget allocation for contextual ranking and selection with quadratic models

  2025cites this paper
• Sampling from Gaussian processes: a tutorial and applications in global sensitivity analysis and optimization

  2025cites this paper
• Spectral Mixture Kernels for Bayesian Optimization

  2025cites this paper
• A Review of Simulation Optimization with Connection to Artificial Intelligence

  2025cites this paper
• Targeted Variance Reduction: Effective Bayesian Optimization of Black-Box Simulators with Noise Parameters

  2025cites this paper
• Knowledge Gradient Procedure to Select the Best System Under Pairwise Comparisons

  2025cites this paper
• A multimodal robotic platform for multi-element electrocatalyst discovery

  2025cites this paper
• Ranking and Selection with Simultaneous Input Data Collection

  2025cites this paper
• Bayesian Optimization for Intrinsically Noisy Response Surfaces

  2025cites this paper
• FigBO: A Generalized Acquisition Function Framework with Look-Ahead Capability for Bayesian Optimization

  2025cites this paper
• Additive Distributionally Robust Ranking and Selection

  2025cites this paper
• Multi-Objectivising Acquisition Functions in Bayesian Optimisation

  2025influential citation
• Efficient optimization of expensive black-box simulators via marginal means, with application to neutrino detector design

  2025cites this paper
• Budget Allocation Exploiting Label Correlation between Instances

  2025influential citation
• Adaptive Gaussian Process Surrogate Models for Efficient Uncertainty Quantification of the Flutter Boundary

  2025cites this paper
• Metareasoning in uncertain environments: a meta-BAMDP framework

  2024cites this paper
• Thompson Sampling Procedures for Ranking and Selection with Pairwise Comparisons and Binary Outcomes

  2024cites this paper
• Estimating Value of Information Arm Allocation Indices in Contextual Ranking and Selection Problems

  2024cites this paper
• Finding the optimal exploration-exploitation trade-off online through Bayesian risk estimation and minimization

  2024cites this paper
• Sample-Efficient Clustering and Conquer Procedures for Parallel Large-Scale Ranking and Selection

  2024cites this paper
• Epidemiological model calibration via graybox Bayesian optimization

  2024cites this paper
• Machine Learning-Engineered Nanozyme System for Synergistic Anti-Tumor Ferroptosis/Apoptosis Therapy.

  2024cites this paper
• PINN-BO: A Black-box Optimization Algorithm using Physics-Informed Neural Networks

  2024cites this paper
• Series Expansion of Probability of Correct Selection for Improved Finite Budget Allocation in Ranking and Selection

  2024cites this paper
• Selection of the Most Probable Best

  2024cites this paper
• Large-scale sandwich structures optimization using Bayesian method

  2024cites this paper
• Sequential Selection for Minimizing the Variance with Application to Crystallization Experiments

  2024influential citation
• Variational Entropy Search for Adjusting Expected Improvement

  2024cites this paper
• Fixed-Precision Ranking and Selection as Markov Decision Process

  2024cites this paper
• A Simple and Efficient Approach to Batch Bayesian Optimization

  2024cites this paper
• Expected Diverse Utility (EDU): Diverse Bayesian Optimization of Expensive Computer Simulators

  2024cites this paper
• A simulation optimization approach for weight valuation in analytic hierarchy process

  2024cites this paper
• Efficient Hyperparameter Optimization with Adaptive Fidelity Identification

  2024cites this paper
• Offline Contextual Learning with Correlated Bayesian Beliefs

  2024cites this paper
• Nonstationary A/B Tests: Optimal Variance Reduction, Bias Correction, and Valid Inference

  2024cites this paper
• Marginal improvement procedures for top-m selection

  2024cites this paper
• Identifying the Best Arm in the Presence of Global Environment Shifts

  2024cites this paper
• Feature Misspecification in Sequential Learning Problems

  2024cites this paper
• Expected Improvement applied to an industrialcontext -- Prediction of new geometries increasing theefficiency of fans

  2024cites this paper
• Metareasoning in uncertain environments: a meta-BAMDP framework

  2024cites this paper
• Koopman-Assisted Reinforcement Learning

  2024cites this paper
• Reference alternatives based knockout-tournament procedure for ranking and selection

  2024cites this paper
• Calibration of Compartmental Epidemiological Models via Graybox Bayesian Optimization

  2024cites this paper
• Robust Entropy Search for Safe Efficient Bayesian Optimization

  2024cites this paper
• Optimistic Bayesian Optimization with Unknown Constraints

  2024cites this paper
• Efficient constitutive parameter identification through optimisation-based techniques: A Comparative analysis and novel composite Bayesian optimisation strategy

  2024cites this paper
• Evolve Cost-aware Acquisition Functions Using Large Language Models

  2024cites this paper
• FunBO: Discovering Acquisition Functions for Bayesian Optimization with FunSearch

  2024cites this paper
• Simulation Budget Allocation for Improving Scheduling and Routing of Automated Guided Vehicles in Warehouse Management

  2024cites this paper
• Taking another step: A simple approach to high-dimensional Bayesian optimization

  2024cites this paper
• Bayesian Optimization of Environmentally Sustainable Graphene Inks Produced by Wet Jet Milling.

  2024cites this paper
• AExGym: Benchmarks and Environments for Adaptive Experimentation

  2024cites this paper
• Asynchronous Multi-Information Source Bayesian Optimization

  2024cites this paper
• Information-directed policy sampling for episodic Bayesian Markov decision processes

  2024cites this paper
• Run-indexed time-varying Bayesian optimization with positional encoding for auto-tuning of controllers: Application to a plasma-assisted deposition process with run-to-run drifts

  2024cites this paper
• An OCBA-Based Method for Efficient Sample Collection in Reinforcement Learning

  2024cites this paper
• Ranking and Selection with Two-Stage Decision

  2024cites this paper
• Adaptive Labeling for Efficient Out-of-distribution Model Evaluation

  2024cites this paper
• Targeted Variance Reduction: Robust Bayesian Optimization of Black-Box Simulators with Noise Parameters

  2024cites this paper
• Time-Varying Bayesian Optimization for MPC Calibration for Run-to-Run Drifting Systems: A Study on Discrete-Temporal Kernels

  2024cites this paper
• Epsilon-Greedy Thompson Sampling to Bayesian Optimization

  2024cites this paper
• GP+: A Python library for kernel-based learning via Gaussian processes

  2024cites this paper
• The Online Shortest Path Problem: Learning Travel Times Using a Multiarmed Bandit Framework

  2024cites this paper
• Multi-Fidelity Bayesian Optimization With Across-Task Transferable Max-Value Entropy Search

  2024cites this paper
• Expected Coordinate Improvement for High-Dimensional Bayesian Optimization

  2024cites this paper
• Ranking and selection with two-stage decision

  2024cites this paper
• Sequential Learning with a Similarity Selection Index

  2023cites this paper
• A Study of Bayesian Neural Network Surrogates for Bayesian Optimization

  2023cites this paper