Adaptive Experimental Design for Optimizing Combinatorial Structures

Scientists and engineers in diverse domains need to perform expensive experiments to optimize combinatorial spaces, where each candidate input is a discrete structure (e.g., sequence, tree, graph) or a hybrid structure (mixture of discrete and continuous design variables). For example, in hardware design optimization over locations of processing cores and communication links for data transfer, design evaluation involves performing a computationally-expensive simulation. These experiments are often performed in a heuristic manner by humans and without any formal reasoning. In this paper, we first describe the key challenges in solving these problems in the framework of Bayesian optimization (BO) and our progress over the last five years in addressing these challenges. We also discuss exciting sustainability applications in domains such as electronic design automation, nanoporous materials science, biological sequence design, and electric transportation systems.

• Publication year

  2021

• Venue

  International Joint Conference on Artificial Intelligence

• Publication date

  2021-08-01

• Fields of study

  Mathematics, Computer Science, Engineering

• Identifiers
  DOI 10.24963/ijcai.2021/699
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• High-Throughput Training of Deep CNNs on ReRAM-Based Heterogeneous Architectures via Optimized Normalization Layers

  2022cited by this paper
• Bayesian optimization of nanoporous materials

  2021influential reference
• 3D++: Unlocking the Next Generation of High-Performance and Energy-Efficient Architectures using M3D Integration

  2021cited by this paper
• Bayesian Optimization over Hybrid Spaces

  2021influential reference
• Learning Pareto-Frontier Resource Management Policies for Heterogeneous SoCs: An Information-Theoretic Approach

  2021influential reference
• ReGraphX: NoC-enabled 3D Heterogeneous ReRAM Architecture for Training Graph Neural Networks

  2021cited by this paper
• Information-Theoretic Multi-Objective Bayesian Optimization with Continuous Approximations

  2020influential reference
• Scalable Combinatorial Bayesian Optimization with Tractable Statistical models

  2020cited by this paper
• Design of Multi-Output Switched-Capacitor Voltage Regulator via Machine Learning

  2020cited by this paper
• Power, Performance, and Thermal Trade-offs in M3D-enabled Manycore Chips

  2020cited by this paper
• Uncertainty-Aware Search Framework for Multi-Objective Bayesian Optimization

  2020influential reference
• Optimizing Discrete Spaces via Expensive Evaluations: A Learning to Search Framework

  2020influential reference
• Multi-Fidelity Multi-Objective Bayesian Optimization: An Output Space Entropy Search Approach

  2020cited by this paper
• An Energy-aware Online Learning Framework for Resource Management in Heterogeneous Platforms

  2020cited by this paper
• Uncertainty aware Search Framework for Multi-Objective Bayesian Optimization with Constraints

  2020influential reference
• Mixed-Variable Bayesian Optimization

  2020cited by this paper
• Mercer Features for Efficient Combinatorial Bayesian Optimization

  2020influential reference
• Max-value Entropy Search for Multi-objective Bayesian Optimization with Constraints

  2020influential reference
• AccuReD: High Accuracy Training of CNNs on ReRAM/GPU Heterogeneous 3-D Architecture

  2020cited by this paper
• Machine Learning Enabled Fast Multi-Objective Optimization for Electrified Aviation Power System Design

  2020influential reference
• Max-value Entropy Search for Multi-Objective Bayesian Optimization

  2020cited by this paper
• Combinatorial Bayesian Optimization using the Graph Cartesian Product

  2019influential reference
• Dynamic Resource Management of Heterogeneous Mobile Platforms via Imitation Learning

  2019cited by this paper
• Autonomous Design Space Exploration of Computing Systems for Sustainability: Opportunities and Challenges

  2019cited by this paper
• Impact of Electrostatic Coupling on Monolithic 3D-enabled Network on Chip

  2019cited by this paper
• Hybrid On-Chip Communication Architectures for Heterogeneous Manycore Systems

  2018cited by this paper
• Bayesian Optimization of Combinatorial Structures

  2018cited by this paper
• A new golden age for computer architecture: Domain-specific hardware/software co-design, enhanced security, open instruction sets, and agile chip development

  2018cited by this paper
• Performance and Thermal Tradeoffs for Energy-Efficient Monolithic 3D Network-on-Chip

  2018cited by this paper
• Learning-Based Application-Agnostic 3D NoC Design for Heterogeneous Manycore Systems

  2018cited by this paper
• Machine-learning-guided directed evolution for protein engineering

  2018cited by this paper
• Leiden University

  2018cited by this paper
• Max-value Entropy Search for Efficient Bayesian Optimization

  2017cited by this paper
• On-Chip Communication Network for Efficient Training of Deep Convolutional Networks on Heterogeneous Manycore Systems

  2017cited by this paper
• Monolithic 3D-Enabled High Performance and Energy Efficient Network-on-Chip

  2017cited by this paper
• Imitation Learning for Dynamic VFI Control in Large-Scale Manycore Systems

  2017cited by this paper
• Automatic Chemical Design Using a Data-Driven Continuous Representation of Molecules

  2016cited by this paper
• Arch2030: A Vision of Computer Architecture Research over the Next 15 Years

  2016cited by this paper
• Taking the Human Out of the Loop: A Review of Bayesian Optimization

  2016cited by this paper
• Output-Space Predictive Entropy Search for Flexible Global Optimization

  2016cited by this paper
• Bayesian Optimization with Resource Constraints and Production

  2016cited by this paper
• Design-Space Exploration and Optimization of an Energy-Efficient and Reliable 3-D Small-World Network-on-Chip

  2016cited by this paper
• Predictive Entropy Search for Multi-objective Bayesian Optimization

  2015cited by this paper
• Submodularization for Binary Pairwise Energies

  2014cited by this paper
• Structured prediction via output space search

  2014cited by this paper
• HC-Search: A Learning Framework for Search-based Structured Prediction

  2014cited by this paper
• HC-Search for Multi-Label Prediction: An Empirical Study

  2014cited by this paper
• Auto-WEKA: combined selection and hyperparameter optimization of classification algorithms

  2012cited by this paper
• Practical Bayesian Optimization of Machine Learning Algorithms

  2012cited by this paper
• Additive Gaussian Processes

  2011cited by this paper
• Sequential Model-Based Optimization for General Algorithm Configuration

  2011cited by this paper
• Search-based structured prediction

  2009cited by this paper
• The computation of the expected improvement in dominated hypervolume of Pareto front approximations

  2008cited by this paper
• Nature Methods

  2007cited by this paper
• ParEGO: a hybrid algorithm with on-line landscape approximation for expensive multiobjective optimization problems

  2006cited by this paper
• Integer linear programming inference for conditional random fields

  2005cited by this paper
• Performance

  2005cited by this paper
• Support vector machine learning for interdependent and structured output spaces

  2004cited by this paper
• Max-Margin Markov Networks

  2003cited by this paper
• Diffusion kernels on graphs and other discrete structures

  2002cited by this paper
• Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data

  2001cited by this paper

• PathWise: Planning through World Model for Automated Heuristic Design via Self-Evolving LLMs

  2026cites this paper
• Adaptive Combinatorial Maximization: Beyond Approximate Greedy Policies

  2024cites this paper
• Tree Search-Based Evolutionary Bandits for Protein Sequence Optimization

  2024cites this paper
• Bayesian Optimization over High-Dimensional Combinatorial Spaces via Dictionary-based Embeddings

  2023cites this paper
• Bandit Theory and Thompson Sampling-Guided Directed Evolution for Sequence Optimization

  2022cites this paper
• Machine Learning Enabled Design Automation and Multi-Objective Optimization for Electric Transportation Power Systems

  2022cites this paper
• Simulation-Assisted Optimization for Large-Scale Evacuation Planning with Congestion-Dependent Delays

  2022cites this paper
• Bayesian optimization of nanoporous materials

  2021cites this paper
• Combining Latent Space and Structured Kernels for Bayesian Optimization over Combinatorial Spaces

  2021cites this paper
• Output Space Entropy Search Framework for Multi-Objective Bayesian Optimization

  2021cites this paper
• Bayesian Optimization over Permutation Spaces

  2021influential citation