HyperSHAP: Shapley Values and Interactions for Explaining Hyperparameter Optimization

Hyperparameter optimization (HPO) is a crucial step in achieving strong predictive performance. Yet, the impact of individual hyperparameters on model generalization is highly context-dependent, prohibiting a one-size-fits-all solution and requiring opaque HPO methods to find optimal configurations. However, the black-box nature of most HPO methods undermines user trust and discourages adoption. To address this, we propose a game-theoretic explainability framework for HPO based on Shapley values and interactions. Our approach provides an additive decomposition of a performance measure across hyperparameters, enabling local and global explanations of hyperparameters'contributions and their interactions. The framework, named HyperSHAP, offers insights into ablation studies, the tunability of learning algorithms, and optimizer behavior across different hyperparameter spaces. We demonstrate HyperSHAP's capabilities on various HPO benchmarks to analyze the interaction structure of the corresponding HPO problems, demonstrating its broad applicability and actionable insights for improving HPO.

• Publication year

  2025

• Venue

  Unknown venue

• Publication date

  2025-02-03

• Fields of study

  Mathematics, Computer Science

• Identifiers
  arXiv 2502.01276
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Dynamic Priors in Bayesian Optimization for Hyperparameter Optimization

  2025cited by this paper
• KernelSHAP-IQ: Weighted Least-Square Optimization for Shapley Interactions

  2024cited by this paper
• Position: Why We Must Rethink Empirical Research in Machine Learning

  2024cited by this paper
• Beyond TreeSHAP: Efficient Computation of Any-Order Shapley Interactions for Tree Ensembles

  2024cited by this paper
• shapiq: Shapley Interactions for Machine Learning

  2024influential reference
• Hyperparameter Importance Analysis for Multi-Objective AutoML

  2024cited by this paper
• Position: A Call to Action for a Human-Centered AutoML Paradigm

  2024cited by this paper
• PED-ANOVA: Efficiently Quantifying Hyperparameter Importance in Arbitrary Subspaces

  2023cited by this paper
• Hyperparameter Optimization for Large Language Model Instruction-Tuning

  2023cited by this paper
• A General Framework for User-Guided Bayesian Optimization

  2023cited by this paper
• Practitioner Motives to Select Hyperparameter Optimization Methods

  2023cited by this paper
• Cost-Effective Hyperparameter Optimization for Large Language Model Generation Inference

  2023cited by this paper
• JAHS-Bench-201: A Foundation For Research On Joint Architecture And Hyperparameter Search

  2022influential reference
• HPO X ELA: Investigating Hyperparameter Optimization Landscapes by Means of Exploratory Landscape Analysis

  2022cited by this paper
• Enhancing Explainability of Hyperparameter Optimization via Bayesian Algorithm Execution

  2022cited by this paper
• Faith-Shap: The Faithful Shapley Interaction Index

  2022influential reference
• The Shapley Value in Machine Learning

  2022cited by this paper
• YAHPO Gym - An Efficient Multi-Objective Multi-Fidelity Benchmark for Hyperparameter Optimization

  2021influential reference
• AutoML for Multi-Label Classification: Overview and Empirical Evaluation

  2021cited by this paper
• SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimization

  2021cited by this paper
• Explaining Hyperparameter Optimization via Partial Dependence Plots

  2021cited by this paper
• Whither AutoML? Understanding the Role of Automation in Machine Learning Workflows

  2021cited by this paper
• To tune or not to tune? An Approach for Recommending Important Hyperparameters

  2021cited by this paper
• Pre-trained Gaussian Processes for Bayesian Optimization

  2021cited by this paper
• AutoTinyBERT: Automatic Hyper-parameter Optimization for Efficient Pre-trained Language Models

  2021cited by this paper
• NetGAN without GAN: From Random Walks to Low-Rank Approximations

  2020influential reference
• Trust in AutoML: exploring information needs for establishing trust in automated machine learning systems

  2020cited by this paper
• LiBRe: Label-Wise Selection of Base Learners in Binary Relevance for Multi-label Classification

  2020cited by this paper
• Visualizing the Impact of Feature Attribution Baselines

  2020cited by this paper
• Auto-PyTorch Tabular: Multi-Fidelity MetaLearning for Efficient and Robust AutoDL

  2020influential reference
• Golden parameter search: exploiting structure to quickly configure parameters in parallel

  2020influential reference
• DeepLine: AutoML Tool for Pipelines Generation using Deep Reinforcement Learning and Hierarchical Actions Filtering

  2019cited by this paper
• The Shapley Taylor Interaction Index

  2019cited by this paper
• A Human-in-the-loop Perspective on AutoML: Milestones and the Road Ahead

  2019cited by this paper
• The many Shapley values for model explanation

  2019cited by this paper
• Tunability: Importance of Hyperparameters of Machine Learning Algorithms

  2018influential reference
• Hyperparameter Importance Across Datasets

  2017cited by this paper
• A Unified Approach to Interpreting Model Predictions

  2017cited by this paper
• Efficient and Robust Automated Machine Learning

  2015cited by this paper
• Analysing differences between algorithm configurations through ablation

  2015cited by this paper
• An Efficient Approach for Assessing Hyperparameter Importance

  2014influential reference
• Input Warping for Bayesian Optimization of Non-Stationary Functions

  2014cited by this paper
• UpSet: Visualization of Intersecting Sets

  2014cited by this paper
• Measuring algorithm footprints in instance space

  2012cited by this paper
• Random Search for Hyper-Parameter Optimization

  2012cited by this paper
• Variance Components and Generalized Sobol' Indices

  2012cited by this paper
• Axiomatic characterizations of probabilistic and cardinal-probabilistic interaction indices

  2006influential reference
• An axiomatic approach to the concept of interaction among players in cooperative games

  1999cited by this paper
• How Evaluation Guides AI Research: The Message Still Counts More than the Medium

  1988cited by this paper
• A Value for n-person Games

  1988cited by this paper
• Extremal Principle Solutions of Games in Characteristic Function Form: Core, Chebychev and Shapley Value Generalizations

  1988cited by this paper

• Conditional PED-ANOVA: Hyperparameter Importance in Hierarchical & Dynamic Search Spaces

  2026cites this paper