Speeding Up Automatic Hyperparameter Optimization of Deep Neural Networks by Extrapolation of Learning Curves

No abstract is available for this paper.

• Publication year

  2015

• Venue

  International Joint Conference on Artificial Intelligence

• Publication date

  2015-07-25

• Fields of study

  Computer Science

• Identifiers

  No identifiers available.

• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Algorithm Runtime Prediction: Methods and Evaluation (Extended Abstract)

  2015cited by this paper
• Dropout: a simple way to prevent neural networks from overfitting

  2014influential reference
• Deeply-Supervised Nets

  2014influential reference
• Caffe: Convolutional Architecture for Fast Feature Embedding

  2014cited by this paper
• Freeze-Thaw Bayesian Optimization

  2014cited by this paper
• Raiders of the Lost Architecture: Kernels for Bayesian Optimization in Conditional Parameter Spaces

  2014influential reference
• Striving for Simplicity: The All Convolutional Net

  2014influential reference
• Making a Science of Model Search: Hyperparameter Optimization in Hundreds of Dimensions for Vision Architectures

  2013cited by this paper
• Network In Network

  2013influential reference
• Towards an Empirical Foundation for Assessing Bayesian Optimization of Hyperparameters

  2013cited by this paper
• Maxout Networks

  2013influential reference
• New types of deep neural network learning for speech recognition and related applications: an overview

  2013cited by this paper
• DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition

  2013cited by this paper
• Improving deep neural networks for LVCSR using rectified linear units and dropout

  2013cited by this paper
• ImageNet classification with deep convolutional neural networks

  2012influential reference
• Algorithm runtime prediction: Methods & evaluation

  2012cited by this paper
• emcee: The MCMC Hammer

  2012cited by this paper
• Neural Networks: Tricks of the Trade

  2012cited by this paper
• Random Search for Hyper-Parameter Optimization

  2012cited by this paper
• Practical Bayesian Optimization of Machine Learning Algorithms

  2012influential reference
• Prediction of Learning Curves in Machine Translation

  2012cited by this paper
• Sequential Model-Based Optimization for General Algorithm Configuration

  2011cited by this paper
• An Analysis of Single-Layer Networks in Unsupervised Feature Learning

  2011influential reference
• Algorithms for Hyper-Parameter Optimization

  2011influential reference
• Understanding the difficulty of training deep feedforward neural networks

  2010cited by this paper
• A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning

  2010cited by this paper
• What is the best multi-stage architecture for object recognition?

  2009cited by this paper
• Learning Multiple Layers of Features from Tiny Images

  2009influential reference
• On Early Stopping in Gradient Descent Learning

  2007cited by this paper
• Neural Networks: Tricks of the Trade

  2002cited by this paper
• Random Forests

  2001cited by this paper
• Modelling Classification Performance for Large Data Sets

  2001cited by this paper
• Gradient-Based Optimization of Hyperparameters

  2000cited by this paper
• Modeling decision tree performance with the power law

  1999cited by this paper
• Efficient Global Optimization of Expensive Black-Box Functions

  1998cited by this paper
• Hoeffding Races: Accelerating Model Selection Search for Classification and Function Approximation

  1993cited by this paper
• Backpropagation Applied to Handwritten Zip Code Recognition

  1989cited by this paper

• CAPE: Generalized Convergence Prediction Across Architectures Without Full Training

  2026influential citation
• Empirical Gaussian Processes

  2026cites this paper
• Lithium-ion battery state of health estimation using automated machine learning (AutoML): performance comparison

  2026cites this paper
• Evolutionary Neural Architecture Search for Super-Resolution: Benchmarking SynFlow and model-based predictors

  2026cites this paper
• Data Complexity-aware Deep Model Performance Forecasting

  2026cites this paper
• Model-Centric Diagnostics: A Framework for Internal State Readouts

  2026cites this paper
• Training Memory in Deep Neural Networks: Mechanisms, Evidence, and Measurement Gaps

  2026cites this paper
• Takeuchi's Information Criteria as Generalization Measures for DNNs Close to NTK Regime

  2026cites this paper
• Zero-Shot Performance Prediction for Probabilistic Scaling Laws

  2025cites this paper
• Adapting Curricula by Tracking the Gap between Validation and Training Loss

  2025cites this paper
• High-Dimensional Hyperparameter Optimization via Adjoint Differentiation

  2025cites this paper
• Enhancing Knowledge Reusability: A Distributed Multitask Machine Learning Approach

  2025cites this paper
• Bayesian Neural Scaling Law Extrapolation with Prior-Data Fitted Networks

  2025influential citation
• Closed-Loop Parameter Optimization for Robotic Machining Using Physics-Informed Machine Learning and Multiobjective Optimization

  2025cites this paper
• A Multi-Power Law for Loss Curve Prediction Across Learning Rate Schedules

  2025cites this paper
• Comparative Analysis of Automated Machine Learning for Hyperparameter Optimization and Explainable Artificial Intelligence Models

  2025cites this paper
• Learning Learning Curves

  2025cites this paper
• Optimal Resource Allocation for ML Model Training and Deployment under Concept Drift

  2025cites this paper
• EA-HAS-Bench and Language-Enhanced Shrinkage Search for Energy-Aware NAS

  2025cites this paper
• Method for Recognizing Behavioral Variation Within Apis Mellifera Bee Hives Using Deep Learning

  2025cites this paper
• Modular Machine Learning: An Indispensable Path towards New-Generation Large Language Models

  2025cites this paper
• Systematic review on neural architecture search

  2025cites this paper
• In-Context Probing for Membership Inference in Fine-Tuned Language Models

  2025cites this paper
• Investigation of Training-free Metrics for Multi-objective Neural Architecture Search

  2025cites this paper
• Optimising neural network hyperparameters by predicting unseen architectures with learning curve approach

  2025cites this paper
• LCDB 1.1: A Database Illustrating Learning Curves Are More Ill-Behaved Than Previously Thought

  2025cites this paper
• Successive Halving with Learning Curve Prediction via Latent Kronecker Gaussian Processes

  2025cites this paper
• One-Shot Neural Architecture Search with Network Similarity Directed Initialization for Pathological Image Classification

  2025cites this paper
• Data Mixture Optimization: A Multi-fidelity Multi-scale Bayesian Framework

  2025cites this paper
• Esc: An Early-Stopping Checker for Budget-aware Index Tuning

  2025cites this paper
• Scaling with Collapse: Efficient and Predictable Training of LLM Families

  2025cites this paper
• A method for the automated configuration of anytime portfolios of algorithms

  2025cites this paper
• Boiler operation predictions by integrating thermo-fluid principles within an artificial neural network framework

  2025cites this paper
• NNGPT: Rethinking AutoML with Large Language Models

  2025cites this paper
• Efficient Hyperparameter Search for Non-Stationary Model Training

  2025cites this paper
• Context-Aware Search Space Adaptation of Hyperparameters and Architectures for AutoML in Text Classification

  2025cites this paper
• Cost-Sensitive Freeze-thaw Bayesian Optimization for Efficient Hyperparameter Tuning

  2025cites this paper
• Energy-Aware Neural Architecture Search: Leveraging Genetic Algorithms for Balancing Performance and Consumption

  2024cites this paper
• UQ-Guided Hyperparameter Optimization for Iterative Learners

  2024cites this paper
• Architecture-Aware Learning Curve Extrapolation via Graph Ordinary Differential Equation

  2024cites this paper
• Scaling Gaussian Processes for Learning Curve Prediction via Latent Kronecker Structure

  2024cites this paper
• A novel optimal deep learning approach for designing intrusion detection system in wireless sensor networks

  2024cites this paper
• Towards Green AI by Reducing Training Effort of Recurrent Neural Networks Using Hyper-Parameter Optimization with Dynamic Stopping Criteria

  2024cites this paper
• Learning with Adaptive Resource Allocation

  2024cites this paper
• ProxyLM: Predicting Language Model Performance on Multilingual Tasks via Proxy Models

  2024cites this paper
• Hyperparameter optimization: Classics, acceleration, online, multi-objective, and tools.

  2024cites this paper
• Aioli: A Unified Optimization Framework for Language Model Data Mixing

  2024cites this paper
• Dynamic Anisotropic Smoothing for Noisy Derivative-Free Optimization

  2024cites this paper
• Towards Fair and Rigorous Evaluations: Hyperparameter Optimization for Top-N Recommendation Task with Implicit Feedback

  2024cites this paper
• Meta-learning from learning curves for budget-limited algorithm selection

  2024cites this paper
• Best Practices of Successive Halving on Neural Machine Translation and Large Language Models

  2024cites this paper
• Efficient Hyperparameter Optimization with Adaptive Fidelity Identification

  2024cites this paper
• A Survey on Neural Architecture Search Based on Reinforcement Learning

  2024cites this paper
• Which classifiers are connected to others? An optimal connection framework for multi-layer ensemble systems

  2024cites this paper
• ConceptGlassbox: Guided Concept-Based Explanation for Deep Neural Networks

  2024cites this paper
• A Training-Free Neural Architecture Search Algorithm Based on Search Economics

  2024cites this paper
• Cost-Sensitive Multi-Fidelity Bayesian Optimization with Transfer of Learning Curve Extrapolation

  2024cites this paper
• Online Resource Allocation for Edge Intelligence with Colocated Model Retraining and Inference

  2024cites this paper
• ESC-NAS: Environment Sound Classification Using Hardware-Aware Neural Architecture Search for the Edge

  2024cites this paper
• Primitive Agentic First-Order Optimization

  2024cites this paper
• BTTackler: A Diagnosis-based Framework for Efficient Deep Learning Hyperparameter Optimization

  2024cites this paper
• Enhancing classification of lung diseases by optimizing training hyperparameters of the deep learning network

  2024cites this paper
• GI-NAS: Boosting Gradient Inversion Attacks Through Adaptive Neural Architecture Search

  2024cites this paper
• FastTuning: Enabling Fast and Efficient Hyper-Parameter Tuning With Partitioning and Parallelism of Search Space

  2024cites this paper
• The Unreasonable Effectiveness Of Early Discarding After One Epoch In Neural Network Hyperparameter Optimization

  2024influential citation
• In-Context Freeze-Thaw Bayesian Optimization for Hyperparameter Optimization

  2024influential citation
• Network properties determine neural network performance

  2024cites this paper
• MIGAN: A Privacy Leakage Evaluation Scheme for CIoT-Based Federated Learning Users

  2024cites this paper
• Use of edge resources for DNN model maintenance in 5G IoT networks

  2024cites this paper
• UniSched: A Unified Scheduler for Deep Learning Training Jobs With Different User Demands

  2024influential citation
• Evaluating Learning Potential with Internal States in Deep Neural Networks

  2024cites this paper
• Using reinforcement learning to autonomously identify sources of error for agents in group missions

  2024cites this paper
• CE-NAS: An End-to-End Carbon-Efficient Neural Architecture Search Framework

  2024cites this paper
• AZ-NAS: Assembling Zero-Cost Proxies for Network Architecture Search

  2024cites this paper
• Improved Search in Neuroevolution Using a Neural Architecture Classifier With the CNN Architecture Encoding as Feature Vector

  2024cites this paper
• RL-Based Hyperparameter Selection for Spectrum Sensing With CNNs

  2024cites this paper
• Fast Model Selection and Hyperparameter Tuning for Generative Models

  2024cites this paper
• FlexHB: a More Efficient and Flexible Framework for Hyperparameter Optimization

  2024cites this paper
• GraphNAS++: Distributed Architecture Search for Graph Neural Networks

  2023cites this paper
• Practitioner Motives to Select Hyperparameter Optimization Methods

  2023cites this paper
• PriorBand: Practical Hyperparameter Optimization in the Age of Deep Learning

  2023cites this paper
• Using Pipeline Performance Prediction to Accelerate AutoML Systems

  2023cites this paper
• EA-HAS-Bench: Energy-aware Hyperparameter and Architecture Search Benchmark

  2023influential citation
• Scaling Laws for Hyperparameter Optimization

  2023cites this paper
• Newton design: designing CNNs with the family of Newton’s methods

  2023cites this paper
• OpenBox: A Python Toolkit for Generalized Black-box Optimization

  2023cites this paper
• Hyper-parameter Tuning for Adversarially Robust Models

  2023cites this paper
• An improved hyperparameter optimization framework for AutoML systems using evolutionary algorithms

  2023cites this paper
• LayerNAS: Neural Architecture Search in Polynomial Complexity

  2023cites this paper
• MSINet: Twins Contrastive Search of Multi-Scale Interaction for Object ReID

  2023cites this paper
• A feature-wise attention module based on the difference with surrounding features for convolutional neural networks

  2023cites this paper
• GP-NAS-ensemble: a model for NAS Performance Prediction

  2023cites this paper
• Optimizing Hyperparameters with Conformal Quantile Regression

  2023cites this paper
• MASIF: Meta-learned Algorithm Selection using Implicit Fidelity Information

  2023cites this paper
• Learning to Rank Normalized Entropy Curves with Differentiable Window Transformation

  2023cites this paper
• Efficient Evaluation Methods for Neural Architecture Search: A Survey

  2023cites this paper
• Learning Reliable Neural Networks with Distributed Architecture Representations

  2023cites this paper
• Neural Architecture Search: Insights from 1000 Papers

  2023influential citation
• Deep Power Laws for Hyperparameter Optimization

  2023cites this paper
• Gray-Box Gaussian Processes for Automated Reinforcement Learning

  2023cites this paper