On First-Order Meta-Learning Algorithms

This paper considers meta-learning problems, where there is a distribution of tasks, and we would like to obtain an agent that performs well (i.e., learns quickly) when presented with a previously unseen task sampled from this distribution. We analyze a family of algorithms for learning a parameter initialization that can be fine-tuned quickly on a new task, using only first-order derivatives for the meta-learning updates. This family includes and generalizes first-order MAML, an approximation to MAML obtained by ignoring second-order derivatives. It also includes Reptile, a new algorithm that we introduce here, which works by repeatedly sampling a task, training on it, and moving the initialization towards the trained weights on that task. We expand on the results from Finn et al. showing that first-order meta-learning algorithms perform well on some well-established benchmarks for few-shot classification, and we provide theoretical analysis aimed at understanding why these algorithms work.

• Publication year

  2018

• Venue

  arXiv.org

• Publication date

  2018-03-08

• Fields of study

  Computer Science

• Identifiers
  arXiv 1803.02999
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Recasting Gradient-Based Meta-Learning as Hierarchical Bayes

  2018cited by this paper
• Meta-Learning and Universality: Deep Representations and Gradient Descent can Approximate any Learning Algorithm

  2017cited by this paper
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks

  2017influential reference
• Few-shot Autoregressive Density Estimation: Towards Learning to Learn Distributions

  2017cited by this paper
• RL$^2$: Fast Reinforcement Learning via Slow Reinforcement Learning

  2016cited by this paper
• Learning to learn by gradient descent by gradient descent

  2016cited by this paper
• Meta-Learning with Memory-Augmented Neural Networks

  2016cited by this paper
• Optimization as a Model for Few-Shot Learning

  2016cited by this paper
• Matching Networks for One Shot Learning

  2016influential reference
• Human-level concept learning through probabilistic program induction

  2015cited by this paper
• Dueling Network Architectures for Deep Reinforcement Learning

  2015cited by this paper
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

  2015cited by this paper
• Part-Based R-CNNs for Fine-Grained Category Detection

  2014cited by this paper
• Adam: A Method for Stochastic Optimization

  2014cited by this paper
• One-Shot Learning with a Hierarchical Nonparametric Bayesian Model

  2011cited by this paper
• One shot learning of simple visual concepts

  2011cited by this paper
• Parallelized Stochastic Gradient Descent

  2010cited by this paper
• ImageNet: A large-scale hierarchical image database

  2009cited by this paper
• Meaning and compositionality as statistical induction of categories and constraints

  2009cited by this paper
• The CMA Evolution Strategy: A Comparing Review

  2006cited by this paper
• Using fast weights to deblur old memories

  1987cited by this paper
• Learning curves for stochastic gradient descent in linear feedforward networks

  year unknowncited by this paper

• Few-shot Medical Classification: Methods, Challenges and Future Directions

  2026cites this paper
• Fully First-Order Algorithms for Online Bilevel Optimization

  2026cites this paper
• Self-adaptive Resource Allocation in Fog-Cloud Systems Using Multi-agent Deep Reinforcement Learning with Meta-learning

  2026cites this paper
• Test-Time Mixture of World Models for Embodied Agents in Dynamic Environments

  2026cites this paper
• TorsoTrack: Fine-Grained Nonintrusive Torso Inclination Assessment Using mmwave Radar

  2026cites this paper
• MePo: Meta Post-Refinement for Rehearsal-Free General Continual Learning

  2026cites this paper
• Meta-Learning Empowered Receiver Design for MIMO-OFDM System Under HPA Nonlinearity

  2026cites this paper
• Self-Regressive Prototype Refinement: Stepping from Local to Global Prototypes in Few-Shot Image Classification

  2026cites this paper
• Improving Interactive In-Context Learning from Natural Language Feedback

  2026cites this paper
• 1S-DAug: One-Shot Data Augmentation for Robust Few-Shot Generalization

  2026cites this paper
• Prompt tuning with preference ranking for few-shot pre-trained decision transformer

  2026cites this paper
• Intelligent Omni-Surface-Aided Multi-Objective ISAC: A Meta Hybrid Deep Reinforcement Learning Approach

  2026influential citation
• Physics Informed Differentiable Solvers for Learning Parametric Solution Manifolds in Heterogeneous Physical Systems

  2026cites this paper
• Beyond route-specific forecasting: An empirical test of two cross-series transfer learning strategies for airline demand with short-data constraints

  2026cites this paper
• AgentCompress: Task-Aware Compression for Affordable Large Language Model Agents

  2026cites this paper
• Toward Adaptive Grid Resilience: A Gradient-Free Meta-RL Framework for Critical Load Restoration

  2026cites this paper
• Incremental Learning in Medical Imaging: A Comprehensive Survey of Deep Neural Network Advances and Challenges

  2026cites this paper
• Meta-Learned Zero-Shot Sketch-Based Point Cloud Retrieval via Perspective-Predicted Feature Learning

  2026cites this paper
• FactorMiner: A Self-Evolving Agent with Skills and Experience Memory for Financial Alpha Discovery

  2026cites this paper
• Complex emotion recognition system using basic emotions via facial expression, electroencephalogram, and electrocardiogram signals: a review

  2026cites this paper
• Hybrid Granularity Distribution Estimation for Few-Shot Learning: Statistics Transfer From Categories and Instances

  2026cites this paper
• Trust Region Continual Learning as an Implicit Meta-Learner

  2026cites this paper
• FAMF: Robust Feature-Level Adversarial Attack on Metric-Based Few-Shot Learning Models

  2026cites this paper
• FedPML: Federated Prototype Meta-Learning for Adaptation and Generalization in Distributed Environments

  2026cites this paper
• Domain Knowledge-Informed Few-Shot Fault Diagnosis of Multiple Low-Severity Faults in PMSGs Under Imbalanced Loads

  2026influential citation
• A 40-nm Training-Inference STT-MRAM Near-Memory Computing Macro for Memory-Augmented Neural Network Acceleration

  2026cites this paper
• Readability-Robust Code Summarization via Meta Curriculum Learning

  2026cites this paper
• Semantic Modulated Prompting for Few-Shot Audio-Visual Classification

  2026cites this paper
• Teaching Models to Teach Themselves: Reasoning at the Edge of Learnability

  2026influential citation
• When Does Adaptation Win? Scaling Laws for Meta-Learning in Quantum Control

  2026influential citation
• Heterogeneity-Aware Federated Meta Learning for Personalized Edge Devices

  2026influential citation
• A Meta-Knowledge-Driven Approach for Adaptive Security Provisioning in Industrial IoT

  2026cites this paper
• Meta-learning to Address Data Shift in Time Series Classification

  2026cites this paper
• Heterogeneous GNN-driven and meta-learned architecture for transparent multimodal string similarity estimation

  2026cites this paper
• Cross-Domain Deep Reinforcement Learning for Real-Time Resource Allocation in Transportation Hubs: From Airport Gates to Seaport Berths

  2026cites this paper
• Influence Guided Sampling for Domain Adaptation of Text Retrievers

  2026cites this paper
• Make Anything Match Your Target: Universal Adversarial Perturbations against Closed-Source MLLMs via Multi-Crop Routed Meta Optimization

  2026cites this paper
• EMA Policy Gradient: Taming Reinforcement Learning for LLMs with EMA Anchor and Top-k KL

  2026cites this paper
• Towards Remote Sensing Change Detection with Neural Memory

  2026cites this paper
• Meta-Sel: Efficient Demonstration Selection for In-Context Learning via Supervised Meta-Learning

  2026cites this paper
• From Text to Diagnosis: Exploring LLM-Assisted Prompt Strategies for Multimodal Few-Shot Medical Image Learning

  2026cites this paper
• VISA: Value Injection via Shielded Adaptation for Personalized LLM Alignment

  2026cites this paper
• Intelligent Read Framework With Meta-Learning for NAND Flash Memory Under Process Variation

  2026cites this paper
• Eternal-MAML: a meta-learning framework for cross-domain defect recognition

  2025cites this paper
• Few-Shot Learning Based on Multimodal Information Processing

  2025cites this paper
• GPRN: GAN-based prototype refinement network for few-shot learning

  2025cites this paper
• Meta-AlignNN: A meta-learning framework for stable brain-computer interface performance across subjects, time, and tasks

  2025cites this paper
• MetaMolGen: A Neural Graph Motif Generation Model for De Novo Molecular Design

  2025influential citation
• CAMeL: Cross-Modality Adaptive Meta-Learning for Text-Based Person Retrieval

  2025cites this paper
• LiFT: Learning to Fine-Tune via Bayesian Parameter Efficient Meta Fine-Tuning

  2025influential citation
• Federated Neural Architecture Search with Model-Agnostic Meta Learning

  2025cites this paper
• Meta-Continual Learning of Neural Fields

  2025cites this paper
• SaRoHead: Detecting Satire in a Multi-Domain Romanian News Headline Dataset

  2025influential citation
• Meta-Learning With Task-Adaptive Selection

  2025cites this paper
• In-Context Policy Adaptation via Cross-Domain Skill Diffusion

  2025cites this paper
• Optimizing Data Distribution and Kernel Performance for Efficient Training of Chemistry Foundation Models: A Case Study with MACE

  2025cites this paper
• MISE: Meta-knowledge Inheritance for Social Media-Based Stressor Estimation

  2025cites this paper
• An adaptive quantitative trading strategy optimization framework based on meta reinforcement learning and cognitive game theory

  2025cites this paper
• Legilimens: Performant Video Analytics on the System-on-Chip Edge

  2025cites this paper
• STDA: Spatio-Temporal Deviation Alignment Learning for Cross-City Fine-Grained Urban Flow Inference

  2025cites this paper
• Meta-Reinforcement Learning With Evolving Gradient Regularization

  2025cites this paper
• Data-Based Modeling and Control of a Single Link Soft Robotic Arm

  2025cites this paper
• Learning From Natural Images in Few-Shot SAR Target Classification

  2025cites this paper
• Free Lunch to Meet the Gap: Intermediate Domain Reconstruction for Cross-Domain Few-Shot Learning

  2025cites this paper
• Automatic test-time adaptation for heterogeneous contexts in meta-learning

  2025cites this paper
• Balanced Direction from Multifarious Choices: Arithmetic Meta-Learning for Domain Generalization

  2025cites this paper
• Adaptive Physics-informed Neural Networks: A Survey

  2025cites this paper
• Meta-learning for cosmological emulation: Rapid adaptation to new lensing kernels

  2025cites this paper
• Wireless Perceptual Space Modeling Method for Cross-Domain Human Activity Recognition

  2025cites this paper
• Handling Domain Shifts for Anomalous Sound Detection: A Review of DCASE-Related Work

  2025cites this paper
• MetaFAP: Meta-Learning for Frequency Agnostic Prediction of Metasurface Properties

  2025cites this paper
• The Self-Learning Agent with a Progressive Neural Network Integrated Transformer

  2025cites this paper
• Meta-Learning to Teach Semantic Prompts for Open Domain Generalization in Vision-Language Models

  2025cites this paper
• Large Pre-Trained Models and Few-Shot Fine-Tuning for Virtual Metrology: A Framework for Uncertainty-Driven Adaptive Process Control in Semiconductor Manufacturing

  2025cites this paper
• Asynchronous Personalized Federated Learning through Global Memorization

  2025cites this paper
• Learning to Generalize without Bias for Open-Vocabulary Action Recognition

  2025cites this paper
• Model-Agnostic Meta-Policy Optimization via Zeroth-Order Estimation: A Linear Quadratic Regulator Perspective

  2025cites this paper
• MetaSym: A Symplectic Meta-learning Framework for Physical Intelligence

  2025cites this paper
• Deep-sea pipeline damage identification using digital twin-assisted enhanced meta-transfer learning

  2025cites this paper
• Contrastive Visual Data Augmentation

  2025cites this paper
• FLCL: Feature-Level Contrastive Learning for Few-Shot Image Classification

  2025cites this paper
• Meta-Learning-Assisted Untrained Neural Network for Electromagnetic Inverse Scattering Problems

  2025cites this paper
• Generalizable speech deepfake detection via meta-learned LoRA

  2025cites this paper
• Multi-task classification network for few-shot learning

  2025cites this paper
• Global–Local Decomposition of Contextual Representations in Meta-Reinforcement Learning

  2025cites this paper
• Scalable Fingerprinting of Large Language Models

  2025cites this paper
• Federated Learning via Meta-Variational Dropout

  2025cites this paper
• Tailored meta-learning for dual trajectory transformer: advancing generalized trajectory prediction

  2025cites this paper
• Remote Sensing Image Classification Method Based on MAML Improvement

  2025cites this paper
• Sparks of cognitive flexibility: self-guided context inference for flexible stimulus-response mapping by attentional routing

  2025cites this paper
• Distilling Reinforcement Learning Algorithms for In-Context Model-Based Planning

  2025cites this paper
• Between Circuits and Chomsky: Pre-pretraining on Formal Languages Imparts Linguistic Biases

  2025cites this paper
• A Spatially Aware Few-Shot Approach to Classification of Radar Sounder Data

  2025cites this paper
• Foundation Models Secretly Understand Neural Network Weights: Enhancing Hypernetwork Architectures with Foundation Models

  2025cites this paper
• Context-Aware Constrained Reinforcement Learning-Based Energy-Efficient Power Scheduling for Non-Stationary XR Data Traffic

  2025cites this paper
• Meta-learning characteristics and dynamics of quantum systems

  2025cites this paper
• Ensemble-Based Model-Agnostic Meta-Learning with Operational Grouping for Intelligent Sensory Systems

  2025cites this paper
• Conservative Offline Meta-Reinforcement Learning with Task Similarity Measurement

  2025cites this paper
• HyperNVD: Accelerating Neural Video Decomposition via Hypernetworks

  2025cites this paper
• Meta-INR: Efficient Encoding of Volumetric Data via Meta-Learning Implicit Neural Representation

  2025cites this paper