Personalized Federated Learning using Hypernetworks

Personalized federated learning is tasked with training machine learning models for multiple clients, each with its own data distribution. The goal is to train personalized models in a collaborative way while accounting for data disparities across clients and reducing communication costs. We propose a novel approach to this problem using hypernetworks, termed pFedHN for personalized Federated HyperNetworks. In this approach, a central hypernetwork model is trained to generate a set of models, one model for each client. This architecture provides effective parameter sharing across clients, while maintaining the capacity to generate unique and diverse personal models. Furthermore, since hypernetwork parameters are never transmitted, this approach decouples the communication cost from the trainable model size. We test pFedHN empirically in several personalized federated learning challenges and find that it outperforms previous methods. Finally, since hypernetworks share information across clients we show that pFedHN can generalize better to new clients whose distributions differ from any client observed during training.

• Publication year

  2021

• Venue

  International Conference on Machine Learning

• Publication date

  2021-03-08

• Fields of study

  Computer Science

• Identifiers
  arXiv 2103.04628
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• A survey on security and privacy of federated learning

  2021cited by this paper
• FedU: A Unified Framework for Federated Multi-Task Learning with Laplacian Regularization

  2021cited by this paper
• Personalized Federated Learning with First Order Model Optimization

  2020cited by this paper
• Personalized Cross-Silo Federated Learning on Non-IID Data

  2020cited by this paper
• Delta-STN: Efficient Bilevel Optimization for Neural Networks using Structured Response Jacobians

  2020cited by this paper
• Learning the Pareto Front with Hypernetworks

  2020cited by this paper
• Personalized Federated Learning with Moreau Envelopes

  2020influential reference
• Adaptive Personalized Federated Learning

  2020cited by this paper
• Survey of Personalization Techniques for Federated Learning

  2020cited by this paper
• Acceleration for Compressed Gradient Descent in Distributed and Federated Optimization

  2020cited by this paper
• Three Approaches for Personalization with Applications to Federated Learning

  2020cited by this paper
• Personalized Federated Learning for Intelligent IoT Applications: A Cloud-Edge Based Framework

  2020cited by this paper
• Personalized Federated Learning: A Meta-Learning Approach

  2020cited by this paper
• Federated Learning of a Mixture of Global and Local Models

  2020cited by this paper
• Think Locally, Act Globally: Federated Learning with Local and Global Representations

  2020cited by this paper
• Faster On-Device Training Using New Federated Momentum Algorithm

  2020cited by this paper
• Meta-Learning via Hypernetworks

  2020cited by this paper
• Efficient Meta Learning via Minibatch Proximal Update

  2019cited by this paper
• Self-Tuning Networks: Bilevel Optimization of Hyperparameters using Structured Best-Response Functions

  2019cited by this paper
• Federated Heavy Hitters Discovery with Differential Privacy

  2019cited by this paper
• Hypernetwork Functional Image Representation

  2019cited by this paper
• Alpha MAML: Adaptive Model-Agnostic Meta-Learning

  2019cited by this paper
• Continual learning with hypernetworks

  2019cited by this paper
• Qsparse-Local-SGD: Distributed SGD With Quantization, Sparsification, and Local Computations

  2019cited by this paper
• Lookahead Optimizer: k steps forward, 1 step back

  2019cited by this paper
• Federated Learning: Challenges, Methods, and Future Directions

  2019cited by this paper
• On the Convergence Theory of Gradient-Based Model-Agnostic Meta-Learning Algorithms

  2019cited by this paper
• Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification

  2019cited by this paper
• Hyper-Graph-Network Decoders for Block Codes

  2019cited by this paper
• FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization

  2019cited by this paper
• SCAFFOLD: Stochastic Controlled Averaging for On-Device Federated Learning

  2019cited by this paper
• On the Convergence of Local Descent Methods in Federated Learning

  2019cited by this paper
• Hyper-Sphere Quantization: Communication-Efficient SGD for Federated Learning

  2019cited by this paper
• Federated Learning with Personalization Layers

  2019cited by this paper
• Advances and Open Problems in Federated Learning

  2019cited by this paper
• Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection.

  2019cited by this paper
• Local SGD Converges Fast and Communicates Little

  2018cited by this paper
• cpSGD: Communication-efficient and differentially-private distributed SGD

  2018cited by this paper
• Federated Learning with Non-IID Data

  2018cited by this paper
• Don't Use Large Mini-Batches, Use Local SGD

  2018cited by this paper
• Cooperative SGD: A unified Framework for the Design and Analysis of Communication-Efficient SGD Algorithms

  2018cited by this paper
• Stochastic Hyperparameter Optimization through Hypernetworks

  2018cited by this paper
• On the Convergence of Federated Optimization in Heterogeneous Networks

  2018cited by this paper
• On First-Order Meta-Learning Algorithms

  2018cited by this paper
• Spectral Normalization for Generative Adversarial Networks

  2018cited by this paper
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks

  2017cited by this paper
• Federated Multi-Task Learning

  2017influential reference
• Learning Differentially Private Recurrent Language Models

  2017influential reference
• Meta-SGD: Learning to Learn Quickly for Few Shot Learning

  2017cited by this paper
• On the convergence properties of a K-step averaging stochastic gradient descent algorithm for nonconvex optimization

  2017cited by this paper
• Character-Level Language Modeling with Recurrent Highway Hypernetworks

  2017cited by this paper
• Communication-Efficient Learning of Deep Networks from Decentralized Data

  2016influential reference
• Conditioned Regression Models for Non-blind Single Image Super-Resolution

  2015cited by this paper
• A Dynamic Convolutional Layer for short rangeweather prediction

  2015cited by this paper
• Human-level concept learning through probabilistic program induction

  2015cited by this paper
• Privacy Aware Learning

  2012cited by this paper
• Learning Multiple Layers of Features from Tiny Images

  2009influential reference
• Visualizing Data using t-SNE

  2008cited by this paper
• Adaptive principal components and image denoising

  2003cited by this paper
• GradientBased Learning Applied to Document Recognition

  2001cited by this paper
• A Model of Inductive Bias Learning

  2000influential reference
• Gradient-based learning applied to document recognition

  1998influential reference
• Maximum Likelihood Estimation of Misspecified Models

  1982cited by this paper

• Enhancing Bandit Algorithms with LLMs for Time-varying User Preferences in Streaming Recommendations

  2026cites this paper
• PHH-FL: Perceptual Hashing Hypernetwork Personalized Federated Learning for Heterogeneous Medical Image Analysis Tasks

  2026cites this paper
• FLSC: A semantic-guided heterogeneous federated learning with cascading heterogeneity compensation

  2026cites this paper
• Personalized Federated Learning with Hierarchical Two-Branch Aggregation for Few-Shot Scenarios

  2026cites this paper
• Efficient Split Learning With Overlapping Areas: Handling Distribution Shift in Multi-Cell Networks

  2026cites this paper
• Per-FedDMA: A personalized federated learning method based on deep multisampling and hypernetwork dynamic adaptation

  2026cites this paper
• FedSpikeViT: Energy-Adaptive Federated Learning Framework for Spiking Vision Transformers with Dynamic Drift Detection in Solar-Harvested Edge Networks

  2026cites this paper
• Federated learning in oncology: Bridging artificial intelligence innovation and privacy protection

  2026cites this paper
• Toward Personalized Federated Meta-Learning With Constrained Hypernetwork on Non-IID Data

  2026influential citation
• SuperSFL: Resource-Heterogeneous Federated Split Learning with Weight-Sharing Super-Networks

  2026cites this paper
• Communication-Efficient Personalized Adaptation via Federated-Local Model Merging

  2026cites this paper
• FedHypeVAE: Federated Learning with Hypernetwork Generated Conditional VAEs for Differentially Private Embedding Sharing

  2026cites this paper
• PP-CLS-FL: Privacy-Preserving CLS-Transfer Federated Learning on Heterogeneous Clients

  2026cites this paper
• Domain Adaptive Mixture-of-Experts for Cross-domain Fake News Detection

  2026cites this paper
• Key-Value Mapping-Based Text-to-Image Diffusion Model Backdoor Attacks

  2026cites this paper
• Heterogeneity-Aware Client Selection Methodology For Efficient Federated Learning

  2026cites this paper
• From local bias to global consensus: Group-wise prototype federated learning under heterogeneous and cross-domain settings

  2026cites this paper
• Patient-Conditioned Adaptive Offsets for Reliable Diagnosis across Subgroups

  2026cites this paper
• Neighborhood and Global Perturbations Supported Sharpness-Aware Minimization in Federated Learning: From Local Tweaks to Global Awareness

  2026cites this paper
• Personalized Federated Learning via Gradient-Fusion and Gradient-Decoupling for Heterogeneous Data

  2026cites this paper
• Trustworthy distributed mirror learning for secure and private multi-agent coordination

  2026cites this paper
• Zero-Training Task-Specific Model Synthesis for Few-Shot Medical Image Classification

  2025cites this paper
• HyperShield: A personalized hypernetwork-based defense against poisoning attacks in federated learning for healthcare

  2025cites this paper
• FedRE: A Representation Entanglement Framework for Model-Heterogeneous Federated Learning

  2025cites this paper
• EPFL: Toward Elastic Personalized Federated Learning With Seamless Client Joining and Quitting

  2025influential citation
• SVQ-VAE: Federated-Learning-Based Semantic-Aware Communication for Vehicular Networks

  2025cites this paper
• FEELPGen: Data-Free Knowledge Distillation for Personalized Federated Learning Across Heterogeneous Edge Silos

  2025cites this paper
• Game-theoretic distributed learning of generative models for heterogeneous data collections

  2025cites this paper
• Hy2CRE: Hypernetworks with Hybrid Data Augmentation for Continual Relation Extraction

  2025cites this paper
• Personalized Federated Learning Based on Feature Filtering

  2025cites this paper
• A personalized federated hypernetworks based aggregation approach for intrusion detection systems

  2025cites this paper
• FedCM: Client Clustering and Migration in Federated Learning via Gradient Path Similarity and Update Direction Deviation

  2025cites this paper
• Model-Oriented Orchestration for Federated Learning: Personalization Through Intelligent Client Matching

  2025cites this paper
• Hypernetwork Aggregation for Decentralized Personalized Federated Learning

  2025cites this paper
• KApFL: Kolmogorov-Arnold Networks for Personalized Federated Learning

  2025cites this paper
• Adaptive Federated Distillation with Dual-LoRA for Personalized Representation Learning

  2025cites this paper
• Knowledge-distillation based personalized federated learning with distribution constraints

  2025cites this paper
• Whom to Trust? Adaptive Collaboration in Personalized Federated Learning

  2025cites this paper
• Deploying Models to Non-participating Clients in Federated Learning without Fine-tuning: A Hypernetwork-based Approach

  2025cites this paper
• Dynamic Personalized Federated Learning via Representation-Driven Clustering

  2025cites this paper
• Hypernetwork-Enhanced Hierarchical Federated Learning for Long-Term Traffic Prediction with Transformer

  2025cites this paper
• UniVarFL: Uniformity and Variance Regularized Federated Learning for Heterogeneous Data

  2025cites this paper
• HyperMem: Hypernetwork with memory for forgetting problem in federated reinforcement learning

  2025cites this paper
• Model Weights Protection in Federated Learning using Hypernetworks

  2025cites this paper
• Decoupling Neural Networks to Leverage Uniform Representation and Balance Personalization and Collaboration in Federated Learning

  2025cites this paper
• pFedDSH: Enabling Knowledge Transfer in Personalized Federated Learning through Data-free Sub-Hypernetwork

  2025cites this paper
• Federated Learning for Large Models in Medical Imaging: A Comprehensive Review

  2025cites this paper
• Tackling Device Data Distribution Real-time Shift via Prototype-based Parameter Editing

  2025cites this paper
• Learning Across the Divide: Personalised Federated Learning for Robust Clinical Modelling Under Data-View Heterogeneity

  2025cites this paper
• Sample-Level Prototypical Federated Learning

  2025cites this paper
• Byzantine-Robust Federated Learning Framework via a Server-Client Defense Mechanisms

  2025cites this paper
• Decentralized Dynamic Cooperation of Personalized Models for Federated Continual Learning

  2025cites this paper
• Layer-Wise Decoupling for Personalized Federated Learning in Web-Sourced Non-Iid Data

  2025cites this paper
• PFL-TEnet: Personalized federated learning for wind power forecasting via time-series embedding and hypernetwork modeling

  2025cites this paper
• Hot-pluggable Federated Learning: Bridging General and Personalized FL via Dynamic Selection

  2025cites this paper
• Improving personalized federated learning to optimize site-specific performance in computer-aided detection/diagnosis

  2025cites this paper
• Prototype-Based Communication Topology Optimization for Decentralized Federated Learning

  2025cites this paper
• Privacy-Preserving Data-Driven Learning Models for Emerging Communication Networks: A Comprehensive Survey

  2025cites this paper
• Federated Learning with Unlabeled Clients: Personalization Can Happen in Low Dimensions

  2025cites this paper
• FedPSFV: Personalized Federated Learning via Prototype Sharing for Finger Vein Recognition

  2025cites this paper
• FedPTR: Enhancing Federated Prompt Learning with Server-Side Retraining for Non-IID Data

  2025cites this paper
• Towards An Unsupervised Federated Hypernetwork Method for Distributed Anomaly Identification in Industrial IoT Environment

  2025cites this paper
• Beyond Personalization: Federated Recommendation with Calibration via Low-rank Decomposition

  2025cites this paper
• Personalized Subgraph Federated Learning with Sheaf Collaboration

  2025influential citation
• TTA-FedDG: Leveraging Test-Time Adaptation to Address Federated Domain Generalization

  2025influential citation
• Federated Learning with Domain Shift Eraser

  2025cites this paper
• Generalizing Personalized Federated Graph Augmentation via Min-max Adversarial Learning

  2025cites this paper
• GradPFL: Gradient-Driven Adaptive Clustering in Personalized Federated Learning

  2025cites this paper
• PFed-NS: An Adaptive Personalized Federated Learning Scheme Through Neural Network Segmentation

  2025cites this paper
• Personalized federated learning via decoupling self-knowledge distillation and global adaptive aggregation

  2025cites this paper
• Personalized Federated Learning via Learning Dynamic Graphs

  2025cites this paper
• Towards Personalized Federated Learning via Contrastive-Augmented Local Memorization Retrieval

  2025cites this paper
• HyDA: Hypernetworks for Test Time Domain Adaptation in Medical Imaging Analysis

  2025cites this paper
• A self-organized MoE framework for distributed federated learning

  2025cites this paper
• Transitioning From Federated Learning to Quantum Federated Learning in Internet of Things: A Comprehensive Survey

  2025cites this paper
• Personalized Lightweight Federated Learning for Efficient and Private Model Training in Heterogeneous Data Environments

  2025cites this paper
• pFedCal: Lightweight Personalized Federated Learning With Adaptive Calibration Strategy

  2025cites this paper
• Reducing Contextual Stochastic Bilevel Optimization via Structured Function Approximation

  2025cites this paper
• FedLFP: Communication-Efficient Personalized Federated Learning on Non-IID Data in Mobile Edge Computing Environments

  2025influential citation
• A Client-level Assessment of Collaborative Backdoor Poisoning in Non-IID Federated Learning

  2025influential citation
• FedRAV: Hierarchically Federated Region-Learning for Traffic Object Classification of Personalized Autonomous Vehicles With Guaranteed Efficiency

  2025cites this paper
• Lazy But Effective: Collaborative Personalized Federated Learning with Heterogeneous Data

  2025cites this paper
• Enhanced Traffic Carbon Emissions Data Sharing and Modeling via Blockchain and Personalized Federated Learning in IIoT Ecosystem

  2025cites this paper
• Avoid Forgetting by Preserving Global Knowledge Gradients in Federated Learning with Non-IID Data

  2025cites this paper
• Graph-Relational Federated Learning: Enhanced Personalization and Robustness

  2025influential citation
• Personalized federated learning through self-knowledge distillation in vehicular edge computing

  2025cites this paper
• Not All Clients Are Equal: Personalized Federated Learning on Heterogeneous Multi-Modal Clients

  2025cites this paper
• Intermediate features matter in prototype-guided personalized federated learning

  2025cites this paper
• FedBKD: Distilled Federated Learning to Embrace Gerneralization and Personalization on Non-IID Data

  2025cites this paper
• FedCALM: Conflict-aware Layer-wise Mitigation for Selective Aggregation in Deeper Personalized Federated Learning

  2025cites this paper
• Adaptive Federated Learning With Reinforcement Learning-Based Client Selection for Heterogeneous Environments

  2025cites this paper
• Federated Fine-Tuning on Heterogeneous LoRAs With Error-Compensated Aggregation

  2025cites this paper
• Hypernetworks for Model-Heterogeneous Personalized Federated Learning

  2025cites this paper
• ALSA: Context-Sensitive Prompt Privacy Preservation in Large Language Models

  2025cites this paper
• Towards Instance-wise Personalized Federated Learning via Semi-Implicit Bayesian Prompt Tuning

  2025influential citation
• Personalized Federated Learning Based on Dynamic Parameter Fusion and Prototype Alignment

  2025cites this paper
• Personalized Split Federated Learning With Early Exit: Pretraining and Online Learning Against Label Shifts

  2025cites this paper
• PFGL-Net: A Personalized Federated Graph Learning Framework for Privacy-Preserving Disease Prediction

  2025cites this paper
• FedMcon: an adaptive aggregation method for federated learning via meta controller

  2025cites this paper
• REACT: Residual-Adaptive Contextual Tuning for Fast Model Adaptation in Threat Detection

  2025cites this paper