Knowledge-distillation based personalized federated learning with distribution constraints

Personalized federated learning PFL seeks to develop models that are tailored to the unique data distributions of individual clients. While some methods rely on a global server model to guide personalization, more recent methods focus on directly learn personalized models. Among these, leveraging inter-client correlations has become a widely adopted strategy for personalized model generation. PFedGraph exemplifies this by constructing client relationships based on model similarity for personalized model aggregation, achieving outstanding personalized performance. However, pFedgraph overlooks category distribution information, a critical aspect reflecting data distribution heterogeneity, although it has been extensively applied in machine learning and federated learning tasks. Category distribution can serve as a direct and informative metric for measuring inter-client data divergence. Furthermore, pFedGraph underutilizes global knowledge derived from diverse client datasets, limiting its personalized ability. To address these limitations, we incorporate category distribution constraints into the computation of client-specific aggregation weights, enabling the generation of personalized models enriched with distribution-aware information. Additionally, to mitigate the risk of overfitting to local data and enhance the use of global knowledge, we align the outputs of personalized models with those of the global model, which is obtained through the classical federated averaging algorithm, to effectively transfer shared global knowledge to personalized models. The proposed method consistently outperforms state-of-the-art approaches across diverse data types and distribution scenarios, demonstrating its effectiveness.

• Publication year

  2025

• Venue

  Neural Networks

• Publication date

  2025-08-01

• Fields of study

  Medicine, Computer Science

• Identifiers
  DOI 10.1016/j.neunet.2025.107951PMID 40840295
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar, PubMed

• No claims are published for this paper.

• No concepts are published for this paper.

• Probabilistic Federated Prompt-Tuning with Non-IID and Imbalanced Data

  2025cited by this paper
• FedCAFE: Federated Cross-Modal Hashing with Adaptive Feature Enhancement

  2024cited by this paper
• MemberShield: A framework for federated learning with membership privacy

  2024cited by this paper
• FedEAN: Entity-Aware Adversarial Negative Sampling for Federated Knowledge Graph Reasoning

  2024cited by this paper
• DKD-pFed: A novel framework for personalized federated learning via decoupling knowledge distillation and feature decorrelation

  2024cited by this paper
• Addressing unreliable local models in federated learning through unlearning

  2024cited by this paper
• Data-free knowledge distillation via generator-free data generation for Non-IID federated learning

  2024cited by this paper
• To be global or personalized: Generalized federated learning with cooperative adaptation for data heterogeneity

  2024cited by this paper
• Subgraph-level federated graph neural network for privacy-preserving recommendation with meta-learning

  2024cited by this paper
• Self-balancing Incremental Broad Learning System with privacy protection

  2024cited by this paper
• FedAS: Bridging Inconsistency in Personalized Federated Learning

  2024cited by this paper
• Exploring personalization via federated representation Learning on non-IID data

  2023cited by this paper
• Federated Learning via Input-Output Collaborative Distillation

  2023cited by this paper
• Exploiting Label Skews in Federated Learning with Model Concatenation

  2023cited by this paper
• Unlocking the Potential of Prompt-Tuning in Bridging Generalized and Personalized Federated Learning

  2023cited by this paper
• Bold but Cautious: Unlocking the Potential of Personalized Federated Learning through Cautiously Aggressive Collaboration

  2023cited by this paper
• Personalized Federated Learning with Inferred Collaboration Graphs

  2023cited by this paper
• Efficient Personalized Federated Learning via Sparse Model-Adaptation

  2023cited by this paper
• Self-Supervised Temporal Graph Learning With Temporal and Structural Intensity Alignment

  2023cited by this paper
• Federated Learning on Heterogeneous and Long-Tailed Data via Classifier Re-Training with Federated Features

  2022cited by this paper
• FEDIC: Federated Learning on Non-IID and Long-Tailed Data via Calibrated Distillation

  2022cited by this paper
• Layer-wised Model Aggregation for Personalized Federated Learning

  2022cited by this paper
• FedTP: Federated Learning by Transformer Personalization

  2022cited by this paper
• FedALA: Adaptive Local Aggregation for Personalized Federated Learning

  2022cited by this paper
• On Bridging Generic and Personalized Federated Learning for Image Classification

  2022cited by this paper
• Exploiting Shared Representations for Personalized Federated Learning

  2021cited by this paper
• Federated Learning Based on Dynamic Regularization

  2021cited by this paper
• Personalized Federated Learning through Local Memorization

  2021cited by this paper
• Parameterized Knowledge Transfer for Personalized Federated Learning

  2021cited by this paper
• Model-Contrastive Federated Learning

  2021cited by this paper
• Personalized Federated Learning using Hypernetworks

  2021cited by this paper
• Ditto: Fair and Robust Federated Learning Through Personalization

  2020cited by this paper
• Empirical Studies of Institutional Federated Learning For Natural Language Processing

  2020cited by this paper
• Group Knowledge Transfer: Federated Learning of Large CNNs at the Edge

  2020cited by this paper
• Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization

  2020cited by this paper
• Personalized Federated Learning with Moreau Envelopes

  2020cited by this paper
• Personalized Cross-Silo Federated Learning on Non-IID Data

  2020cited by this paper
• Personalized Federated Learning with Theoretical Guarantees: A Model-Agnostic Meta-Learning Approach

  2020cited by this paper
• PyTorch: An Imperative Style, High-Performance Deep Learning Library

  2019cited by this paper
• SCAFFOLD: Stochastic Controlled Averaging for Federated Learning

  2019cited by this paper
• Advances and Open Problems in Federated Learning

  2019cited by this paper
• Clustered Federated Learning: Model-Agnostic Distributed Multitask Optimization Under Privacy Constraints

  2019cited by this paper
• Federated Optimization in Heterogeneous Networks

  2018cited by this paper
• An Affect-Rich Neural Conversational Model with Biased Attention and Weighted Cross-Entropy Loss

  2018cited by this paper
• Attention is All you Need

  2017cited by this paper
• Communication-Efficient Learning of Deep Networks from Decentralized Data

  2016cited by this paper
• Deep Residual Learning for Image Recognition

  2015cited by this paper
• Character-level Convolutional Networks for Text Classification

  2015cited by this paper
• Reading Digits in Natural Images with Unsupervised Feature Learning

  2011cited by this paper

• No citing papers are available for this paper.