Integrating low-rank and group-sparse structures for robust multi-task learning

Multi-task learning (MTL) aims at improving the generalization performance by utilizing the intrinsic relationships among multiple related tasks. A key assumption in most MTL algorithms is that all tasks are related, which, however, may not be the case in many real-world applications. In this paper, we propose a robust multi-task learning (RMTL) algorithm which learns multiple tasks simultaneously as well as identifies the irrelevant (outlier) tasks. Specifically, the proposed RMTL algorithm captures the task relationships using a low-rank structure, and simultaneously identifies the outlier tasks using a group-sparse structure. The proposed RMTL algorithm is formulated as a non-smooth convex (unconstrained) optimization problem. We propose to adopt the accelerated proximal method (APM) for solving such an optimization problem. The key component in APM is the computation of the proximal operator, which can be shown to admit an analytic solution. We also theoretically analyze the effectiveness of the RMTL algorithm. In particular, we derive a key property of the optimal solution to RMTL; moreover, based on this key property, we establish a theoretical bound for characterizing the learning performance of RMTL. Our experimental results on benchmark data sets demonstrate the effectiveness and efficiency of the proposed algorithm.

• Publication year

  2011

• Venue

  Knowledge Discovery and Data Mining

• Publication date

  2011-08-21

• Fields of study

  Mathematics, Computer Science

• Identifiers
  DOI 10.1145/2020408.2020423
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• I and J

  2012cited by this paper
• SLEP: Sparse Learning with Efficient Projections

  2011cited by this paper
• Robust PCA via Outlier Pursuit

  2010cited by this paper
• An efficient algorithm for a class of fused lasso problems

  2010cited by this paper
• Learning incoherent sparse and low-rank patterns from multiple tasks

  2010cited by this paper
• Multi-Task Learning using Generalized t Process

  2010cited by this paper
• A Dirty Model for Multi-task Learning

  2010cited by this paper
• Efficient and Robust Feature Selection via Joint ℓ2, 1-Norms Minimization

  2010cited by this paper
• Robust Matrix Decomposition with Outliers

  2010cited by this paper
• Multi-Task Feature Learning Via Efficient l2, 1-Norm Minimization

  2009cited by this paper
• An accelerated gradient method for trace norm minimization

  2009cited by this paper
• Convergence of Fixed-Point Continuation Algorithms for Matrix Rank Minimization

  2009cited by this paper
• A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems

  2009influential reference
• A convex formulation for learning shared structures from multiple tasks

  2009cited by this paper
• Heterogeneous multitask learning with joint sparsity constraints

  2009cited by this paper
• Taking Advantage of Sparsity in Multi-Task Learning

  2009cited by this paper
• Large-scale sparse logistic regression

  2009cited by this paper
• Tree-Guided Group Lasso for Multi-Task Regression with Structured Sparsity

  2009cited by this paper
• Clustered Multi-Task Learning: A Convex Formulation

  2008cited by this paper
• SIMULTANEOUS ANALYSIS OF LASSO AND DANTZIG SELECTOR

  2008cited by this paper
• A Singular Value Thresholding Algorithm for Matrix Completion

  2008cited by this paper
• Convex multi-task feature learning

  2008influential reference
• Robust multi-task learning with t-processes

  2007cited by this paper
• EFFICIENT METHODS IN CONVEX PROGRAMMING

  2007influential reference
• Multi-Task Learning for Classification with Dirichlet Process Priors

  2007cited by this paper
• Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization

  2007cited by this paper
• Multi-task feature selection

  2006cited by this paper
• Learning Multiple Related Tasks using Latent Independent Component Analysis

  2005cited by this paper
• A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data

  2005cited by this paper
• Learning Gaussian processes from multiple tasks

  2005cited by this paper
• Learning Multiple Tasks with Kernel Methods

  2005cited by this paper
• Learning to learn with the informative vector machine

  2004cited by this paper
• Learning Gaussian Process Kernels via Hierarchical Bayes

  2004cited by this paper
• Task Clustering and Gating for Bayesian Multitask Learning

  2003cited by this paper
• Multitask Learning

  1997cited by this paper
• Discovering Structure in Multiple Learning Tasks: The TC Algorithm

  1996cited by this paper
• Proximité et dualité dans un espace hilbertien

  1965cited by this paper
• Bounds on Normal Approximations to Student's and the Chi-Square Distributions

  1959cited by this paper

• Robust Multi-Task Regression with Shifting Low-Rank Patterns

  2025cites this paper
• Late Fusion Multi-task Learning for Semiparametric Inference with Nuisance Parameters

  2025cites this paper
• Precision Fertilization via Spatio-temporal Tensor Multi-task Learning and One-Shot Learning

  2025cites this paper
• Robust-Multi-Task Gradient Boosting

  2025cites this paper
• Convex Clustering Redefined: Robust Learning with the Median of Means Estimator

  2025cites this paper
• Multi-task federated learning with encoder–decoder structure: enabling collaborative learning across different tasks

  2025cites this paper
• Regularized multi-task learning with individual-feature-based task correlations for Alzheimer's cognitive score prediction

  2025cites this paper
• Leveraging large-scale aerial data for accurate urban rooftop solar potential estimation via multitask learning

  2025cites this paper
• Adaptive sieving with semismooth Newton proximal augmented Lagrangian algorithm for multi-task Lasso problems

  2025cites this paper
• Computationally efficient and statistically optimal robust high-dimensional linear regression

  2025cites this paper
• Method for Knowledge Transfer via Multi-Task Semi-Supervised Self-Paced

  2025cites this paper
• Tensor Completion for Alzheimer's Disease Prediction From Diffusion Tensor Imaging

  2024cites this paper
• Multi-task learning via robust regularized clustering with non-convex group penalties

  2024cites this paper
• Machine Learning System Aware Optimization

  2024cites this paper
• Assessing the Capabilities of UV-NIR Spectroscopy for Predicting Macronutrients in Hydroponic Solutions with Single-Task and Multi-Task Learning

  2024cites this paper
• ESSR: Evolving Sparse Sharing Representation for Multitask Learning

  2024cites this paper
• Adaptive dual graph regularization for clustered multi-task learning

  2024cites this paper
• Informative relationship multi-task learning: Exploring pairwise contribution across tasks' sharing knowledge

  2024cites this paper
• Elaborate multi-task subspace learning with discrete group constraint

  2023cites this paper
• Fast Brillouin Frequency Shift Retrieval by Sparse Frequency Enhanced Neural Network

  2023cites this paper
• Robust Temporal Smoothness in Multi-Task Learning

  2023cites this paper
• Additive multi-task learning models and task diagnostics

  2023cites this paper
• Automatic Temporal Relation in Multi-Task Learning

  2023cites this paper
• Decoding working-memory load during n-back task performance from high channel fNIRS data

  2023cites this paper
• Multi-view shaker detection: Insights from a noise-immune influence analysis Perspective

  2023cites this paper
• Sustainable fertilisation management via tensor multi-task learning using multi-dimensional agricultural data

  2023cites this paper
• Efficient multi-task learning with adaptive temporal structure for progression prediction

  2023cites this paper
• Deep Robust Low Rank Correlation With Unifying Clustering Structure for Cross Domain Adaptation

  2023cites this paper
• Calibrated multi-task subspace learning via binary group structure constraint

  2023cites this paper
• A cross-domain recommendation model by unified modelling high-order information and rating information

  2023cites this paper
• Multilinear multitask learning by transformed tensor singular value decomposition

  2023cites this paper
• Computationally Efficient and Statistically Optimal Robust High-Dimensional Linear Regression

  2023influential citation
• Multi-Task Learning with Prior Information

  2023cites this paper
• Multi-Task Learning with Calibrated Mixture of Insightful Experts

  2022cites this paper
• Representation learning with deep sparse auto-encoder for multi-task learning

  2022cites this paper
• Meta-Knowledge and Multi-Task Learning-Based Multi-Scene Adaptive Crowd Counting

  2022influential citation
• Saliency-Regularized Deep Multi-Task Learning

  2022cites this paper
• Path Signature Neural Network of Cortical Features for Prediction of Infant Cognitive Scores

  2022cites this paper
• Multi-task longitudinal forecasting with missing values on Alzheimer's Disease

  2022cites this paper
• Computationally Efficient and Statistically Optimal Robust Low-rank Matrix and Tensor Estimation

  2022cites this paper
• Multitask transfer learning with kernel representation

  2022cites this paper
• Mixed-integer quadratic programming reformulations of multi-task learning models

  2022cites this paper
• Computationally Efficient and Statistically Optimal Robust Low-rank Matrix Estimation

  2022cites this paper
• Modeling Alzheimer’s Disease Progression via Amalgamated Magnitude-Direction Brain Structure Variation Quantification and Tensor Multi-task Learning

  2022cites this paper
• Spatio-temporal Tensor Multi-Task Learning for Precision Fertilisation with Real-world Agricultural Data

  2022cites this paper
• Adaptive and Robust Multi-task Learning

  2022influential citation
• Adaptive Data Fusion for Multi-task Non-smooth Optimization

  2022influential citation
• Task Modeling: Approximating Multitask Predictions for Cross-Task Transfer

  2022cites this paper
• PaCo: Parameter-Compositional Multi-Task Reinforcement Learning

  2022cites this paper
• Exploiting task relationships for Alzheimer's disease cognitive score prediction via multi-task learning

  2022cites this paper
• Sparse assortment personalization in high dimensions

  2022cites this paper
• Multi-task Learning with Adaptive Global Temporal Structure for Predicting Alzheimer's Disease Progression

  2022cites this paper
• Sign-regularized Multi-task Learning

  2021cites this paper
• FedU: A Unified Framework for Federated Multi-Task Learning with Laplacian Regularization

  2021cites this paper
• A Scalable, Adaptive and Sound Nonconvex Regularizer for Low-rank Matrix Learning

  2021cites this paper
• Multi-Task Learning Via Sharing Inexact Low-Rank Subspace

  2021cites this paper
• Seed Stocking Via Multi-Task Learning

  2021cites this paper
• Spatio-Temporal Multi-Task Learning via Tensor Decomposition

  2021cites this paper
• Dual feature correlation guided multi-task learning for Alzheimer's disease prediction

  2021cites this paper
• S3 DMT-Net: improving soft sharing based multi-task CNN using task-specific distillation and cross-task interactions

  2021influential citation
• Multi-task Learning of Order-Consistent Causal Graphs

  2021cites this paper
• Unveiling Groups of Related Tasks in Multi - Task Learning

  2021cites this paper
• Multitask Learning for Classification Problem via New Tight Relaxation of Rank Minimization

  2021influential citation
• Transfer Vision Patterns for Multi-Task Pixel Learning

  2021cites this paper
• New Tight Relaxations of Rank Minimization for Multi-Task Learning

  2021cites this paper
• Auto-weighted centralised multi-task learning via integrating functional and structural connectivity for subjective cognitive decline diagnosis

  2021cites this paper
• A new deep belief network-based multi-task learning for diagnosis of Alzheimer’s disease

  2021cites this paper
• A Multi-Task Learning Approach to Personalized Progression Modeling

  2020cites this paper
• Multi-Task Logistic Low-Ranked Dirty Model for Fault Detection in Power Distribution System

  2020influential citation
• Transfer Learning in Recommender Systems

  2020cites this paper
• Heterogeneous Transfer Learning

  2020cites this paper
• Byzantine Resilient Distributed Multi-Task Learning

  2020cites this paper
• A Metric-Learning-Based Nonlinear Modeling Algorithm and Its Application in Key-Performance-Indicator Prediction

  2020influential citation
• Transfer Learning

  2020cites this paper
• Multitask Bayesian Spatiotemporal Gaussian Processes for Short-Term Load Forecasting

  2020influential citation
• Comparison of machine learning approaches for classification of invoices

  2020cites this paper
• Low-Rank Matrix Recovery from Noisy via an MDL Framework-based Atomic Norm

  2020cites this paper
• Multi-task Learning

  2020cites this paper
• Lifelong Machine Learning

  2020cites this paper
• Towards Interpretable Multi-Task Learning Using Bilevel Programming

  2020cites this paper
• Federated Multi-task Learning with Hierarchical Attention for Sensor Data Analytics

  2020cites this paper
• Infant Cognitive Scores Prediction with Multi-stream Attention-Based Temporal Path Signature Features

  2020cites this paper
• SD-MTCNN: Self-Distilled Multi-Task CNN

  2020cites this paper
• Efficient Low-Rank Matrix Learning by Factorizable Nonconvex Regularization

  2020cites this paper
• Transfer Learning in Dialogue Systems

  2020cites this paper
• Low-Rank Matrix Recovery from Noise via an MDL Framework-Based Atomic Norm

  2020cites this paper
• Transitive Transfer Learning

  2020cites this paper
• Multilinear Multitask Learning by Rank-Product Regularization

  2020cites this paper
• Few-Shot Learning

  2020cites this paper
• Generalization Bounds of Multitask Learning From Perspective of Vector-Valued Function Learning

  2020cites this paper
• Unified Multi-Domain Learning and Data Imputation using Adversarial Autoencoder

  2020cites this paper
• Personalized Federated Learning: An Attentive Collaboration Approach

  2020cites this paper
• Transfer Learning in Computer Vision

  2020cites this paper
• Failure prediction of tasks in the cloud at an earlier stage: a solution based on domain information mining

  2020cites this paper
• Relation-Based Transfer Learning

  2020cites this paper
• A multi-task multi-class learning method for automatic identification of heavy minerals from river sand

  2020cites this paper
• AdaMT-Net: An Adaptive Weight Learning Based Multi-Task Learning Model For Scene Understanding

  2020cites this paper
• Instance-Based Transfer Learning

  2020cites this paper
• Introduction

  2020cites this paper
• Self-calibrated brain network estimation and joint non-convex multi-task learning for identification of early Alzheimer's disease

  2020cites this paper