Group-sparse Embeddings in Collective Matrix Factorization

Abstract: CMF is a technique for simultaneously learning low-rank representations based on a collection of matrices with shared entities. A typical example is the joint modeling of user-item, item-property, and user-feature matrices in a recommender system. The key idea in CMF is that the embeddings are shared across the matrices, which enables transferring information between them. The existing solutions, however, break down when the individual matrices have low-rank structure not shared with others. In this work we present a novel CMF solution that allows each of the matrices to have a separate low-rank structure that is independent of the other matrices, as well as structures that are shared only by a subset of them. We compare MAP and variational Bayesian solutions based on alternating optimization algorithms and show that the model automatically infers the nature of each factor using group-wise sparsity. Our approach supports in a principled way continuous, binary and count observations and is efficient for sparse matrices involving missing data. We illustrate the solution on a number of examples, focusing in particular on an interesting use-case of augmented multi-view learning.

• Publication year

  2013

• Venue

  International Conference on Learning Representations

• Publication date

  2013-12-20

• Fields of study

  Mathematics, Computer Science

• Identifiers
  arXiv 1312.5921
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Matrix Factorization Techniques for Recommender Systems

  2014cited by this paper
• Learning New Facts From Knowledge Bases With Neural Tensor Networks and Semantic Word Vectors

  2013cited by this paper
• Connecting comments and tags: improved modeling of social tagging systems

  2013cited by this paper
• A semantic matching energy function for learning with multi-relational data

  2013cited by this paper
• Bayesian Canonical correlation analysis

  2013influential reference
• Convex Collective Matrix Factorization

  2013influential reference
• Fast Variational Bayesian Inference for Non-Conjugate Matrix Factorization Models

  2012cited by this paper
• A Three-Way Model for Collective Learning on Multi-Relational Data

  2011cited by this paper
• Bayesian Group Factor Analysis

  2011influential reference
• Matrix co-factorization for recommendation with rich side information and implicit feedback

  2011cited by this paper
• Factorized Latent Spaces with Structured Sparsity

  2010cited by this paper
• A Bayesian Matrix Factorization Model for Relational Data

  2010cited by this paper
• Multi-set canonical correlation analysis for the fusion of concurrent single trial ERP and functional MRI

  2010cited by this paper
• Bilingual sentence matching using Kernel CCA

  2010cited by this paper
• Sparse canonical correlation analysis, with applications to genomic data

  2009cited by this paper
• Matchbox: large scale online bayesian recommendations

  2009cited by this paper
• Matrix Factorization Techniques for Recommender Systems

  2009influential reference
• Relational learning via collective matrix factorization

  2008cited by this paper
• Relation Prediction in Multi-Relational Domains using Matrix Factorization

  2008cited by this paper
• Probabilistic Matrix Factorization

  2007cited by this paper
• Microarray analysis reveals a major direct role of DNA copy number alteration in the transcriptional program of human breast tumors

  2002cited by this paper
• Algorithms for Non-negative Matrix Factorization

  2000influential reference
• Application of Dimensionality Reduction in Recommender System - A Case Study

  2000cited by this paper
• Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values†

  1994cited by this paper
• Tkk Reports in Information and Computer Science Practical Approaches to Principal Component Analysis in the Presence of Missing Values Tkk Reports in Information and Computer Science Practical Approaches to Principal Component Analysis in the Presence of Missing Values

  year unknowncited by this paper

• Towards Realistic Error Models for Tabular Data

  2025cites this paper
• Sparse and Orthogonal Low-rank Collective Matrix Factorization (solrCMF): Efficient data integration in flexible layouts

  2024influential citation
• Large-scale Data Integration using Matrix Denoising and Geometric Factor Matching

  2024cites this paper
• Neural Collective Matrix Factorization for Integrated Analysis of Heterogeneous Biomedical Data

  2022cites this paper
• A tensor-network-based big data fusion framework for Cyber-Physical-Social Systems (CPSS)

  2021cites this paper
• A tensor decomposition-based integrated analysis applicable to multiple gene expression profiles without sample matching

  2021influential citation
• Multi-way Clustering and Discordance Analysis through Deep Collective Matrix Tri-Factorization

  2021cites this paper
• THESIS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY Network modeling and integrative analysis of high-dimensional genomic data

  2020influential citation
• Structured Bayesian Latent Factor Models with Meta-data

  2019cites this paper
• MM-PCA: Integrative Analysis of Multi-group and Multi-view Data.

  2019cites this paper
• Mining urban lifestyles: urban computing, human behavior and recommender systems

  2019cites this paper
• Deep Augmented Multiview Clustering

  2019cites this paper
• Predicting Synthetic Lethal Interactions using Heterogeneous Data Sources

  2019cites this paper
• Deep collective matrix factorization for augmented multi-view learning

  2018influential citation
• Bayesian Multi-label Learning with Sparse Features and Labels, and Label Co-occurrences

  2018cites this paper
• Learning Social Circles in Ego-Networks Based on Multi-View Network Structure

  2017cites this paper
• Cross-Domain Recommendation for Large-Scale Data

  2017cites this paper
• On the Unreported-Profile-is-Negative Assumption for Predictive Cheminformatics

  2017cites this paper
• Bayesian Hybrid Matrix Factorisation for Data Integration

  2017cites this paper
• Bayesian matrix factorisation : inference, priors, and data integration

  2017cites this paper
• Information Projection and Approximate Inference for Structured Sparse Variables

  2016cites this paper
• GFA: Exploratory Analysis of Multiple Data Sources with Group Factor Analysis

  2016cites this paper
• World Knowledge as Indirect Supervision for Document Clustering

  2016cites this paper
• Predicting Complications in Critical Care Using Heterogeneous Clinical Data

  2016influential citation
• Towards a Theoretical Understanding of Negative Transfer in Collective Matrix Factorization

  2016cites this paper
• Learning with Positive and Unknown Features

  2016cites this paper
• Bayesian group factor analysis with structured sparsity

  2016influential citation
• A distributed and privatized framework for drug-target interaction prediction

  2016cites this paper
• Co-Regularized Collective Matrix Factorization for Joint Matrix Completion

  2016influential citation
• Probabilistic non-negative matrix factorisation and extensions

  2016cites this paper
• A PAC bound for joint matrix completion based on Partially Collective Matrix Factorization

  2016cites this paper
• Clinical Decision Support for Stroke Using Multi-view Learning Based Models for NIHSS Scores

  2016cites this paper
• Partial Collective Matrix Factorization and its PAC Bound

  2016influential citation
• Canonical Correlation Analysis in Cross-Domain Recommendation

  2016cites this paper
• New Advancements of Scalable Statistical Methods for Learning Latent Structures in Big Data

  2016influential citation
• A Survey of Multi-View Representation Learning

  2016cites this paper
• Personalized Messaging Engine: The Next Step in Employee Engagement

  2015cites this paper
• Integrating Features and Similarities: Flexible Models for Heterogeneous Multiview Data

  2015cites this paper
• Fusing Heterogeneous Data for Alzheimer's Disease Classification

  2015cites this paper
• Multi-tensor Completion with Common Structures

  2015cites this paper
• Multi-tensor factorization

  2015cites this paper
• Group Factor Analysis

  2014cites this paper
• Bayesian latent variable models for learning dependencies between multiple data sources

  2014cites this paper
• Bayesian multi-tensor factorization

  2014cites this paper
• Multi-tensor completion for estimating missing values in video data

  2014cites this paper
• Polya-gamma augmentations for factor models

  2014cites this paper