Learning Mixtures of Separable Dictionaries for Tensor Data: Analysis and Algorithms

This work addresses the problem of learning sparse representations of tensor data using structured dictionary learning. It proposes learning a mixture of separable dictionaries to better capture the structure of tensor data by generalizing the separable dictionary learning model. Two different approaches for learning mixture of separable dictionaries are explored and sufficient conditions for local identifiability of the underlying dictionary are derived in each case. Moreover, computational algorithms are developed to solve the problem of learning mixture of separable dictionaries in both batch and online settings. Numerical experiments are used to show the usefulness of the proposed model and the efficacy of the developed algorithms.

• Publication year

  2019

• Venue

  IEEE Transactions on Signal Processing

• Publication date

  2019-03-22

• Fields of study

  Mathematics, Computer Science, Engineering

• Identifiers
  DOI 10.1109/TSP.2019.2952046arXiv 1903.09284
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Sample Complexity Bounds for Dictionary Learning from Vector- and Tensor-Valued Data

  2021cited by this paper
• Nonlinear Programming

  2021cited by this paper
• Sample Complexity Bounds for Low-Separation-Rank Dictionary Learning

  2019cited by this paper
• Global Optimality in Separable Dictionary Learning with Applications to the Analysis of Diffusion MRI

  2019cited by this paper
• Learning Fast Dictionaries for Sparse Representations Using Low-Rank Tensor Decompositions

  2018cited by this paper
• Separable Dictionary Learning with Global Optimality and Applications to Diffusion MRI

  2018cited by this paper
• Algorithm

  2018cited by this paper
• Learning Dictionaries as a Sum of Kronecker Products

  2017cited by this paper
• STARK: Structured dictionary learning through rank-one tensor recovery

  2017cited by this paper
• Identifiability of Kronecker-Structured Dictionaries for Tensor Data

  2017influential reference
• Tensor Decomposition for Signal Processing and Machine Learning

  2016cited by this paper
• Learning brain regions via large-scale online structured sparse dictionary learning

  2016cited by this paper
• Minimax Lower Bounds on Dictionary Learning for Tensor Data

  2016cited by this paper
• Denoising and Completion of 3D Data via Multidimensional Dictionary Learning

  2015cited by this paper
• Tensorizing Neural Networks

  2015cited by this paper
• Learning Sparsely Used Overcomplete Dictionaries

  2014cited by this paper
• Tensor-based algorithms for learning multidimensional separable dictionaries

  2014cited by this paper
• Multitask learning meets tensor factorization: task imputation via convex optimization

  2014cited by this paper
• Sparse and Spurious: Dictionary Learning With Noise and Outliers

  2014influential reference
• Multidimensional compressed sensing and their applications

  2013influential reference
• Separable Dictionary Learning

  2013influential reference
• New Algorithms for Learning Incoherent and Overcomplete Dictionaries

  2013cited by this paper
• When are overcomplete topic models identifiable? uniqueness of tensor tucker decompositions with structured sparsity

  2013cited by this paper
• Sample Complexity of Dictionary Learning and Other Matrix Factorizations

  2013cited by this paper
• Multilinear Multitask Learning

  2013cited by this paper
• Covariance Estimation in High Dimensions Via Kronecker Product Expansions

  2013cited by this paper
• On the Identifiability of Overcomplete Dictionaries via the Minimisation Principle Underlying K-SVD

  2013cited by this paper
• Tensor dictionary learning with sparse TUCKER decomposition

  2013cited by this paper
• The Lasso Problem and Uniqueness

  2012cited by this paper
• A survey of multilinear subspace learning for tensor data

  2011cited by this paper
• Dictionary Learning

  2011cited by this paper
• Tight Oracle Inequalities for Low-Rank Matrix Recovery From a Minimal Number of Noisy Random Measurements

  2011cited by this paper
• Tensor completion and low-n-rank tensor recovery via convex optimization

  2011cited by this paper
• Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers

  2011cited by this paper
• Tensor-Train Decomposition

  2011cited by this paper
• Recursive Least Squares Dictionary Learning Algorithm

  2010cited by this paper
• Batch and Adaptive PARAFAC-Based Blind Separation of Convolutive Speech Mixtures

  2010cited by this paper
• Image Quality Metrics: PSNR vs. SSIM

  2010cited by this paper
• A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems

  2009cited by this paper
• Online Learning for Matrix Factorization and Sparse Coding

  2009influential reference
• Tensor Decompositions and Applications

  2009influential reference
• A Singular Value Thresholding Algorithm for Matrix Completion

  2008cited by this paper
• Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit

  2007cited by this paper
• Tensor Rank and the Ill-Posedness of the Best Low-Rank Approximation Problem

  2006influential reference
• $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation

  2006cited by this paper
• K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation

  2005influential reference
• Dictionary Learning Algorithms for Sparse Representation

  2003cited by this paper
• Numerical operator calculus in higher dimensions

  2002cited by this paper
• The ubiquitous Kronecker product

  2000cited by this paper
• Classes of nonseparable, spatio-temporal stationary covariance functions

  1999cited by this paper
• Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition

  1993cited by this paper
• Parallel and distributed computation

  1989cited by this paper
• Tensor Rank is NP-Complete

  1989cited by this paper
• Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-model factor analysis

  1970cited by this paper
• Principles of mathematical analysis

  1964cited by this paper

• Low Separation Rank in Tensor Generalized Linear Models: An Asymptotic Analysis

  2024cites this paper
• CircWaveDL: Modeling of optical coherence tomography images based on a new supervised tensor-based dictionary learning for classification of macular abnormalities

  2024cites this paper
• Update on the CIC Multistage Decimation Filter With a Minimum Number of Additions per Output Sample (APOS): Can We Still Decrease the Number of APOS? [Tips & Tricks]

  2023cites this paper
• Structured Low-Rank Tensors for Generalized Linear Models

  2023influential citation
• MRI Image Fusion Based on Sparse Representation with Measurement of Patch-Based Multiple Salient Features

  2023cites this paper
• Low-Rank Phase Retrieval with Structured Tensor Models

  2022cites this paper
• A Guide to Reproducible Research in Signal Processing and Machine Learning

  2021cites this paper
• A Guide to Computational Reproducibility in Signal Processing and Machine Learning [Tips & Tricks]

  2021cites this paper
• Sparse Bayesian learning algorithm for separable dictionaries

  2021cites this paper
• A spectral PALM algorithm for matrix and tensor-train based Dictionary Learning

  2021cites this paper
• Classification-assisted Deep Sparse Image Recognition

  2021cites this paper
• Nonconvex matrix and tensor recovery with applications in machine learning

  2021cites this paper
• Online Nonnegative CP-dictionary Learning for Markovian Data

  2020cites this paper
• Efficient and Parallel Separable Dictionary Learning

  2020influential citation
• Revisiting Sparse Channel Estimation in Massive MIMO-OFDM Systems

  2019cites this paper
• Deep Sparse Representation-Based Classification

  2019cites this paper