Non-Negative Stiefel Approximating Flow: Orthogonalish Matrix Optimization for Interpretable Embeddings

Interpretable representation learning is a central challenge in modern machine learning, particularly in high-dimensional settings such as neuroimaging, genomics, and text analysis. Current methods often struggle to balance the competing demands of interpretability and model flexibility, limiting their effectiveness in extracting meaningful insights from complex data. We introduce Non-negative Stiefel Approximating Flow (NSA-Flow), a general-purpose matrix estimation framework that unifies ideas from sparse matrix factorization, orthogonalization, and constrained manifold learning. NSA-Flow enforces structured sparsity through a continuous balance between reconstruction fidelity and column-wise decorrelation, parameterized by a single tunable weight. The method operates as a smooth flow near the Stiefel manifold with proximal updates for non-negativity and adaptive gradient control, yielding representations that are simultaneously sparse, stable, and interpretable. Unlike classical regularization schemes, NSA-Flow provides an intuitive geometric mechanism for manipulating sparsity at the level of global structure while simplifying latent features. We demonstrate that the NSA-Flow objective can be optimized smoothly and integrates seamlessly with existing pipelines for dimensionality reduction while improving interpretability and generalization in both simulated and real biomedical data. Empirical validation on the Golub leukemia dataset and in Alzheimer's disease demonstrate that the NSA-Flow constraints can maintain or improve performance over related methods with little additional methodological effort. NSA-Flow offers a scalable, general-purpose tool for interpretable ML, applicable across data science domains.

• Publication year

  2025

• Venue

  arXiv.org

• Publication date

  2025-11-09

• Fields of study

  Mathematics, Computer Science

• Identifiers
  DOI 10.48550/arXiv.2511.06425arXiv 2511.06425
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Machine-learning models for Alzheimer’s disease diagnosis using neuroimaging data: survey, reproducibility, and generalizability evaluation

  2025cited by this paper
• λ-Orthogonality Regularization for Compatible Representation Learning

  2025cited by this paper
• Orthogonal graph regularized non-negative matrix factorization under sparse constraints for clustering

  2024influential reference
• An Introduction to Optimization on Smooth Manifolds

  2023influential reference
• Towards Better Orthogonality Regularization with Disentangled Norm in Training Deep CNNs

  2023cited by this paper
• Group Orthogonalization Regularization For Vision Models Adaptation and Robustness

  2023cited by this paper
• Variational Bayesian Orthogonal Nonnegative Matrix Factorization Over the Stiefel Manifold

  2022cited by this paper
• Feedback Gradient Descent: Efficient and Stable Optimization with Orthogonality for DNNs

  2022cited by this paper
• Orthogonal Nonnegative Matrix Factorization using a novel deep Autoencoder Network

  2021cited by this paper
• Assessing the Changes of Cortical Thickness in Alzheimer Disease With MRI Using Freesurfer Software

  2021cited by this paper
• The Restricted Isometry of ReLU Networks: Generalization through Norm Concentration

  2020cited by this paper
• Large Batch Optimization for Deep Learning: Training BERT in 76 minutes

  2019cited by this paper
• A Frobenius norm regularization method for convolutional kernels to avoid unstable gradient problem

  2019cited by this paper
• The personalized Alzheimer's disease cortical thickness index predicts likely pathology and clinical progression in mild cognitive impairment

  2018cited by this paper
• Efficient Orthogonal Non-negative Matrix Factorization over Stiefel Manifold

  2016cited by this paper
• Proximal Algorithms

  2013influential reference
• Manopt, a matlab toolbox for optimization on manifolds

  2013cited by this paper
• Proximal alternating linearized minimization for nonconvex and nonsmooth problems

  2013cited by this paper
• RTRMC: A Riemannian trust-region method for low-rank matrix completion

  2011influential reference
• Convex Analysis and Monotone Operator Theory in Hilbert Spaces

  2011cited by this paper
• Automatic classification of patients with Alzheimer's disease from structural MRI: A comparison of ten methods using the ADNI database

  2011cited by this paper
• Unsupervised Learning of Sparse Features for Scalable Audio Classification

  2011cited by this paper
• Matrix Factorization Techniques for Recommender Systems

  2009cited by this paper
• Proximal Splitting Methods in Signal Processing

  2009cited by this paper
• A penalized matrix decomposition, with applications to sparse principal components and canonical correlation analysis.

  2009cited by this paper
• Optimization algorithms on matrix manifolds

  2009influential reference
• Latent Dirichlet Allocation

  2009cited by this paper
• Orthogonal nonnegative matrix t-factorizations for clustering

  2006influential reference
• Sparse Principal Component Analysis

  2006cited by this paper
• Independent component analysis: algorithms and applications

  2000cited by this paper
• Algorithms for Non-negative Matrix Factorization

  2000cited by this paper
• Molecular classification of cancer: class discovery and class prediction by gene expression monitoring.

  1999influential reference
• Acceleration of stochastic approximation by averaging

  1992cited by this paper
• A generalized solution of the orthogonal procrustes problem

  1966cited by this paper

• No citing papers are available for this paper.