Automatic Symmetry Discovery with Lie Algebra Convolutional Network

Existing equivariant neural networks require prior knowledge of the symmetry group and discretization for continuous groups. We propose to work with Lie algebras (infinitesimal generators) instead of Lie groups. Our model, the Lie algebra convolutional network (L-conv) can automatically discover symmetries and does not require discretization of the group. We show that L-conv can serve as a building block to construct any group equivariant feedforward architecture. Both CNNs and Graph Convolutional Networks can be expressed as L-conv with appropriate groups. We discover direct connections between L-conv and physics: (1) group invariant loss generalizes field theory (2) Euler-Lagrange equation measures the robustness, and (3) equivariance leads to conservation laws and Noether current.These connections open up new avenues for designing more general equivariant networks and applying them to important problems in physical sciences

• Publication year

  2021

• Venue

  Neural Information Processing Systems

• Publication date

  2021-09-15

• Fields of study

  Mathematics, Physics, Computer Science

• Identifiers
  arXiv 2109.07103
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• A Practical Method for Constructing Equivariant Multilayer Perceptrons for Arbitrary Matrix Groups

  2021influential reference
• Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges

  2021cited by this paper
• Learning Invariances in Neural Networks

  2020influential reference
• Universal Equivariant Multilayer Perceptrons

  2020cited by this paper
• Neural Mechanics: Symmetry and Broken Conservation Laws in Deep Learning Dynamics

  2020cited by this paper
• Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Data

  2020influential reference
• On Learning Sets of Symmetric Elements

  2020cited by this paper
• Incorporating Symmetry into Deep Dynamics Models for Improved Generalization

  2020cited by this paper
• Meta-Learning Symmetries by Reparameterization

  2020cited by this paper
• Disentangling by Subspace Diffusion

  2020cited by this paper
• Lorentz Group Equivariant Neural Network for Particle Physics

  2020influential reference
• Symmetry-adapted representation learning

  2019cited by this paper
• Gauge Equivariant Convolutional Networks and the Icosahedral CNN

  2019influential reference
• General E(2)-Equivariant Steerable CNNs

  2019cited by this paper
• On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups

  2018influential reference
• Spherical CNNs

  2018cited by this paper
• Deep Models of Interactions Across Sets

  2018cited by this paper
• Clebsch-Gordan Nets: a Fully Fourier Space Spherical Convolutional Neural Network

  2018cited by this paper
• Random Walks on Simplicial Complexes and the normalized Hodge Laplacian

  2018influential reference
• A General Theory of Equivariant CNNs on Homogeneous Spaces

  2018influential reference
• Neural Message Passing for Quantum Chemistry

  2017cited by this paper
• Learning Steerable Filters for Rotation Equivariant CNNs

  2017cited by this paper
• Deep Sets

  2017cited by this paper
• Equivariance Through Parameter-Sharing

  2017cited by this paper
• Group Equivariant Convolutional Networks

  2016cited by this paper
• Harmonic Networks: Deep Translation and Rotation Equivariance

  2016cited by this paper
• Semi-Supervised Classification with Graph Convolutional Networks

  2016influential reference
• Learning the Irreducible Representations of Commutative Lie Groups

  2014cited by this paper
• Deep Symmetry Networks

  2014cited by this paper
• An Unsupervised Algorithm For Learning Lie Group Transformations

  2010cited by this paper
• Manifolds and Differential Geometry

  2009cited by this paper
• Lie Groups, Lie Algebras, and Representations

  2003cited by this paper
• Gradient-based learning applied to document recognition

  1998influential reference
• Learning Lie Groups for Invariant Visual Perception

  1998influential reference
• Infinite Conformal Symmetry in Two-Dimensional Quantum Field Theory

  1996cited by this paper
• Backpropagation Applied to Handwritten Zip Code Recognition

  1989influential reference
• Approximation by superpositions of a sigmoidal function

  1989cited by this paper
• Multilayer feedforward networks are universal approximators

  1989cited by this paper
• Classical theory of fields

  1952cited by this paper
• Quantum Theory of Fields

  1949cited by this paper
• UvA-DARE (Digital Academic Repository) 3D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Data 3D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Data

  year unknowncited by this paper

• Invariant-Net: Enhancing generalization in physics-informed neural networks via lie symmetry reduction

  2026cites this paper
• Lie Flow: Video Dynamic Fields Modeling and Predicting with Lie Algebra as Geometric Physics Principle

  2026cites this paper
• Governing Equation Discovery from Data Based on Differential Invariants

  2025cites this paper
• Learning dissection trajectories from expert surgical videos via imitation learning with equivariant diffusion

  2025cites this paper
• AtlasD: Automatic Local Symmetry Discovery

  2025cites this paper
• Learning finite symmetry groups of dynamical systems via equivariance detection

  2025cites this paper
• LieCConv: An Image Classification Algorithm Based on Lie Group Convolutional Neural Network

  2025cites this paper
• Diffusion Generative Modeling on Lie Group Representations

  2025cites this paper
• MatrixNet: Learning over symmetry groups using learned group representations

  2025cites this paper
• The Generalized Skew Spectrum of Graphs

  2025cites this paper
• Symmetries-enhanced Multi-Agent Reinforcement Learning

  2025cites this paper
• EquiReg: Equivariance Regularized Diffusion for Inverse Problems

  2025cites this paper
• Discovering Symmetry Groups with Flow Matching

  2025cites this paper
• Achieving Approximate Symmetry Is Exponentially Easier than Exact Symmetry

  2025cites this paper
• Symmetry-Driven Discovery of Dynamical Variables in Molecular Simulations

  2025cites this paper
• A Robust Kernel Statistical Test of Invariance: Detecting Subtle Asymmetries

  2025cites this paper
• Equivariance by Contrast: Identifiable Equivariant Embeddings from Unlabeled Finite Group Actions

  2025cites this paper
• Explicit Discovery of Nonlinear Symmetries from Dynamic Data

  2025influential citation
• Type-II neural symmetry detection with Lie theory

  2025influential citation
• Automatic Discovery of One Parameter Subgroups of SO(n)

  2025cites this paper
• Learning Equivariant Functions via Quadratic Forms

  2025cites this paper
• A Computer Graphic (CG)-Based Novel Framework for High-Precision Analysis of Physical Experimental Data

  2025cites this paper
• Digital optics and optical intelligent agent

  2025cites this paper
• Machine learning for detection of equivariant finite symmetry groups in dynamical systems

  2025cites this paper
• Symmetry Discovery for Different Data Types

  2024cites this paper
• Learning Nonholonomic Dynamics with Constraint Discovery

  2024cites this paper
• On the VC dimension of deep group convolutional neural networks

  2024cites this paper
• Break the Bias: Delving Semantic Transform Invariance for Few-Shot Segmentation

  2024cites this paper
• Learning Infinitesimal Generators of Continuous Symmetries from Data

  2024influential citation
• SymmetryLens: A new candidate paradigm for unsupervised symmetry learning via locality and equivariance

  2024cites this paper
• Learning Symmetries via Weight-Sharing with Doubly Stochastic Tensors

  2024cites this paper
• Lie group convolution neural networks with scale-rotation equivariance

  2024cites this paper
• CFASL: Composite Factor-Aligned Symmetry Learning for Disentanglement in Variational AutoEncoder

  2024cites this paper
• Data-driven Lie point symmetry detection for continuous dynamical systems

  2024cites this paper
• A Generative Model of Symmetry Transformations

  2024cites this paper
• ILPO-NET: Network for the invariant recognition of arbitrary volumetric patterns in 3D

  2024cites this paper
• On the Fourier analysis in the SO(3) space : EquiLoPO Network

  2024cites this paper
• Symmetry-Informed Governing Equation Discovery

  2024cites this paper
• Neural Isometries: Taming Transformations for Equivariant ML

  2024cites this paper
• Symmetry Discovery Beyond Affine Transformations

  2024cites this paper
• Neural Manifold Operators for Learning the Evolution of Physical Dynamics

  2024cites this paper
• SymmetryLens: Unsupervised Symmetry Learning via Locality and Density Preservation

  2024cites this paper
• Latent Space Symmetry Discovery

  2023cites this paper
• Hamiltonian Neural Networks with Automatic Symmetry Detection

  2023cites this paper
• Generative Adversarial Symmetry Discovery

  2023influential citation
• A General Theory of Correct, Incorrect, and Extrinsic Equivariance

  2023cites this paper
• SO(2) and O(2) Equivariance in Image Recognition with Bessel-Convolutional Neural Networks

  2023cites this paper
• Learning to Extrapolate: A Transductive Approach

  2023cites this paper
• Lie Group Algebra Convolutional Filters

  2023cites this paper
• Policy Gradient Methods in the Presence of Symmetries and State Abstractions

  2023cites this paper
• Data Symmetries and Learning in Fully Connected Neural Networks

  2023cites this paper
• Learning Linear Groups in Neural Networks

  2023cites this paper
• Regularizing Towards Soft Equivariance Under Mixed Symmetries

  2023cites this paper
• Neural Discovery of Permutation Subgroups

  2023cites this paper
• Scale-Rotation-Equivariant Lie Group Convolution Neural Networks (Lie Group-CNNs)

  2023influential citation
• Learning Lie Group Symmetry Transformations with Neural Networks

  2023cites this paper
• Solution Methods for Nonlinear Ordinary Di(cid:27)erential Equations Using Lie Symmetry Groups

  2023cites this paper
• Non-parametric Hypothesis Tests for Distributional Group Symmetry

  2023cites this paper
• Exploiting Code Symmetries for Learning Program Semantics

  2023cites this paper
• On genuine invariance learning without weight-tying

  2023cites this paper
• Using and Abusing Equivariance

  2023cites this paper
• Symmetric Models for Visual Force Policy Learning

  2023cites this paper
• A Unified Framework for Discovering Discrete Symmetries

  2023influential citation
• Unveiling Invariances via Neural Network Pruning

  2023cites this paper
• Reflection Invariance Learning for Few-shot Semantic Segmentation

  2023cites this paper
• Discovering Symmetry Breaking in Physical Systems with Relaxed Group Convolution

  2023cites this paper
• Learning Layer-wise Equivariances Automatically using Gradients

  2023cites this paper
• Almost Equivariance via Lie Algebra Convolutions

  2023cites this paper
• Symmetric derivatives of parametrized quantum circuits

  2023cites this paper
• Learning hyperelastic anisotropy from data via a tensor basis neural network

  2022cites this paper
• Structuring Representations Using Group Invariants

  2022cites this paper
• Physics-guided Data Augmentation for Learning the Solution Operator of Linear Differential Equations

  2022cites this paper
• Equivariant Networks for Crystal Structures

  2022cites this paper
• Discrete Lagrangian Neural Networks with Automatic Symmetry Discovery

  2022cites this paper
• The Surprising Effectiveness of Equivariant Models in Domains with Latent Symmetry

  2022cites this paper
• Homomorphic Self-Supervised Learning

  2022cites this paper
• Warped Convolutional Networks: Bridge Homography to sl(3) algebra by Group Convolution

  2022influential citation
• Learning Symmetric Rules with SATNet

  2022cites this paper
• Warped Convolution Networks for Homography Estimation

  2022cites this paper
• G AUGE E QUIVARIANT D EEP Q-L EARNING ON D IS - CRETE M ANIFOLDS

  2022cites this paper
• Learning Symmetric Embeddings for Equivariant World Models

  2022cites this paper
• Learning Linear Symmetries in Data Using Moment Matching

  2022cites this paper
• A tradeoff between universality of equivariant models and learnability of symmetries

  2022cites this paper
• Invariance-adapted decomposition and Lasso-type contrastive learning

  2022cites this paper
• Structure-preserving GANs

  2022cites this paper
• Unsupervised Learning of Equivariant Structure from Sequences

  2022cites this paper
• On scientific understanding with artificial intelligence

  2022cites this paper
• LieGG: Studying Learned Lie Group Generators

  2022cites this paper
• FINDE: Neural Differential Equations for Finding and Preserving Invariant Quantities

  2022cites this paper
• Homomorphism Autoencoder - Learning Group Structured Representations from Observed Transitions

  2022cites this paper
• Learning Partial Equivariances From Data

  2021cites this paper
• Variational learning of Euler-Lagrange dynamics from data

  2021cites this paper
• Information-theoretic symmetry classifications of crystal patterns in the presence of noise and strong Fedorov type pseudosymmetries for an optimal subsequent crystallographic processing of these patterns

  2021cites this paper
• Physics-Guided Deep Learning for Dynamical Systems: A Survey

  2021cites this paper
• Boosting deep neural networks with geometrical prior knowledge: a survey

  2020cites this paper
• A Group-Theoretic Approach to Computational Abstraction: Symmetry-Driven Hierarchical Clustering

  2018cites this paper
• Representation Learning in Sensory Cortex: A Theory

  2014cites this paper
• Neural Symmetry Detection for Learning Neural Network Constraints

  year unknowncites this paper
• Quantifying Lie Group Learning with Local Symmetry Error

  year unknowncites this paper
• Finding Symmetry in Neural Network Parameter Spaces

  year unknowncites this paper