On a Model of Associative Memory with Huge Storage Capacity

In Krotov et al. (in: Lee (eds) Advances in Neural Information Processing Systems, Curran Associates, Inc., Red Hook, 2016) Krotov and Hopfield suggest a generalized version of the well-known Hopfield model of associative memory. In their version they consider a polynomial interaction function and claim that this increases the storage capacity of the model. We prove this claim and take the ”limit” as the degree of the polynomial becomes infinite, i.e. an exponential interaction function. With this interaction we prove that model has an exponential storage capacity in the number of neurons, yet the basins of attraction are almost as large as in the standard Hopfield model.

• Publication year

  2017

• Venue

  Journal of statistical physics

• Publication date

  2017-02-07

• Fields of study

  Mathematics, Computer Science

• Identifiers
  DOI 10.1007/s10955-017-1806-yarXiv 1702.01929
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Dense Associative Memory for Pattern Recognition

  2016cited by this paper
• Accepted manuscript THE CAPACITY OF q-STATE POTTS NEURAL NETWORKS WITH PARALLEL RETRIEVAL DYNAMICS

  2007cited by this paper
• The storage capacity of the Hopfield model and moderate deviations

  2005cited by this paper
• THE SPIN-GLASS PHASE-TRANSITION IN THE HOPFIELD MODEL WITH p-SPIN INTERACTIONS

  2001cited by this paper
• Sharp upper bounds on perfect retrieval in the Hopfield model

  1999cited by this paper
• On the storage capacity of Hopfield models with correlated patterns

  1998cited by this paper
• Large Deviations Techniques and Applications

  1998cited by this paper
• Rigorous results for the Hopfield model with many patterns

  1998cited by this paper
• Lower bounds on the restitution error in the Hopfield Model

  1997cited by this paper
• Memory capacity in neural network models: Rigorous lower bounds

  1988cited by this paper
• The capacity of the Hopfield associative memory

  1987cited by this paper
• Spin-glass models of neural networks.

  1985cited by this paper
• Storing infinite numbers of patterns in a spin-glass model of neural networks.

  1985cited by this paper
• Neural networks and physical systems with emergent collective computational abilities.

  1982cited by this paper
• The Storage Capacity of Generalized Hoppeld Models with Semantically Correlated Patterns

  year unknowncited by this paper

• High-Temperature Increasing Propagation of Chaos and its breakdown for the Hopfield Model

  2026cites this paper
• Graph Hopfield Networks: Energy-Based Node Classification with Associative Memory

  2026cites this paper
• Stochastic Thermodynamics of Associative Memory

  2026cites this paper
• A Dynamical Theory of Sequential Retrieval in Input-Driven Hopfield Networks

  2026cites this paper
• A Convergent Method for Energy Optimization in Modern Hopfield Networks

  2026cites this paper
• Memory-guided Prototypical Co-occurrence Learning for Mixed Emotion Recognition

  2026cites this paper
• Deep Clustering with Associative Memories

  2026cites this paper
• Fast variable selection under ℓ0\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\ell _0$$\end{document} regularization

  2026cites this paper
• Origin of non-zero overlap in Hopfield neural networks beyond storage capacity

  2026cites this paper
• Bridging Diffusion Guidance and Anderson Acceleration via Hopfield Dynamics

  2026cites this paper
• Optical Hopfield neural networks with enhanced storage capacity.

  2026cites this paper
• A hardware-adaptive learning algorithm for superlinear-capacity associative memory on memristor crossbars.

  2026cites this paper
• A Biologically Plausible Dense Associative Memory with Exponential Capacity

  2026cites this paper
• Temporal Complexity and Self-Organization in an Exponential Dense Associative Memory Model

  2026influential citation
• Neuromodulation–inspired gated associative memory networks: extended memory retrieval and emergent multistability

  2025cites this paper
• Adaptive Hopfield Network: Rethinking Similarities in Associative Memory

  2025cites this paper
• Classifying States of the Hopfield Network with Improved Accuracy, Generalization, and Interpretability

  2025cites this paper
• Beyond Disorder: Unveiling Cooperativeness in Multidirectional Associative Memories

  2025cites this paper
• Dense Associative Memory with Epanechnikov Energy

  2025cites this paper
• Where and How to Perturb: On the Design of Perturbation Guidance in Diffusion and Flow Models

  2025cites this paper
• Dynamic Manifold Hopfield Networks for Context-Dependent Associative Memory

  2025cites this paper
• Dynamical Properties of Dense Associative Memory

  2025cites this paper
• Amplitude equations of associative memory patterns in spatially distributed systems

  2025cites this paper
• Auto-Associative Memories for Direct Signalling of Visual Angle During Object Approaches

  2025cites this paper
• A Hetero-Associative Sequential Memory Model Utilizing Neuromorphic Signals: Validated on a Mobile Manipulator

  2025cites this paper
• Self-Organization and Spectral Mechanism of Attractor Landscapes in High-Capacity Kernel Hopfield Networks

  2025cites this paper
• Higher-Order Kuramoto Oscillator Network for Dense Associative Memory

  2025influential citation
• Effects of Feature Correlations on Associative Memory Capacity

  2025cites this paper
• Topological constraints on self-organisation in locally interacting systems

  2025cites this paper
• Yet another exponential Hopfield model

  2025cites this paper
• Dense associative memory for Gaussian distributions

  2025cites this paper
• Global universal scaling and ultrasmall parameterization in machine-learning interatomic potentials with superlinearity

  2025cites this paper
• Content-Addressable Memory with a Content-Free Energy Function.

  2025cites this paper
• The Capacity of Modern Hopfield Networks under the Data Manifold Hypothesis

  2025cites this paper
• A Framework for Non-Linear Attention via Modern Hopfield Networks

  2025cites this paper
• In-context denoising with one-layer transformers: connections between attention and associative memory retrieval

  2025cites this paper
• What does memory retrieval leave on the table? Modelling the Cost of Semi-Compositionality with MINERVA2 and sBERT

  2025cites this paper
• Cancer Survival Prognosis From Whole Slide Images Using Hopfield Network

  2025cites this paper
• Attractor Regimes of Boolean Recurrent Neural Networks subject to STDP and Global Plasticity

  2025cites this paper
• A Hybrid Multi-Well Hopfield-CNN with Feature Extraction and K-Means for MNIST Classification

  2025cites this paper
• Long-Sequence Memory with Temporal Kernels and Dense Hopfield Functionals

  2025influential citation
• Memorization and generalization in generative diffusion under the manifold hypothesis

  2025cites this paper
• Memory-specific E-I balance supports diverse replay and mitigates catastrophic forgetting

  2025cites this paper
• Modern Hopfield-Heuristic Physical Computing Architecture for Diabetes Prediction

  2025cites this paper
• All-to-All Connected Oscillator Ising Machines and Their Application as Associative Memory

  2025cites this paper
• Dense associative memory in a nonlinear optical Hopfield neural network

  2025cites this paper
• DAM-Seg: Anatomically accurate cardiac segmentation using Dense Associative Networks

  2025cites this paper
• Modern Hopfield Networks with Continuous-Time Memories

  2025cites this paper
• PLADIS: Pushing the Limits of Attention in Diffusion Models at Inference Time by Leveraging Sparsity

  2025cites this paper
• Hopfield Networks as Models of Emergent Function in Biology

  2025influential citation
• Implicit Bias and Invariance: How Hopfield Networks Efficiently Learn Graph Orbits

  2025cites this paper
• Statistical mechanics of vector Hopfield network near and above saturation

  2025cites this paper
• A 63.3 μW Modern Hopfield Network Hardware Accelerator Enabling Dense Associative Memory

  2025cites this paper
• On the Role of Hidden States of Modern Hopfield Network in Transformer

  2025cites this paper
• Hardware-Adaptive and Superlinear-Capacity Memristor-based Associative Memory

  2025cites this paper
• Self-attention as an attractor network: transient memories without backpropagation

  2024cites this paper
• Modern Hopfield Networks meet Encoded Neural Representations - Addressing Practical Considerations

  2024cites this paper
• Online Training of Hopfield Networks using Predictive Coding

  2024cites this paper
• Machine learning in biological physics: From biomolecular prediction to design

  2024cites this paper
• Sequential Learning in the Dense Associative Memory

  2024cites this paper
• CRS-Diff: Controllable Remote Sensing Image Generation With Diffusion Model

  2024cites this paper
• Flipped Classroom: Aligning Teacher Attention with Student in Generalized Category Discovery

  2024cites this paper
• Theoretical framework for quantum associative memories

  2024cites this paper
• An emergent attractor network in a passive resistive switching circuit

  2024cites this paper
• Semantically-correlated memories in a dense associative model

  2024influential citation
• Provably Optimal Memory Capacity for Modern Hopfield Models: Transformer-Compatible Dense Associative Memories as Spherical Codes

  2024cites this paper
• Hierarchical Associative Memory, Parallelized MLP-Mixer, and Symmetry Breaking

  2024cites this paper
• Prototype Analysis in Hopfield Networks With Hebbian Learning

  2024cites this paper
• Associative Memories in the Feature Space

  2024cites this paper
• Bridging Associative Memory and Probabilistic Modeling

  2024cites this paper
• Improved Robustness and Hyperparameter Selection in the Dense Associative Memory

  2024cites this paper
• On the capacity of continuous-time Hopfield models

  2024cites this paper
• Dense Hopfield Networks in the Teacher-Student Setting

  2024cites this paper
• Hopfield networks for asset allocation

  2024cites this paper
• Smoothed Energy Guidance: Guiding Diffusion Models with Reduced Energy Curvature of Attention

  2024cites this paper
• Associative memory inspires improvements for in-context learning using a novel attention residual stream architecture

  2024cites this paper
• Outlier-Efficient Hopfield Layers for Large Transformer-Based Models

  2024cites this paper
• Computational models of learning and synaptic plasticity

  2024cites this paper
• Modern Hopfield Networks Require Chain-of-Thought to Solve $\mathsf{NC}^1$-Hard Problems

  2024cites this paper
• Conv-CoA: Improving Open-domain Question Answering in Large Language Models via Conversational Chain-of-Action

  2024cites this paper
• Waddington landscape for prototype learning in generalized Hopfield networks

  2024cites this paper
• Explosive neural networks via higher-order interactions in curved statistical manifolds

  2024cites this paper
• Uniform Memory Retrieval with Larger Capacity for Modern Hopfield Models

  2024cites this paper
• Hierarchical cell identities emerge from animal gene regulatory mechanisms

  2024cites this paper
• Random Features Hopfield Networks generalize retrieval to previously unseen examples

  2024cites this paper
• Input-driven dynamics for robust memory retrieval in Hopfield networks

  2024cites this paper
• BiSHop: Bi-Directional Cellular Learning for Tabular Data with Generalized Sparse Modern Hopfield Model

  2024cites this paper
• High-order rotor Hopfield neural networks for associative memory

  2024cites this paper
• Exact Replica Symmetric Solution for Transverse Field Hopfield Model under Finite Trotter Size

  2024cites this paper
• Daydreaming Hopfield Networks and their surprising effectiveness on correlated data

  2024cites this paper
• Improved Robustness and Hyperparameter Selection in Modern Hopfield Networks

  2024cites this paper
• Analysis of discrete modern Hopfield networks in open quantum system

  2024cites this paper
• On the Feasibility of Single-Pass Full-Capacity Learning in Linear Threshold Neurons with Binary Input Vectors

  2024cites this paper
• Hebbian learning from first principles

  2024cites this paper
• Improving Out-of-Distribution Data Handling and Corruption Resistance via Modern Hopfield Networks

  2024cites this paper
• Beyond Scaling Laws: Understanding Transformer Performance with Associative Memory

  2024cites this paper
• Attractor Memory for Long-Term Time Series Forecasting: A Chaos Perspective

  2024cites this paper
• Sparse and Structured Hopfield Networks

  2024cites this paper
• Toward a Model of Associative Memory via Predictive Coding

  2024cites this paper
• Binary Associative Memory Networks: A Review of Mathematical Framework and Capacity Analysis

  2024cites this paper