Probabilistic Neural Circuits

Probabilistic circuits (PCs) have gained prominence in recent years as a versatile framework for discussing probabilistic models that support tractable queries and are yet expressive enough to model complex probability distributions. Nevertheless, tractability comes at a cost: PCs are less expressive than neural networks. In this paper we introduce probabilistic neural circuits (PNCs), which strike a balance between PCs and neural nets in terms of tractability and expressive power. Theoretically, we show that PNCs can be interpreted as deep mixtures of Bayesian networks. Experimentally, we demonstrate that PNCs constitute powerful function approximators.

• Publication year

  2024

• Venue

  AAAI Conference on Artificial Intelligence

• Publication date

  2024-03-10

• Fields of study

  Mathematics, Computer Science

• Identifiers
  DOI 10.1609/aaai.v38i15.29675arXiv 2403.06235
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Sparse Probabilistic Circuits via Pruning and Growing

  2022influential reference
• Continuous Mixtures of Tractable Probabilistic Models

  2022influential reference
• Training and Inference on Any-Order Autoregressive Models the Right Way

  2022cited by this paper
• A Compositional Atlas of Tractable Circuit Operations for Probabilistic Inference

  2021influential reference
• HyperSPNs: Compact and Expressive Probabilistic Circuits

  2021influential reference
• Random probabilistic circuits

  2021cited by this paper
• Conditional sum-product networks: Modular probabilistic circuits via gate functions

  2021cited by this paper
• Elevating Perceptual Sample Quality in PCs through Differentiable Sampling

  2021cited by this paper
• Tractable Regularization of Probabilistic Circuits

  2021influential reference
• Juice: A Julia Package for Logic and Probabilistic Circuits

  2021cited by this paper
• Einsum Networks: Fast and Scalable Learning of Tractable Probabilistic Circuits

  2020influential reference
• Integer Discrete Flows and Lossless Compression

  2019cited by this paper
• Random Sum-Product Networks: A Simple and Effective Approach to Probabilistic Deep Learning

  2019influential reference
• Learning Logistic Circuits

  2019cited by this paper
• On the Expressive Power of Deep Polynomial Neural Networks

  2019cited by this paper
• Fixing Weight Decay Regularization in Adam

  2017cited by this paper
• Sum-Product-Quotient Networks

  2017influential reference
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms

  2017cited by this paper
• A Unified Approach for Learning the Parameters of Sum-Product Networks

  2016cited by this paper
• Conditional Image Generation with PixelCNN Decoders

  2016cited by this paper
• A note on the evaluation of generative models

  2015cited by this paper
• On Theoretical Properties of Sum-Product Networks

  2015cited by this paper
• On the Expressive Efficiency of Sum Product Networks

  2014cited by this paper
• Adam: A Method for Stochastic Optimization

  2014influential reference
• Probabilistic Sentential Decision Diagrams

  2014cited by this paper
• A Deep and Tractable Density Estimator

  2013cited by this paper
• The MNIST Database of Handwritten Digit Images for Machine Learning Research [Best of the Web]

  2012influential reference
• Sum-product networks: A new deep architecture

  2011cited by this paper
• Shallow vs. Deep Sum-Product Networks

  2011cited by this paper
• Decomposable negation normal form

  2001influential reference
• A differential approach to inference in Bayesian networks

  2000influential reference
• A stochastic version of the delta rule

  1990cited by this paper
• Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence SDD: A New Canonical Representation of Propositional Knowledge Bases

  year unknowncited by this paper

• A Quantum Information Theoretic Approach to Tractable Probabilistic Models

  2025cites this paper
• QuantTPM: Efficient Mixed-Precision Quantization Framework for Tractable Probabilistic Models

  2025cites this paper
• Independence Is Not an Issue in Neurosymbolic AI

  2025cites this paper
• Tractable Representation Learning with Probabilistic Circuits

  2025cites this paper
• Learning Tractable Distributions Of Language Model Continuations

  2025cites this paper
• Building Expressive and Tractable Probabilistic Generative Models: A Review

  2024cites this paper
• Approach to Tractable Probabilistic

  year unknowncites this paper