f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization

Generative neural samplers are probabilistic models that implement sampling using feedforward neural networks: they take a random input vector and produce a sample from a probability distribution defined by the network weights. These models are expressive and allow efficient computation of samples and derivatives, but cannot be used for computing likelihoods or for marginalization. The generative-adversarial training method allows to train such models through the use of an auxiliary discriminative neural network. We show that the generative-adversarial approach is a special case of an existing more general variational divergence estimation approach. We show that any f-divergence can be used for training generative neural samplers. We discuss the benefits of various choices of divergence functions on training complexity and the quality of the obtained generative models.

• Publication year

  2016

• Venue

  Neural Information Processing Systems

• Publication date

  2016-06-02

• Fields of study

  Mathematics, Computer Science

• Identifiers
  arXiv 1606.00709
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• GENERATIVE ADVERSARIAL NETS

  2018influential reference
• Improved Techniques for Training GANs

  2016cited by this paper
• Fundamentals Of Convex Analysis

  2016cited by this paper
• Deep Unsupervised Learning using Nonequilibrium Thermodynamics

  2015cited by this paper
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks

  2015influential reference
• Generative Moment Matching Networks

  2015cited by this paper
• Adversarial Autoencoders

  2015cited by this paper
• Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)

  2015cited by this paper
• How (not) to Train your Generative Model: Scheduled Sampling, Likelihood, Adversary?

  2015cited by this paper
• LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop

  2015cited by this paper
• Training generative neural networks via Maximum Mean Discrepancy optimization

  2015cited by this paper
• A note on the evaluation of generative models

  2015cited by this paper
• Chainer : a Next-Generation Open Source Framework for Deep Learning

  2015influential reference
• Conditional generative adversarial nets for convolutional face generation

  2015cited by this paper
• Neural Variational Inference and Learning in Belief Networks

  2014cited by this paper
• Adam: A Method for Stochastic Optimization

  2014influential reference
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models

  2014cited by this paper
• Identifying and attacking the saddle point problem in high-dimensional non-convex optimization

  2014cited by this paper
• On distinguishability criteria for estimating generative models

  2014cited by this paper
• Auto-Encoding Variational Bayes

  2013influential reference
• Generating Sequences With Recurrent Neural Networks

  2013cited by this paper
• On the chi square and higher-order chi distances for approximating f-divergences

  2013cited by this paper
• RNADE: The real-valued neural autoregressive density-estimator

  2013cited by this paper
• The Neural Autoregressive Distribution Estimator

  2011cited by this paper
• Noise-contrastive estimation: A new estimation principle for unnormalized statistical models

  2010influential reference
• Information, Divergence and Risk for Binary Experiments

  2009cited by this paper
• Hilbert Space Embeddings and Metrics on Probability Measures

  2009cited by this paper
• Estimating Divergence Functionals and the Likelihood Ratio by Convex Risk Minimization

  2008influential reference
• Strictly Proper Scoring Rules, Prediction, and Estimation

  2007influential reference
• A Kernel Statistical Test of Independence

  2007influential reference
• On Divergences and Informations in Statistics and Information Theory

  2006cited by this paper
• Divergence measures and message passing

  2005cited by this paper
• Information Theory and Statistics: A Tutorial

  2004cited by this paper
• GTM: The Generative Topographic Mapping

  1998cited by this paper
• Bayesian neural networks and density networks

  1995cited by this paper
• Mixture Density Networks

  1994cited by this paper
• A General Class of Coefficients of Divergence of One Distribution from Another

  1966cited by this paper

• Surprisal-R\'enyi Free Energy

  2026cites this paper
• Identification of antimicrobial compounds in Dipsacus inermis via phytochemical profiling, in vitro assessment, and advanced computational techniques

  2026cites this paper
• Generalization bounds for a generator-regularized InfoGAN-inspired adversarial objective.

  2026cites this paper
• RSTG: Robust Generation of High Quality Spatial Transcriptomics Data using Beta Divergence Based AutoEncoder

  2026cites this paper
• Regularized f-Divergence Kernel Tests

  2026influential citation
• BandPO: Bridging Trust Regions and Ratio Clipping via Probability-Aware Bounds for LLM Reinforcement Learning

  2026cites this paper
• Group Orthogonalized Policy Optimization:Group Policy Optimization as Orthogonal Projection in Hilbert Space

  2026cites this paper
• CMCL-DDI: Pharmacophore-aware cross-view contrastive learning for drug-drug interaction prediction.

  2026cites this paper
• Causal discovery for linear causal model with correlated noise: an Adversarial Learning Approach

  2026influential citation
• A Unified Density Operator View of Flow Control and Merging

  2026influential citation
• RelA-Diffusion: Relativistic Adversarial Diffusion for Multi-Tracer PET Synthesis from Multi-Sequence MRI

  2026cites this paper
• Workflow for Volumetric Uncertainty and Sensitivity Analysis of Machine Learning‐Interpreted Seismic Horizons at the Smeaheia CO2 Project Site

  2026cites this paper
• Generative data augmentation for improving state estimation and prognostics in lithium-ion batteries: Advances, Challenges, and Future directions

  2026cites this paper
• Generative adversarial networks: A comprehensive survey

  2026cites this paper
• HP-GAN: Harnessing pretrained networks for GAN improvement with FakeTwins and discriminator consistency.

  2026cites this paper
• Orthogonalized Policy Optimization:Policy Optimization as Orthogonal Projection in Hilbert Space

  2026cites this paper
• Harnessing Synthetic Data from Generative AI for Statistical Inference

  2026cites this paper
• Revolutionizing sentiment analysis with generative AI: techniques, trends, and challenges

  2026cites this paper
• Graph-constructed adaptive semantic fusion for heterophilic graph contrastive learning

  2026cites this paper
• Information Bottleneck-Based Subgraphs Defending Against Inference Attacks in Federated Graph Learning Systems

  2026cites this paper
• BlindU: Blind Machine Unlearning without Revealing Erasing Data

  2026cites this paper
• GAS: Improving Discretization of Diffusion ODEs via Generalized Adversarial Solver

  2025cites this paper
• Rate-Distortion Optimized Communication for Collaborative Perception

  2025cites this paper
• Enhanced BiGAN for multi-task learning in business process prediction: next activity and timestamp prediction tasks

  2025cites this paper
• Graph Representation Learning with Massive Unlabeled Data for Rumor Detection

  2025cites this paper
• Measuring Time-Series Dataset Similarity using Wasserstein Distance

  2025cites this paper
• Dynamic graph representation learning with disentangled information bottleneck

  2025cites this paper
• A Unified Framework for Diffusion Model Unlearning with f-Divergence

  2025cites this paper
• Physics-Based Motion Imitation with Adversarial Differential Discriminators

  2025cites this paper
• Backdoor or Manipulation? Graph Mixture of Experts Can Defend Against Various Graph Adversarial Attacks

  2025cites this paper
• Identifying nonequilibrium degrees of freedom in high-dimensional stochastic systems

  2025cites this paper
• Implicit Reward as the Bridge: A Unified View of SFT and DPO Connections

  2025cites this paper
• Neural Total Variation Distance Estimators for Changepoint Detection in News Data

  2025cites this paper
• Generalized Dual Discriminator GANs

  2025cites this paper
• MCINet: Fusing Low-Light Visible-Infrared Image via Max-Merge Complementary Information

  2025cites this paper
• Relieving the Over-Smoothing Problem using Graph Disentanglement with Latent Bottlenecks

  2025cites this paper
• Preference Optimization by Estimating the Ratio of the Data Distribution

  2025cites this paper
• ADD: Physics-Based Motion Imitation with Adversarial Differential Discriminators

  2025cites this paper
• Joint-stochastic-approximation Autoencoders with Application to Semi-supervised Learning

  2025cites this paper
• Zero-Shot Whole-Body Humanoid Control via Behavioral Foundation Models

  2025cites this paper
• Generalization in VAE and Diffusion Models: A Unified Information-Theoretic Analysis

  2025cites this paper
• Benchmarking Mutual Information-based Loss Functions in Federated Learning

  2025cites this paper
• Generalized Score Matching: Bridging $f$-Divergence and Statistical Estimation Under Correlated Noise

  2025cites this paper
• Boosting Statistic Learning with Synthetic Data from Pretrained Large Models

  2025cites this paper
• α-GAN by Rényi Cross Entropy

  2025cites this paper
• Disentanglement-based compression and generative reconstruction of fixed-scene videos

  2025cites this paper
• Graph Prompting for Graph Learning Models: Recent Advances and Future Directions

  2025cites this paper
• Exploring the Limitations of Federated Learning: A Novel Wasserstein Metric-Based Poisoning Attack on Traffic Sign Classification

  2025cites this paper
• The Hidden Link Between RLHF and Contrastive Learning

  2025cites this paper
• A Markov Categorical Framework for Language Modeling

  2025cites this paper
• Inverse Reinforcement Learning Meets Large Language Model Post-Training: Basics, Advances, and Opportunities

  2025cites this paper
• Harnessing Diffusion-Yielded Score Priors for Image Restoration

  2025cites this paper
• Imbalance-Robust and Sampling-Efficient Continuous Conditional GANs via Adaptive Vicinity and Auxiliary Regularization

  2025cites this paper
• Symmetric Behavior Regularized Policy Optimization

  2025influential citation
• Semi-Supervised Bayesian GANs with Log-Signatures for Uncertainty-Aware Credit Card Fraud Detection

  2025cites this paper
• A Graph Laplacian Eigenvector-based Pre-training Method for Graph Neural Networks

  2025cites this paper
• The Choice of Divergence: A Neglected Key to Mitigating Diversity Collapse in Reinforcement Learning with Verifiable Reward

  2025cites this paper
• Convergent motifs of early olfactory processing are recapitulated by layer-wise efficient coding

  2025cites this paper
• Nested Annealed Training Scheme for Generative Adversarial Networks

  2025cites this paper
• Coupling Generative Modeling and an Autoencoder with the Causal Bridge

  2025cites this paper
• A Neural Network Algorithm for KL Divergence Estimation with Quantitative Error Bounds

  2025cites this paper
• XIPHOS: Adaptive In-Vehicle Intrusion Detection via Unsupervised Graph Contrastive Learning

  2025influential citation
• Topology-preserving and structure-aware (hyper)graph contrastive learning for node classification

  2025cites this paper
• Graph Prompt Clustering

  2025cites this paper
• CKGAN: Training Generative Adversarial Networks Using Characteristic Kernel Integral Probability Metrics

  2025cites this paper
• Two-Dimensional Stochastic Structural Geomodeling with Deep Generative Adversarial Networks

  2025cites this paper
• AMOR: Adaptive Character Control through Multi-Objective Reinforcement Learning

  2025cites this paper
• A Deep Bayesian Nonparametric Framework for Robust Mutual Information Estimation

  2025cites this paper
• Wind Farm Control via Offline Reinforcement Learning With Adversarial Training

  2025cites this paper
• Text-to-3D Generation using Jensen-Shannon Score Distillation

  2025cites this paper
• One-step Diffusion Models with f-Divergence Distribution Matching

  2025cites this paper
• GNN-transformer contrastive learning explores homophily

  2025cites this paper
• INFO-SEDD: Continuous Time Markov Chains as Scalable Information Metrics Estimators

  2025cites this paper
• Meta-Statistical Learning: Supervised Learning of Statistical Inference

  2025cites this paper
• A Fused Gromov-Wasserstein Approach to Subgraph Contrastive Learning

  2025cites this paper
• An Adaptive Underwater Image Enhancement Framework via Multidomain Fusion and Color Compensation

  2025cites this paper
• Generative Adversarial Network Based on Self-Attention Mechanism for Automatic Page Layout Generation

  2025cites this paper
• NRGAN: A Noise-resilient GAN with adaptive feature modulation for SAR image segmentation

  2025cites this paper
• Graph-weighted contrastive learning for semi-supervised hyperspectral image classification

  2025cites this paper
• Improving Discriminator Guidance in Diffusion Models

  2025cites this paper
• Unifying Attribute and Structure Preservation for Enhanced Graph Contrastive Learning

  2025cites this paper
• Disentangling Reasoning Factors for Natural Language Inference

  2025cites this paper
• Robust Classification with Noisy Labels Based on Posterior Maximization

  2025cites this paper
• Cybersecurity Multi-Dimensional Few-Shot Data Generation on Malicious Enhancement

  2025cites this paper
• Universal Rate-Distortion-Classification Representations for Lossy Compression

  2025cites this paper
• A Theory of Universal Rate-Distortion-Classification Representations for Lossy Compression

  2025cites this paper
• Mutual Information Neural-Estimation-Driven Constellation Shaping Design and Performance Analysis

  2025cites this paper
• Learning Brenier Potentials with Convex Generative Adversarial Neural Networks

  2025cites this paper
• Robust Representation Learning Based on Deep Mutual Information for Scene Classification Against Adversarial Perturbations

  2025cites this paper
• Multi-task learning for joint prediction of breast cancer histological indicators in dynamic contrast-enhanced magnetic resonance imaging

  2025cites this paper
• Bounding Neyman-Pearson Region with f-Divergences

  2025cites this paper
• Proximal optimal transport divergences

  2025cites this paper
• Comprehensive Information Extraction With Separable Representation Learning for Multi-View Clustering

  2025cites this paper
• Symmetric Regularized Sequential Latent Variable Models With Adversarial Neural Networks

  2025cites this paper
• Implications From the Analogous Relationship Between Evolutionary and Learning Processes

  2025cites this paper
• End-to-End Abnormal Subgraph Detection via Subgraph-Level Contrastive Learning

  2025cites this paper
• A Robust Local Fr\'echet Regression Using Unbalanced Neural Optimal Transport with Applications to Dynamic Single-cell Genomics Data

  2025cites this paper
• Bayesian inverse problems with conditional Sinkhorn generative adversarial networks in least volume latent spaces

  2025cites this paper
• Weakly Supervised Object Segmentation by Background Conditional Divergence

  2025cites this paper
• Score-of-Mixture Training: One-Step Generative Model Training Made Simple via Score Estimation of Mixture Distributions

  2025influential citation