A Recurrent Latent Variable Model for Sequential Data

In this paper, we explore the inclusion of latent random variables into the hidden state of a recurrent neural network (RNN) by combining the elements of the variational autoencoder. We argue that through the use of high-level latent random variables, the variational RNN (VRNN)1 can model the kind of variability observed in highly structured sequential data such as natural speech. We empirically evaluate the proposed model against other related sequential models on four speech datasets and one handwriting dataset. Our results show the important roles that latent random variables can play in the RNN dynamics.

• Publication year

  2015

• Venue

  Neural Information Processing Systems

• Publication date

  2015-06-07

• Fields of study

  Computer Science

• Identifiers
  arXiv 1506.02216
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• DRAW: A Recurrent Neural Network For Image Generation

  2015cited by this paper
• Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation

  2014cited by this paper
• Adam: A Method for Stochastic Optimization

  2014cited by this paper
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models

  2014cited by this paper
• Learning Stochastic Recurrent Networks

  2014influential reference
• Variational Recurrent Auto-Encoders

  2014cited by this paper
• Speech Synthesis Based on Hidden Markov Models

  2013cited by this paper
• The Blizzard Challenge 2013

  2013cited by this paper
• Generating Sequences With Recurrent Neural Networks

  2013influential reference
• Auto-Encoding Variational Bayes

  2013influential reference
• Modeling Temporal Dependencies in High-Dimensional Sequences: Application to Polyphonic Music Generation and Transcription

  2012influential reference
• Learning visual motion in recurrent neural networks

  2012cited by this paper
• Theano: new features and speed improvements

  2012cited by this paper
• Rectified Linear Units Improve Restricted Boltzmann Machines

  2010cited by this paper
• Unsupervised feature learning for audio classification using convolutional deep belief networks

  2009cited by this paper
• Unsupervised learning of auditory filter banks using non-negative matrix factorisation

  2008cited by this paper
• IAM-OnDB - an on-line English sentence database acquired from handwritten text on a whiteboard

  2005cited by this paper
• Long Short-Term Memory

  1997cited by this paper

• Density Matrix RNN (DM-RNN): A Quantum Information Theoretic Framework for Modeling Musical Context and Polyphony

  2026cites this paper
• LatentTrack: Sequential Weight Generation via Latent Filtering

  2026cites this paper
• Stabilizing autoregressive forecasts in chaotic systems via multi-rate latent recurrence

  2026cites this paper
• Latent Matters: Learning Deep State-Space Models

  2026cites this paper
• Variational Deep Alliance: A Generative Auto-Encoding Approach to Longitudinal Data Analysis

  2026cites this paper
• Variational Distributional Neuron

  2026cites this paper
• Latent Probabilistic Dynamic Embedding Supervised Deep Networks With Graph-Guiding for Soft Sensing in Industrial Process

  2026cites this paper
• Uncertainty-Aware Bayesian Time Series framework for probabilistic imputation

  2026cites this paper
• RDSAD: Robust Threat Detection in Evolving Data Streams via Adaptive Latent Dynamics

  2026influential citation
• SELDON: Supernova Explosions Learned by Deep ODE Networks

  2026cites this paper
• Switching Spatio-Temporal Deep Variational Model for Radar Target Detection in Complex Clutter

  2026cites this paper
• CrossTopoNet: A cross-attention framework on topological latent feature space for time-series forecasting

  2025cites this paper
• Geo-lucid Conditional Diffusion Models for High Physical Fidelity Trajectory Generation

  2025cites this paper
• DVAE: A Dynamic Variational Autoencoder for Structured Causal Discovery with Application in Biomedical Time Series

  2025cites this paper
• Soft sensor for nonuniform sampling nonlinear dynamic process using irregular-time-interval latent probabilistic predictability embedding supervised deep network.

  2025cites this paper
• Vector Quantization in the Brain: Grid-like Codes in World Models

  2025cites this paper
• Contrastive Denoising Variational Recurrent Neural Network for Noise-Agnostic State Modeling of Soft Robotic Systems

  2025cites this paper
• Predicting Slowly-Varying Mean and Variance in Gaussian-Type Time-Series: A Comprehensive Deep-Learning Benchmark

  2025influential citation
• Wavelet-based Disentangled Adaptive Normalization for Non-stationary Times Series Forecasting

  2025cites this paper
• A health monitoring and early fault detection method of rotating machines based on latent variables of diffusion model

  2025cites this paper
• Recovering hidden degrees of freedom using Gaussian processes.

  2025cites this paper
• A Generalizable Physics-Enhanced State Space Model for Long-Term Dynamics Forecasting in Complex Environments

  2025cites this paper
• Neural Bayesian Filtering

  2025cites this paper
• CAMNet: Leveraging Cooperative Awareness Messages for Vehicle Trajectory Prediction

  2025influential citation
• Social Spatio-Temporal Graph Convolutional Neural Network for Pedestrian Trajectory Prediction

  2025cites this paper
• DANet: A RAG-inspired Dual Attention Model for Few-shot Time Series Prediction

  2025cites this paper
• SCMDM: User-Oriented Service Migration Based on State Cognition for Mobile Edge Computing

  2025cites this paper
• A Review of Pedestrian Trajectory Prediction Methods Based on Deep Learning Technology

  2025cites this paper
• Meta Variational Memory Transformer for Anomaly Detection of Multivariate Time Series

  2025cites this paper
• Autoregressivity in the Latent Space of a GP-VAE Language Model: An Empirical Ablation Study

  2025cites this paper
• Long-Term Individual Causal Effect Estimation via Identifiable Latent Representation Learning

  2025cites this paper
• Bayesian Meta-Reinforcement Learning with Laplace Variational Recurrent Networks

  2025cites this paper
• L-GTA: Latent Generative Modeling for Time Series Augmentation

  2025cites this paper
• Spatial-temporal context-aware network for 3D-Craft generation

  2025cites this paper
• Characterizing the sense of agency in human–robot interaction based on the free energy principle

  2025cites this paper
• Versatile Physics‐based Character Control with Hybrid Latent Representation

  2025cites this paper
• Spiking Generative Models Based on Variational Autoencoder and Adversarial Training

  2025cites this paper
• A Dynamic, Ordinal Gaussian Process Item Response Theoretic Model

  2025cites this paper
• Towards Unsupervised Time-Series Anomaly Detection for Virtual Cloud Networks

  2025influential citation
• Leveraging subdomain alignment for enhanced anomaly detection in time series

  2025cites this paper
• Time-DisCo: A Disentangled Contrastive Autoencoder Approach to Enhancing Multivariate Probabilistic Time Series Forecasting From Multisensor Data

  2025cites this paper
• Optimizing latency for caching with delayed hits in non-stationary environment

  2025cites this paper
• Enhancing Uncertainty Estimation and Interpretability with Bayesian Non-negative Decision Layer

  2025cites this paper
• Chaotic Time Series Forecasting Optimisation Based on Hybrid PSO Algorithm

  2025cites this paper
• Flight trajectory modeling reveals species-specific obstacle avoidance policies in echolocating bats

  2025influential citation
• Towards Foundation Auto-Encoders for Time-Series Anomaly Detection

  2025cites this paper
• Disentangling Speaker and Content in Pre-trained Speech Models with Latent Diffusion for Robust Speaker Verification

  2025cites this paper
• Dynamical system reconstruction from partial observations using stochastic dynamics

  2025cites this paper
• Memory-Augmented Generative AI for Real-time Wireless Prediction in Dynamic Industrial Environments

  2025cites this paper
• UF-RNN: Real-Time Adaptive Motion Generation Using Uncertainty-Driven Foresight Prediction

  2025cites this paper
• Towards Identifiability of Hierarchical Temporal Causal Representation Learning

  2025cites this paper
• Neural Stochastic Flows: Solver-Free Modelling and Inference for SDE Solutions

  2025cites this paper
• Gaussian Mixture Conditional Variational Recurrent Neural Network for Unified Trajectory Imputation and Prediction

  2025cites this paper
• Model-free learning of random dynamical system from noisy observations

  2025cites this paper
• Deep Belief Markov Models for POMDP Inference

  2025cites this paper
• BRAID: Input-driven nonlinear dynamical modeling of neural-behavioral data

  2025cites this paper
• Variational Hierarchical N-BEATS Model for Long-Term Time-Series Forecasting

  2025cites this paper
• Inference-time Stochastic Refinement of GRU-Normalizing Flow for Real-time Video Motion Transfer

  2025cites this paper
• Deep Dynamic Probabilistic Canonical Correlation Analysis

  2025cites this paper
• BP-SGCN: Behavioral Pseudo-Label Informed Sparse Graph Convolution Network for Pedestrian and Heterogeneous Trajectory Prediction

  2025cites this paper
• Control-ITRA: Controlling the Behavior of a Driving Model

  2025cites this paper
• Predictive Learning in Energy-based Models with Attractor Structures

  2025influential citation
• Human-Robot Kinaesthetic Interactions Based on the Free-Energy Principle

  2025cites this paper
• Disentangling the latent space of an end2end generative VRNN model for structural health condition diagnosis

  2025cites this paper
• Interactive sequential generative models for team sports

  2025cites this paper
• Machine learning for human mobility during disasters: A systematic literature review

  2025cites this paper
• Does Augmented Reality Apps Trigger Loyalty Toward the Brand? A Myth or Reality: The Case of UK and UAE Consumers

  2025cites this paper
• The Alpha-Alternator: Dynamic Adaptation To Varying Noise Levels In Sequences Using The Vendi Score For Improved Robustness and Performance

  2025cites this paper
• Habitizing Diffusion Planning for Efficient and Effective Decision Making

  2025cites this paper
• Effectiveness of board of directors and risk-taking: the mediating role of Shari’ah supervisory board of Bangladesh Islamic financial institutions

  2025cites this paper
• Bayesian Computation in Deep Learning

  2025cites this paper
• iDANSE: Iterative Data-driven Nonlinear State Estimation of Model-free Hidden Sequences

  2025cites this paper
• A Hierarchical Taxonomy For Deep State Space Models

  2025influential citation
• MTMDC-GAN: Self-Attention Driven Multi-Scale Temporal Synthesis with Multi-Domain Analysis and Contrastive Learning

  2025cites this paper
• Stochastic Trajectory Prediction Under Unstructured Constraints

  2025cites this paper
• BEAC: Imitating Complex Exploration and Task-oriented Behaviors for Invisible Object Nonprehensile Manipulation

  2025cites this paper
• Time-Series Forecasting in Healthcare: A Dual-Framework Approach with Latent Class Integration

  2025cites this paper
• Analyzing homogenous and heterogeneous multi-server queues via neural networks

  2025cites this paper
• Towards efficient real-time video motion transfer via generative time series modeling

  2025influential citation
• Stochastic Diffusion: A Diffusion Based Model for Stochastic Time Series Forecasting

  2025cites this paper
• Data augmentation for forecasting industrial aging processes via conditional multimodal generative time-series models

  2025cites this paper
• An Anomaly Detection Method for Multivariate Time Series Data Based on Variational Autoencoders and Association Discrepancy

  2025cites this paper
• Computing the steady-state probabilities of the number of customers in the system of a tandem queueing system, a Machine Learning approach

  2025cites this paper
• Latent Diffeomorphic Dynamic Mode Decomposition

  2025cites this paper
• Alternators With Noise Models

  2025cites this paper
• Indirect core-driven cross-channel collaborative fusion soft sensor modeling

  2025cites this paper
• Multi-Graph Spatio-Temporal Convolution for Traffic Flow Prediction Focusing on Edge Derived Imbalanced Data From Highway Electronics

  2025cites this paper
• Learning Latent Variable Models via Jarzynski-adjusted Langevin Algorithm

  2025cites this paper
• SimProcess: High Fidelity Simulation of Noisy ICS Physical Processes

  2025cites this paper
• Exploring the application of Time Series Foundation Models to network monitoring tasks

  2025cites this paper
• InfoDPCCA: Information-Theoretic Dynamic Probabilistic Canonical Correlation Analysis

  2025cites this paper
• The sound of malware: an audio fingerprinting malware detection method

  2025cites this paper
• ITCNet: long sequence time-series forecasting of underwater excavation states for cutter suction dredgers

  2025cites this paper
• Deep switching state space model for nonlinear time series forecasting with regime switching

  2025cites this paper
• FairDRL-ST: Disentangled Representation Learning for Fair Spatio-Temporal Mobility Prediction

  2025cites this paper
• Rényi divergence in hidden Markov models

  2025cites this paper
• Time Series Domain Adaptation via Latent Invariant Causal Mechanism

  2025cites this paper
• EgoTraj-Bench: Towards Robust Trajectory Prediction Under Ego-view Noisy Observations

  2025cites this paper
• Curiosity-Driven Development of Action and Language in Robots Through Self-Exploration

  2025cites this paper
• On the use of LSTM-based estimation components for tokamak gas actuator control

  2025cites this paper