Learning long-term dependencies with gradient descent is difficult

No abstract is available for this paper.

• Publication year

  1994

• Venue

  IEEE Trans. Neural Networks

• Publication date

  1994-03-01

• Fields of study

  Medicine, Computer Science

• Identifiers
  DOI 10.1109/72.279181PMID 18267787
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar, PubMed

• No claims are published for this paper.

• No concepts are published for this paper.

• Unified Integration of Explicit Knowledge and Learning by Example in Recurrent Networks

  1995influential reference
• A solvable connectionist model of immediate recall of ordered lists

  1994cited by this paper
• Iterative solution of non‐linear equations using a computer algebra package

  1994cited by this paper
• A method of training multi-layer networks with heaviside characteristics using internal representations

  1993cited by this paper
• The problem of learning long-term dependencies in recurrent networks

  1993influential reference
• Local Feedback Multilayered Networks

  1992cited by this paper
• Inserting rules into recurrent neural networks

  1992cited by this paper
• Using random weights to train multilayer networks of hard-limiting units

  1992cited by this paper
• Induction of Multiscale Temporal Structure

  1991cited by this paper
• Nonlinear dynamics and stability of analog neural networks

  1991cited by this paper
• Artificial neural networks and their application to sequence recognition

  1991cited by this paper
• Global optimization of a neural network-hidden Markov model hybrid

  1991cited by this paper
• Improving the convergence of back-propagation learning with second-order methods

  1989cited by this paper
• The "Moving Targets" Training Algorithm

  1989influential reference
• BPS: a learning algorithm for capturing the dynamic nature of speech

  1989cited by this paper
• A Learning Algorithm for Continually Running Fully Recurrent Neural Networks

  1989cited by this paper
• A Focused Backpropagation Algorithm for Temporal Pattern Recognition

  1989cited by this paper
• Learning by Choice of Internal Representations

  1988cited by this paper
• Minimizing multimodal functions of continuous variables with the “simulated annealing” algorithmCorrigenda for this article is available here

  1987cited by this paper
• Learning processes in an asymmetric threshold network

  1986cited by this paper
• Learning internal representations by error propagation

  1986cited by this paper
• Learning Process in an Asymmetric Threshold Network

  1986cited by this paper
• Optimization by Simulated Annealing

  1983cited by this paper
• AND T

  year unknowncited by this paper

• Stacked ML-GARCH for Bitcoin Risk Forecasting: A Novel Ensemble Approach for Superior Value-at-Risk Estimation

  2026cites this paper
• Sparse assemblies of recurrent neural networks with stability guarantees

  2026cites this paper
• AI-powered analysis of viral metagenomic sequencing data for rapid outbreak investigation and novel pathogen discovery

  2026cites this paper
• Tuning the burn-in phase in training recurrent neural networks improves their performance

  2026cites this paper
• Dynamic Kannada Sign Language Recognition on Resource Constrained Devices.

  2026cites this paper
• Optimizing the Bounds of Neural Networks Using a Novel Simulated Annealing Method

  2026cites this paper
• Online Learning Robust Control Design for a Class of Uncertain Nonlinear Systems

  2026cites this paper
• Deep Learning-Based Skin Care Detection with Multi-method Explainability: Grad-CAM, Lime, and Occlusion Sensitivity

  2026influential citation
• Textual Equilibrium Propagation for Deep Compound AI Systems

  2026cites this paper
• From Classical to Topological Neural Networks Under Uncertainty

  2026cites this paper
• Exploring the Schumpeterian Entrepreneur’s Personality: A Natural Language Approach with Neural Networks Estimates

  2026cites this paper
• Deep learning-based prediction of the hysteretic behavior of buckling-restrained braces for seismic design using analysis-of-mean-based optimal hyperparameters

  2026cites this paper
• Preference adaptive multi-objective HVAC control using a vectorized deep transformer Q-network

  2026cites this paper
• ShipSeer: Pushing accuracy-performance boundaries in ship motion prediction with spectral and multi-component analysis

  2026cites this paper
• Machine learning based sensing of particle shape and size using passive artificial cilia

  2026cites this paper
• Inverse design of cement and cement paste composition for targeted tensile strength using sequential neural networks and hydration based micromechanical modelling

  2026cites this paper
• Using ANN and association rules to analyze maternal health-related variables and child growth in India: a cross-sectional study

  2026cites this paper
• Modulation-Based Feature Extraction for Robust Sleep Stage Classification Across Apnea-Based Cohorts

  2026cites this paper
• Enhancing the Generalization Ability of Learning-Based Optimizers through GRU and Multi-Head Attention Mechanisms

  2026cites this paper
• A Unified Transformer-Based Harmonic Detection Network for Distorted Power Systems

  2026cites this paper
• Reinforcement Learning for Control Systems with Time Delays: A Comprehensive Survey

  2026cites this paper
• When to Memorize and When to Stop: Gated Recurrent Memory for Long-Context Reasoning

  2026cites this paper
• A flexible framework for hyperparameter optimization using homotopy and surrogate models.

  2026cites this paper
• Cardinality constrained index tracking in an end-to-end way: Soft or hard?

  2026cites this paper
• CCCaption: Dual-Reward Reinforcement Learning for Complete and Correct Image Captioning

  2026cites this paper
• Prediction of transient thermal deformation in press by detrending time-series variations

  2026cites this paper
• Modelling the Actual Prices of Kenyan Exchange Rates Using GARCH-Transformer Hybrid Model

  2026cites this paper
• Deep learning enhanced crack identification on rocks

  2026cites this paper
• Constructing ESG-driven portfolios using genetic algorithms based on LSTM-predicted data

  2026cites this paper
• Qnnformer: quantum-enhanced neural networks with multi-attention mechanisms for long-term time series forecasting

  2026cites this paper
• Trend-weighted multi-resolution transformer for multi-parametric glucose prediction

  2026cites this paper
• Blind identification of collective motion criticality using sequence model predictive entropy variance

  2026cites this paper
• Enhancing knee contact force prediction using transformer networks: A comparative study with LSTM and MLP

  2026cites this paper
• Tracking with attention: A review of transformer-based object tracking

  2026cites this paper
• An Introduction to Pathology Foundation Models

  2026cites this paper
• Enhancing exchange rate forecasting with contextual sentiment indices: A fine-tuned FinBERT approach

  2026cites this paper
• Hybrid residual reservoir computing using dynamic memristor for time series prediction

  2026cites this paper
• CSLLM: Code-Specific Large Language Models - A Survey

  2026cites this paper
• OpenITI Tabanlı RAG Sistemi ile Doğal Dil İşleme

  2026cites this paper
• MLG-Net: A hybrid framework for bridge cable damage identification using acoustic emission technology

  2026cites this paper
• Automating Driving Behavior Anomaly Qualification in Real-Time for Electric Two-Wheelers Using GRU Autoencoder

  2026cites this paper
• Attention-Based Bidirectional Gated Recurrent Unit Neural Networks for Lithology Identification from Well-Logging Data

  2026cites this paper
• A Directional Multi-LSTM Framework Integrated BERT for S-sulfhydration Sites Prediction

  2026cites this paper
• Sequential pattern transformer (SPT): a generative and interpretable framework for predicting disease trajectories

  2026cites this paper
• Simplified Deep Learning Driven Design of Blue Light Grating Couplers

  2026cites this paper
• Prior-Knowledge-Guided Missing Data Imputation for Bridge Cracks: A Temperature-Driven SP-VMD-CNN-GRU Framework

  2026cites this paper
• Machine learning assisted forward and reverse mapping of micro-additive manufacturing process

  2026cites this paper
• Continuous-Time Piecewise-Linear Recurrent Neural Networks

  2026cites this paper
• An intelligent method for Buoy meteorological data restoration using a Spatio-Temporal Dual-Attention Network with transformer and GAT.

  2026cites this paper
• Position: Why a Dynamical Systems Perspective is Needed to Advance Time Series Modeling

  2026cites this paper
• Why ReLU? A Bit-Model Dichotomy for Deep Network Training

  2026cites this paper
• Surrogate modeling for three-phase flow in porous media based on a temporal-attention-enhanced multiple-input operator network

  2026cites this paper
• Prediction of safety limits for rail weld irregularities in high-speed turnouts using a CNN-LSTM hybrid model

  2026cites this paper
• A data-driven framework for three-dimensional landslide 2 displacement prediction based on multi-temporal InSAR and 3 machine learning techniques

  2026cites this paper
• A comprehensive review of machine learning and deep learning approaches for rainfall forecasting: current progress, challenges, and future directions

  2026cites this paper
• Towards efficient and reliable artificial intelligence through neuromorphic principles.

  2026cites this paper
• Embedding interpretable $\ell_1$-regression into neural networks for uncovering temporal structure in cell imaging

  2026influential citation
• The Volterra signature

  2026cites this paper
• A hybrid learning approach integrating ARIMA-GARCH-LSTM for value-at-risk forecasting

  2026cites this paper
• AI-driven isotope detection: applying deep learning to gamma spectral analysis

  2026cites this paper
• A data-driven inelasticity framework enhanced by neural network based propagators – application to trusses

  2026cites this paper
• Nonparametric prediction of ship maneuvering motions based on the heterogeneous integration model

  2026cites this paper
• DADIGAN: A dual attention blocks-based disentangled iterative Generative Adversarial Network for cloud and shadow removal on SAR and optical images

  2026cites this paper
• An EEMD-based LSTM method for reconstructing the attenuated interference signals in a laser doppler vibrometry system

  2026cites this paper
• Bridging the safety-specific language model gap: Domain-adaptive pretraining of transformer-based models across several industrial sectors for occupational safety applications

  2026cites this paper
• Unidirectional LSTM-based 3D reaching trajectory prediction using selected magnetometer-only SQUID-MEG channels

  2026cites this paper
• An innovative neural network architecture designed for industrial fault diagnosis with hierarchical adaptive attention mechanism

  2026cites this paper
• Uncertainty-Aware Bayesian Time Series framework for probabilistic imputation

  2026cites this paper
• Rotary Position Embedding-Based Transformer Hawkes Process for Event-Type Big Data

  2026cites this paper
• A dual-module TCNN–LSTM model with self-attention for predicting tunnel fires under mobile ventilation control

  2026cites this paper
• LION-DG: Layer-Informed Initialization with Deep Gradient Protocols for Accelerated Neural Network Training

  2026cites this paper
• Robust Reasoning as a Symmetry-Protected Topological Phase

  2026cites this paper
• Multivariate CO2 Emissions Forecasting Using Deep Neural Network Architectures

  2026cites this paper
• The quantum-inspired adaptive superposition optimization for neural network training

  2026cites this paper
• A hierarchical decomposition and component-specific framework for forecasting non-stationary mine water inflow: Pingdingshan mining region, China

  2026cites this paper
• A Review of Online Diffusion Policy RL Algorithms for Scalable Robotic Control

  2026influential citation
• Genetic risk predictions using deep learning models with summary data

  2026cites this paper
• Computer-assisted Diagnosis of Obstructive Sleep Apnea in Adults: A Narrative Review

  2026cites this paper
• An automated slot stowage optimization method for container ship based on improved Actor-Critic algorithm

  2026cites this paper
• Surrogate-assisted dynamic response prediction and intelligent fault management for gas–cooled reactor–Brayton energy systems

  2026cites this paper
• A Novel Algorithm for Determining the Window Size in Power Load Prediction

  2026cites this paper
• Prediction of electrical and thermal properties of cement-based composites driven by microscopic structural data

  2026cites this paper
• Optimization via the strategic law of large numbers.

  2026cites this paper
• A comparative study on stochastic and AI-based approaches for predicting meteorological droughts

  2026cites this paper
• From Fuzzy to Exact: The Halo Architecture for Infinite-Depth Reasoning via Rational Arithmetic

  2026cites this paper
• Lameness detection in dairy cows using pose estimation and bidirectional LSTMs

  2026cites this paper
• Pattern-guided forecasting framework for metal price prediction with grouping decomposed series

  2026cites this paper
• Large language models for clinical artificial intelligence in healthcare a systematic review

  2026cites this paper
• Composite laminate damage identification via Gabor-based anisotropic filter enhanced attention ConvLSTM network

  2026cites this paper
• ParalESN: Enabling parallel information processing in Reservoir Computing

  2026influential citation
• G3DCT: An Interpretable Spatial Grid-based Framework with Temporal Convolution-Transformer for EEG Artifact Identification

  2026cites this paper
• Balanced Grey Wolf Optimizer Algorithm for Backpropagation Neural Networks

  2026cites this paper
• Asymptotic evaluation of the information processing capacity in reservoir computing

  2025cites this paper
• Dynamical phases of short-term memory mechanisms in RNNs

  2025cites this paper
• GradientStabilizer:Fix the Norm, Not the Gradient

  2025cites this paper
• Neurons as autonomous agents: A biologically inspired framework for cognitive architectures in artificial intelligence

  2025cites this paper
• Performance evaluation of efficient interpretable CNN-transformer model for redshift prediction

  2025cites this paper
• A new ensemble learning method based on signal source driver for GNSS coordinate time series prediction

  2025cites this paper
• Self-Adjust Softmax

  2025cites this paper
• Shale Gas Production Prediction Based on Modified Graph Attention and Memory-Augmented Neural Networks

  2025cites this paper