Beyond Feedforward Models Trained by Backpropagation: A Practical Training Tool for a More Efficient Universal Approximator

Cellular simultaneous recurrent neural network (SRN) has been shown to be a function approximator more powerful than the multilayer perceptron (MLP). This means that the complexity of MLP would be prohibitively large for some problems while SRN could realize the desired mapping with acceptable computational constraints. The speed of training of complex recurrent networks is crucial to their successful application. This work improves the previous results by training the network with extended Kalman filter (EKF). We implemented a generic cellular SRN (CSRN) and applied it for solving two challenging problems: 2-D maze navigation and a subset of the connectedness problem. The speed of convergence has been improved by several orders of magnitude in comparison with the earlier results in the case of maze navigation, and superior generalization has been demonstrated in the case of connectedness. The implications of this improvements are discussed.

• Publication year

  2007

• Venue

  IEEE Transactions on Neural Networks

• Publication date

  2007-10-22

• Fields of study

  Medicine, Computer Science, Engineering

• Identifiers
  DOI 10.1109/TNN.2008.2000396arXiv 0710.4182PMID 18541494
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar, PubMed

• No claims are published for this paper.

• No concepts are published for this paper.

• Neurodynamics of cognition and consciousness

  2009cited by this paper
• Efficient Learning in Cellular Simultaneous Recurrent Neural Networks - The Case of Maze Navigation Problem

  2007cited by this paper
• Cellular SRN Trained by Extended Kalman Filter Shows Promise for ADP

  2006cited by this paper
• Quantum-inspired Evolutionary Algorithms and Binary Particle Swarm Optimization for Training MLP and SRN Neural Networks

  2005cited by this paper
• A Self-Learning Evolutionary Chess Program

  2004cited by this paper
• Approximation and estimation bounds for artificial neural networks

  2004cited by this paper
• An Introduction to Multivariate Statistical Analysis

  2004cited by this paper
• Kalman Filtering and Neural Networks

  2001cited by this paper
• ALGORITHMS FOR THE DETECTION OF CONNECTEDNESS AND THEIR NEURAL IMPLEMENTATION

  2001cited by this paper
• Evolving an expert checkers playing program without using human expertise

  2001cited by this paper
• The cellular simultaneous recurrent network adaptive critic design for the generalized maze problem has a simple closed-form solution

  2000cited by this paper
• IMPLEMENTING BACK-PROPAGATION- THROUGH-TIME LEARNING ALGORITHM USING CELLULAR NEURAL NETWORKS

  1999cited by this paper
• Understanding the Kalman filter: An object oriented programming perspective

  1999cited by this paper
• Neural network design for J function approximation in dynamic programming

  1998cited by this paper
• Enhanced Multi-Stream Kalman Filter Training for Recurrent Networks

  1998cited by this paper
• Nonlinear modeling : advanced black-box techniques

  1998cited by this paper
• Neuronal Information Processing - From Biological Data to Modelling and Applications (World Scientific, 1998). REDUCING THE COMPLEXITY OF NEURAL NETS FOR INDUSTRIAL APPLICATIONS AND BIOLOGICAL MODELS

  1998cited by this paper
• Generalized maze navigation: SRN critics solve what feedforward or Hebbian nets cannot

  1996influential reference
• Neurocontrol of nonlinear dynamical systems with Kalman filter trained recurrent networks

  1994cited by this paper
• Neural Networks: A Comprehensive Foundation

  1994cited by this paper
• Handbook of Intelligent Control: Neural, Fuzzy, and Adaptive Approaches

  1992cited by this paper
• Backpropagation Through Time: What It Does and How to Do It

  1990cited by this paper
• Consistency of HDP applied to a simple reinforcement learning problem

  1990cited by this paper
• Perceptrons: expanded edition

  1988cited by this paper
• An Introduction to Multivariate Statistical Analysis

  1959cited by this paper

• A Comprehensive Review on RSM-Coupled Optimization Techniques and Its Applications

  2023cites this paper
• Analysis of Learning Heuristic Estimates for Grid Planning with Cellular Simultaneous Recurrent Networks

  2023influential citation
• Data‐driven performance metrics for neural network learning

  2023cites this paper
• Ornithopter Trajectory Optimization with Neural Networks and Random Forest

  2022cites this paper
• Learning Heuristic Estimates for Planning in Grid Domains by Cellular Simultaneous Recurrent Networks

  2022influential citation
• Integrating Elman recurrent neural network with particle swarm optimization algorithms for an improved hybrid training of multidisciplinary datasets

  2021cites this paper
• MODELLING OF GENE EXPRESSION DATA USING EVOLVING SPIKING NEURAL NETWORK-A CASE STUDY ON NUTRIGENOMICS PROBLEM

  2020cites this paper
• Online Model-Free n-Step HDP With Stability Analysis

  2020cites this paper
• An Improved N-Step Value Gradient Learning Adaptive Dynamic Programming Algorithm for Online Learning

  2020cites this paper
• Face spoofing detection via ensemble of classifiers toward low-power devices

  2020cites this paper
• Personalised Modelling for Integrated Static and Dynamic Data. Applications in Neuroinformatics

  2019cites this paper
• Design of a K-Winners-Take-All Model With a Binary Spike Train

  2019cites this paper
• Use of Levenberg-Marquardt Method to Train Cellular Simultaneous Recurrent Network to Perform Maze Traversal Task

  2019cites this paper
• CLASSIFICATION OF GALAXIES IN SHAPLEY CONCENTRATION REGION WITH MACHINE LEARNING

  2019cites this paper
• The Boundedness Conditions for Model-Free HDP( $\lambda$ )

  2019cites this paper
• The New AI

  2019cites this paper
• A Computational Framework for Personalised Modelling. Applications in Bioinformatics

  2018cites this paper
• Efficient Learning of Data Distribution using Simultaneous Recurrent Belief Network

  2018influential citation
• Sparse Simultaneous Recurrent Deep Learning for Robust Facial Expression Recognition

  2018cites this paper
• Novel deep generative simultaneous recurrent model for efficient representation learning

  2018cites this paper
• Deep Recurrent Learning for Efficient Image Recognition Using Small Data

  2018cites this paper
• Object recognition using cellular simultaneous recurrent networks and convolutional neural network

  2017cites this paper
• Constrained versus unconstrained learning in generalized recurrent network for image processing

  2017influential citation
• Abstraction hierarchy in deep learning neural networks

  2017cites this paper
• Contemporary deep recurrent learning for recognition

  2017cites this paper
• Extended Kalman filtering to design optimal controllers of fronts generated by level set methods

  2016cites this paper
• On spectral component estimation using neural networks for rolling bearing fault diagnosis

  2016cites this paper
• AFRL-OSR-VA-TR-2012-1229 Investigation of Large Scale Cortical Models on Clustered Multi-core Processors

  2016cites this paper
• Efficient feature extraction with simultaneous recurrent network for metric learning

  2016cites this paper
• Deep SRN for robust object recognition: A case study with NAO humanoid robot

  2016cites this paper
• Commentary by Paul J. Werbos

  2016cites this paper
• Reinforcement Learning of Linking and Tracing Contours in Recurrent Neural Networks

  2015influential citation
• City Research Online

  2015cites this paper
• Novel hierarchical Cellular Simultaneous Recurrent neural Network for object detection

  2015cites this paper
• Training Recurrent Neural Networks With the Levenberg–Marquardt Algorithm for Optimal Control of a Grid-Connected Converter

  2015cites this paper
• Heuristic dynamic programming with internal goal representation Report Title

  2015influential citation
• Cellular recurrent deep network for image registration

  2015cites this paper
• Goal representation adaptive dynamic programming for machine intelligence

  2015cites this paper
• A boundedness theoretical analysis for GrADP design: A case study on maze navigation

  2015cites this paper
• Application of Artificial Neural Networks for Modelling of Nicolsky-Eisenman Equation and Determination of Ion Activities in Mixtures

  2014cites this paper
• Using recurrent networks for non-temporal classification tasks

  2014cites this paper
• FACIAL RECOGNITION BASED ON BACK PROPAGATION TECHNIQUE

  2014cites this paper
• Chaotic neocortical dynamics.

  2013cites this paper
• Goal Representation Heuristic Dynamic Programming on Maze Navigation

  2013cites this paper
• RETRACTED ARTICLE: A hierarchical learning architecture with multiple-goal representations and multiple timescale based on approximate dynamic programming

  2013cites this paper
• Learning topological image transforms using cellular simultaneous recurrent networks

  2013cites this paper
• Supervised Neural Network using Maximum-Margin (MM) Principle

  2013cites this paper
• Heuristic dynamic programming with internal goal representation

  2013cites this paper
• Common Nature of Learning Between Back-Propagation and Hopfield-Type Neural Networks for Generalized Matrix Inversion With Simplified Models

  2013cites this paper
• Fuzziness, Probability, Uncertainty and the Foundations of Knowledge

  2013cites this paper
• Reinforcement Learning and Approximate Dynamic Programming (RLADP)Â -Â Foundations, Common Misconceptions, and the Challenges Ahead

  2013cites this paper
• Large pose invariant face recognition using feature-based recurrent neural network

  2012cites this paper
• Embedding evolutionary strategy in ordinal optimization for hard optimization problems

  2012cites this paper
• Metamodeling and the Critic-based approach to multi-level optimization

  2012cites this paper
• Estimation Aspects of Signal Spectral Components Using Neural Networks

  2012cites this paper
• Decentralized Asynchronous Learning in Cellular Neural Networks

  2012cites this paper
• Modified cellular simultaneous recurrent networks with cellular particle swarm optimization

  2012cites this paper
• Accelerating Pattern Recognition Algorithms On Parallel Computing Architectures

  2011influential citation
• Metamodeling for large-scale optimization tasks based on object networks

  2011cites this paper
• GPGPU acceleration of Cellular Simultaneous Recurrent Networks adapted for maze traversals

  2011cites this paper
• Local Context Discrimination in Signature Neural Networks

  2011cites this paper
• Adaptive Dynamic Programming for Machine Intelligence

  2011cites this paper
• Handwritten Numeral Recognition Using Modified BP ANN Structure

  2011cites this paper
• Accelerating CSRN based face recognition on an NVIDIA GPGPU

  2011influential citation
• An online actor-critic learning approach with Levenberg-Marquardt algorithm

  2011cites this paper
• Signature Neural Networks: Definition and Application to Multidimensional Sorting Problems

  2011cites this paper
• Neural networks as a path to self-awareness

  2011cites this paper
• Nonlinear model identification and adaptive model predictive control using neural networks.

  2011cites this paper
• Large-scale pose-invariant face recognition using cellular simultaneous recurrent network.

  2010cites this paper
• Recursive and Non-Recursive Algorithms for the Group Size Counting Problem in Computer Go

  2010cites this paper
• Optimization via efficient learning in CNNs: Cognitively-motivated temporal discount functions in SRNNs

  2010cites this paper
• Improved learning in grid-to-grid neural network via clustering

  2010cites this paper
• Novel Maximum-Margin Training Algorithms for Supervised Neural Networks

  2010cites this paper
• Cognitively Inspired Neural Network for Recognition of Situations

  2010cites this paper
• Cognitively Inspired Neural Network for Situation Recognition

  2010cites this paper
• Uncovering the Hidden Issues in E-Government Adoption in a Least Developed Country: The Case of Bangladesh

  2010cites this paper
• A hierarchical learning architecture with multiple-goal representations based on adaptive dynamic programming

  2010cites this paper
• Self-organization in CNN-based Object Nets

  2010influential citation
• Optimal Feed Forward MLPArchitecture for Off-Line Cursive Numeral Recognition

  2010cites this paper
• Performance of generalized multilayered perceptons trained using the Levenberg-Marquardt method

  2009cites this paper
• Recurrent network-based face recognition using image sequences

  2009cites this paper
• Pose invariant face recognition using Cellular Simultaneous Recurrent Networks

  2009cites this paper
• Neural Network for Graphs: A Contextual Constructive Approach

  2009cites this paper
• A linear ridgelet network

  2009cites this paper
• Putting more brain-like intelligence into the electric power grid: What we need and how to do it

  2009cites this paper
• Clustering for Improved Learning in Maze Traversal Problem

  2009cites this paper
• Performance of generalized multi-layered perceptrons and layered arbitrarily connected networks trained using the Levenberg-Marquardt method

  2009cites this paper
• R-POPTVR: A Novel Reinforcement-Based POPTVR Fuzzy Neural Network for Pattern Classification

  2009cites this paper
• 2009 Special Issue: Intelligence in the brain: A theory of how it works and how to build it

  2009cites this paper
• Prediction of Parametric Value of Drinking Water of Hyderabad City by Artificial Neural Network Modeling

  2008cites this paper
• Recurrent Neural Networks

  2008cites this paper