Deep, Big, Simple Neural Nets for Handwritten Digit Recognition

Good old online backpropagation for plain multilayer perceptrons yields a very low 0.35 error rate on the MNIST handwritten digits benchmark. All we need to achieve this best result so far are many hidden layers, many neurons per layer, numerous deformed training images to avoid overfitting, and graphics cards to greatly speed up learning.

• Publication year

  2010

• Venue

  Neural Computation

• Publication date

  2010-03-01

• Fields of study

  Mathematics, Computer Science, Medicine

• Identifiers
  DOI 10.1162/NECO_a_00052arXiv 1003.0358PMID 20858131
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar, PubMed

• No claims are published for this paper.

• No concepts are published for this paper.

• Accelerating Large-Scale Convolutional Neural Networks with Parallel Graphics Multiprocessors

  2010cited by this paper
• Deep Belief Networks for phone recognition

  2009cited by this paper
• Implicit Mixtures of Restricted Boltzmann Machines

  2008cited by this paper
• A trainable feature extractor for handwritten digit recognition

  2007influential reference
• Computational neuroscience : theoretical insights into brain function

  2007cited by this paper
• Learning a Nonlinear Embedding by Preserving Class Neighbourhood Structure

  2007cited by this paper
• To recognize shapes, first learn to generate images.

  2007cited by this paper
• Unsupervised Learning of Invariant Feature Hierarchies with Applications to Object Recognition

  2007influential reference
• Efficient Learning of Sparse Representations with an Energy-Based Model

  2006influential reference
• Greedy Layer-Wise Training of Deep Networks

  2006cited by this paper
• Supporting Online Material for Reducing the Dimensionality of Data with Neural Networks

  2006cited by this paper
• High Performance Convolutional Neural Networks for Document Processing

  2006cited by this paper
• Introduction to real-time ray tracing

  2005cited by this paper
• Using GPUs for machine learning algorithms

  2005cited by this paper
• Best practices for convolutional neural networks applied to visual document analysis

  2003influential reference
• Artificial intelligence - a modern approach, 2nd Edition

  2003cited by this paper
• Gradient Flow in Recurrent Nets: the Difficulty of Learning Long-Term Dependencies

  2001cited by this paper
• Gradient-based learning applied to document recognition

  1998influential reference
• Long Short-Term Memory

  1997cited by this paper
• Untersuchungen zu dynamischen neuronalen Netzen

  1991cited by this paper
• Learning internal representations by error propagation

  1986cited by this paper
• Learning Internal Representations by Error Propagation, Parallel Distributed Processing

  1986cited by this paper
• Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations

  1986cited by this paper
• Une procedure d'apprentissage pour reseau a seuil asymmetrique (A learning scheme for asymmetric threshold networks)

  1985cited by this paper
• Beyond Regression : "New Tools for Prediction and Analysis in the Behavioral Sciences

  1974influential reference
• c ○ 2002 Kluwer Academic Publishers. Manufactured in The Netherlands. Training Invariant Support Vector Machines

  year unknowninfluential reference

• Learning-based approaches for wireless PHY layers from the perspective of conventional machine learning to foundation models: A comprehensive survey

  2026cites this paper
• Secondary use of health data: applications, models, algorithms, and ethical considerations

  2026cites this paper
• Denoising Telerik RadCaptcha: A Comparative Evaluation of the Effectiveness of Pre-Processing Techniques and Deep Learning Methods Using a Novel Dataset

  2026cites this paper
• LogSumExp: Efficient Approximate Logarithm Acceleration for Embedded Tractable Probabilistic Reasoning

  2026influential citation
• Heuristic selection through neural networks: an extended analysis on the pod allocation problem within robotic mobile fulfillment systems

  2026cites this paper
• Comparison of metakaolin blended mortar compressive strength from multi non-linear regression analysis, artificial neural network and experimental work

  2026cites this paper
• A Semi-Supervised-Learning-Aided Explainable Belief Rule-Based Approach to Predict the Energy Consumption of Buildings

  2025cites this paper
• Embedding physical symmetries into machine-learned reduced plasma physics models via data augmentation

  2025cites this paper
• Oya: Deep Learning for Accurate Global Precipitation Estimation

  2025cites this paper
• Deep-to-broad technique to accelerate deep neural networks

  2025cites this paper
• Spike-based time-domain analog weighted-sum calculation model for extremely low power VLSI implementation of multi-layer neural networks

  2025cites this paper
• Nondestructive reading technique and refreshment circuit for symmetric and asymmetric stochastic memristors

  2025cites this paper
• AI-Driven Equalization for Compressive Optical Fronthaul in Next-generation Radio Access Network (RAN)

  2025cites this paper
• Mitigating Overfitting in Fine-Pattern Quality Classification: Enabling Automated Projection Image Quality Inspection

  2025cites this paper
• State of health estimation of lithium-ion battery cell based on optical thermometry with physics-informed machine learning

  2025cites this paper
• AI Safety Assurance for Automated Vehicles: A Survey on Research, Standardization, Regulation

  2025cites this paper
• LiCAR: pseudo-RGB LiDAR image for CAR segmentation

  2025cites this paper
• Eucalyptus diameter and volume prediction with deep neural networks: A Long Short-Term Memory model approach

  2025cites this paper
• A metatheory of classical and modern connectionism.

  2025cites this paper
• Dual discriminator GAN-based synthetic crop disease image generation for precise crop disease identification

  2025cites this paper
• On projective stochastic-gradient type methods for solving large scale systems of nonlinear ill-posed equations: applications to machine learning

  2025influential citation
• Automation and Optimization of Manual Marks Entry with Smart Scanning App

  2025cites this paper
• Bitcoin Price Prediction Using LSTM Networks: A Deep Learning Approach

  2025cites this paper
• Artificial Intelligence in Pulmonary Imaging: A Review of Techniques

  2025cites this paper
• AdaptoVision: A Multi-Resolution Image Recognition Model for Robust and Scalable Classification

  2025cites this paper
• A Survey of Sampling Methods for Hyperspectral Remote Sensing: Addressing Bias Induced by Random Sampling

  2025cites this paper
• Improved YOLOv5 network for recognizing fuzzy image detection

  2025cites this paper
• Advantage of Different Data Dimensions and Dual-Beam for Radar Low Resolution Doppler Classification

  2025cites this paper
• Students' Conceptual Explanations of Neural Networks Enabled by Extended Reality Learning: A Multiple Methods Approach

  2025cites this paper
• Breaking the Performance Gap of Fully and Semisupervised Learning in Electromagnetic Signature Recognition

  2024cites this paper
• A self-supervised learning framework based on masked autoencoder for complex wafer bin map classification

  2024cites this paper
• TCP-RBA: Semi-supervised learning for traditional chinese painting classification with random brushwork augment

  2024cites this paper
• BNVGLENET: Hypercomplex Bangla handwriting character recognition with hierarchical class expansion using Convolutional Neural Networks

  2024cites this paper
• Fine Granularity Is Critical for Intelligent Neural Network Pruning

  2024cites this paper
• Resource-efficient Medical Image Analysis with Self-adapting Forward-Forward Networks

  2024cites this paper
• An Advanced Explainable Belief Rule-Based Framework to Predict the Energy Consumption of Buildings

  2024cites this paper
• Towards a More Robust and Accurate OCR Model with Adversarial Techniques in HMI Testing Scenarios

  2024cites this paper
• Chemical signature characterization with hyperspectral imagery: novel deep learning model architectures and physically-motivated data augmentation techniques

  2024cites this paper
• Density Vision: Crowd Classification using Deep Learning

  2024cites this paper
• Mathematical Engineering of Deep Learning

  2024cites this paper
• Highway Value Iteration Networks

  2024cites this paper
• A deep neural network based surrogate model for damage identification in full-scale structures with incomplete noisy measurements

  2024cites this paper
• Using particle size distributions to identify indoor emissions: a machine learning method for source recognition

  2024cites this paper
• Monitoring framework development for a network of multiple laboratory structures

  2024cites this paper
• Low Doppler Resolution Radar-Based Target Classification for Clutter Suppression

  2024cites this paper
• A Comparative Study of Open Source Computer Vision Models for Application on Small Data: The Case of CFRP Tape Laying

  2024cites this paper
• A review of research and development of semi-supervised learning strategies for medical image processing

  2024cites this paper
• Towards Integration of Artificial Intelligence into Medical Devices as a Real-Time Recommender System for Personalised Healthcare: State-of-the-Art and Future Prospects

  2024cites this paper
• CNN Architectures for Image Classification: A Comparative Study Using ResNet50V2, ResNet152V2, InceptionV3, Xception, and MobileNetV2

  2024cites this paper
• Fault Identification of Rotating Machinery Based on Dynamic Feature Reconstruction Signal Graph

  2024cites this paper
• Byzantine-robust Federated Learning via Cosine Similarity Aggregation

  2024cites this paper
• Deep-ER: Deep Learning ECCENTRIC Reconstruction for fast high-resolution neurometabolic imaging

  2024cites this paper
• A Theoretically Grounded Extension of Universal Attacks from the Attacker's Viewpoint

  2024cites this paper
• A Novel Deep Learning Method for Segmenting the Left Ventricle in Cardiac Cine MRI

  2024cites this paper
• Analyzing and Filtering Multimedia Digital Evidence with Artificial Intelligence

  2023cites this paper
• Optimization of Sensors for Structure Damage Detection Using Deep Learning Approach

  2023cites this paper
• Handwritten Character Recognition Using Machine Learning

  2023cites this paper
• Convolutional Neural Network Model for Traffic Sign Recognition

  2023cites this paper
• Pruning Distorted Images in MNIST Handwritten Digits

  2023cites this paper
• Survey on Activation Functions for Optical Neural Networks

  2023cites this paper
• Exploding the myths: An introduction to artificial neural networks for prediction and forecasting

  2023cites this paper
• Performing Melanoma Diagnosis by an Effective Multi-view Convolutional Network Architecture

  2023cites this paper
• SAAL: Sharpness-Aware Active Learning

  2023cites this paper
• A Graph Reconstruction by Dynamic Signal Coefficient for Fault Classification

  2023cites this paper
• Towards a bilipschitz invariant theory

  2023influential citation
• Intelligent Tools and Whole-process Control System Based on Artificial Intelligence and Internet of Things Technology

  2023cites this paper
• A fully-automatic semi-supervised deep learning model for difficult airway assessment

  2023cites this paper
• Adopting Two Supervisors for Efficient Use of Large-Scale Remote Deep Neural Networks

  2023cites this paper
• Classification of lemon quality using hybrid model based on Stacked AutoEncoder and convolutional neural network

  2023cites this paper
• Progressive growing of Generative Adversarial Networks for improving data augmentation and skin cancer diagnosis

  2023cites this paper
• A neural network for phenotyping Fusarium‐damaged kernels (FDKs) in wheat and its impact on genomic selection accuracy

  2023cites this paper
• Machine Learning in Unmanned Systems for Chemical Synthesis

  2023cites this paper
• Spiking neural P systems with polarizations and astrocytes

  2023cites this paper
• Deep Learning Based Distributed Meta-Learning for Fast and Accurate Online Adaptive Powertrain Fuel Consumption Modeling

  2023cites this paper
• Toward Federated Learning With Byzantine and Inactive Users: A Game Theory Approach

  2023influential citation
• TT-MLP: Tensor Train Decomposition on Deep MLPs

  2023cites this paper
• Learning sparse and smooth functions by deep Sigmoid nets

  2023cites this paper
• Hardware Prototyping of Handwritten Character Recognition using VEGA Soft Core Processor

  2023cites this paper
• Evaluating the Performance of Deep Learning Models in Handwritten Digit Recognition

  2023cites this paper
• A Comparative Study of Machine Learning and Deep Learning Algorithm for Handwritten Digit Recognition

  2023cites this paper
• A semi-supervised learning approach with consistency regularization for tumor histopathological images analysis

  2023cites this paper
• The influence of different parameters on classical neural network fitting

  2023cites this paper
• A Deep Learning Approach for Classification of Medicare Beneficiaries Based on Gender and being Affected with Cancer

  2023cites this paper
• Toward a Brain-Inspired Theory of Artificial Learning

  2023cites this paper
• Machine learning insight into h-BN growth on Pt(111) from atomic states

  2023cites this paper
• A Biologist’s Guide to the Galaxy: Leveraging Artificial Intelligence and Very High-Resolution Satellite Imagery to Monitor Marine Mammals from Space

  2023cites this paper
• The Performance of a Lip-Sync Imagery Model, New Combinations of Signals, a Supplemental Bond Graph Classifier, and Deep Formula Detection as an Extraction and Root Classifier for Electroencephalograms and Brain–Computer Interfaces

  2023cites this paper
• Prediction of Gender-Biased Perceptions of Learners and Teachers Using Machine Learning

  2023cites this paper
• Robustness of Visual Explanations to Common Data Augmentation Methods

  2023cites this paper
• Quantum machine learning for image classification

  2023cites this paper
• Caenorhabditis elegans Connectomes of both Sexes as Image Classifiers

  2023cites this paper
• Transforming Information Extraction: AI and Machine Learning in Optical Character Recognition Systems and Applications Across Industries

  2023cites this paper
• Emergent representations in networks trained with the Forward-Forward algorithm

  2023cites this paper
• High-Resolution Volumetric Imaging and Classification of Organisms with Standard Optical Microscopy.

  2023cites this paper
• Using CNN-LSTMs and Transformer RNNs for COVID19 Impact Prediction

  2023cites this paper
• Self-ChakmaNet: A deep learning framework for indigenous language learning using handwritten characters

  2023cites this paper
• N EUROEVOLUTION T ICKET S EARCH — F INDING S PARSE , T RAINABLE DNN I NITIALISATIONS

  2023cites this paper
• Finding Sparse Initialisations using Neuroevolutionary Ticket Search (NeTS)

  2023cites this paper
• Pretrained back propagation based adaptive resonance theory network for adaptive learning

  2023cites this paper
• Concrete and steel bridge Structural Health Monitoring—Insight into choices for machine learning applications

  2023cites this paper