Maxout Networks

We consider the problem of designing models to leverage a recently introduced approximate model averaging technique called dropout. We define a simple new model called maxout (so named because its output is the max of a set of inputs, and because it is a natural companion to dropout) designed to both facilitate optimization by dropout and improve the accuracy of dropout's fast approximate model averaging technique. We empirically verify that the model successfully accomplishes both of these tasks. We use maxout and dropout to demonstrate state of the art classification performance on four benchmark datasets: MNIST, CIFAR-10, CIFAR-100, and SVHN.

• Publication year

  2013

• Venue

  International Conference on Machine Learning

• Publication date

  2013-02-18

• Fields of study

  Mathematics, Computer Science

• Identifiers
  arXiv 1302.4389
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• Maxout and dropout achieve state-of-the-art classification performance on MNIST, CIFAR-10, CIFAR-100, and SVHN.

  Confidence 0.95

  박진우 (dztg5apj7m) extractionAnonymous (12632b8b5f) review
• Maxout and dropout are reported to make optimization easier under dropout while also improving the accuracy of dropout's approximate model averaging procedure.

  Confidence 0.91

  박진우 (dztg5apj7m) extractionAnonymous (12632b8b5f) review
• Maxout is introduced as a simple model built around taking the maximum over a set of inputs, with the aim of working well with dropout and improving dropout's fast approximate model averaging.

  Confidence 0.96

  박진우 (dztg5apj7m) extractionAnonymous (12632b8b5f) review

• approximate model averaging

  method, inference technique

  A fast technique for approximating the prediction of an ensemble of many dropout-thinned models.

  Aliases: fast approximate model averaging

  박진우 (dztg5apj7m) extractionAnonymous (12632b8b5f) review
• classification performance

  evaluation metric

  The quality of a model's class prediction accuracy on evaluation data.

  Aliases: classification accuracy

  박진우 (dztg5apj7m) extractionAnonymous (12632b8b5f) review
• dropout

  training technique, regularization method

  A training and approximate model-averaging technique that randomly omits units during learning.

  Aliases: dropout regularization

  박진우 (dztg5apj7m) extractionAnonymous (12632b8b5f) review
• maxout

  model, neural network component

  A neural network unit that outputs the maximum value from a set of learned inputs.

  Aliases: maxout networks, maxout unit

  박진우 (dztg5apj7m) extractionAnonymous (12632b8b5f) review
• mnist, cifar-10, cifar-100, and svhn

  benchmark datasets

  Four standard image classification benchmarks used to evaluate the reported models.

  Aliases: MNIST, CIFAR-10, CIFAR-100, SVHN

  박진우 (dztg5apj7m) extractionAnonymous (12632b8b5f) review
• optimization

  process

  The process of fitting the model parameters during training.

  Aliases: training optimization

  박진우 (dztg5apj7m) extractionAnonymous (12632b8b5f) review

• Joint Training Deep Boltzmann Machines for Classification

  2013cited by this paper
• Stochastic Pooling for Regularization of Deep Convolutional Neural Networks

  2013cited by this paper
• Improving Neural Networks with Dropout

  2013cited by this paper
• Learnable Pooling Regions for Image Classification

  2013cited by this paper
• Improving neural networks by preventing co-adaptation of feature detectors

  2012influential reference
• Practical Bayesian Optimization of Machine Learning Algorithms

  2012cited by this paper
• Joint Training of Deep Boltzmann Machines

  2012cited by this paper
• Convolutional neural networks applied to house numbers digit classification

  2012cited by this paper
• Theano: new features and speed improvements

  2012cited by this paper
• ImageNet classification with deep convolutional neural networks

  2012cited by this paper
• Deep Convex Net: A Scalable Architecture for Speech Pattern Classification

  2011cited by this paper
• The Manifold Tangent Classifier

  2011cited by this paper
• Reading Digits in Natural Images with Unsupervised Feature Learning

  2011cited by this paper
• Deep Sparse Rectifier Neural Networks

  2011cited by this paper
• Deep, Big, Simple Neural Nets for Handwritten Digit Recognition

  2010cited by this paper
• What is the best multi-stage architecture for object recognition?

  2009cited by this paper
• Deep Boltzmann Machines

  2009cited by this paper
• Learning Multiple Layers of Features from Tiny Images

  2009influential reference
• Rank, Trace-Norm and Max-Norm

  2005cited by this paper
• General constructive representations for continuous piecewise-linear functions

  2004influential reference
• On the piecewise analysis of networks of linear threshold neurons

  1998cited by this paper
• Gradient-based learning applied to document recognition

  1998cited by this paper
• Bagging Predictors

  1996influential reference
• A model of multiplicative neural responses in parietal cortex.

  1996cited by this paper

• Swish-T: Enhancing Swish Activation With Tanh-Based Bias for Improved Neural Network Performance

  2026cites this paper
• Exploring cultural commonsense in multilingual large language models: A survey

  2026cites this paper
• Adversarially robust neural network decision boundaries via tropical geometry.

  2026cites this paper
• Numerical exploration of the range of shape functionals using neural networks

  2026cites this paper
• Rational ANOVA Networks

  2026cites this paper
• On the Expressive Power of Transformers for Maxout Networks and Continuous Piecewise Linear Functions

  2026cites this paper
• ACD2W-InceptionNeXt: adjacent class distinguished and class distance weighted InceptionNeXt-based computer-aided mayo endoscopic scoring system for still images and video segments

  2026cites this paper
• A comprehensive review of convolutional neural networks: foundations, enhancements and applications

  2026cites this paper
• CalamanCy: A Tagalog Natural Language Processing Toolkit

  2025cites this paper
• A Method Based on Heterogeneous Graph and Contrastive Learning in Named Entity Recognition

  2025cites this paper
• A Fast Neural Architecture Search Method for Multi-Modal Classification via Knowledge Sharing

  2025cites this paper
• RoSwish: A novel Rotating Swish activation function with adaptive rotation around zero

  2025cites this paper
• Geometric Mixture Classifier (GMC): A Discriminative Per-Class Mixture of Hyperplanes

  2025cites this paper
• A Framework Based on Graph Cellular Automata for Similarity Evaluation in Urban Spatial Networks

  2025cites this paper
• On the Representation of Piecewise Quadratic Functions by Neural Networks

  2025cites this paper
• DropKAN: Dropout Kolmogorov–Arnold Networks

  2025cites this paper
• Activation Control of Multiple Piecewise Linear Neural Networks

  2025cites this paper
• A well‐posed variational approach to the identification and convergent approximation of material laws from boundary data

  2025cites this paper
• Multi-Scale Features Are Effective for Multi-Modal Classification: An Architecture Search Viewpoint

  2025cites this paper
• Introducing Euclidean distance optimization into Softmax loss under neural collapse

  2025cites this paper
• A discrete convolutional network for entity relation extraction

  2025cites this paper
• On the Upper Bounds of Number of Linear Regions and Generalization Error of Deep Convolutional Neural Networks

  2025cites this paper
• Advancing Constrained Monotonic Neural Networks: Achieving Universal Approximation Beyond Bounded Activations

  2025cites this paper
• A demonstration on the construction of modular neural network using elevator system that operates based on reinforcement learning

  2025cites this paper
• Mechanical Property Prediction of Materials via a Literature Mining Approach

  2025cites this paper
• Automatic Extraction of Road Networks by Using Teacher–Student Adaptive Structural Deep Belief Network and Its Application to Landslide Disaster

  2025cites this paper
• Integrating Task-Specific and Universal Adapters for Pre-Trained Model-based Class-Incremental Learning

  2025cites this paper
• Research on Optimization of Cigarette Stick Weight Parameters of Cigarette Machine Based on Neural Network

  2025cites this paper
• DeepGESI: A Non-Intrusive Objective Evaluation Model for Predicting Speech Intelligibility in Hearing-Impaired Listeners

  2025cites this paper
• Multifractal Recalibration of Neural Networks for Medical Imaging Segmentation

  2025cites this paper
• Low-Power CMOS Design of ReLU Activation Module for Neural Networks

  2025cites this paper
• Training Deep Morphological Neural Networks as Universal Approximators

  2025cites this paper
• MLP Fusion: Revisiting Convolutional Networks with Transformer-Based Insights

  2025cites this paper
• Modality-Aware SAM: Sharpness-Aware-Minimization Driven Gradient Modulation for Harmonized Multimodal Learning

  2025cites this paper
• LoTAS: A Novel Activation Function for Deep Neural Networks

  2025cites this paper
• View-invariant person re-identification in camera network by pose-aware AE-GAN and attention-driven feature fusion

  2025cites this paper
• Near-optimal delta-convex estimation of Lipschitz functions

  2025cites this paper
• An Information-Theoretic Lower Bound on the Generalization Error of Autoencoders

  2025cites this paper
• Deep embedded lightweight CNN network for indoor objects detection on FPGA

  2025cites this paper
• A Highly Nonlinear Survival Network for Hospital Readmission Prediction of Cardiac Patients

  2025cites this paper
• Virtual Reality‐Based Attention Prediction Model in Gaming for Autistic Children

  2025cites this paper
• Self-Supervised Neural Architecture Search for Multimodal Deep Neural Networks

  2025cites this paper
• Efficient Reachability Analysis for Convolutional Neural Networks Using Hybrid Zonotopes

  2025cites this paper
• Binary Lightweight Neural Networks for Arbitrary Scale Super-Resolution of Remote Sensing Images

  2025cites this paper
• A Neural Network-Based Approach to Determining the Mechanical Design Dimensions of Asynchronous Machines

  2025cites this paper
• PEAFusion: Parameter-efficient Adaptation for RGB-Thermal fusion-based semantic segmentation

  2025cites this paper
• Efficient Reinforcement Learning by Reducing Forgetting with Elephant Activation Functions

  2025cites this paper
• A mixed-integer framework for analyzing neural network-based controllers for piecewise affine systems with bounded disturbances

  2025cites this paper
• Feature and classifier-level domain adaptation in DistilHuBERT for cross-corpus speech emotion recognition

  2025cites this paper
• V2X-MGHD: A Collaborative Perception Network for Multiview LiDAR Sensors via Masked Generative Heterogeneous Distillation

  2025cites this paper
• Representation Learning in Complex Logical Query Answering on Knowledge Graphs: A Survey

  2025cites this paper
• Learning to Associate: Multimodal Inference with Fully Missing Modalities

  2025cites this paper
• CPD-KD: a cooperative perception network for discrepancy feature fusion through knowledge distillation

  2025cites this paper
• RoCooper: Robust Cooperative Perception Under Vehicle-to-Vehicle Communication Impairments

  2025cites this paper
• Leveraging environmental information for enhanced prediction of cardiac readmissions

  2025cites this paper
• Corn and Weed Object Detection Based on Improverd YOLOv8n

  2025cites this paper
• Enhancing Multimodal Learning via Hierarchical Fusion Architecture Search With Inconsistency Mitigation

  2025cites this paper
• STReLU: Spatio–Temporal Rectified Linear Unit for Traffic Forecasting

  2025cites this paper
• STCON NIST SRE24 System: Composite Speaker Recognition Solution for Challenging Scenarios

  2025cites this paper
• Sparse Hybrid Linear-Morphological Networks

  2025cites this paper
• SwinUFormer: A U-Net based on Swin Transformer for Ultrasound Imaging

  2025cites this paper
• Efficient Non-Blind Image Deblurring With Discriminative Shrinkage Deep Networks

  2025influential citation
• Neuroplasticity in Artificial Intelligence - An Overview and Inspirations on Drop In & Out Learning

  2025cites this paper
• P-SiLU: a trigonometric-based activation function for enhanced convolutional neural networks

  2025cites this paper
• Mamba Modulation: On the Length Generalization of Mamba

  2025cites this paper
• Hölder network for improved adversarial robustness

  2025cites this paper
• On the expressivity of sparse maxout networks

  2025cites this paper
• Convergence of Gradient Descent for Recurrent Neural Networks: A Nonasymptotic Analysis

  2024cites this paper
• Exploring the Relationship: Transformative Adaptive Activation Functions in Comparison to Other Activation Functions

  2024cites this paper
• Efficient learning of Scale-Adaptive Nearly Affine Invariant Networks

  2024cites this paper
• CompeteSMoE - Effective Training of Sparse Mixture of Experts via Competition

  2024cites this paper
• On the Number of Linear Regions of Convolutional Neural Networks With Piecewise Linear Activations

  2024cites this paper
• Achieving Sharp Upper Bounds on the Expressive Power of Neural Networks via Tropical Polynomials

  2024cites this paper
• Boundedness and Convergence of Mini-batch Gradient Method with Cyclic Dropconnect and Penalty

  2024cites this paper
• Tropical Decision Boundaries for Neural Networks Are Robust Against Adversarial Attacks

  2024cites this paper
• Adaptive deep neural networks for solving corner singular problems

  2024cites this paper
• Energy Supply Control of Wireless Powered Piecewise Linear Neural Network

  2024cites this paper
• Enhancing Educational Insights Through Convolutional Neural Networks: A Comparative Analysis on CIFAR-10 and CIFAR-100 Datasets

  2024cites this paper
• LanguageFlow: Advancing Diffusion Language Generation with Probabilistic Flows

  2024cites this paper
• Order Theory in the Context of Machine Learning: an application

  2024cites this paper
• Real-Time Multi-Task Environmental Perception System for Traffic Safety Empowered by Edge Artificial Intelligence

  2024cites this paper
• AttCON-Homo: Attention and PLMs-enhanced Neural Networks for Predicting Inter-chain Contacts and Distances in Homo-oligomeric Protein Complexes

  2024cites this paper
• Landmine detection with a mobile application

  2024cites this paper
• B-cosification: Transforming Deep Neural Networks to be Inherently Interpretable

  2024cites this paper
• Fully Automatic Bayesian Method for Trainable Activation Function and Deep Neural Networks

  2024cites this paper
• Reachability analysis for piecewise affine systems with neural network-based controllers

  2024cites this paper
• V2X-ViTv2: Improved Vision Transformers for Vehicle-to-Everything Cooperative Perception

  2024cites this paper
• Feature Stacking Fusion in Multimodal Neural Architecture Search

  2024cites this paper
• Neural networks to approximate capacity consumption in lot-sizing models

  2024cites this paper
• Skewed Neuronal Heterogeneity Enhances Efficiency On Various Computing Systems

  2024cites this paper
• LAND USE CLASSIFICATION OF SATELLITE IMAGES WITH CONVOLUTIONAL NEURAL NETWORKS (CNNS)

  2024cites this paper
• Dynamic Activation Function Based on the Branching Process and its Application in Image Classification

  2024cites this paper
• Security risks and countermeasures of adversarial attacks on AI-driven applications in 6G networks: A survey

  2024cites this paper
• Image Discrimination and Parameter Analysis Based on Convolutional Neural Networks (CNN)

  2024cites this paper
• Shrinkage Initialization for Smooth Learning of Neural Networks

  2024cites this paper
• Learning adversarially robust kernel ensembles with kernel average pooling

  2024cites this paper
• Image augmentation approaches for small and tiny object detection in aerial images: a review

  2024cites this paper
• Speech Signal Recognition for Parkinson's Disease Diagnosis Based on Heterogeneous Broad Balanced Domain Adaptation

  2024cites this paper
• Sampling Foundational Transformer: A Theoretical Perspective

  2024cites this paper
• Fixed-Time Adaptive Parameter Estimation for Hammerstein Systems Subject to Dead-Zone

  2024cites this paper