The relative performance of ensemble methods with deep convolutional neural networks for image classification

Artificial neural networks have been successfully applied to a variety of machine learning tasks, including image recognition, semantic segmentation, and machine translation. However, few studies fully investigated ensembles of artificial neural networks. In this work, we investigated multiple widely used ensemble methods, including unweighted averaging, majority voting, the Bayes Optimal Classifier, and the (discrete) Super Learner, for image recognition tasks, with deep neural networks as candidate algorithms. We designed several experiments, with the candidate algorithms being the same network structure with different model checkpoints within a single training process, networks with same structure but trained multiple times stochastically, and networks with different structure. In addition, we further studied the overconfidence phenomenon of the neural networks, as well as its impact on the ensemble methods. Across all of our experiments, the Super Learner achieved best performance among all the ensemble methods in this study.

• Publication year

  2017

• Venue

  Journal of Applied Statistics

• Publication date

  2017-04-05

• Fields of study

  Medicine, Computer Science, Mathematics

• Identifiers
  DOI 10.1080/02664763.2018.1441383arXiv 1704.01664PMID 31631918PMCID PMC6800663
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar, PubMed

• No claims are published for this paper.

• No concepts are published for this paper.

• and s

  2019cited by this paper
• Using Super Learner Prediction Modeling to Improve High-dimensional Propensity Score Estimation

  2018cited by this paper
• Online cross‐validation‐based ensemble learning

  2018cited by this paper
• Stacked Generalization

  2017influential reference
• Propensity score prediction for electronic healthcare databases using super learner and high-dimensional propensity score methods

  2017cited by this paper
• Data-adaptive Inference of the Optimal Treatment Rule and its Mean Reward. The Masked Bandit

  2016cited by this paper
• Optimal Spatial Prediction Using Ensemble Machine Learning

  2016cited by this paper
• Residual Networks Behave Like Ensembles of Relatively Shallow Networks

  2016cited by this paper
• node2vec: Scalable Feature Learning for Networks

  2016cited by this paper
• Super-Learning of an Optimal Dynamic Treatment Rule

  2016cited by this paper
• Deep Learning

  2016cited by this paper
• Residual Networks are Exponential Ensembles of Relatively Shallow Networks

  2016cited by this paper
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

  2015influential reference
• Effective Approaches to Attention-based Neural Machine Translation

  2015cited by this paper
• Deep Residual Learning for Image Recognition

  2015influential reference
• Mortality prediction in intensive care units with the Super ICU Learner Algorithm (SICULA): a population-based study.

  2015cited by this paper
• Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification

  2015influential reference
• Dropout: a simple way to prevent neural networks from overfitting

  2014cited by this paper
• Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation

  2014cited by this paper
• The Loss Surfaces of Multilayer Networks

  2014influential reference
• DeepWalk: online learning of social representations

  2014cited by this paper
• ImageNet Large Scale Visual Recognition Challenge

  2014cited by this paper
• Mitosis detection in breast cancer pathology images by combining handcrafted and convolutional neural network features

  2014cited by this paper
• Cascaded ensemble of convolutional neural networks and handcrafted features for mitosis detection

  2014cited by this paper
• Very Deep Convolutional Networks for Large-Scale Image Recognition

  2014influential reference
• Going deeper with convolutions

  2014influential reference
• Network In Network

  2013cited by this paper
• Visualizing and Understanding Convolutional Networks

  2013cited by this paper
• ImageNet classification with deep convolutional neural networks

  2012cited by this paper
• Statistical Applications in Genetics and Molecular Biology Super Learner

  2011influential reference
• Super Learner In Prediction

  2010cited by this paper
• Learning Multiple Layers of Features from Tiny Images

  2009influential reference
• Hierarchical Ensemble of Global and Local Classifiers for Face Recognition

  2009cited by this paper
• Et al

  2008cited by this paper
• Super Learning: An Application to the Prediction of HIV-1 Drug Resistance

  2007cited by this paper
• Survival ensembles.

  2006cited by this paper
• Stacked regressions

  2004influential reference
• Unified Cross-Validation Methodology For Selection Among Estimators and a General Cross-Validated Adaptive Epsilon-Net Estimator: Finite Sample Oracle Inequalities and Examples

  2003cited by this paper
• Limits on the majority vote accuracy in classifier fusion

  2003cited by this paper
• Greedy function approximation: A gradient boosting machine.

  2001cited by this paper
• Random Forests

  2001cited by this paper
• Multiple Classifier Systems

  2000cited by this paper
• A decision-theoretic generalization of on-line learning and an application to boosting

  1997cited by this paper
• Bagging Predictors

  1996influential reference
• Experiments with a New Boosting Algorithm

  1996cited by this paper
• Stacked generalization

  1992cited by this paper
• A James-Stein Type Estimator for Combining Unbiased and Possibly Biased Estimators

  1991cited by this paper
• Combining Independent Normal Mean Estimation Problems with Unknown Variances

  1976cited by this paper
• THE VARIANCE OF A LINEAR COMBINATION OF INDEPENDENT ESTIMATORS USING ESTIMATED WEIGHTS

  1974cited by this paper
• Combining Possibly Related Estimation Problems

  1973cited by this paper
• Combining Independent Estimators and Estimation in Linear Regression with Unequal Variances

  1971cited by this paper

• Feasibility of predicting vertical cephalometric angles from panoramic radiographs using deep learning.

  2026cites this paper
• Ensemble deep learning architectures for detecting pulmonary tuberculosis in chest X-rays

  2026cites this paper
• Inverse design of photonic integrated devices: Optimization and machine learning

  2025cites this paper
• High-Fidelity Tuning of Olfactory Mixture Distances in the Perceptual Space of Smell Through a Community Effort

  2025cites this paper
• An accurate prediction of the physicochemical properties of traditional Chinese medicine granules using a multi-source data model fusion strategy based on deep ensemble learning algorithms

  2025cites this paper
• Decorrelating structure via adapters makes ensemble learning practical for Semi-supervised Learning

  2025cites this paper
• FedSC: Federated Learning with Semantic-Aware Collaboration

  2025cites this paper
• On the Optimal Ensemble of Distributed DNN Models

  2025cites this paper
• TransembleNet: Enhancing vector mosquito species classification through transfer learning-based ensemble model

  2025cites this paper
• Attention-Based Inception-Residual CNN: Skin Cancer Diagnosis with Attention-Based Inception-Residual CNN Model

  2025cites this paper
• Deep learning-based classification of benign and malignant breast microcalcifications in mammography

  2025cites this paper
• Tertiary lymphoid structure drives allograft rejection via IFN-γ-JAK-STAT-dependent atypical memory B cell differentiation

  2025cites this paper
• Residual Feature Integration is Sufficient to Prevent Negative Transfer

  2025cites this paper
• High-Throughput Estimation of Sugarcane Phenotypic Traits Using UAV Multispectral Data Under High-Density Planting Conditions

  2025cites this paper
• Leveraging transfer learning from Acute Lymphoblastic Leukemia (ALL) pretraining to enhance Acute Myeloid Leukemia (AML) prediction

  2025cites this paper
• Decision Strategies in AI-Based Ensemble Models in Opportunistic Alzheimer's Detection from Structural MRI.

  2025cites this paper
• Convergent multi-modular architecturefor adaptive learning in Drosophila and artificial intelligence

  2025cites this paper
• Ensemble of Efficient Vision Transformers for Insect Classification

  2025cites this paper
• TranSEF: Transformer Enhanced Self-Ensemble Framework for Damage Assessment in Canola Crops

  2025cites this paper
• EnsembleEdgeFusion: advancing semantic segmentation in microvascular decompression imaging with innovative ensemble techniques

  2025cites this paper
• High-throughput method for improving rice AGB estimation based on UAV multi-source remote sensing image feature fusion and ensemble learning

  2025cites this paper
• Propagation of Chaos for Mean-Field Langevin Dynamics and its Application to Model Ensemble

  2025cites this paper
• Ensemble learning using Gompertz function for leukemia classification

  2025cites this paper
• Brain Tumor Classification Based on Machine Learning and Deep Learning Technologies: A Comprehensive Review

  2025cites this paper
• Explainable Deep Ensemble Learning Framework for Lung Cancer Classification

  2025cites this paper
• FedDifRC: Unlocking the Potential of Text-to-Image Diffusion Models in Heterogeneous Federated Learning

  2025cites this paper
• Leveraging an ensemble of EfficientNetV1 and EfficientNetV2 models for classification and interpretation of breast cancer histopathology images

  2025influential citation
• Convolutional Neural Networks for Seizure Detection: A Study on Training Strategies

  2025cites this paper
• SynthSecureNet: An Improved Deep Learning Architecture with Application to Intelligent Violence Detection

  2025cites this paper
• FedSKC: Federated Learning with Non-IID Data via Structural Knowledge Collaboration

  2025cites this paper
• Trans-defense: Transformer-based Denoiser for Adversarial Defense with Spatial-Frequency Domain Representation

  2025cites this paper
• Metacognitive Sensitivity for Test-Time Dynamic Model Selection

  2025cites this paper
• Kinematic Skeleton Extraction from 3D Model Based on Hierarchical Segmentation

  2025cites this paper
• Radiomics-Driven Tumor Prognosis Prediction Across Imaging Modalities: Advances in Sampling, Feature Selection, and Multi-Omics Integration

  2025cites this paper
• Real-Time Efficiency Prediction in Nonlinear Fractional-Order Systems via Multimodal Fusion

  2025cites this paper
• zERExtractor:An Automated Platform for Enzyme-Catalyzed Reaction Data Extraction from Scientific Literature

  2025cites this paper
• High dimensional predictions of suicide risk in 4.2 million US Veterans using ensemble transfer learning

  2024cites this paper
• Blending Is All You Need: Cheaper, Better Alternative to Trillion-Parameters LLM

  2024cites this paper
• Toward Improving Ensemble-Based Collaborative Inference at the Edge

  2024influential citation
• DISSECT: deep semi-supervised consistency regularization for accurate cell type fraction and gene expression estimation

  2024cites this paper
• Multi-objective optimization of sealing structure of subsea pipeline connector based on developed FE model, sensitivity analysis, surrogate model and NSGA-II

  2024cites this paper
• A fundamental overview of ensemble deep learning models and applications: systematic literature and state of the art

  2024cites this paper
• Ensemble and Transformer Encoder-based Models for the Cervical Cancer Classification Using Pap-smear Images

  2024cites this paper
• A Learnable Prior Improves Inverse Tumor Growth Modeling

  2024cites this paper
• Applying the Forward-Forward Algorithm to Event-Based Sensing

  2024cites this paper
• A dynamic uncertainty-aware ensemble model: Application to lung cancer segmentation in digital pathology

  2024cites this paper
• Leveraging Sample Complementarity: A Novel Ensemble Strategy

  2024cites this paper
• Robust Model Selection for Plant Leaf Image Recognition Based on Evolutionary Ant Colony Optimization With Learning Rate Schedule

  2024cites this paper
• Ensemble CNN Models for Accurate Pneumonia Classification in Chest X-Ray Images

  2024cites this paper
• Enhancing Breast Cancer Survival Prognosis Through Omic and Non-Omic Data Integration.

  2024cites this paper
• Multi-Step-Ahead Time Series Forecasting using Deep Learning and Fuzzy Time Series-based Error Correction Method

  2024cites this paper
• Automated lung cancer detection using histopathological images

  2024cites this paper
• Priority-Encoder Ensemble for Speech Recognition

  2024cites this paper
• A fragmented neural network ensemble method and its application to image classification

  2024cites this paper
• Application of neural networks ensemble method for the Kazakh sign language recognition

  2024cites this paper
• Classification of Diabetic Retinopathy Severity Using Deep Learning Techniques on Retinal Images

  2024cites this paper
• Towards safe and reliable deep learning for lung nodule malignancy estimation using out-of-distribution detection

  2024cites this paper
• Development of the Demand Forecasting and Product Recommendation Method to Support the Small and Medium Distribution Companies based on the Product Recategorization

  2024cites this paper
• Automatic segmentation of deep endometriosis in the rectosigmoid using deep learning

  2024cites this paper
• Diffractive deep neural networks: Theories, optimization, and applications

  2024cites this paper
• Comprehensive hepatotoxicity prediction: ensemble model integrating machine learning and deep learning

  2024cites this paper
• An ensemble deep learning framework for foetal plane identification

  2024cites this paper
• Generalized Super-Resolution 4D Flow MRI - Using Ensemble Learning to Extend Across the Cardiovascular System

  2024cites this paper
• Accuracy Assessment and Reinforcement Training of Deep Neural Networks in the Context of Accelerating a Simulation-Based Toolpath Generation Method

  2024cites this paper
• Synergy and Diversity in CLIP: Enhancing Performance Through Adaptive Backbone Ensembling

  2024cites this paper
• phylaGAN: data augmentation through conditional GANs and autoencoders for improving disease prediction accuracy using microbiome data

  2024cites this paper
• Exploring Insights in Biomass and Waste Gasification via Ensemble Machine Learning Models and Interpretability Techniques

  2024cites this paper
• Early prediction of chronic kidney disease based on ensemble of deep learning models and optimizers

  2024cites this paper
• Decorrelating Structure via Adapters Makes Ensemble Learning Practical for Semi-supervised Learning

  2024cites this paper
• Improving soybean yield prediction by integrating UAV nadir and cross-circling oblique imaging

  2024cites this paper
• Just Say the Name: Online Continual Learning with Category Names Only via Data Generation

  2024cites this paper
• Ensemble learning for integrative prediction of genetic values with genomic variants

  2024cites this paper
• Solving a Typical Small Sample Size MRSM Dataset Problem Using a Flexible Hybrid Ensemble Approach for Credibility

  2024cites this paper
• Beyond Amyloid: A Machine Learning-Driven Approach Reveals Properties of Potent GSK-3β Inhibitors Targeting Neurofibrillary Tangles

  2024cites this paper
• Development and multi-institutional validation of a deep learning model for grading of vesicoureteral reflux on voiding cystourethrogram: a retrospective multicenter study

  2024cites this paper
• Convolutional Neural Networks for Glioma Segmentation and Prognosis: A Systematic Review.

  2024cites this paper
• Satellite Still Image Classification Using CNN

  2024cites this paper
• A Learnable Prior Improves Inverse Tumor Growth Modeling

  2024cites this paper
• A method of multidimensional software aging prediction based on ensemble learning: A case of Android OS

  2024cites this paper
• Ensemble Deep Learning for Enhanced Seismic Data Reconstruction

  2024cites this paper
• Improving cardiovascular risk prediction with machine learning: a focus on perivascular adipose tissue characteristics

  2024cites this paper
• End-to-End Optimization and Learning for Multiagent Ensembles

  2023cites this paper
• Machine-learning assisted swallowing assessment: a deep learning-based quality improvement tool to screen for post-stroke dysphagia

  2023cites this paper
• Deep-kidney: an effective deep learning framework for chronic kidney disease prediction

  2023cites this paper
• Classification of First-Episode Psychosis with EEG Signals: ciSSA and Machine Learning Approach

  2023cites this paper
• Hierarchical Pruning of Deep Ensembles with Focal Diversity

  2023cites this paper
• Generalized super-resolution 4D Flow MRI-using ensemble learning to extend across the cardiovascular system

  2023cites this paper
• Scene Text Recognition via Dual-path Network with Shape-driven Attention Alignment

  2023cites this paper
• Integrated radiogenomics models predict response to neoadjuvant chemotherapy in high grade serous ovarian cancer

  2023cites this paper
• Identification of Plant Disease Based on Multi-Task Continual Learning

  2023cites this paper
• Comparison of Different Methods for Building Ensembles of Convolutional Neural Networks

  2023cites this paper
• Semi-supervised generative adversarial networks with spatial coevolution for enhanced image generation and classification

  2023cites this paper
• Improved Crop and Weed Detection with Diverse Data Ensemble Learning

  2023cites this paper
• Automatic semantic segmentation of the lumbar spine: Clinical applicability in a multi-parametric and multi-center study on magnetic resonance images

  2023cites this paper
• An Efficient Deep Convolutional Neural Network for Automatic Detection of FDG Uptakes in Fused PET/CT Images

  2023cites this paper
• Decision Layer: Enhancing Multi-model, Multi Timescale Decisions on the Fly with Online Feedback

  2023cites this paper
• Faster, Lighter, More Accurate: A Deep Learning Ensemble for Content Moderation

  2023cites this paper
• Minimum Bayes’ Risk Decoding for System Combination of Grammatical Error Correction Systems

  2023cites this paper
• Hydraulic-Pump Fault-Diagnosis Method Based on Mean Spectrogram Bar Graph of Voiceprint and ResNet-50 Model Transfer

  2023cites this paper
• Modularity in Deep Learning: A Survey

  2023cites this paper