Benchmarking open source deep learning frameworks

Deep Learning (DL) is one of the hottest fields. To foster the growth of DL, several open source frameworks appeared providing implementations of the most common DL algorithms. These frameworks vary in the algorithms they support and in the quality of their implementations. The purpose of this work is to provide a qualitative and quantitative comparison among three such frameworks: TensorFlow, Theano and CNTK. To ensure that our study is as comprehensive as possible, we consider multiple benchmark datasets from different fields (image processing, NLP, etc.) and measure the performance of the frameworks' implementations of different DL algorithms. For most of our experiments, we find out that CNTK's implementations are superior to the other ones under consideration.

• Publication year

  2020

• Venue

  International Journal of Electrical and Computer Engineering (IJECE)

• Publication date

  2020-10-01

• Fields of study

  Computer Science

• Identifiers
  DOI 10.11591/ijece.v10i5.pp5479-5486
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• A detailed comparative study of open source deep learning frameworks

  2019cited by this paper
• A comparative study of open source deep learning frameworks

  2018cited by this paper
• DAWNBench : An End-to-End Deep Learning Benchmark and Competition

  2017cited by this paper
• Benchmarking State-of-the-Art Deep Learning Software Tools

  2016cited by this paper
• Deep learning with theano, torch, caffe, tensorflow, and deeplearning4J: which one is the best in speed and accuracy?

  2016cited by this paper
• TensorFlow: A system for large-scale machine learning

  2016cited by this paper
• Theano: A Python framework for fast computation of mathematical expressions

  2016cited by this paper
• A Tour of TensorFlow

  2016cited by this paper
• The Computational Network Toolkit [Best of the Web]

  2015cited by this paper
• Comparative Study of Deep Learning Software Frameworks

  2015cited by this paper
• An introduction to computational networks and the computational network toolkit (invited talk)

  2014cited by this paper
• Recurrent Neural Network Regularization

  2014cited by this paper
• Theano-based Large-Scale Visual Recognition with Multiple GPUs

  2014cited by this paper
• ImageNet classification with deep convolutional neural networks

  2012cited by this paper
• Theano: new features and speed improvements

  2012cited by this paper
• Learning Word Vectors for Sentiment Analysis

  2011cited by this paper
• Theano: A CPU and GPU Math Compiler in Python

  2010cited by this paper
• Learning Multiple Layers of Features from Tiny Images

  2009cited by this paper
• ImageNet: A large-scale hierarchical image database

  2009cited by this paper
• Gradient-based learning applied to document recognition

  1998cited by this paper
• Building a Large Annotated Corpus of English: The Penn Treebank

  1993cited by this paper
• First Results in Robot Road-Following

  1985cited by this paper

• Determining critical factors for the success of machine learning libraries considering fuzzy interrelationships

  2026cites this paper
• Accuracy Comparison Between Feedforward Neural Network, Support Vector Machine and Boosting Ensembles for Financial Risk Evaluation

  2025cites this paper
• The Engineer's Dilemma: A Review of Establishing a Legal Framework for Integrating Machine Learning in Construction by Navigating Precedents and Industry Expectations

  2025cites this paper
• Evaluating the deep learning software tools for large-scale enterprises using a novel TODIFFA-MCDM framework

  2024cites this paper
• A Comprehensive Crucial Review of Re-Purposing DNN-Based Systems: Significance, Challenges, and Future Directions

  2024cites this paper
• The Importance of High Speed Storage in Deep Learning Training

  2023cites this paper
• An experimental comparison of the widely used pre‐trained deep neural networks for image classification tasks towards revealing the promise of transfer‐learning

  2022cites this paper
• A powerful comparison of deep learning frameworks for Arabic sentiment analysis

  2021cites this paper
• Atar: Attention-based LSTM for Arabizi transliteration

  2021cites this paper
• Deepenz: prediction of enzyme classification by deep learning

  2021cites this paper