Survey on Improved Scheduling in Hadoop MapReduce in Cloud Environments

Cloud Computing is emerging as a new computational paradigm shift. Hadoop-MapReduce has become a powerful Computation Model for processing large data on distributed commodity hardware clusters such as Clouds. In all Hadoop implementations, the default FIFO scheduler is available where jobs are scheduled in FIFO order with support for other priority based schedulers also. In this paper we study various scheduler improvements possible with Hadoop and also provided some guidelines on how to improve the scheduling in Hadoop in Cloud Environments.

• Publication year

  2012

• Venue

  arXiv.org

• Publication date

  2012-07-03

• Fields of study

  Computer Science, Engineering

• Identifiers
  arXiv 1207.0780
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Genetic Algorithm for Energy Efficient Placement of Virtual Machines in MapReduce Based Cloud Environment

  2012cited by this paper
• Performance Issues of Heterogeneous Hadoop Clusters in Cloud Computing

  2012cited by this paper
• The Hadoop Distributed File System

  2010influential reference
• Delay scheduling: a simple technique for achieving locality and fairness in cluster scheduling

  2010cited by this paper
• Scheduling Hadoop Jobs to Meet Deadlines

  2010cited by this paper
• Dynamic Proportional Share Scheduling in Hadoop

  2010cited by this paper
• MapReduce: simplified data processing on large clusters

  2008influential reference
• Improving MapReduce Performance in Heterogeneous Environments

  2008cited by this paper
• The Google file system

  2003cited by this paper

• Towards High-Performance Data Loading in Cloud-Native Deep Learning Systems

  2024cites this paper
• An emergency task scheduling method based on YARN capacity scheduler

  2024cites this paper
• Performance of Distributed File Systems on Cloud Computing Environment: An Evaluation for Small-File Problem

  2023cites this paper
• Big Data Complexity, Dependability and Emerging Applications

  2023cites this paper
• Past, Present and Future of Hadoop: A Survey

  2022cites this paper
• Improving Hadoop Performance through scheduling and prefetching

  2022influential citation
• Optimized scheduling of multi‐user Map‐Reduce jobs in heterogeneous environment

  2022cites this paper
• Optimizing job completion time with fairness in large-scale data centers

  2021cites this paper
• Simplified ciphertext-policy attribute-based encryption scheme with attribute level collusion resistance for cloud storage

  2021cites this paper
• Survey on improving the performance of MapReduce in Hadoop

  2021cites this paper
• Factors affecting cloud data-center efficiency: a scheduling algorithm-based analysis

  2021cites this paper
• Research on CM_HI_pApriori Algorithm Based on Hyperthyroidism

  2021cites this paper
• A Comprehensive Survey on Coded Distributed Computing: Fundamentals, Challenges, and Networking Applications

  2021cites this paper
• Optimizing Quality-Aware Big Data Applications in the Cloud

  2021cites this paper
• SPO: A Secure and Performance-aware Optimization for MapReduce Scheduling

  2021cites this paper
• Security Enhancement Framework For Cloud Computing Environment

  2020cites this paper
• Using Hadoop Technology to Overcome Big Data Problems by Choosing Proposed Cost-efficient Scheduler Algorithm for Heterogeneous Hadoop System (BD3)

  2020influential citation
• Implementing Hadoop Container Migrations in OpenNebula Private Cloud Environment

  2020cites this paper
• Analysis of hadoop MapReduce scheduling in heterogeneous environment

  2020cites this paper
• A Survey of Coded Distributed Computing

  2020cites this paper
• Task Allocation in Hybrid Big Data Analytics for Urban IoT Applications

  2020cites this paper
• Jargon of Hadoop MapReduce scheduling techniques: a scientific categorization

  2019cites this paper
• A Survey of Big Data Machine Learning Applications Optimization in Cloud Data Centers and Networks

  2019cites this paper
• A Study on Different Types of Scheduling Algorithm

  2019cites this paper
• An improved integrated Grid and MapReduce‐Hadoop architecture for spatial data: Hilbert TGS R‐Tree–based IGSIM

  2019cites this paper
• A New Approach for Scheduling Tasks and/or Jobs in Big Data Cluster

  2019cites this paper
• Parallel Implementation of Improved K-Means Based on a Cloud Platform

  2019cites this paper
• Task Scheduling in Data Stream Processing Systems

  2019cites this paper
• DEVELOPMENT OF A DISTRIBUTED BIG DATA FUSION ARCHITECTURE FOR MACHINE-TO-MACHINE COMMUNICATION USING ENSEMBLE LEARNING

  2019cites this paper
• Load Balancing in Heterogeneous Cluster Cloud Computing

  2019cites this paper
• A survey of scheduling frameworks in big data systems

  2018cites this paper
• An optimization framework for the capacity allocation and admission control of MapReduce jobs in cloud systems

  2018cites this paper
• An Enhanced and Efficient Approach for Improving the Performance of HPC Environment Using Map-Reduce With MARIANE

  2018cites this paper
• An improved hadoop load rebalancer

  2018cites this paper
• MapReduce Scheduling for Deadline-Constrained Jobs in Heterogeneous Cloud Computing Systems

  2018cites this paper
• An Overview of Hadoop Scheduler Algorithms

  2018cites this paper
• Optimising Cloud-Based Hadoop 2.x Applications

  2018cites this paper
• MapReduce a Comprehensive Review

  2018cites this paper
• Towards Efficient Big Data: Hadoop Data Placing and Processing

  2018cites this paper
• Big Data Meet Cyber-Physical Systems: A Panoramic Survey

  2018influential citation
• D-SPACE4Cloud: Towards Quality-Aware Data Intensive Applications in the Cloud

  2018cites this paper
• How Heterogeneity Affects the Design of Hadoop MapReduce Schedulers: A State-of-the-Art Survey and Challenges

  2018cites this paper
• Load-balancing algorithms in cloud computing: A survey

  2017cites this paper
• Improving Fair Scheduling Performance on Hadoop

  2017cites this paper
• A scalable Map Reduce tasks scheduling: a threading-based approach

  2017influential citation
• MPLEX: In-Situ Big Data Processing with Compute-Storage Multiplexing

  2017cites this paper
• An Efficient Bandwidth Aware Scheduling Algorithm in Hadoop

  2017cites this paper
• Job Schedulers in Mapreduce

  2017cites this paper
• IOTSim: A simulator for analysing IoT applications

  2017cites this paper
• Task Scheduling in Big Data Platforms: A Systematic Literature Review

  2017cites this paper
• Journal of Network and Computer Applications

  2017cites this paper
• BAHS: A Bandwidth-Aware Heterogeneous Scheduling Approach for SDN-Based Cluster Systems

  2017cites this paper
• An improved optimal task selection strategy for hadoop scheduling

  2017cites this paper
• Energy-efficient hadoop for big data analytics and computing: A systematic review and research insights

  2017cites this paper
• Resources Management and Performance Analysis in Datacenter Networks

  2017cites this paper
• Optimisation model for scheduling MapReduce jobs in big data processing / Ibrahim Abaker Targio Hashem

  2017cites this paper
• Job schedulers for Big data processing in Hadoop environment: testing real-life schedulers using benchmark programs

  2017influential citation
• MapReduce : From Elementary Circuits to Cloud

  2017cites this paper
• Survey on Join Algorithms and Task Scheduling Algorithms for Hadoop MapReduce Platform

  2017cites this paper
• Improving the Performance of Fair Scheduler in Hadoop

  2017cites this paper
• Enabling Cyber-Physical Communication in 5G Cellular Networks: Challenges, Solutions and Applications

  2017cites this paper
• Toward integrating grid and cloud-based concepts for an enhanced deployment of spatial data warehouses in cyber-physical system applications

  2016cites this paper
• Optimised Method of Resource Allocation for Hadoop on Cloud

  2016cites this paper
• Proficiency of Hadoop Integrants in Virtualization Milieu by wielding Word Count Doctrine

  2016cites this paper
• A Survey on Job Scheduling in Big Data

  2016cites this paper
• Energy-Efficient Hadoop Green Scheduler

  2016cites this paper
• Fuzzy based map reduce technique for energy efficiency in multi core cloud computing

  2016cites this paper
• A Modified FIFO with Distributed and Pipelining Scheduling in Hadoop

  2016influential citation
• Performance evaluation of job schedulers on Hadoop YARN

  2016cites this paper
• Soft Real-Time Hadoop Scheduler for Big Data Processing in Smart Cities

  2016cites this paper
• Private Cloud Deployment Model in Open-Source Mobile Robots Ecosystem

  2016cites this paper
• A Scheduling Algorithm for Hadoop MapReduce Workflows with Budget Constraints in the Heterogeneous Cloud

  2016cites this paper
• Simulation of the Performance of MapReduce Jobs

  2016cites this paper
• System Optimization for Big Data Processing

  2016cites this paper
• RUSH: A RobUst ScHeduler to Manage Uncertain Completion-Times in Shared Clouds

  2016cites this paper
• Shortest Processing Time First and Hadoop

  2016cites this paper
• A SIMPLE AND EFFICIENT PROTOTYPE FOR DYNAMIC SLOT ALLOCATION FOR HADOOP CLUSTER

  2016cites this paper
• PFT: A Performance-Fairness scheduler on Hadoop YARN

  2016cites this paper
• Evaluating Performance of Rescheduling Strategies in Cloud System

  2016cites this paper
• Critique of Wordcount Blueprint By Virtue of MapReduce Postulate

  2016cites this paper
• Profit-oriented task scheduling algorithm in Hadoop cluster

  2016cites this paper
• Shortest Processing Time First Algorithm for Hadoop

  2016cites this paper
• D-SPACE4Cloud: A Design Tool for Big Data Applications

  2016cites this paper
• Survey on MapReduce and Scheduling Algorithms in Hadoop

  2015cites this paper
• Performance evaluation of fair and capacity scheduling in Hadoop YARN

  2015cites this paper
• Running genetic algorithms on Hadoop for solving high dimensional optimization problems

  2015cites this paper
• On the Feasibility of MapReduce to Compute Phase Space Properties of Graphical Dynamical Systems: An Empirical Study

  2015cites this paper
• A Zero-Penalty Container-Based Execution Infrastructure for Hadoop Framework

  2015cites this paper
• The Improved Job Scheduling Algorithm of Hadoop Platform

  2015cites this paper
• Classification Framework of MapReduce Scheduling Algorithms

  2015cites this paper
• Modified Delay Scheduling: A Heuristic Approach for Hadoop Scheduling to Improve Fairness and Response Time

  2015cites this paper
• Parallel random forest with IPython cluster

  2015cites this paper
• Evaluating map reduce tasks scheduling algorithms over cloud computing infrastructure

  2015influential citation
• Need for speed: CORA scheduler for optimizing completion-times in the cloud

  2015cites this paper
• Improving Job Scheduling in Hadoop

  2015cites this paper
• A Review on Need of MapReduce in Big Data Application

  2015cites this paper
• Performance Enhancement of MapReduce Framework in Big Data Application Using Load Balancing with Cache

  2015cites this paper
• Private search over big data leveraging distributed file system and parallel processing

  2015cites this paper
• Task Scheduling in Hadoop

  2015cites this paper
• A survey on job scheduling algorithms in Big data processing

  2015cites this paper