Improving MapReduce performance through data placement in heterogeneous Hadoop clusters

No abstract is available for this paper.

• Publication year

  2010

• Venue

  IEEE International Symposium on Parallel & Distributed Processing, Workshops and Phd Forum

• Publication date

  2010-04-19

• Fields of study

  Computer Science, Engineering

• Identifiers
  DOI 10.1109/IPDPSW.2010.5470880
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• A Dynamic MapReduce Scheduler for Heterogeneous Workloads

  2009cited by this paper
• A proactive low-overhead file replication scheme for structured P2P content delivery networks

  2009cited by this paper
• Supporting MapReduce on large-scale asymmetric multi-core clusters

  2009cited by this paper
• Data-intensive File Systems for Internet Services: A Rose by Any Other Name... (CMU-PDL-08-114)

  2008cited by this paper
• Mars: A MapReduce Framework on graphics processors

  2008cited by this paper
• Improving MapReduce Performance in Heterogeneous Environments

  2008cited by this paper
• MapReduce: simplified data processing on large clusters

  2008cited by this paper
• Pig latin: a not-so-foreign language for data processing

  2008cited by this paper
• Map-reduce-merge: simplified relational data processing on large clusters

  2007cited by this paper
• Evaluating MapReduce for Multi-core and Multiprocessor Systems

  2007cited by this paper
• Interpreting the data: Parallel analysis with Sawzall

  2005cited by this paper
• Interpreting the data

  2005cited by this paper
• The Google file system

  2003cited by this paper
• Parallel virtual file system

  1998cited by this paper
• A scalable high-performance I/O system

  1994cited by this paper
• By other name.

  1971cited by this paper

• Catalyzing big data excellence: the enhanced Data Placement Policy (EDPP) revolution

  2026cites this paper
• Optimizing MapReduce efficiency and reducing complexity with enhanced particle Swarm Optimization (MR-MPSO)

  2025cites this paper
• Integrated Data, Task and Resource Management to Speed Up Processing Small Files in Hadoop Cluster

  2024cites this paper
• RETRACTED: Improving big data analytics data processing speed through map reduce scheduling and replica placement with HDFS using genetic optimization techniques

  2024cites this paper
• Proactive, Accuracy-aware Straggler Mitigation in Machine Learning Clusters

  2024cites this paper
• IDaPS - Improved data-locality aware data placement strategy based on Markov clustering to enhance MapReduce performance on Hadoop

  2024cites this paper
• Smart Data Placement Using Storage-as-a-Service Model for Big Data Pipelines

  2023cites this paper
• Survey on Resource Management Solutions to Speed up Processing Small Files in Hadoop Cluster

  2023cites this paper
• Multi-Terminal Collaborative Distributed Garden Planning System Integrating CG Technology

  2023cites this paper
• Map-Reduce (Hadoop) based Data Clustering for BigData: A Survey

  2023cites this paper
• Early straggler tasks detection by recurrent neural network in a heterogeneous environment

  2022influential citation
• Research On A Data-Aware RPC Method For Improving Storage Performance Of Big Data System

  2022cites this paper
• A taxonomy of tools and approaches for distributed genomic analyses

  2022cites this paper
• Forseti: Dynamic chunk-level reshaping for data processing on heterogeneous clusters

  2022cites this paper
• Task Scheduling for Spark Applications With Data Affinity on Heterogeneous Clusters

  2022cites this paper
• A classification framework for straggler mitigation and management in a heterogeneous Hadoop cluster: A state-of-art survey

  2022cites this paper
• Optimization for resource management in heterogeneous spark clusters

  2022cites this paper
• Smart Data Placement for Big Data Pipelines: An Approach based on the Storage-as-a-Service Model

  2022cites this paper
• Data Locality in High Performance Computing, Big Data, and Converged Systems: An Analysis of the Cutting Edge and a Future System Architecture

  2022cites this paper
• Historical data based approach to mitigate stragglers from the Reduce phase of MapReduce in a heterogeneous Hadoop cluster

  2022cites this paper
• DQ-DPS Data Partition Strategy Based on Distributed Machine Learning

  2021cites this paper
• Optimizing Internal Overlaps by Self-Adjusting Resource Allocation in Multi-Stage Computing Systems

  2021cites this paper
• Historical data based approach for straggler avoidance in a heterogeneous Hadoop cluster

  2021influential citation
• New Data Placement Strategy in the HADOOP Framework

  2021cites this paper
• Experimental Analysis in Hadoop MapReduce: A Closer Look at Fault Detection and Recovery Techniques

  2021cites this paper
• Optimization Research and Exploration of New Rural Cooperative Medical Information System

  2021cites this paper
• Sync-Switch: Hybrid Parameter Synchronization for Distributed Deep Learning

  2021cites this paper
• A survey of domains in workflow scheduling in computing infrastructures: Community and keyword analysis, emerging trends, and taxonomies

  2021cites this paper
• Application of digital image processing technology in online education under COVID-19 epidemic

  2021cites this paper
• Survey on improving the performance of MapReduce in Hadoop

  2021cites this paper
• An Analytical Performance Evaluation of MapReduce Model Using Transient Queuing Model

  2021cites this paper
• Application and Storage-Aware Data Placement and Job Scheduling for Hadoop Clusters

  2020cites this paper
• Load Balancing Mechanism Based on Linear Regression Partition Prediction in Spark

  2020cites this paper
• Heterogeneous Job Allocation Scheduler for Hadoop MapReduce Using Dynamic Grouping Integrated Neighboring Search

  2020cites this paper
• Investigating Genome Analysis Pipeline Performance on GATK with Cloud Object Storage

  2020cites this paper
• Improvement of job completion time in data-intensive cloud computing applications

  2020cites this paper
• Design and Implementation of System Which Efficiently Retrieve Useful Data for Detection of Dementia Disease

  2020cites this paper
• Implementing Hadoop Container Migrations in OpenNebula Private Cloud Environment

  2020cites this paper
• CORE-An optimal data placement strategy in Hadoop for data intensive applications based on cohesion relation

  2020cites this paper
• Density-based Algorithms for Big Data Clustering Using MapReduce Framework

  2020cites this paper
• HadoopSec 2.0: Prescriptive analytics-based multi-model sensitivity-aware constraints centric block placement strategy for Hadoop

  2020cites this paper
• Retrieval of Frequent Itemset Using Improved Mining Algorithm in Hadoop

  2020cites this paper
• A Content-wise Data Placement Policy for Improving the Performance of MapReduce-based Video Processing Applications in Cloud Computing

  2020cites this paper
• A heuristic method towards deadline-aware energy-efficient mapreduce scheduling problem in Hadoop YARN

  2020cites this paper
• Performance Analysis of Distributed Computing Frameworks for Big Data Analytics: Hadoop Vs Spark

  2020cites this paper
• Earlier stage for straggler detection and handling using combined CPU test and LATE methodology

  2020cites this paper
• MapReduce processing challenges in cloud environment

  2020cites this paper
• Novel data‐placement scheme for improving the data locality of Hadoop in heterogeneous environments

  2020cites this paper
• GeoLocalitySim: Geographical Cloud Simulator with Data Locality

  2020cites this paper
• An Optimal Locality-Aware Task Scheduling Algorithm Based on Bipartite Graph Modelling for Spark Applications

  2020cites this paper
• Efficient scheme for compressing and transferring data in hadoop clusters

  2020cites this paper
• Job failure prediction in Hadoop based on log file analysis

  2020cites this paper
• Critical literature review on straggler node

  2020cites this paper
• A Survey and Annotated Bibliography of Workflow Scheduling in Computing Infrastructures: Community, Keyword, and Article Reviews - Extended Technical Report

  2020cites this paper
• A Novel Snake-like Data Placement Policy for Improving Video Processing in Hadoop Clusters

  2020cites this paper
• Improving Performance in Hadoop Cluster Over Cloud Computing Environment Using Hold & Release Mechanism

  2019cites this paper
• Improving MapReduce scheduler for heterogeneous workloads in a heterogeneous environment

  2019influential citation
• Impact of Small Files on Hadoop Performance: Literature Survey and Open Points

  2019cites this paper
• Survey of Data Locality in Apache Hadoop

  2019influential citation
• HaRD: a heterogeneity-aware replica deletion for HDFS

  2019influential citation
• Towards an Energy Efficient Computing With Coordinated Performance-Aware Scheduling in Large Scale Data Clusters

  2019cites this paper
• MapReduce Data Skewness Handling: A Systematic Literature Review

  2019cites this paper
• A Survey on Data Placement Strategy in Big Data Heterogeneous Environments

  2019cites this paper
• Energy- and locality-efficient multi-job scheduling based on MapReduce for heterogeneous datacenter

  2019cites this paper
• Dynamic ranking-based MapReduce job scheduler to exploit heterogeneous performance in a virtualized environment

  2019cites this paper
• Big Data for Smart Infrastructure Design: Opportunities and Challenges

  2019cites this paper
• Adaptive Data Center Network Traffic Management for Distributed High Speed Storage

  2019cites this paper
• A Survey of Big Data Machine Learning Applications Optimization in Cloud Data Centers and Networks

  2019cites this paper
• A Study on Different Types of Scheduling Algorithm

  2019cites this paper
• Jargon of Hadoop MapReduce scheduling techniques: a scientific categorization

  2019cites this paper
• Improving the Performance of Heterogeneous Hadoop Clusters Using Map Reduce

  2019cites this paper
• To Minimize Fault Report and Bug Fixing Time using an Efficient Integration of Instance and Aspect Preferment Algorithm

  2019cites this paper
• Dynamic Load Balancing Algorithm for Heterogeneous Hadoop Cluster

  2019cites this paper
• Workload-Aware Scheduling for Data Analytics upon Heterogeneous Storage

  2019cites this paper
• Fine tuning of MapReduce jobs using parallel K Map clustering

  2019cites this paper
• CNR:A Technique for Data Replication Organization in BigData

  2019cites this paper
• Improvement of Data-Intensive Applications Running on Cloud Computing Clusters

  2019cites this paper
• Content Based Video Retrieval by Using Distributed Real-Time System Based on Storm

  2019cites this paper
• Research on Load Balancing Algorithm Optimization Based on Spark Platform

  2019cites this paper
• CoBell: Runtime Prediction for Distributed Dataflow Jobs in Shared Clusters

  2018cites this paper
• solution for the future: small file management by optimizing Hadoop

  2018cites this paper
• Improving Content Based Video Retrieval Performance by Using Hadoop-MapReduce Model

  2018cites this paper
• Optimizing the performance of hadoop clusters through efficient cluster management techniques

  2018cites this paper
• Enhanced Bond Energy Algorithm for Data Placement in Hadoop Framework

  2018cites this paper
• Load Balancing through Block Rearrangement Policy for Hadoop Heterogeneous Cluster

  2018cites this paper
• Performance Evaluation of Load Balancing Algorithms in Hadoop

  2018cites this paper
• Replicas Strategy and Cache Optimization of Video Surveillance Systems Based on Cloud Storage

  2018cites this paper
• Optimizing distributed file storage and processing engines for CERN's Large Hadron Collider using multi criteria partitioned replication

  2018cites this paper
• H2Hadoop: Improving Hadoop Performance Using the Metadata of Related Jobs

  2018cites this paper
• A Task-Based Greedy Scheduling Algorithm for Minimizing Energy of MapReduce Jobs

  2018cites this paper
• A Hadoop Yarn Scheduling Based on Node Computing Capability and Data Locality in Heterogeneous Environments

  2018cites this paper
• On a Dynamic Data Placement Strategy for Heterogeneous Hadoop Clusters

  2018cites this paper
• Experimental Performance Evaluation of Cloud-Based Analytics-asa-Service

  2018cites this paper
• LAS: Logical-Block Affinity Scheduling in Big Data Analytics Systems

  2018cites this paper
• HaDaap: A hotness‐aware data placement strategy for improving storage efficiency in heterogeneous Hadoop clusters

  2018cites this paper
• Towards Efficient Big Data: Hadoop Data Placing and Processing

  2018influential citation
• Block Placement in Distributed File Systems Based on Block Access Frequency

  2018cites this paper
• Improving I/O Performance Through Colocating Interrelated Input Data and Near-Optimal Load Balancing

  2018cites this paper
• Improving Late Scheduler and Result Analysis in Heterogeneous Environment

  2018influential citation
• A Review on Data locality in Hadoop MapReduce

  2018influential citation