dSpark: Deadline-Based Resource Allocation for Big Data Applications in Apache Spark

No abstract is available for this paper.

• Publication year

  2017

• Venue

  IEEE International Conference on e-Science

• Publication date

  2017-10-01

• Fields of study

  Computer Science

• Identifiers
  DOI 10.1109/eScience.2017.21
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Dynamic Configuration of Partitioning in Spark Applications

  2017cited by this paper
• Apache Spark

  2016cited by this paper
• Stage Aware Performance Modeling of DAG Based in Memory Analytic Platforms

  2016cited by this paper
• A Novel Method for Tuning Configuration Parameters of Spark Based on Machine Learning

  2016cited by this paper
• OptEx: A Deadline-Aware Cost Optimization Model for Spark

  2016cited by this paper
• Spark Parameter Tuning via Trial-and-Error

  2016cited by this paper
• Performance Prediction for Apache Spark Platform

  2015cited by this paper
• Spark deployment and performance evaluation on the MareNostrum supercomputer

  2015cited by this paper
• BigDataBench: A big data benchmark suite from internet services

  2014cited by this paper
• Apache Hadoop YARN: yet another resource negotiator

  2013cited by this paper
• Performance Modeling of MapReduce Jobs in Heterogeneous Cloud Environments

  2013cited by this paper
• Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing

  2012cited by this paper
• Deadline-based workload management for MapReduce environments: Pieces of the performance puzzle

  2012cited by this paper
• ARIA: automatic resource inference and allocation for mapreduce environments

  2011cited by this paper
• Resource Provisioning Framework for MapReduce Jobs with Performance Goals

  2011cited by this paper
• HBase - The Definitive Guide: Random Access to Your Planet-Size Data

  2011cited by this paper
• Mesos: A Platform for Fine-Grained Resource Sharing in the Data Center

  2011cited by this paper
• The Hadoop Distributed File System

  2010cited by this paper
• Cassandra: a decentralized structured storage system

  2010cited by this paper
• MapReduce: simplified data processing on large clusters

  2008cited by this paper
• 2016 Ieee International Conference on Big Data (big Data) a Workload Aware Model of Computational Resource Selection for Big Data Applications

  year unknowncited by this paper

• Predicting Big Data Applications Runtime on Spark Cluster using Machine Learning Techniques

  2025cites this paper
• Workload Characterization for Resource Optimization of Big Data Analytics: Best Practices, Trends, and Opportunities

  2025cites this paper
• Enhancing Task Management in Apache Spark Through Energy-Efficient Data Segregation and Time-Based Scheduling

  2024cites this paper
• Cloud-integrated cyber–physical systems: Reliability, performance and power consumption with shared-servers and parallelized services

  2024cites this paper
• Scheduling of Intermittent Query Processing

  2023cites this paper
• KORDI: A Framework for Real-Time Performance and Cost Optimization of Apache Spark Streaming

  2023cites this paper
• Juggler: Autonomous Cost Optimization and Performance Prediction of Big Data Applications

  2022cites this paper
• Resource Provisioning Techniques in Multi-Access Edge Computing Environments: Outlook, Expression, and Beyond

  2022cites this paper
• Optimization for resource management in heterogeneous spark clusters

  2022cites this paper
• A Novel Reinforcement Learning Approach for Spark Configuration Parameter Optimization

  2022cites this paper
• Job scheduling for big data analytical applications in clouds: A taxonomy study

  2022cites this paper
• Diaspore: Diagnosing Performance Interference in Apache Spark

  2021cites this paper
• SMConf: One-Size-Fit-Bunch, Automated Memory Capacity Configuration for In-memory Data Analytic Platform

  2021cites this paper
• Stargazer: Toward efficient data analytics scheduling via task completion time inference

  2021cites this paper
• Prediction-Based Cost-Efficient Resource Allocation Scheme for Big Data Streams in Cloud Systems

  2021cites this paper
• A Machine Learning-Based Elastic Strategy for Operator Parallelism in a Big Data Stream Computing System

  2021cites this paper
• Streaming Data Priority Scheduling Framework for Autonomous Driving by Edge

  2021cites this paper
• Machine Learning in Apache Spark Environment for Diagnosis of Diabetes

  2021cites this paper
• Exponentially‐spider monkey optimization based allocation of resource in cloud

  2021cites this paper
• Benchmarking and Performance Modelling of Dataflow with Cycles

  2021cites this paper
• Cost-efficient Management of Cloud Resources for Big Data Applications

  2020cites this paper
• A Survey: Benchmarking and Performance Modelling of Data Intensive Applications

  2020influential citation
• Configuring Parallelism for Hybrid Layouts Using Multi-Objective Optimization

  2020cites this paper
• COCOS: A Scalable Architecture for Containerized Heterogeneous Systems

  2020cites this paper
• Workload Forecasting Based on Big Data Characteristics in Cloud Systems

  2020cites this paper
• Handbook of Research on Cloud Computing and Big Data Applications in IoT

  2019cites this paper
• Time Estimation and Resource Minimization Scheme for Apache Spark and Hadoop Big Data Systems With Failures

  2019cites this paper
• Storage Format Selection and Optimization for Materialized Intermediate Results in Data-Intensive Flows

  2019cites this paper
• Big-Data Applications as Self-Adaptive Systems of Systems

  2019cites this paper
• Quick Execution Time Predictions for Spark Applications

  2019cites this paper
• Symbolic execution-driven extraction of the parallel execution plans of Spark applications

  2019influential citation
• An energy-aware scheduling algorithm for big data applications in Spark

  2019cites this paper
• Automatically Configuring Parallelism for Hybrid Layouts

  2019cites this paper
• Fast and Lightweight Execution Time Predictions for Spark Applications

  2019cites this paper
• Resource Management and Scheduling for Big Data Applications in Cloud Computing Environments

  2018cites this paper
• Comparative Analysis of Energy-Efficient Scheduling Algorithms for Big Data Applications

  2018influential citation