Jockey: guaranteed job latency in data parallel clusters

No abstract is available for this paper.

• Publication year

  2012

• Venue

  European Conference on Computer Systems

• Publication date

  2012-04-10

• Fields of study

  Computer Science

• Identifiers
  DOI 10.1145/2168836.2168847
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• SLO-driven right-sizing and resource provisioning of MapReduce jobs

  2011cited by this paper
• No one (cluster) size fits all: automatic cluster sizing for data-intensive analytics

  2011influential reference
• Scarlett: coping with skewed content popularity in mapreduce clusters

  2011cited by this paper
• CIEL: A Universal Execution Engine for Distributed Data-Flow Computing

  2011cited by this paper
• Delay scheduling: a simple technique for achieving locality and fairness in cluster scheduling

  2010cited by this paper
• CloudCmp: comparing public cloud providers

  2010cited by this paper
• Reining in the Outliers in Map-Reduce Clusters using Mantri

  2010influential reference
• ParaTimer: a progress indicator for MapReduce DAGs

  2010cited by this paper
• FlumeJava: easy, efficient data-parallel pipelines

  2010cited by this paper
• The Impact of Virtualization on Network Performance of Amazon EC2 Data Center

  2010cited by this paper
• Quincy: fair scheduling for distributed computing clusters

  2009cited by this paper
• MapReduce optimization using regulated dynamic prioritization

  2009cited by this paper
• Hive - A Warehousing Solution Over a Map-Reduce Framework

  2009influential reference
• SCOPE: easy and efficient parallel processing of massive data sets

  2008influential reference
• Pig latin: a not-so-foreign language for data processing

  2008influential reference
• Online Optimization for Latency Assignment in Distributed Real-Time Systems

  2008cited by this paper
• MapReduce: simplified data processing on large clusters

  2008cited by this paper
• A statistical approach to risk mitigation in computational markets

  2007cited by this paper
• Prediction-Based Enforcement of Performance Contracts

  2007cited by this paper
• Dynamic Provisioning of Multi-tier Internet Applications

  2005cited by this paper
• Value-maximizing deadline scheduling and its application to animation rendering

  2005cited by this paper
• The Google file system

  2003cited by this paper
• Adaptive resource management in asynchronous real-time distributed systems using feedback control functions

  2001cited by this paper
• Utilization-based admission control for real-time applications

  2000cited by this paper
• Validity of the single processor approach to achieving large scale computing capabilities

  1967cited by this paper

• Managed Resource Scaling in Amazon EMR

  2025cites this paper
• PnM: Efficient Intra-Datacenter Calls Packing for Large Conferencing Services

  2025cites this paper
• Efficient Far Memory-Aware Scheduling With FaMAS

  2025cites this paper
• RedunCut: Measurement-Driven Sampling and Accuracy Performance Modeling for Low-Cost Live Video Analytics

  2025cites this paper
• Optimizing Fog Computing Performance: A Machine Learning-Based Task Scheduling Approach Using Fuzzy Logic and Clustering

  2025cites this paper
• Reducing Makespan via Optimizing Service Applications Scheduling Without Runtime Estimation

  2025cites this paper
• Ekko: Fully Decentralized Scheduling for Serverless Edge Computing

  2025cites this paper
• Tetris: Efficient Intra-Datacenter Calls Packing for Large Conferencing Services

  2025cites this paper
• MPDA: A Massively Parallel Learning and Dependency-Aware Scheduling Algorithm for Data Processing Clusters

  2025cites this paper
• Falcon: An Efficient Dependency-Aware Scheduling for High Throughput and Resource Utilization in Clouds

  2025cites this paper
• Distributed Multi-Level Task Scheduling Framework for Heterogeneous workloads

  2025cites this paper
• Formal and Empirical Study of Metadata-Based Profiling for Resource Management in the Computing Continuum

  2025cites this paper
• LAVA: Lifetime-Aware VM Allocation with Learned Distributions and Adaptation to Mispredictions

  2024cites this paper
• GPARS: Graph predictive algorithm for efficient resource scheduling in heterogeneous GPU clusters

  2024cites this paper
• Analytically-Driven Resource Management for Cloud-Native Microservices

  2024cites this paper
• Ursa: Lightweight Resource Management for Cloud-Native Microservices

  2024cites this paper
• A House United Within Itself: SLO-Awareness for On-Premises Containerized ML Inference Clusters via Faro

  2024influential citation
• Measurement-based Resource Allocation and Control in Data Centers: A Survey

  2024cites this paper
• Towards providing reliable job completion time predictions using PCS

  2024cites this paper
• Lucid: A Non-intrusive, Scalable and Interpretable Scheduler for Deep Learning Training Jobs

  2023cites this paper
• PolarisProfiler: A Novel Metadata-Based Profiling Approach for Optimizing Resource Management in the Edge-Cloud Continnum

  2023influential citation
• SAC: Dynamic Caching upon Sketch for In-Memory Big Data Analytics

  2023cites this paper
• Serverless Computing: State-of-the-Art, Challenges and Opportunities

  2023cites this paper
• SLearn: A Case for Task Sampling Based Learning for Cluster Job Scheduling

  2023influential citation
• Cooperative Job Scheduling and Data Allocation in Data-Intensive Parallel Computing Clusters

  2023influential citation
• Sailfish: A Dependency-Aware and Resource Efficient Scheduling for Low Latency in Clouds

  2023cites this paper
• Cost-Intelligent Data Analytics in the Cloud

  2023influential citation
• Smartpick: Workload Prediction for Serverless-enabled Scalable Data Analytics Systems

  2023cites this paper
• Energy-aware scheduling for spark job based on deep reinforcement learning in cloud

  2023cites this paper
• Cilantro: Performance-Aware Resource Allocation for General Objectives via Online Feedback

  2023cites this paper
• Cougar: A General Framework for Jobs Optimization In Cloud

  2023cites this paper
• Tail Prediction for Heterogeneous Data Center Clusters

  2023cites this paper
• Runtime Variation in Big Data Analytics

  2023cites this paper
• CODEC: Cost-Effective Duration Prediction System for Deadline Scheduling in the Cloud

  2023cites this paper
• Multi-level Explanation of Deep Reinforcement Learning-based Scheduling

  2022cites this paper
• PB3Opt: Profile‐based biased Bayesian optimization to select computing clusters on the cloud

  2022cites this paper
• Zero-Shot Cost Models for Out-of-the-box Learned Cost Prediction

  2022cites this paper
• Kuber: Cost-Efficient Microservice Deployment Planner

  2022cites this paper
• Multi Resource Scheduling with Task Cloning in Heterogeneous Clusters

  2022cites this paper
• Search-based Methods for Multi-Cloud Configuration

  2022cites this paper
• Juggler: Autonomous Cost Optimization and Performance Prediction of Big Data Applications

  2022cites this paper
• Blink: Lightweight Sample Runs for Cost Optimization of Big Data Applications

  2022cites this paper
• Towards Optimal Resource Allocation for Big Data Analytics

  2022influential citation
• Truthful Online Scheduling of Cloud Workloads under Uncertainty

  2022cites this paper
• Accelerating DAG-Style Job Execution via Optimizing Resource Pipeline Scheduling

  2022cites this paper
• Less Provisioning: A Hybrid Resource Scaling Engine for Long-Running Services With Tail Latency Guarantees

  2022cites this paper
• IFLustre: Towards Interference-Free and Efficient Storage Allocation in Distributed File System

  2022cites this paper
• Low-Latency Speculative Inference on Distributed Multi-Modal Data Streams

  2022cites this paper
• A scalable and low latency probe-based scheduler for data analytics frameworks

  2021cites this paper
• Improving Learning-Based DAG Scheduling by Inserting Deliberate Idle Slots

  2021cites this paper
• RIBBON: Cost-Effective and QoS-Aware Deep Learning Model Inference using a Diverse Pool of Cloud Computing Instances

  2021cites this paper
• Morphling: Fast, Near-Optimal Auto-Configuration for Cloud-Native Model Serving

  2021cites this paper
• Understanding, Predicting and Scheduling Serverless Workloads under Partial Interference

  2021cites this paper
• Performance Improvement of DAG-Aware Task Scheduling Algorithms with Efficient Cache Management in Spark

  2021cites this paper
• Faster and Cheaper Serverless Computing on Harvested Resources

  2021cites this paper
• Para: Harvesting CPU time fragments in Big Data Analytics

  2021cites this paper
• A Case for Task Sampling based Learning for Cluster Job Scheduling

  2021cites this paper
• Task scheduling based on computing resource usage prediction in large-scale cluster

  2021influential citation
• INFaaS: Automated Model-less Inference Serving

  2021influential citation
• KEA: Tuning an Exabyte-Scale Data Infrastructure

  2021cites this paper
• Optimal Resource Allocation for Serverless Queries

  2021cites this paper
• Enel: Context-Aware Dynamic Scaling of Distributed Dataflow Jobs using Graph Propagation

  2021cites this paper
• Bellamy: Reusing Performance Models for Distributed Dataflow Jobs Across Contexts

  2021cites this paper
• CCA: Cost-Capacity-Aware Caching for In-Memory Data Analytics Frameworks

  2021cites this paper
• Low-latency speculative inference on distributed multi-modal data streams

  2021cites this paper
• WIRE: Resource-efficient Scaling with Online Prediction for DAG-based Workflows

  2021cites this paper
• Performance Models of Data Parallel DAG Workflows for Large Scale Data Analytics

  2021cites this paper
• Llama: A Heterogeneous & Serverless Framework for Auto-Tuning Video Analytics Pipelines

  2021cites this paper
• Characterization and Prediction of Deep Learning Workloads in Large-Scale GPU Datacenters

  2021cites this paper
• FirePlace: Placing Firecraker Virtual Machines with Hindsight Imitation

  2021cites this paper
• A divide and conquer approach to deadline constrained cost-optimization workflow scheduling for the cloud

  2021cites this paper
• On the Scalability of Big Data Cyber Security Analytics Systems

  2021cites this paper
• An Efficient Framework for Resource Allocation and Dynamic Pricing Scheme for Completion Time Failure in Cloud Computing

  2021cites this paper
• Razor: Scaling Backend Capacity for Mobile Applications

  2020cites this paper
• MonkeyKing: Adaptive Parameter Tuning on Big Data Platforms with Deep Reinforcement Learning

  2020cites this paper
• Improved Intermediate Data Management for MapReduce Frameworks

  2020cites this paper
• Fast and Low-cost Search for Efficient Cloud Configurations for HPC Workloads

  2020cites this paper
• Multi-resource Low-latency Cluster Scheduling without Execution Time Estimation

  2020cites this paper
• uPredict: A User-Level Profiler-Based Predictive Framework in Multi-Tenant Clouds

  2020cites this paper
• Mary, Hugo, and Hugo*: Learning to schedule distributed data‐parallel processing jobs on shared clusters

  2020cites this paper
• A Black-Box Fork-Join Latency Prediction Model for Data-Intensive Applications

  2020cites this paper
• Cost Models for Big Data Query Processing: Learning, Retrofitting, and Our Findings

  2020cites this paper
• OPTIMUSCLOUD: Heterogeneous Configuration Optimization for Distributed Databases in the Cloud

  2020cites this paper
• Improving the Efficiency of Heterogeneous Clouds

  2020cites this paper
• Shuffle Scheduling for MapReduce Jobs Based on Periodic Network Status

  2020cites this paper
• Deadline Flow Scheduling in Datacenters with Time-Varying Bandwidth Allocations

  2020influential citation
• Fair Scheduling for Deadline-Driven, Resource-Constrained, Multi-Analytics Workloads

  2020influential citation
• A Shared Memory Cache Layer across Multiple Executors in Apache Spark

  2020cites this paper
• Toward Designing Cost-Optimal Policies to Utilize IaaS Clouds with Online Learning

  2020cites this paper
• Finding the right cloud configuration for analytics clusters

  2020cites this paper
• Container Cluster Scheduling Strategy Based on Delay Decision Under Multidimensional Constraints

  2020cites this paper
• SCYLLA: QoE-aware Continuous Mobile Vision with FPGA-based Dynamic Deep Neural Network Reconfiguration

  2020cites this paper
• Towards Plan-aware Resource Allocation in Serverless Query Processing

  2020cites this paper
• AutoToken

  2020cites this paper
• NLC: An Efficient Caching Algorithm Based on Non-critical Path Least Counts for In-Memory Computing

  2020cites this paper
• INFaaS: A Model-less and Managed Inference Serving System

  2019cites this paper
• Semi-Clairvoyant Scheduling in Data Analytics Systems

  2019cites this paper
• Lynceus: Tuning and Provisioning Data Analytic Jobs on a Budget

  2019cites this paper
• SOPHIA: Online Reconfiguration of Clustered NoSQL Databases for Time-Varying Workloads

  2019cites this paper
• Efficient Workload Colocation in Modern Data Centers

  2019cites this paper