The future of scientific workflows

E. Deelman,T. Peterka,I. Altintas,C. Carothers,K. K. Dam,K. Moreland,M. Parashar,Lavanya Ramakrishnan,M. Taufer,J. Vetter

Published 2018 in The international journal of high performance computing applications

ABSTRACT

No abstract is available for this paper.

PUBLICATION RECORD

Publication year
2018
Venue
The international journal of high performance computing applications
Publication date
Unknown publication date
Fields of study
Physics, Computer Science, Engineering, Environmental Science
Identifiers
DOI 10.1177/1094342017704893
External record
Open on Semantic Scholar
Source metadata
Semantic Scholar

CITATION MAP

EXTRACTION MAP

CLAIMS

No claims are published for this paper.

CONCEPTS

No concepts are published for this paper.

REFERENCES

PANORAMA: An approach to performance modeling and diagnosis of extreme-scale workflows
2017influential reference
A failure of prediction?
2016cited by this paper
Bredala: Semantic Data Redistribution for In Situ Applications
2016cited by this paper
Self-Adaptive Density Estimation of Particle Data
2016cited by this paper
Dictionary Based Secure Provenance Compression for Wireless Sensor Networks
2016cited by this paper
Enhancing reproducibility for computational methods
2016cited by this paper
Komadu: A Capture and Visualization System for Scientific Data Provenance
2015cited by this paper
Open Software for (Open) Science
2015cited by this paper
Adaptive data placement for staging-based coupled scientific workflows
2015cited by this paper
Enabling Structured Exploration of Workflow Performance Variability in Extreme-Scale Environments
2015cited by this paper
The End of Error: Unum Computing
2015cited by this paper
Storage Systems and Input/Output to Support Extreme Scale Science
2015cited by this paper
The Future of Scientific Workflows. Report of the DOE NGNS/CS Scientific Workflows Workshop (Sandia contributions)
2015cited by this paper
Resource Discovery for Extreme Scale Collaboration (RDESC) Final Report - RPI/TWC - Year 3
2015cited by this paper
D2WORM: A Management Infrastructure for Distributed Data-centric Workflows
2015cited by this paper
In-Memory Graph Databases for Web-Scale Data
2015cited by this paper
Distributed Monitoring and Management of Exascale Systems in the Argo Project
2015cited by this paper
Lessons Learned from Building In Situ Coupling Frameworks
2015cited by this paper
Pegasus, a workflow management system for science automation
2015influential reference
Parallel Reproducible Summation
2015cited by this paper
Climate Science Performance, Data and Productivity on Titan
2015cited by this paper
On the Need for Reproducible Numerical Accuracy through Intelligent Runtime Selection of Reduction Algorithms at the Extreme Scale
2015cited by this paper
Secure data provenance compression using arithmetic coding in wireless sensor networks
2014cited by this paper
Combining Workflow Templates with a Shared Space-Based Execution Model
2014cited by this paper
Toward Local Failure Local Recovery Resilience Model using MPI-ULFM
2014cited by this paper
Using Application Skeletons to Improve eScience Infrastructure
2014cited by this paper
Designing Bit-Reproducible Portable High-Performance Applications
2014cited by this paper
noWorkflow: Capturing and Analyzing Provenance of Scripts
2014cited by this paper
Flexible Scheduling and Control of Bandwidth and In-transit Services for End-to-End Application Workflows
2014cited by this paper
Exploring Automatic, Online Failure Recovery for Scientific Applications at Extreme Scales
2014cited by this paper
Flexpath: Type-Based Publish/Subscribe System for Large-Scale Science Analytics
2014cited by this paper
Online
2014cited by this paper
A Tale of Two Data-Intensive Paradigms: Applications, Abstractions, and Architectures
2014cited by this paper
Experiences with User-Centered Design for the Tigres Workflow API
2014influential reference
HACC: Simulating Sky Surveys on State-of-the-Art Supercomputing Architectures
2014cited by this paper
Storing provenance data of genome project workflows using graph database
2014cited by this paper
Towards Supporting Provenance Gathering and Querying in Different Database Approaches
2014cited by this paper
1 Towards HPC-ABDS : An Initial High-Performance Big Data Stack
2014cited by this paper
Similarity assessment and efficient retrieval of semantic workflows
2014cited by this paper
Likelihood-Based Gene Annotations for Gap Filling and Quality Assessment in Genome-Scale Metabolic Models
2014cited by this paper
Extreme-Scale Viability of Collective Communication for Resilient Task Scheduling and Work Stealing
2014cited by this paper
D-PROV: Extending the PROV Provenance Model with Workflow Structure
2013cited by this paper
I/O Containers: Managing the Data Analytics and Visualization Pipelines of High End Codes
2013cited by this paper
Storing, Indexing and Querying Large Provenance Data Sets as RDF Graphs in Apache HBase
2013cited by this paper
Provenance from log files: a BigData problem
2013cited by this paper
VisTrails provenance traces for benchmarking
2013cited by this paper
Reusability in Science: From Initial User Engagement to Dissemination of Results
2013cited by this paper
A file provenance system
2013cited by this paper
Characterizing and profiling scientific workflows
2013cited by this paper
Towards a provenance-aware distributed filesystem
2013cited by this paper
On the Reproducibility of MPI Reduction Operations
2013cited by this paper
FIRST STEPS TOWARDS MORE NUMERICAL REPRODUCIBILITY
2013cited by this paper
The Taverna workflow suite: designing and executing workflows of Web Services on the desktop, web or in the cloud
2013cited by this paper
Production I / O Characterization on the Cray XE 6
2013cited by this paper
Hobbes: composition and virtualization as the foundations of an extreme-scale OS/R
2013cited by this paper
Secure Provenance Transmission for Streaming Data
2013cited by this paper
Toward a data scalable solution for facilitating discovery of scientific data resources
2013cited by this paper
Determinism and Reproducibility in Large-Scale HPC Systems
2013cited by this paper
ADIOS Visualization Schema: A First Step Towards Improving Interdisciplinary Collaboration in High Performance Computing
2013cited by this paper
WGL – A Workflow Generator Language and Utility
2013cited by this paper
Effective End-to-end Management of Data Acquisition and Analysis for X-ray Photon Correlation Spectroscopy
2013cited by this paper
Combining in-situ and in-transit processing to enable extreme-scale scientific analysis
2012cited by this paper
Understanding I/O Performance Using I/O Skeletal Applications
2012cited by this paper
On the role of burst buffers in leadership-class storage systems
2012cited by this paper
A Provenance-based Adaptive Scheduling Heuristic for Parallel Scientific Workflows in Clouds
2012cited by this paper
A hybrid approach for efficient provenance storage
2012cited by this paper
Provenance analysis: Towards quality provenance
2012cited by this paper
ProvManager: a provenance management system for scientific workflows
2012cited by this paper
Towards Open Publication of Reusable Scientific Workflows : Abstractions , Standards and Linked Data
2012cited by this paper
The RAMDISK storage accelerator: a method of accelerating I/O performance on HPC systems using RAMDISKs
2012cited by this paper
Exascale Operating Systems and Runtime Software Report
2012cited by this paper
Modeling and Querying Scientific Workflow Provenance in the D-OPM
2012cited by this paper
Hi-Fi: collecting high-fidelity whole-system provenance
2012cited by this paper
A General-Purpose Provenance Library
2012cited by this paper
A Noisy 10GB Provenance Database
2011cited by this paper
A Method to Mine Workflows from Provenance for Assisting Scientific Workflow Composition
2011cited by this paper
Design and Evaluation of Multiple-Level Data Staging for Blue Gene Systems
2011cited by this paper
Moving the Code to the Data - Dynamic Code Deployment Using ActiveSpaces
2011cited by this paper
Failure prediction and localization in large scientific workflows
2011cited by this paper
In-situ I/O processing: a case for location flexibility
2011cited by this paper
The Open Provenance Model core specification (v1.1)
2011cited by this paper
Scalable SQL and NoSQL data stores
2011cited by this paper
Swift: A language for distributed parallel scripting
2011influential reference
The Architecture of SciDB
2011cited by this paper
Using active NVRAM for I/O staging
2011cited by this paper
CyberShake: A Physics-Based Seismic Hazard Model for Southern California
2011cited by this paper
Storing, reasoning, and querying OPM-compliant scientific workflow provenance using relational databases
2011cited by this paper
Enabling high‐speed asynchronous data extraction and transfer using DART
2010cited by this paper
Weaver: integrating distributed computing abstractions into scientific workflows using Python
2010cited by this paper
PerfExpert: An Easy-to-Use Performance Diagnosis Tool for HPC Applications
2010cited by this paper
Improving numerical reproducibility and stability in large-scale numerical simulations on GPUs
2010cited by this paper
Galaxy: a comprehensive approach for supporting accessible, reproducible, and transparent computational research in the life sciences
2010influential reference
DataSpaces: an interaction and coordination framework for coupled simulation workflows
2010cited by this paper
PreDatA – preparatory data analytics on peta-scale machines
2010cited by this paper
Pregel: a system for large-scale graph processing
2010cited by this paper
Network Analysis of Scientific Workflows: A Gateway to Reuse
2010cited by this paper
VizSchema - a Unified Visualization of Computational Accelerator Physics Data
2010cited by this paper
Accuracy versus time: a case study with summation algorithms
2010cited by this paper
Design, Modeling, and Evaluation of a Scalable Multi-level Checkpointing System
2010cited by this paper
DataStager: scalable data staging services for petascale applications
2009cited by this paper

CITED BY

A Greedy Constructive Heuristic for Executing Cloud-Based Workflows with Data Confidentiality Restrictions
2026cites this paper
Energy‐Aware Multi‐Objective Workflow Scheduling Using Proximal Policy Optimization and Kolmogorov–Arnold Networks in Heterogeneous HPC Environments
2026cites this paper
A Unified, Cross-Platform Framework for Automatic GUI and Plugin Generation in Structural Bioinformatics and Beyond
2026cites this paper
Lessons learned from building Confluence: Cloud-based global river discharge estimation software for the SWOT satellite
2026cites this paper
EWoPe: A light Embeddable WOrkflow PErsistence tool for geoscientific pipeline reproducibility
2026cites this paper
Categorizing methods and approaches for generating and identifying paradata
2025cites this paper
Access Control Policies Specification and Analysis for Multi-Institutional Collaborative Projects
2025cites this paper
Enabling Lightweight Performance Analysis of Complex Scientific Workflows with PerfFlowAspect
2025cites this paper
Enabling Scientific Workflow Scheduling Research in Non-Uniform Memory Access Architectures
2025cites this paper
A Deep Look into the Temporal I/O Behavior of HPC Applications
2025cites this paper
From Prompt to Pipeline: Large Language Models for Scientific Workflow Development in Bioinformatics
2025cites this paper
Detecting Silent Data Corruption from Hardware Counters
2025cites this paper
PT-ESM: A Parameter-Testing and Integration Framework for Earth System Models Oriented towards High-Performance Computing
2024cites this paper
MAESTRO: a lightweight ontology-based framework for composing and analyzing script-based scientific experiments
2024cites this paper
Towards Highly Compatible I/O-Aware Workflow Scheduling on HPC Systems
2024cites this paper
Empowering bioinformatics communities with Nextflow and nf-core
2024cites this paper
A workflow management system for reproducible and interoperable high-throughput self-driving experiments
2024cites this paper
How Do Users Design Scientific Workflows? The Case of Snakemake and Nextflow
2024cites this paper
A surrogate model-based ESM parameter tuning scientific workflow management framework for HPC
2024cites this paper
PerSSD: Persistent, Shared, and Scalable Data with Node-Local Storage for Scientific Workflows in Cloud Infrastructure
2024cites this paper
Integrating big data with KNIME as an alternative without programming code: an application to the PATSTAT patent database
2024cites this paper
DaYu: Optimizing Distributed Scientific Workflows by Decoding Dataflow Semantics and Dynamics
2024cites this paper
Software-defined Support for the Execution of Task-graph-based Applications on Cloud Environments
2024cites this paper
Trends, Approaches, and Gaps in Scientific Workflow Scheduling: A Systematic Review
2024cites this paper
A Workflow Roofline Model for End-to-End Workflow Performance Analysis
2024cites this paper
Validity constraints for data analysis workflows
2024cites this paper
ASA - The Adaptive Scheduling Algorithm
2024cites this paper
Benesh: a Framework for Choreographic Coordination of In Situ Workflows
2023cites this paper
Workflow Engines in the Compute Continuum: a Comparative Analysis
2023cites this paper
Application of classical and advanced machine learning models to predict personality on social media
2023cites this paper
GraspCC-LB: Dimensionamento de Recursos para Execução de Workflows em Ambientes de Computação de Alto Desempenho
2023cites this paper
Simulating IoT Workflows in DISSECT-CF-Fog
2023cites this paper
Design by Contract Revisited in the Context of Scientific Data Analysis Workflows
2023cites this paper
e-Science workflow: A semantic approach for airborne pollen prediction
2023cites this paper
A qualitative assessment of using ChatGPT as large language model for scientific workflow development
2023cites this paper
A learning‐based approach to regression analysis for climate data–A case of Northeast China
2023cites this paper
SciOps: Achieving Productivity and Reliability in Data-Intensive Research
2023cites this paper
CellHeap: A scRNA-seq workflow for large-scale bioinformatics data analysis
2023cites this paper
Efficient in-situ workflow planning for geographically distributed heterogeneous environments
2023cites this paper
Automatic Reproduction of Workflows in the Snakemake Workflow Catalog and nf-core Registries
2023cites this paper
How do users design scientific workflows? The Case of Snakemake
2023cites this paper
On Irregularity Localization for Scientific Data Analysis Workflows
2023cites this paper
A derived information framework for a dynamic knowledge graph and its application to smart cities
2023cites this paper
Signal Processing Based Method for Real-Time Anomaly Detection in High-Performance Computing
2023cites this paper
An improved Priority-Based Level Algorithm for just-in-time scheduling
2023cites this paper
Julia as a unifying end-to-end workflow language on the Frontier exascale system
2023cites this paper
Scientific workflow execution in the cloud using a dynamic runtime model
2023cites this paper
Validity Constraints for Data Analysis Workflows
2023cites this paper
Cybershuttle: An End-to-End Cyberinfrastructure Continuum to Accelerate Discovery in Science and Engineering
2023cites this paper
Design and modeling of moving target defense in workflow-based applications
2023cites this paper
Extending Composable Data Services into SmartNICs
2023cites this paper
Contract-Driven Design of Scientific Data Analysis Workflows
2023cites this paper
End-to-End Workflows for Climate Science: Integrating HPC Simulations, Big Data Processing, and Machine Learning
2023cites this paper
Predicting Dynamic Memory Requirements for Scientific Workflow Tasks
2023cites this paper
Reusability Challenges of Scientific Workflows: A Case Study for Galaxy
2023cites this paper
Real-World Experiences Adopting Workflows at Exascale on the ExaAM Project
2022cites this paper
Workflow Anomaly Detection with Graph Neural Networks
2022cites this paper
Automating Multivariable Workflow Composition for Model-to-Model Integration
2022cites this paper
Drones, virtual reality, and modeling: communicating catastrophic dam failure
2022cites this paper
GEO-WMS: an improved approach to geoscientific workflow management system on HPC
2022cites this paper
SparkFlow: Towards High-Performance Data Analytics for Spark-based Genome Analysis
2022cites this paper
A Perspective on Digital Knowledge Representation in Materials Science and Engineering
2022cites this paper
Ten simple rules for making a software tool workflow-ready
2022cites this paper
Framework para a Construção de Redes Filogenéticas em Ambiente de Computação de Alto Desempenho
2022influential citation
S-ProvFlow. Storing and Exploring Lineage Data as a Service
2022cites this paper
A flexible deadline-driven resource provisioning and scheduling algorithm for multiple workflows with VM sharing protocol on WaaS-cloud
2022cites this paper
Outcome-Preserving Input Reduction for Scientific Data Analysis Workflows
2022cites this paper
DayDream: Executing Dynamic Scientific Workflows on Serverless Platforms with Hot Starts
2022cites this paper
The Exascale Framework for High Fidelity coupled Simulations (EFFIS): Enabling whole device modeling in fusion science
2021cites this paper
A Review of Serverless Use Cases and their Characteristics
2021cites this paper
CellHeap: A Workflow for Optimizing COVID-19 Single-Cell RNA-Seq Data Processing in the Santos Dumont Supercomputer
2021cites this paper
A provenance model for control-flow driven scientific workflows
2021cites this paper
UNIVERSIDADE FEDERAL DE JUIZ DE FORA INSTITUTO DE CIÊNCIAS EXATAS PROGRAMA DE PÓS-GRADUAÇÃO EM CIÊNCIA DA COMPUTAÇÃO
2021cites this paper
Distributed workflows with Jupyter
2021cites this paper
WfCommons: A Framework for Enabling Scientific Workflow Research and Development
2021cites this paper
Distributed in-memory data management for workflow executions
2021influential citation
Scalable adaptive optimizations for stream-based workflows in multi-HPC-clusters and cloud infrastructures
2021cites this paper
Design considerations for workflow management systems use in production genomics research and the clinic
2021cites this paper
Executing cyclic scientific workflows in the cloud
2021cites this paper
Workflows Community Summit: Bringing the Scientific Workflows Community Together
2021cites this paper
Design and Performance Characterization of RADICAL-Pilot on Leadership-Class Platforms
2021cites this paper
A Taxonomy and Survey on Energy-Aware Scientific Workflows Scheduling in Large-Scale Heterogeneous Architecture
2021cites this paper
in Simulation of Weather and Climate in Europe Phase 2 Report on the ESDM runtime extensions for parallel in-flight analytics Deliverable
2021cites this paper
User Steering Support in Large-scale Workflows
2021cites this paper
Efficient Scheduling of Scientific Workflow Actions in the Cloud Based on Required Capabilities
2021cites this paper
Workflow Critical Path: A data-oriented critical path metric for Holistic HPC Workflows
2021cites this paper
DYFLOW: A flexible framework for orchestrating scientific workflows on supercomputers
2021cites this paper
A Community Roadmap for Scientific Workflows Research and Development
2021cites this paper
Mitigating Uncertainty in Developing Scientific Applications in Integrated Environment
2021cites this paper
Modular performance prediction for scientific workflows using Machine Learning
2021cites this paper
The Collaborative Research Center FONDA
2021cites this paper
Enabling discovery data science through cross-facility workflows
2021cites this paper
Store Scientific Workflows Data in SSHOC Repository
2020cites this paper
Data Flow Dependent Component Placement of Data Processing Cloud Applications
2020cites this paper
An edge-aware autonomic runtime for data streaming and in-transit processing
2020cites this paper
Priority research directions for in situ data management: Enabling scientific discovery from diverse data sources
2020influential citation
Computational Experiment Design for Operations Model Simulation
2020cites this paper
A Black-Box Fork-Join Latency Prediction Model for Data-Intensive Applications
2020cites this paper
Multiple Workflows Scheduling in Multi-tenant Distributed Systems
2020cites this paper
Polyflow: a Polystore-compliant Mechanism to Provide Interoperability to Heterogeneous Provenance Graphs
2020cites this paper