Open Access Repository

Orchestrating big data analysis workflows in the cloud: research challenges, survey, and future directions

Barika, MSM ORCID: 0000-0002-9146-2459, Garg, S ORCID: 0000-0003-3510-2464, Zomaya, AY, Wang, L, van Moorsel, A and Ranjan, R 2019 , 'Orchestrating big data analysis workflows in the cloud: research challenges, survey, and future directions' , ACM Computing Surveys , pp. 1-37 .

Full text not available from this repository.


Interest in processing big data has increased rapidly to gain insights that can transform businesses, government policies and research outcomes. This has led to advancement in communication, programming and processing technologies, including Cloud computing services and technologies such as Hadoop, Spark and Storm. This trend also affects the needs of analytical applications, which are no longer monolithic but composed of several individual analytical steps running in the form of a workflow. These Big Data Workflows are vastly different in nature from traditional workflows. Researchers are currently facing the challenge of how to orchestrate and manage the execution of such workflows. In this paper, we discuss in detail orchestration requirements of these workflows as well as the challenges in achieving these requirements. We also survey current trends and research that supports orchestration of big data workflows and identify open research challenges to guide future developments in this area.

Item Type: Article
Authors/Creators:Barika, MSM and Garg, S and Zomaya, AY and Wang, L and van Moorsel, A and Ranjan, R
Keywords: cloud computing, big data, map reduce, workflow orchestration, requirements, approaches
Journal or Publication Title: ACM Computing Surveys
Publisher: Assoc Computing Machinery
ISSN: 0360-0300
Copyright Information:

Copyright 2019 Association for Computing Machinery

Related URLs:
Item Statistics: View statistics for this item

Actions (login required)

Item Control Page Item Control Page