Multi-objective optimization-based workflow scheduling for applications with data locality and deadline constraints in geo-distributed clouds

Geo-distributed clouds have emerged as a new generation of cloud computing paradigm, in which each cloud is operated and managed by independent cloud service providers (CSPs). By enhancing cooperation among CSPs, it can offer efficient cross-cloud services. In geo-distributed clouds, the resources offered by CSPs are heterogeneous with different billing mechanisms and the data required by workflow applications are geographically distributed with locality characteristics. As such, it is significantly challenging for cloud users to select the appropriate resources to execute their workflow applications. In this paper, we model the constrained multi-objective workflow scheduling problem (CMWSP) in geo-distributed clouds as a constrained multi-objective optimization problem that minimizes both workflow makespan and resource rental costs. To solve the CMWSP, we propose a multi-objective multi-workflow scheduling mechanism (MOMWS), which integrates workflow preprocessing, evolutionary multi-objective optimization and intensification strategy while explicitly considering the data locality characteristics, deadline requirements, and rental period reuse. Specifically, we first design a task preprocessing algorithm for workflow applications to reduce transferred data volume by merging tasks with the same original datasets. Based on this algorithm, we introduce a priority assignment algorithm to decide the scheduling sequence of workflow applications. We next propose a makespan and cost-aware workflow scheduling algorithm to determine a set of high-quality approximations of the Pareto front to the CMWSP. Based on real-world CSPs and workflow applications, extensive experiments are carried out to demonstrate the effectiveness and efficiency of MOMWS.