A Proposal to Orchestrate Test Cases

This paper presents the concept of test orchestration, understood as a novel way to select, order, and execute in parallel a group of tests. Our view of test orchestration can be seen as a process in which different test cases are organized, assembled and executed following a topology that determines how their executions coordinate. We distinguish two types of orchestrations techniques: i) verdict-driven, which organizes tests using their outcome (i.e., passed or failed) to drive the workflow; and ii) data-driven, in which test data (input) and test outcomes (output) are handled within the graph. Both approaches are being implemented in the project ElasTest, an open source platform aimed to simplify the end-to-end test process of large software systems.