Adaptive Parallelization of Queries Calling Dependent Data Providing Web Services

A common pattern in queries calling web services is that the output of one web service call is the input for another. The challenge addressed in this chapter is to develop methods to speed up such dependent calls by parallelization. We developed a system. WSMED, to provide a web service query service. WSMED automatically parallelizes the web service calls by starting separate query processes, each managing a parameterized sub-query, a plan function. The parallel plan is defined in terms of an operator. FF_APPLYP, to ship in parallel to other query processes the same plan function for different parameters. By using FF_APPLYP we first investigated ways to set up different process trees manually to find a best one. To automatically achieve the optimal process tree we modified FF_APPLYP to an operator AFF_APPLYP that adapts a parallel plan locally in each query process until an optimized performance is achieved. The query execution time obtained with AFF_APPLYP is shown to be close to the best process tree obtained with FF_APPLYP.