The Michigan Benchmark: Towards Xml Query Performance Diagnostics

We propose a micro-benchmark for XML data management to aid engineers in designing improved XML processing engines. This benchmark is inherently different from application-level benchmarks, which are designed to help users choose between alternative products. We primarily attempt to capture the rich variety of data structures and distributions possible in XML, and to isolate their effects, without imitating any particular application. The benchmark specifies a single data set against which carefully specified queries can be used to evaluate system performance for XML data with various characteristics.We have used the benchmark to analyze the performance of three database systems: two native XML DBMSs, and a commercial ORDBMS. The benchmark reveals key strengths and weaknesses of these systems. We find that commercial relational techniques are effective for XML query processing in many cases, but are sensitive to query rewriting, and require better support for efficiently determining indirect structural containment. In addition, the benchmark also played an important role in helping the development team of Timber (our native XML DBMS) devise a more effective access method, and fine tune the implementation of the structural join algorithms. (c) 2004 Elsevier B.V. All rights reserved.