Combinatorial Testing on Implementations of HTML5 Support

The fifth version of HTML standard, which is widely accepted in the diverse landscape of browser vendors and their continuously upgrading releases, requires web browsers to support playback of multimedia natively, instead of by third-party plugins. Works on testing browsers' implementations of the HTML5 support, however, are not efficient enough till now. Regarding a browser's processing on HTML5 element tags of web pages and attributes of multimedia files, this paper treats the browser as a parameterized black-box and employs the combinatorial testing technique to design web pages to test its system behavior. Experiments are conducted on two sets of browsers. The first set includes nine popular ones in both desktop and mobile for discovering the distribution of multimedia related errors, and the second set contains five versions of the same browser for revealing the evolution of such errors. The experimental results indicate that the proposed approach is promising as it can reveal errors in browsers by various vendors and in various versions, and that the upgrades of the browser will not necessarily fix the existing bugs, and even introduce new ones, due to inefficient testing.