Combinatorial Testing on MP3 for Audio Players

Audio players form a type of frequently used software, in which there are still bugs. In this paper, we investigate the test on MP3 for audio player, a mainstream audio format. An MP3 file consists of ID3v2 tag and MPEG layer III frames, with which there are standards for them. We first introduce the standards through the official documents, then summarize several potentially error-prone points for audio players. For our purpose, we use two black-box test design methods, combinatorial testing namely (CT) and random testing (RT). We test 10 popular audio players and 4 web browser built-in players for supporting audio tag in html5 with these test cases and find 7 types of bugs. Our experimental results show that, although there are standards for ID3v2 tag and MPEG frames, different audio player implementors have different understanding and implementations. We also compare the bugs detected by the test cases with CT and RT.