Effect of TV Content in Subjective Assessment of Video Quality on Mobile Devices

Selection of test materials in subjective assessment methodology recommendations is based mainly on technical parameters. Materials should test the ability of the codec to cope with spatial and temporal redundancy. However consumers watch TV for a reason - one of the main criteria is the interesting content. In this study we examined whether the content recognition and subjects' personal interests have an effect on quality assessment. We also studied subjective assessment criteria for video quality. The study was done using small resolution and low bit rate video in mobile phones in a laboratory environment. Altogether 135 subjects, aged 18-65 years, participated in the tests. The test started with a subjective assessment of video quality using well-known TV content. Afterwards a survey was done to measure content recognition and level of interests in the content. The test session ended up with a qualitative interview about evaluation criteria. Our studies showed that there is a connection between interest in content and given quality score with TV content. Therefore we raise a concern on content selection and recommend measuring the evaluator's interest in content in subjective assessment studies. The study on subjective evaluation criteria revealed that subjects pay attention on content and quality impairments especially in regions of interest.