Reproducibility and Experimental Design for Machine Learning on Audio and Multimedia Data

This tutorial provides an actionable perspective on the experimental design for machine learning experiments on multimedia data. The tutorial consists of lectures and hands-on exercises. The lectures provide an engineering introduction to machine learning design. By understanding the information flow and quantities in the scientific process, machine learners can be designed to be more efficient and their limits can be easier understood. The thought framework presented is derived from the traditional experimental sciences which require published results to be self-contained with regards to reproducibility. In the practical exercises, we will work on calculating and measuring quantities like Memory Equivalent Capacity or generalization ratio for different machine learners and data sets and discuss how these quantities relate to reproducible experimental design.