Transparent Practices for Quantitative Empirical Research

ABSTRACTTransparent research practices enable the research design, materials, analytic methods, and data to be thoroughly evaluated and potentially reproduced. The HCI community has recognized research transparency as one quality aspect of paper submission and review since CHI 2021. This course addresses HCI researchers and students who are already knowledgeable about experiment research design and statistical analysis. Building upon this knowledge, we will present current best practices and tools for increasing research transparency. We will cover relevant concepts and skills in Open Science, frequentist statistics, and Bayesian statistics, and uncertainty visualization. In addition to lectures, there will be hands-on exercises: The course participants will assess transparency practices in excerpts of quantitative reports, interactively explore implications of analytical choices using RStudio Cloud, and discuss their findings in small groups. In the final session, each participant will choose a case study based on their interest and assess its research transparency together with their classmates and instructors.