Recommendations for the Empirical Assessment of Human-AI Work Systems: A Contribution to AI Measurement Science

This Report is a companion to the Report titled "Requirements for the Evaluation of Human-AI Work Systems." Whereas that Report focused on the minimum necessary empirical requirements for the assessment of AI systems, this Report provides additional recommendations and technical details to assist the developers of AI systems. Recommendations are presented covering study design, research methods, measurement, statistical analyses, and online experimentation. This guidance should be applicable to all research intended to evaluate the effectivity of AI systems.