Lessons Learned from EXMOS User Studies: A Technical Report Summarizing Key Takeaways from User Studies Conducted to Evaluate The EXMOS Platform
CoRR(2023)
摘要
In the realm of interactive machine-learning systems, the provision of
explanations serves as a vital aid in the processes of debugging and enhancing
prediction models. However, the extent to which various global model-centric
and data-centric explanations can effectively assist domain experts in
detecting and resolving potential data-related issues for the purpose of model
improvement has remained largely unexplored. In this technical report, we
summarise the key findings of our two user studies. Our research involved a
comprehensive examination of the impact of global explanations rooted in both
data-centric and model-centric perspectives within systems designed to support
healthcare experts in optimising machine learning models through both automated
and manual data configurations. To empirically investigate these dynamics, we
conducted two user studies, comprising quantitative analysis involving a sample
size of 70 healthcare experts and qualitative assessments involving 30
healthcare experts. These studies were aimed at illuminating the influence of
different explanation types on three key dimensions: trust, understandability,
and model improvement. Results show that global model-centric explanations
alone are insufficient for effectively guiding users during the intricate
process of data configuration. In contrast, data-centric explanations exhibited
their potential by enhancing the understanding of system changes that occur
post-configuration. However, a combination of both showed the highest level of
efficacy for fostering trust, improving understandability, and facilitating
model enhancement among healthcare experts. We also present essential
implications for developing interactive machine-learning systems driven by
explanations. These insights can guide the creation of more effective systems
that empower domain experts to harness the full potential of machine learning
更多查看译文
关键词
exmos user studies,user studies,technical report summarizing
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要