Ecosystem-level Analysis of Deployed Machine Learning Reveals Homogeneous Outcomes
arxiv(2023)
摘要
Machine learning is traditionally studied at the model level: researchers
measure and improve the accuracy, robustness, bias, efficiency, and other
dimensions of specific models. In practice, the societal impact of machine
learning is determined by the surrounding context of machine learning
deployments. To capture this, we introduce ecosystem-level analysis: rather
than analyzing a single model, we consider the collection of models that are
deployed in a given context. For example, ecosystem-level analysis in hiring
recognizes that a job candidate's outcomes are not only determined by a single
hiring algorithm or firm but instead by the collective decisions of all the
firms they applied to. Across three modalities (text, images, speech) and 11
datasets, we establish a clear trend: deployed machine learning is prone to
systemic failure, meaning some users are exclusively misclassified by all
models available. Even when individual models improve at the population level
over time, we find these improvements rarely reduce the prevalence of systemic
failure. Instead, the benefits of these improvements predominantly accrue to
individuals who are already correctly classified by other models. In light of
these trends, we consider medical imaging for dermatology where the costs of
systemic failure are especially high. While traditional analyses reveal racial
performance disparities for both models and humans, ecosystem-level analysis
reveals new forms of racial disparity in model predictions that do not present
in human predictions. These examples demonstrate ecosystem-level analysis has
unique strengths for characterizing the societal impact of machine learning.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要