Fairness Feedback Loops: Training on Synthetic Data Amplifies Bias
arxiv(2024)
摘要
Model-induced distribution shifts (MIDS) occur as previous model outputs
pollute new model training sets over generations of models. This is known as
model collapse in the case of generative models, and performative prediction or
unfairness feedback loops for supervised models. When a model induces a
distribution shift, it also encodes its mistakes, biases, and unfairnesses into
the ground truth of its data ecosystem. We introduce a framework that allows us
to track multiple MIDS over many generations, finding that they can lead to
loss in performance, fairness, and minoritized group representation, even in
initially unbiased datasets. Despite these negative consequences, we identify
how models might be used for positive, intentional, interventions in their data
ecosystems, providing redress for historical discrimination through a framework
called algorithmic reparation (AR). We simulate AR interventions by curating
representative training batches for stochastic gradient descent to demonstrate
how AR can improve upon the unfairnesses of models and data ecosystems subject
to other MIDS. Our work takes an important step towards identifying,
mitigating, and taking accountability for the unfair feedback loops enabled by
the idea that ML systems are inherently neutral and objective.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要