Estimating Fr\'echet bounds for validating programmatic weak supervision
arxiv(2023)
摘要
We develop methods for estimating Fr\'echet bounds on (possibly
high-dimensional) distribution classes in which some variables are
continuous-valued. We establish the statistical correctness of the computed
bounds under uncertainty in the marginal constraints and demonstrate the
usefulness of our algorithms by evaluating the performance of machine learning
(ML) models trained with programmatic weak supervision (PWS). PWS is a
framework for principled learning from weak supervision inputs (e.g.,
crowdsourced labels, knowledge bases, pre-trained models on related tasks,
etc), and it has achieved remarkable success in many areas of science and
engineering. Unfortunately, it is generally difficult to validate the
performance of ML models trained with PWS due to the absence of labeled data.
Our algorithms address this issue by estimating sharp lower and upper bounds
for performance metrics such as accuracy/recall/precision/F1 score.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要