# Bounding the fairness and accuracy of classifiers from population statistics

ICML 2020, 2020.

Keywords:

false negative ratefairness definitionpractical procedureinsurance companypositive predictionMore(7+)

Weibo:

Abstract:

We consider the study of a classification model whose properties are impossible to estimate using a validation set, either due to the absence of such a set or because access to the classifier, even as a black-box, is impossible. Instead, only aggregate statistics on the rate of positive predictions in each of several sub-populations are a...More

Code:

Data:

Introduction

- Suppose that a health insurance company uses some unpublished method to decide whether a person should be classified as “at risk” for diabetes, for instance so as to offer diabetes screening.
- The true proportions of diabetes in each state are known
- Based only on these two types of aggregate statistics—the true positive rate in each state, and the predicted positive rate in each state— what can be inferred regarding the fairness and/or accuracy of the classifier?
- Can this information be used to make any inferences about the other property?

Highlights

- Suppose that a health insurance company uses some unpublished method to decide whether a person should be classified as “at risk” for diabetes, for instance so as to offer diabetes screening
- The true proportions of diabetes in each state are known. Based only on these two types of aggregate statistics—the true positive rate in each state, and the predicted positive rate in each state— what can be inferred regarding the fairness and/or accuracy of the classifier? suppose that the company publishes some additional information about the accuracy of the classifier, or about its fairness with respect to the state of residence
- We focus on the notion of equalized odds (Hardt et al, 2016), which requires equal false positive rate and false negative rate in each sub-population, where a sub-population is the set of individuals who share the same value of the attribute of interest
- We showed that useful bounds on fairness and accuracy can be provided for classifiers based only on the aggregate statistics on predicted positive rates and true positive rates
- We defined a new unfairness measure to facilitate the study of classifiers that are not completely fair, and provided an efficient and practical procedure which provably lower-bounds a given trade-off between fairness and accuracy
- Even if the classifier has zero error, upper bounding the error must take into account the possibility of no overlap between the positive predictions and the true positives, it would be at least twice the positive rate

Methods

- The authors show two types of experiments: First, the authors test the tightness of the lower bound the authors obtain for discβ, by comparing it with the true discβ for the classifier, as calculated using labeled data.
- In the second set of experiments, the authors demonstrate possible uses and outcomes of the lower-bounding procedure.
- The authors study several classifiers for which the authors only have aggregate statistics, calculate lower-bound Pareto curves of the trade-off between unfairness and error for each classifier, and discuss how these curves can help in decision making.
- Matlab code for Alg. 1, as well as experiment data and code, are available at https://github.com/sivansabato/bfa

Results

- If the resulting classification problem had more than 99% of the examples assigned to the same label, this classification problem was discarded.

Conclusion

- The authors showed that useful bounds on fairness and accuracy can be provided for classifiers based only on the aggregate statistics on predicted positive rates and true positive rates.
- The authors defined a new unfairness measure to facilitate the study of classifiers that are not completely fair, and provided an efficient and practical procedure which provably lower-bounds a given trade-off between fairness and accuracy.
- Obtaining a meaningful upper bound on the discrepancy is a challenging open problem that may require additional modeling assumptions

Summary

## Introduction:

Suppose that a health insurance company uses some unpublished method to decide whether a person should be classified as “at risk” for diabetes, for instance so as to offer diabetes screening.- The true proportions of diabetes in each state are known
- Based only on these two types of aggregate statistics—the true positive rate in each state, and the predicted positive rate in each state— what can be inferred regarding the fairness and/or accuracy of the classifier?
- Can this information be used to make any inferences about the other property?
## Methods:

The authors show two types of experiments: First, the authors test the tightness of the lower bound the authors obtain for discβ, by comparing it with the true discβ for the classifier, as calculated using labeled data.- In the second set of experiments, the authors demonstrate possible uses and outcomes of the lower-bounding procedure.
- The authors study several classifiers for which the authors only have aggregate statistics, calculate lower-bound Pareto curves of the trade-off between unfairness and error for each classifier, and discuss how these curves can help in decision making.
- Matlab code for Alg. 1, as well as experiment data and code, are available at https://github.com/sivansabato/bfa
## Results:

If the resulting classification problem had more than 99% of the examples assigned to the same label, this classification problem was discarded.## Conclusion:

The authors showed that useful bounds on fairness and accuracy can be provided for classifiers based only on the aggregate statistics on predicted positive rates and true positive rates.- The authors defined a new unfairness measure to facilitate the study of classifiers that are not completely fair, and provided an efficient and practical procedure which provably lower-bounds a given trade-off between fairness and accuracy.
- Obtaining a meaningful upper bound on the discrepancy is a challenging open problem that may require additional modeling assumptions

Related work

- Fairness in classification has been a highly studied topic of research in recent years, due to its importance in legal, financial, and medical decisions (Barocas et al, 2017). This importance has grown in parallel with the wide application of automated (and frequently, opaque) models in multiple areas affecting people. Various notions of fairness have been proposed (see, e.g., Dwork et al, 2012; Grgic-Hlaca et al, 2016; Kusner et al, 2017; Berk et al, 2018; Verma & Rubin, 2018). In this work, we focus on the notion of equalized odds (Hardt et al, 2016), which requires equal FPRs and FNRs in each sub-population, where a sub-population is the set of individuals who share the same value of the attribute of interest. Many works propose methods for learning fair classifiers under the equalizedodds definition (see, e.g., Feldman et al, 2015; Hardt et al, 2016; Goh et al, 2016; Zafar et al, 2017b;a; Woodworth et al, 2017; Wu et al, 2019). Learning methods that guarantee or approximate other definitions of fairness, such as equal opportunity and demographic fairness, have also been widely studied in recent years (e.g., Dwork et al, 2012; Zemel et al, 2013; Calmon et al, 2017b; Donini et al, 2018; Goel et al, 2018; Johndrow & Lum, 2019).

Reference

- Barocas, S., Hardt, M., and Narayanan, A. Fairness in machine learning. NIPS Tutorial, 2017.
- Bellamy, R. K., Dey, K., Hind, M., Hoffman, S. C., Houde, S., Kannan, K., Lohia, P., Martino, J., Mehta, S., Mojsilovic, A., et al. Ai fairness 360: An extensible toolkit for detecting and mitigating algorithmic bias. IBM Journal of Research and Development, 63(4/5):4–1, 2019.
- Berk, R., Heidari, H., Jabbari, S., Kearns, M., and Roth, A. Fairness in criminal justice risk assessments: The state of the art. Sociological Methods & Research, pp. 0049124118782533, 2018.
- Black, E., Yeom, S., and Fredrikson, M. Fliptest: Fairness auditing via optimal transport. CoRR, abs/1906.09218, 2019.
- Calmon, F., Wei, D., Vinzamuri, B., Natesan Ramamurthy, K., and Varshney, K. R. Optimized pre-processing for discrimination prevention. In Guyon, I., Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R. (eds.), Advances in Neural Information Processing Systems 30, pp. 3992–4001. Curran Associates, Inc., 2017a.
- Calmon, F., Wei, D., Vinzamuri, B., Natesan Ramamurthy, K., and Varshney, K. R. Optimized pre-processing for discrimination prevention. In Guyon, I., Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R. (eds.), Advances in Neural Information Processing Systems 30, pp. 3992–4001. Curran Associates, Inc., 2017b.
- CDC and NCI. United states cancer statistics: Data visualizations, 2019. URL https://gis.cdc.gov/ Cancer/USCS/DataViz.html.
- Chen, J., Kallus, N., Mao, X., Svacha, G., and Udell, M. Fairness under unawareness: Assessing disparity when protected class is unobserved. In Proceedings of the Conference on Fairness, Accountability, and Transparency, pp. 339–34ACM, 2019.
- Donini, M., Oneto, L., Ben-David, S., Shawe-Taylor, J. S., and Pontil, M. Empirical risk minimization under fairness constraints. In Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., and Garnett, R. (eds.), Advances in Neural Information Processing Systems 31, pp. 2791–2801. Curran Associates, Inc., 2018.
- Dua, D. and Graff, C. UCI machine learning repository, 2019. URL http://archive.ics.uci.edu/ml.
- Dwork, C., Hardt, M., Pitassi, T., Reingold, O., and Zemel, R. Fairness through awareness. In Proceedings of the 3rd innovations in theoretical computer science conference, pp. 214–226. ACM, 2012.
- Federal Elections Commission. Federal elections 2016, 2016. URL https://transition.fec.gov/pubrec/fe2016/federalelections2016.pdf.
- Feldman, M., Friedler, S. A., Moeller, J., Scheidegger, C., and Venkatasubramanian, S. Certifying and removing disparate impact. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 259–268. ACM, 2015.
- National presidential polls, november 8th, 2016, 2016.
- https://projects.fivethirtyeight.
- Goel, N., Yaghini, M., and Faltings, B. Non-discriminatory machine learning through convex fairness criteria. In Thirty-Second AAAI Conference on Artificial Intelligence, 2018.
- Goh, G., Cotter, A., Gupta, M., and Friedlander, M. P. Satisfying real-world goals with dataset constraints. In Lee, D. D., Sugiyama, M., Luxburg, U. V., Guyon, I., and Garnett, R. (eds.), Advances in Neural Information Processing Systems 29, pp. 2415–2423. Curran Associates, Inc., 2016.
- Grgic-Hlaca, N., Zafar, M. B., Gummadi, K. P., and Weller, A. The case for process fairness in learning: Feature selection for fair decision making. In NIPS Symposium on Machine Learning and the Law, volume 1, pp. 2, 2016.
- Hardt, M., Price, E., and Srebro, N. Equality of opportunity in supervised learning. In Advances in neural information processing systems, pp. 3315–3323, 2016.
- Heidari, H., Ferrari, C., Gummadi, K., and Krause, A. Fairness behind a veil of ignorance: A welfare analysis for automated decision making. In Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., and Garnett, R. (eds.), Advances in Neural Information Processing Systems 31, pp. 1265–1276. Curran Associates, Inc., 2018.
- Jackson, C., Best, N., and Richardson, S. Improving ecological inference using individual-level data. Statistics in medicine, 25(12):2136–2159, 2006.
- Jackson, C., Best, N., and Richardson, S. Hierarchical related regression for combining aggregate and individual data in studies of socio-economic disease risk factors. Journal of the Royal Statistical Society: Series A (Statistics in Society), 171(1):159–178, 2008.
- Johndrow, J. E. and Lum, K. An algorithm for removing sensitive information: application to race-independent recidivism prediction. The Annals of Applied Statistics, 13(1):189–220, 2019.
- Kleinberg, J., Mullainathan, S., and Raghavan, M. Inherent trade-offs in the fair determination of risk scores. In 8th Innovations in Theoretical Computer Science Conference (ITCS 2017). Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik, 2017.
- Kusner, M. J., Loftus, J., Russell, C., and Silva, R. Counterfactual fairness. In Guyon, I., Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R. (eds.), Advances in Neural Information Processing Systems 30, pp. 4066–4076. Curran Associates, Inc., 2017.
- McDuff, D., Ma, S., Song, Y., and Kapoor, A. Characterizing bias in classifiers using generative models. In Advances in Neural Information Processing Systems 32, pp. 5404–5415. Curran Associates, Inc., 2019.
- Menon, A. K. and Williamson, R. C. The cost of fairness in binary classification. In Conference on Fairness, Accountability and Transparency, pp. 107–118, 2018.
- Pleiss, G., Raghavan, M., Wu, F., Kleinberg, J., and Weinberger, K. Q. On fairness and calibration. In Advances in Neural Information Processing Systems, pp. 5680–5689, 2017.
- Soldaini, L. and Yom-Tov, E. Inferring individual attributes from search engine queries and auxiliary information. In Proceedings of the 26th international conference on World Wide Web, pp. 293–301, 2017.
- Speicher, T., Heidari, H., Grgic-Hlaca, N., Gummadi, K. P., Singla, A., Weller, A., and Zafar, M. B. A unified approach to quantifying algorithmic unfairness: Measuring individual &group unfairness via inequality indices. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 2239–2248, 2018.
- Sun, T., Sheldon, D., and OConnor, B. A probabilistic approach for learning with label proportions applied to the us presidential election. In 2017 IEEE International Conference on Data Mining (ICDM), pp. 445–454, Nov 2017. doi: 10.1109/ICDM.2017.54.
- US Census Bureau. Annual estimates of the resident population for the united states, regions, states, and puerto rico: April 1, 2010 to july 1, 2019, 2019. URL https://www2.census.gov/programs-surveys/popest/tables/2010-2019/state/totals/nst-est2019-01.xlsx.
- Verma, S. and Rubin, J. Fairness definitions explained. In 2018 IEEE/ACM International Workshop on Software Fairness (FairWare), pp. 1–7. IEEE, 2018.
- Woodworth, B., Gunasekar, S., Ohannessian, M. I., and Srebro, N. Learning non-discriminatory predictors. In Kale, S. and Shamir, O. (eds.), Proceedings of the 2017 Conference on Learning Theory, volume 65 of Proceedings of Machine Learning Research, pp. 1920–1953, Amsterdam, Netherlands, 07–10 Jul 2017. PMLR.
- Wu, Y., Zhang, L., and Wu, X. On convexity and bounds of fairness-aware classification. In The World Wide Web Conference, pp. 3356–3362. ACM, 2019.
- Zafar, M. B., Valera, I., Gomez Rodriguez, M., and Gummadi, K. P. Fairness beyond disparate treatment & disparate impact: Learning classification without disparate mistreatment. In Proceedings of the 26th International Conference on World Wide Web, pp. 1171–1180. International World Wide Web Conferences Steering Committee, 2017a.
- Zafar, M. B., Valera, I., Rogriguez, M. G., and Gummadi, K. P. Fairness constraints: Mechanisms for fair classification. In Artificial Intelligence and Statistics, pp. 962– 970, 2017b.
- Zemel, R., Wu, Y., Swersky, K., Pitassi, T., and Dwork, C. Learning fair representations. In Dasgupta, S. and McAllester, D. (eds.), Proceedings of the 30th International Conference on Machine Learning, volume 28(3) of Proceedings of Machine Learning Research, pp. 325– 333, Atlanta, Georgia, USA, 17–19 Jun 2013. PMLR.

Tags

Comments