Applying Combinatorial Testing to Verification-Based Fairness Testing.

Fairness testing, given a machine learning classifier, detects discriminatory data contained in it via executing test cases. In this paper, we propose a new approach to fairness testing named V-BT-CT, which applies combinatorial t-way testing (CT) to Verification Based Testing (V-BT). V-BT is a state-of-the-art fairness testing method, which represents a given classifier under test in logical constraints and searches for test cases by solving such constraints. CT is a coverage-based sampling technique, with an ability to sample diverse test data from a search space specified by logical constraints. We implement a proof-of-concept of V-BT-CT, and see its feasibility by experiments. We also discuss its advantages, current limitations, and further research directions.