Combining Binary Classifiers Leads to Nontransitive Paradoxes

We study collections of binary classifiers trained on different pairs of target classes and observe cases of nontransitive disagreement. Noting a similar phenomenon in voting theory, we seek to understand how this nontransitivity emerges from aggregation and the limits of its behavior. We prove that a previously developed framework, called expert graphs, is equivalent to a framework used in voting theory called linear ordering graphs (LOGs). Finally, we show that the curl condition, which limits the degree of nontransitivity in expert graphs and LOGs, precisely characterizes the maximum possible nontransitivity in two types of planar graphs, namely triangulated cycles and wheel graphs, and we conjecture that it does so for all planar graphs.