Information vs. Robustness in Rank Aggregation: Models, Algorithms and a Statistical Framework for Evaluation

The rank aggregation problem has been studied extensively in re- cent years with a focus on how to combine several dierent rankers to obtain a consensus aggregate ranker. We study the rank aggregation problem from a dierent perspective: how the individual input rankers impact the performance of the aggregate ranker. We develop a general statistical framework based on a model of how the individual rankers depend on the ground truth ranker. Within this framework, one can generate synthetic data sets and study the performance of dierent aggregation methods. The individual rankers, which are the inputs to the rank aggregation algorithm, are statistical perturbations of the ground truth ranker. With rigorous experimental evaluation, we study how noise level and the misinformation of the rankers aect the per- formance of the aggregate ranker. We introduce and study a novel