An axiomatic distance methodology for aggregating multimodal evaluations

This work introduces a multimodal data aggregation methodology featuring optimization models and algorithms for jointly aggregating heterogeneous ordinal and cardinal evaluation inputs into a consensus evaluation. Specifically, this work derives mathematical modeling components to enforce three types of logical couplings between the collective ordinal and cardinal evaluations: Rating and ranking preferences, numerical and ordinal estimates, and rating and approval preferences. The proposed methodology is based on axiomatic distances rooted in social choice theory. Moreover, it adequately deals with highly incomplete evaluations, tied values, and other complicating aspects of group decision-making contexts. We illustrate the practicality of the proposed methodology in a case study involving an academic student paper competition. The methodology’s advantages and computational aspects are further explored via synthetic instances sampled from distributions parametrized by ground truths and varying noise levels. These results show that multimodal aggregation effectively extracts a collective truth from noisy information sources and successfully captures the distinctive evaluation qualities of rating and ranking preference data.