Evaluation Mechanism for Decentralized Collaborative Pattern Learning in Heterogeneous Vehicular Networks

Collaborative machine learning, especially Federated Learning (FL), is widely used to build high-quality Machine Learning (ML) models in the Internet of Vehicles (IoV). In this paper, we study the performance evaluation problem in an inherently heterogeneous IoV, where the final models across the network are not identical and are computed on different standards. Previous studies assume that local agents are receiving data from the same phenomenon, and a same final model is fitted to them. However, this "one model fits all" approach leads to a biased performance evaluation of individual agents. We propose a general approach to measure the performance of individual agents, where the common knowledge and correlation between different agents are explored. Experimental results indicate that our evaluation scheme is efficient in these settings.