Recursive Truth Estimation of Time-Varying Sensing Data from Online Open Sources

This paper is motivated by prospective Internet of Things (IoT) applications that exploit inputs from online open sources whose reliability may be uncertain. Unlike physical signal fusion (that can leverage solid analytic foundations derived from physical properties of fused signals), data reliability assessment from arbitrary online open sources is a harder problem. At least two difficulties arise. First, source reliability is harder to estimate from first principles due to lack of visibility into the sensing and subsequent processing stages for published data. Second, by virtue of being open, some sources can be copied by others, leading to correlated errors at a large scale. This paper presents a recursive truth estimator for online public data streams that addresses the above two problems. We focus on categorical data. Many truth-finding systems were developed to cope with unreliable categorical data. Most of them are designed for batch analysis of bulk datasets. This work extends previous efforts by developing an online recursive estimator. Unlike previous recursive fact-finders, ours is the first that can jointly handle (i) changes in the population of sources over time, (ii) changes in the ground-truth state of the physical phenomenon being observed (that result in the appearance of conflicting claims), and (iii) correlated errors due to potential copying among sources. Results show that our algorithm not only outperforms other recursive fact-finders in the case of changing ground-truth state, but also improves estimation accuracy of static state.