Communication Optimality Trade-offs For Distributed Estimation

This paper proposes 𝐂ommunication efficient 𝐑𝐄cursive 𝐃istributed estimati𝐎n algorithm, 𝒞ℛℰ𝒟𝒪, for networked multi-worker setups without a central master node. 𝒞ℛℰ𝒟𝒪 is designed for scenarios in which the worker nodes aim to collaboratively estimate a vector parameter of interest using distributed online time-series data at the individual worker nodes. The individual worker nodes iteratively update their estimate of the parameter by assimilating latest locally sensed information and estimates from neighboring worker nodes exchanged over a (possibly sparse) time-varying communication graph. The underlying inter-worker communication protocol is adaptive, making communications increasingly (probabilistically) sparse as time progresses. Under minimal conditions on the inter-worker information exchange network and the sensing models, almost sure convergence of the estimate sequences at the worker nodes to the true parameter is established. Further, the paper characterizes the performance of 𝒞ℛℰ𝒟𝒪 in terms of asymptotic covariance of the estimate sequences and specifically establishes the achievability of optimal asymptotic covariance. The analysis reveals an interesting interplay between the algorithm's communication cost 𝒞_t (over t time-steps) and the asymptotic covariance. Most notably, it is shown that 𝒞ℛℰ𝒟𝒪 may be designed to achieve a Θ(𝒞_t^-2+ζ) decay of the mean square error (ζ>0, arbitrarily small) at each worker node, which significantly improves over the existing Θ(𝒞_t^-1) rates. Simulation examples on both synthetic and real data sets demonstrate 𝒞ℛℰ𝒟𝒪's communication efficiency.