AoI-Oriented Status Updating in Large-scale Heterogeneous Multi-Channel Systems

In this work, we study the age-optimal status update strategy for a large number of sensors in a wireless system with multiple heterogeneous unreliable channels. Particularly, we first formulate the status update procedure as a Restless Multi-Armed Bandit (RMAB) problem so as to minimize the long-term average Age of Information (AoI) cost. Then, a Deep Whittle index-based Q-Network (DWQN) algorithm is devised to solve it, in which an accurate approximation of the Whittle index can be learned. By applying this algorithm, the challenges from both the unknown of the environmental dynamics and large-scale state space can be addressed. Finally, simulations are conducted to validate the effectiveness of our proposed algorithm by comparing it with baseline strategies.