Efficient D2D content caching using multi-agent reinforcement learning

To address the increase of multimedia traffic dominated by streaming videos, user equipment (UE) can collaboratively cache and share contents to alleviate the burden of base-stations. Since content popularity is regional specific and does not have a stationary distribution, a machine learning based caching strategy that exploits the knowledge of content demand history would be highly promising. In this paper, we design D2D caching strategies using multi-agent reinforcement learning (MARL) with non-perfect content popularity information. Specifically, we model the D2D caching problem as a multi-agent multi-armed bandit problem. Since the joint action space is too large to use traditional MARL methods, a belief-based modified combinatorial upper confidence bound (MCUCB) algorithm is proposed to technically solve the problem. Simulation results show that the belief-based MCUCB caching scheme outperforms other popular caching schemes in terms of cache byte hit rate and average downloading latency.