Multi-Agent DRL for Resource Allocation and Cache Design in Terrestrial-Satellite Networks

In the past few years, satellite communications have greatly affected our daily lives, and the integrated terrestrial-satellite network can combine the advantages of satellite and base stations (BSs) to provide wider coverage and lower cost. Because the resources of terrestrial-satellite network are limited, how to allocate resources of terrestrial-satellite network through effective methods has become a major challenge. This paper proposes a framework for resource allocation of terrestrial-satellite network based on non-orthogonal multiple access (NOMA). Then, a deployment method of local cache pools is given to achieve lower time delay and maximize energy efficiency in terrestrial-satellite network. In the proposed framework, we adopt a multi-agent deep deterministic policy gradient (MADDPG) method to obtain the maximum energy efficiency by user association, power control, and cache design. The MADDPG algorithm is divided into two stages, users and BSs are set as agents to complete the optimization problem in the framework. Finally, the simulation results show that the proposed method has better optimized performance compared with the traditional single-agent deep reinforcement learning algorithm and can efficiently solve the problems of resource allocation and cache design in the integrated terrestrial-satellite network.