Self-Optimization of Cellular Networks Using Deep Reinforcement Learning with Hybrid Action Space

Wireless networks have been going through tremendous proliferation recently. As a result, a continuous configuration and management are necessary to sustain a balanced performance while facing such continued growth and endless changes. A self-managed network is required to replace manual management, which is costly, time-consuming, and error-prone. In this paper, we propose a machine-learning-based cellular network management system. The proposed system aims to enhance the network stability and adaptability to temporal changes (e.g., load imbalances across cells). The presented approach is a deep reinforcement learning scheme that enables a network manager to learn a policy that maximizes the network average sum throughput while trying to minimize the consumed energy and the number of blocked users. In addition to controlling the transmitted power and the cell individual offset, MIMO can be switched ON and OFF to control the consumed energy without affecting the quality of service. This results in a hybrid action space, i.e., our action vector has some binary actions as well as continuous actions. We present a novel algorithm to deal with this hybrid action space. Our results reveal that our proposed algorithm is flexible, efficient, and reliable. We report significant performance gains compared to some baselines (without self-management) and previously proposed algorithms.