Hierarchical Deep Reinforcement Learning for Self-Powered Monitoring and Communication Integrated System in High-Speed Railway Networks

To align with the vision of future intelligent high-speed railway (HSR) networks, integrating sensor monitoring and remote communication are challenging for ensuring the lightweight of train equipment, high-quality transmissions, and dynamic interaction between monitoring and communication. In this paper, we propose a self-powered multisensor monitoring and communication integrated system in HSR. A low-power backscatter communication working framework of the self-powered monitoring system is designed in the monitoring network model, and a finite Gaussian mixture model (GMM) clustering method is used to analyze the communication cell coverage area in the communication network model. Aiming to minimize the total task completion time, we formulate a data monitoring and remote communication problem with the energy transfer constraint, data collection constraint, and transmission data rate constraint. As for the non-convex minimum time optimization problem, we develop a novel option-based hierarchical deep reinforcement learning (OHDRL) method to deal with the complex continuous variation characteristics of the monitoring and communication integrated HSR system. The system learns to select options at a high level, and the action is executed according to the policy of the selected option at a low level. This approach enables us to handle stochastic HSR environments, closed-loop policies, and goals in a temporal abstraction way. Numerical results reveal that the proposed algorithm for the integrated monitoring and communication HSR achieves a significantly higher reward and more stable learning performance than other algorithms in the literature.