An apprenticeship learning scheme based on expert demonstrations for cross-layer routing design in cognitive radio networks

In cognitive radio, Reinforcement Learning (RL) has been widely applied to the construction of cognition engine. However, two crucial challenges remain to be resolved: First, it takes long time to interact with the environment before reaching intelligent decision. Second, agents improve their performance through trial and error, but some applications in Cognitive Radio Networks (CRN) cannot afford the extensive latency and energy expenditure. An apprenticeship learning scheme based on expert demonstrations is adopted for solving above problems. Firstly, the reinforcement responsibility rating is introduced to enhance the efficiency of power allocation by allowing multi-level transition for the transmit power. In order to avoid the failure of expert node identification due to SU’s remote location, an adaptive radius Bregman Ball model is presented. Furthermore, the Multi-Teacher Deep Q-learning from Demonstrations (MT-DQfD) is proposed to accelerate the learning procedure by sharing demonstrations derived from multiple expert nodes. Our experiments illustrate that the proposed cross-layer routing protocol reduces the training period while improves transmission quality compared to the traditional algorithms. Moreover, the newly-joined nodes can achieve better performance than the experts.