Cognitive radar mode control: a comparison of different reinforcement learning algorithms

This paper describes the use of deep reinforcement learning (RL) to apply the concept of cognition in sensing systems to the choice of operational radio frequency (RF) mode (active, bistatic receive, electronic surveillance (ES), electronic protection measure (EPM)) for a multi-function RF system (MFRFS). This is investigated in a simulated air-to-air combat scenario, with the RL on a blue fast jet rewarded for successfully guiding a missile to the opposition, a red fast jet, and penalised if the red jet is successful. Three RL algorithms (deep Q-network (DQN), advantage actor-critic (A2C), and proximal policy optimisation (PPO)) are compared with baselines that include the 4 static modes and a set of fixed rulesets, and it is shown that - after hyperparameter tuning - the algorithms perform comparably to these baselines. It is suggested that PPO might be the optimal algorithm in this context.