Adaptive Neighbor Discovery Scheme for Directional Ad Hoc Network.

This paper proposes an adaptive neighbor discovery scheme for directional ad hoc network, and models the problem of directional ad hoc network neighbor discovery as a distributed multi-agent reinforcement learning model. The nodes in the network are independent agents, using Q learning algorithm whose behavior strategy is UCB (Upper Confidence Bound) to discover one-hop neighbors independently. Due to the estimation deviation of the value function and the non-stationarity of the directional ad hoc network, the behavior strategy of Q learning algorithm needs to explore new state-action pairs. This paper uses the UCB behavior strategy and compares it with the ɛ-greedy strategy, the attenuated ɛ-greedy strategy and the softmax strategy. It is obtained that the UCB strategy has the best performance in the problem of directional ad hoc network neighbor discovery.