Asynchronous Value Iteration Network

Value iteration network (VIN) improves the generalization of a policy-based neural network by embedding a planning module. However, this module performs value iteration on the entire state space of a Markov decision process and all states in the space are updated by sweeping the state space systematically, regardless of their significance. This paper introduces an improved version of VIN with a novel planning module, called asynchronous value iteration network (AVIN), performing value updates on some states more frequently than other states asynchronously, depending on their significance/urgency to improve a policy. The new planning module utilizes the urgency of the states to prioritize updates at important states. We measure the urgency in a way of enhancing the global awareness, leading to an improvement of the generalization ability of policies. AVIN with the new module makes the value updates more efficient and effective, thus significantly demonstrating better generalization on unknown environments.