Ant Colony Optimization Revisited from a Randomized Shortest Path Perspective

In this letter, it is shown that the randomized shortest-path framework (RSP, [15]) provides a theoretical interpretation of a class of ant colony optimization (ACO) algorithms, enjoying some nice properties. According to RSP, ants are sent from some initial node until they either eventually reach the goal node, or abandon and come back unsuccessfully along the same path. During their return travel (backward pass), each node on the trajectory is rewarded if the goal was reached – successful walk. The policy, which takes the form of the probabilities of following arc k → k′ in each node k, is updated periodically at each epoch t, and is set to the previous policy times (1) the proportion of successful walks starting from node k′ (probability of success, the pheromone), and (2) exp[−θckk′ ] (the heuristic function), where ckk′ is the cost associated to arc k → k′. The RSP framework shows that (i) this policy is optimal at any epoch t in that it minimizes the expected cost for reaching the goal node (exploitation) while maintaining a constant relative entropy spread in the graph (exploration), and (ii) the procedure converges to the minimal cost policy when t → ∞, provided the probability of success is well-estimated, that is, enough ants are sent at each epoch (asymptotic convergence). In other words, it provides an optimal trade-off between exploration and exploitation. We therefore decided to bring the RSP framework to the attention of the evolutionary computation community, hoping that it will stimulate the design as well as the empirical evaluation of new ACO algorithms having interesting theoretical properties.