Fictitious Play: Convergence, Smoothness, and Optimism

We consider the dynamics of two-player zero-sum games, with the goal of understanding when such dynamics lead to equilibrium behavior at a fast rate. In particular, we study the dynamic known as \emph{fictitious play} (FP) in which each player simultaneously best-responds to the empirical distribution of the historical plays of their opponent. Nearly 70 years ago it was shown by Robinson that FP does converge to the Nash Equilibrium, although the rate she proved was exponential in the total number of actions of the players. In 1959, Karlin conjectured that FP converges at the more natural rate of $O(1/\epsilon^2)$. However, Daskalakis and Pan disproved a version of this conjecture in 2014, showing that an exponentially-slow rate can occur, although their result relied on adversarial tie-breaking. In this paper, we show that Karlin's conjecture is indeed correct in two major instances if you appropriately handle ties. First, we show that if the game matrix is diagonal and ties are broken lexicographically, then FP converges at a $O(1/\epsilon^2)$ rate, and we also show a matching lower bound under this tie-breaking assumption. Our second result shows that FP converges at a rate of $O(1/\epsilon^2)$ when the players' decision sets are smooth, and $\tilde{O}(1/\epsilon)$ under an additional assumption. In this last case, we also show that a modification of FP, known as Optimistic FP, converges at a rate of $O(1/\epsilon)$.