Stationary equilibria of N-player stochastic games

Abstract We provide a characterisation of the set of stationary equilibria for finite discountedstochastic games using an approach inspired by Atkinson (Academic Press, 1972). In the zero-sum case, this approach has already led to a characterisation of the discounted values and of the limit value (Attia and Oliu-Barton, Proceedings of the National Academy of Sciences, 2019), and to efficient algorithms to compute these quantities (Oliu-Barton, Mathematics of Operations Research, 2021). The extension of these results to the non-zero sum case, that is an efficient computation of stationary equilibria, and a characterisation of the limit set of stationary equilibrium payoffs, which is known to exist (Renault and Ziliotto, Mathematics of Operations Research, 2020), are two very hard open problems, which could be tackled using our characterisation. Interestingly, this characterisation remains valid for a broader class of games, the so-called normal-form games with complementarities, which models the static interaction of several firms on finitely many complementary markets.