Simulations of Ballot Polling Risk-Limiting Audits

In this paper we present simulation results comparing the risk, stopping probability, and number of ballots required over multiple rounds of ballot polling risk-limiting audits (RLAs) Minerva, SelectionOrdered (SO) Bravo, and End-of-Round (EoR) Bravo. Bravo is the most commonly used ballot polling RLA and requires the smallest expected number of ballots when ballots are drawn one at a time and the (true) underlying election is as announced. In real audits, multiple ballots are drawn at a time, and Bravo is implemented as SO Bravo or EoR Bravo. Minerva is a recently proposed ballot polling RLA that requires fewer ballots than either implementation of Bravo in a first round with stopping probability 0.9 but requires a predetermined round schedule. It is an open question how these audits compare over multiple rounds and for lower stopping probabilities. Our simulations use stopping probabilities of 0.9 and 0.25. The results are consistent with predictions of the R2B2 open-source library for ballot polling audits. We observe that both Bravo audits are more conservative than Minerva, which stops with fewer ballots, for both first round stopping probabilities. However, the advantage of using Minerva decreases considerably for the smaller first round stopping probability, as one would expect.