Impact of Modelling Approach on the Estimation of Government Intervention Effects in COVID-19

The efficacy of government interventions in epidemic has become a hot subject since the onset of COVID-19. There is however much variation in the results quantifying the effects of interventions, which is partly related to the varying modelling approaches employed by existing studies. This paper therefore aims to examine how the choice of modelling approach would affect the estimation results of intervention effects, by experimenting with different modelling approaches on a same data set composed of the 500 most affected U.S. counties. We compare the most frequently used methods from the two classes of modelling approaches, which are Bayesian hierarchical model from the class of computational approach and difference-in-difference from the class of natural experimental approach. We find that computational methods are likely to produce larger estimates of intervention effects due to simultaneous voluntary behavioral changes. In contrast, natural experimental methods are more likely to extract the true effect of interventions. Among different difference-in-difference estimators, the two-way fixed effect estimator seems to be an efficient one. Our work can inform the methodological choice of future research on this topic, as well as more robust re-interpretation of existing works, to facilitate both future epidemic response plans and the science of public health.