Machine Learning and Causal Inference : A Modular Approach to Assessing the Effects of the London Bombings of 2005

The design of a randomized study guarantees not only clear and “interpretable comparisons”(Kinder and Palfrey, 1993, page 7) but valid statistical tests even in the absence of large samples or known data generating processes for outcomes (Fisher, 1935, Chap 2). Yet, while design alone yields valid tests the tests could lack power: a valid but wide confidence interval may be more useful than a misleadingly narrow confidence interval, but still shed little light on the theory motivating the study. After a brief demonstration of Fisher’s statistical framework, we show a method by which a researcher may use substantive background knowledge about outcomes in order to increase the power of her statistical tests. Combining substance and design in this particular way enables valid and powerful tests. We combine modern methods of machine learning with Fisher’s conceptual framework and survey sampling based design-based statistical inference originating with Neyman in order to maximize power without compromising the integrity of the resulting statistical inference. We apply our ideas in the context of a natural experiment created by the London subway bombings of 2005. ∗Associate Professor, Dept of Political Science and Statistics, University of Illinois @ UrbanaChampaign (jwbowers@illinois.edu). Acknowledgements:Thanks to Chris Achen, Joe Bowers, Dan Carpenter, Wendy Tam Cho, Tommy Engstrom, Don Green, Danny Hidalgo, Jim Kuklinski, and Cara Wong. Part of this work was funded by NSF Grants SES-0753168 and SES-0753164 (collaborative grants with Ben Hansen). Thanks are also due to participants in seminars, talks, and workshops at the Center for Political Studies at the University of Michigan, the Experiments in Governance and Politics network at Columbia University, the American Sociological Association Methodology Section, the Institute for Government and Public Administration at the University of Illinois at Urbana-Champaign, and the Department of Political Science at the University of Southern California. Version: fa5ac68 †PhD student, Department of Political Science, University of Illinois at Urbana-Champaign. ‡Associate Professor, Dept of Statistics, University of Michigan §Associate Professor, Dept of Political Science, Fordham University