Machine Learning and Deductive Social Science: An Introduction to Predictability Hypotheses

Perhaps due to decades of metatheoretical work done during a time of methodological and computational limitations, it is a widespread belief that machine learning and deductive hypothesis testing are fundamentally incompatible. I argue that the more flexible estimators provided by machine learning are in fact invaluable tools for testing pre-specified social theory, especially sociological ones. Specifically, I argue that machine learning algorithms are helpful when using observational data to deductively test a class of hypotheses I call predictability hypotheses: general theoretical statements which posit an unspecified (although potentially causal) relationship between two or more theoretical constructs. I point to several examples of predictability hypotheses already being used by social scientists and describe when and why they are scientifically valuable. I then consider some methodological implications for testing predictability hypotheses and conclude by making my own predictions about their future in sociology.