The big, the bad, and the ugly: Geographic estimation with flawed psychological data.

The geographic distribution of psychological constructs has long been an area of focus for psydiological researchers. Recently, however, there has been increased interest in investigations of the so-called subnational distribution of psychological variables, which focus on localized groupings of individuals within spatial units, such as counties or states. By estimating the subnational distribution of a given outcome (e.g., estimating its state- or county-level means), researchers have been able to address questions about the spatial variation of a variety of psychological constructs and investigate the regional association between psychological phenomena and real-world outcomes, such as health outcomes, prosocial behavior, and racial inequity. Unfortunately, however, there are many challenges to estimating a construct's subnational distribution, such as those raised by response biases and subnational sparsity. To help psychological researchers address these issues, we provide a comprehensive discussion of subnational estimation and introduce multilevel regression and poststratification (MrP), a method that is widely considered to be the gold standard for subnational estimation with random samples. As psychologists often do not have access to large, national random samples, we also report 3 studies evaluating MrP's performance under simulated and real-world conditions of sample biases. Ultimately, we find that MrP is likely to outperform the subnational estimation methods that psychological researchers currently use. Based on this, we suggest that psychologists interested in understanding how psychological phenomena vary below the nation level use MrP to conduct these investigations. To help facilitate this, we have made all code and data used for the reported studies publicly available. Translational Abstract The geographic distribution of psychological constructs has attracted increasing interest among psychological researchers. Relying on these and other data, psychologists have been able to not only address novel questions about the spatial variation of psychological constructs but also investigate the regional association between psychological phenomena and real-world outcomes, such as outcomes associated with health, prosocial behavior, and racial inequity. Unfortunately, there are many challenges to estimating a construct's regional distribution-so-called subnational estimation-and these challenges are exacerbated by issues of nonrepresentativeness and geographic sparsity. In this work, we provide a comprehensive discussion of major obstacles for subnational estimation and introduce readers to state-of-the-art approaches that rely on multilevel regression and poststratification (MrP) to deal with these obstacles. We also present a novel evaluation of MrP and extensions of MrP under conditions of sample size and response bias via simulations (Study 1) and application to real-world data obtained from a large convenience sample (Study 2). Finally, we investigate how estimated associations between an estimated county-level outcome-racial bias-and a secondary outcome-Barack Obama's 2008 county-level Presidential vote share-vary depending on the method used for subnational estimation (Study 3). In addition to offering a comprehensive introduction to cutting-edge methods for subnational estimation, this work provides strong evidence for the necessity of incorporating more sophisticated techniques for subnational estimation into studies of the geographic distribution of psychological phenomena.