Feasibility and Utility of Multi-Mode Data Collection in Random-Digit Dialing Cellphone Surveys For Young Adults: Evidence From a Survey Experiment (Preprint)

BACKGROUND Young adults’ early adoption of new cellphone technologies have created challenges to survey recruitment but offer opportunities to combine traditional random-digit dialing (RDD) sampling with web-mode data collection. The National Young Adult Health Survey (NYAHS) was designed to test the feasibility of this methodology. OBJECTIVE (1) Compare response rates across telephone- and web-modes, (2) assess sample representativeness, (3) examine the impact of sampling design and sample weighting on statistical precision, and (4) compare cigarette smoking prevalence to a gold standard national survey. METHODS The sampling frame was randomized to: (1) single-mode telephone interviews, (2) telephone to web sequential mixed mode, and (3) single-mode web survey, and 831 young adults (18-34 years) were recruited via RDD at baseline. A soft-launch was conducted prior to main-launch. We compared web-mode to pooled single-mode telephone and mixed-mode (telephone-mode) on the American Association for Public Opinion Research (AAPOR) response rate 3 (RR3) and for the screening (S-RR3) and extended surveys (E-RR3). Base-weighted demographic distributions were compared to the American Community Survey (ACS). Telephone- and web-mode samples were pooled and calibrated to the ACS to calculate design effects (DEFF) and compare cigarette smoking prevalence to the National Health Interview Survey (NHIS). Consistency of estimates was judged by overlap of 95% confidence intervals (CI). RESULTS RR3 was higher in telephone- than web-mode (24% and 30% vs. 6.1% and 12.5% for soft- and main-launch, respectively), which was reflected in S-RR3 and E-RR3. The telephone-mode over-represented men (57.1% vs. 50.9%) and those enrolled in college (40.3% vs. 23.8%) and under-represented those with a BA/BS+ (34.4% vs. 55%). The web-mode over-represented white, non-Latinos (70.7% vs. 54.4%) and those with some college (30.4% vs. 7.6%) and under-represented Latinos (13.6% vs. 20.7%) and those with a HS/GED (15.3% vs. 29.3%). The DEFF was 1.29 (subpopulation range: 0.96 to 1.90). The NYAHS cigarette smoking prevalence was consistent with NHIS overall (15%, CI 12.4% to 18% vs. 13.5%, CI 12.3% to 14.7%), with notable deviation among 18 to 24 year-olds (15.6%, CI 11.3% to 22.2% vs. 8.7%, CI 7.1% to 10.6%), and those with 更多