With Small Power, Comes Great Responsibility: Lessons Learned from an Evaluation of Veteran and Military Mental Health Public Awareness Campaigns

This study was conducted to determine the feasibility of conducting a cost–benefit evaluation of federally-funded media campaigns encouraging mental health help-seeking among United States military personnel and veterans. To calculate the necessary sample size for the evaluation, we obtained campaign costs, and determined the number of treatment seekers needed for the campaign to break even with its cost and the associated population change that an evaluation would need to detect. The sample size needed for an evaluation with 80% power was greater than the total population of U.S. military personnel and veterans. Given that the necessary sample size exceeds the population to be sampled, an appropriately powered outcome evaluation is not feasible. Other programs that would be cost effective with extremely small effect sizes should not be subject to underpowered and thus inaccurate empirical outcome evaluation.