Misinterpretations of P-values and statistical tests persists among researchers and professionals working with statistics and epidemiology

Background: The aim was to investigate inferences of statistically significant test results among persons with more or less statistical education and research experience. Methods: A total of 75 doctoral students and 64 statisticians/epidemiologist responded to a web questionnaire about inferences of statistically significant findings. Participants were asked about their education and research experience, and also whether a 'statistically significant' test result (P = 0.024, alpha-level 0.05) could be inferred as proof or probability statements about the truth or falsehood of the null hypothesis (H-0) and the alternative hypothesis (H-1). Results: Almost all participants reported having a university degree, and among statisticians/epidemiologist, most reported having a university degree in statistics and were working professionally with statistics. Overall, 9.4% of statisticians/epidemiologist and 24.0% of doctoral students responded that the statistically significant finding proved that H-0 is not true, and 73.4% of statisticians/epidemiologists and 53.3% of doctoral students responded that the statistically significant finding indicated that H(0 )is improbable. Corresponding numbers about inferences about the alternative hypothesis (H-1) were 12.0% and 6.2% about proving H-1 being true and 62.7 and 62.5% for the conclusion that H-1 is probable. Correct inferences to both questions, which is that a statistically significant finding cannot be inferred as either proof or a measure of a hypothesis' probability, were given by 10.7% of doctoral students and 12.5% of statisticians/epidemiologists. Conclusions: Misinterpretation of P-values and statistically significant test results persists also among persons who have substantial statistical education and who work professionally with statistics.