Guidelines for Assessing Enological and Statistical Significance of Wine Tasters' Binary Judgments

The purpose of this article is to assess the reliability and accuracy (validity) of hypothetical binary tasting judgments in an oenological framework. The model that is utilized allows for the control of a wide array of variables that would be exceedingly difficult to fully control in the typical oenological investigation. It is shown that results that are judged to be oenologically significant are uniformly judged to be statistically significant as well, whether the level of Wine Taster agreement is set at 70% (Fair); 80% (Good), or 90% (Excellent). However, in a number of instances, results that were statistically significant were not enologically significant by standards that are widely accepted and utilized. This finding is consistent with the bio-statistical fact that given a sufficiently large sample size, even the most trivial of results will prove to be statistically significant. Consistent with expectations, multiple patterns of 80% (Good) and 90% (Excellent) agreement tended to be both statistically and enologically significant.