Hostname: page-component-8448b6f56d-tj2md Total loading time: 0 Render date: 2024-04-18T08:45:45.763Z Has data issue: false hasContentIssue false
  • English
  • Français

The Distribution of Ratings Assigned to Blind Replicates*

Published online by Cambridge University Press:  02 August 2017

Jeffrey C. Bodington
Jeffrey C. Bodington*
Affiliation:
Bodington & Company, 50 California St., San Francisco, CA 94111; e-mail: jcb@bodingtonandcompany.com.
Get access Rights & Permissions [Opens in a new window]

Abstract

The inability of many wine judges to achieve perfect consistency by assigning the same rating to the same wine in a blind tasting is well established. Results for four wine tastings that include blind replicates are examined in this article. Although perfection is rare, the probability distributions of those results show that wine judges do tend to assign closer ratings to replicates than is likely due to chance alone. Approximately one-third of judges assign ratings that are within one rank of perfect consistency, and two-thirds assign ratings within two ranks of perfect consistency. This finding is sensitive to judges’ capabilities, the mechanics of the tasting protocol, and the extent to which the replicate is different from other wines in the tasting. Much wine-related research to date takes judges’ individual ratings as deterministic, yet these results show that those ratings are stochastic. These results yield a probability distribution that may guide future research concerning the uses and economic implications of wine ratings. (JEL Classifications: A10, C10, C00, C12, D12)

Keywords

blindrandomreplicatesstatisticswine tasting
Type
Articles
Information
Journal of Wine Economics , Volume 12 , Issue 4 , November 2017 , pp. 363 - 369
Copyright
Copyright © American Association of Wine Economists 2017 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

*

The author thanks Robert Hodgson for providing California State Fair Commercial Wine Competition data and essential explanations. The author also thanks Deborah Parker Wong and an anonymous reviewer for their perceptive and constructive comments. All remaining errors are the responsibility of the author alone.

References

Alvo, M., and Yu, P. L. H. (2014). Statistical Methods for Ranking Data. New York: Springer.Google Scholar
Ashenfelter, O., and Jones, G. V. (2013). The demand for expert opinion: Bordeaux wine. Journal of Wine Economics, 8(3), 285–293.CrossRefGoogle Scholar
Ashton, R. H. (2012). Reliability and consensus of experienced wine judges: Expertise within and between? Journal of Wine Economics, 7(1), 70–87.Google Scholar
Ashton, R. H. (2013). Is there consensus among wine quality ratings of prominent critics? An empirical analysis of red Bordeaux, 2004–2010. Journal of Wine Economics, 8(2), 225–234.CrossRefGoogle Scholar
Ashton, R. H. (2016). The value of expert opinion in the pricing of Bordeaux wine futures. Journal of Wine Economics, 11(2), 261–288.Google Scholar
Bodington, J. C. (2012). 804 tastes: Evidence on preferences, randomness, and value from double-blind wine tastings. Journal of Wine Economics, 7(2), 181–191.CrossRefGoogle Scholar
Bodington, J. C. (2015). Evaluating wine-tasting results and randomness with a mixture of rank preference models. Journal of Wine Economics, 10(1), 31–46.Google Scholar
Cardebat, J. M., Figuet, J. M., and Paroissien, E. (2014). Expert opinion and Bordeaux wine prices: An attempt to correct biases in subjective judgments. Journal of Wine Economics, 9(3), 282–303.Google Scholar
Cicchetti, D. (2014). Blind tasting of South African wines: A tale of two methodologies. American Association of Wine Economists, Working Paper No. 164, August. Available at www.wine-economics.org/aawe-working-paper-no-164-economics/.Google Scholar
Hodgson, R. T. (2008). An examination of judge reliability at a major U.S. wine competition. Journal of Wine Economics, 3(2), 105–113.Google Scholar
Hodgson, R. T., and Cao, J. (2014). Criteria for accrediting expert wine judges. Journal of Wine Economics, 9(1), 62–74.Google Scholar
Liquid Assets (2002). http://liquidasset.com/.Google Scholar
Marden, J. I. (1995). Analyzing and Modeling Rank Data. London: Chapman & Hall.Google Scholar
Marks, D. (2015). Seeking the veritas about the vino: Fine wine ratings as wine knowledge. Journal of Wine Research, 26(4), 319–335.CrossRefGoogle Scholar
Marley, A. A. J. (1993). Aggregation theorems and the combination of probabilistic rank orders. In: Critchlow, D. E., Fligner, M. A., and Verducci, J. S. (eds.), Probability Models and Statistical Analyses for Ranking Data. New York: Springer-Verlag. Chapter 12.Google Scholar
Masset, P., Weisskopf, J.-P., and Cossutta, M. (2015). Wine tasters, ratings, and en primeur prices. Journal of Wine Economics, 10(1), 75–107.Google Scholar
Niu, S., Lan, Y., Guo, J., and Cheng, X. (2013). Stochastic rank aggregation. Proceedings of the Twenty-Ninth Conference on Uncertainty in Artificial Intelligence, Bellevue, WA, August 11–15.Google Scholar
Oczkowski, E. (2016). Identifying the effects of objective and subjective quality on wine prices. Journal of Wine Economics, 11(2), 249–260.Google Scholar
Stuen, E. T., Miller, J. R., and Stone, R. W. (2015). An analysis of wine critic consensus: A study of Washington and California wines. Journal of Wine Economics, 10(1), 47–61.Google Scholar