Skip to main content
    • Aa
    • Aa
  • Get access
    Check if you have access via personal or institutional login
  • Cited by 35
  • Cited by
    This article has been cited by the following publications. This list is generated based on data provided by CrossRef.

    Ashton, Robert H. 2016. The Value of Expert Opinion in the Pricing of Bordeaux Wine Futures. Journal of Wine Economics, p. 1.

    Chen, Natalie and Juvenal, Luciana 2016. Quality, trade, and exchange rate pass-through. Journal of International Economics, Vol. 100, p. 61.

    Dressler, Marc 2016. Strategic winery reputation management – exploring German wine guides. International Journal of Wine Business Research, Vol. 28, Issue. 1, p. 4.

    Cicchetti, Dom 2015. The Encyclopedia of Clinical Psychology.

    Gergaud, Olivier Storchmann, Karl and Verardi, Vincenzo 2015. EXPERT OPINION AND PRODUCT QUALITY: EVIDENCE FROM NEW YORK CITY RESTAURANTS. Economic Inquiry, Vol. 53, Issue. 2, p. 812.

    Honoré-Chedozeau, Carole Ballester, Jordi Chatelet, Bertrand Lempereur, Valérie and Aurand, Jean-Marie 2015. Wine competition: from between-juries consistency to sensory perception of consumers. BIO Web of Conferences, Vol. 5, p. 03009.

    Keller, M 2015. Environmentally Sustainable Viticulture.

    Miller, Jon R. Stone, Robert W. and Stuen, Eric T. 2015. When is a wine a bargain? A comparison of popular and regression-based approaches. Journal of Wine Research, Vol. 26, Issue. 2, p. 153.

    Oczkowski, E. and Doucouliagos, H. 2015. Wine Prices and Quality Ratings: A Meta-regression Analysis. American Journal of Agricultural Economics, Vol. 97, Issue. 1, p. 103.

    Cacchiarelli, Luca Carbone, Anna Laureti, Tiziana and Sorrentino, Alessandro 2014. The value of quality clues in the wine market: evidences from Lazio, Italy. Journal of Wine Research, Vol. 25, Issue. 4, p. 281.

    Cao, Jing 2014. Quantifying Randomness Versus Consensus in Wine Quality Ratings. Journal of Wine Economics, Vol. 9, Issue. 02, p. 202.

    Cardebat, Jean-Marie Figuet, Jean-Marc and Paroissien, Emmanuel 2014. Expert Opinion and Bordeaux Wine Prices: An Attempt to Correct Biases in Subjective Judgments. Journal of Wine Economics, Vol. 9, Issue. 03, p. 282.

    Hodgson, Robert and Cao, Jing 2014. Criteria for Accrediting Expert Wine Judges. Journal of Wine Economics, Vol. 9, Issue. 01, p. 62.

    Hopfer, Helene and Heymann, Hildegarde 2014. Judging wine quality: Do we need experts, consumers or trained panelists?. Food Quality and Preference, Vol. 32, p. 221.

    Jackson, Ronald S. 2014. Wine Science.

    Ashton, Robert H. 2013. Is There Consensus Among Wine Quality Ratings of Prominent Critics? An Empirical Analysis of Red Bordeaux, 2004–2010. Journal of Wine Economics, Vol. 8, Issue. 02, p. 225.

    Castriota, S. Curzi, D. and Delmastro, M. 2013. Tasters’ bias in wine guides’ quality evaluations. Applied Economics Letters, Vol. 20, Issue. 12, p. 1174.

    Cuellar, Steven S. and Claps, Marianna 2013. Differential effects of brand, ratings and region on willingness to pay: a hedonic price approach. Journal of Wine Research, Vol. 24, Issue. 2, p. 138.

    2013. Quantitative Sensory Analysis.

    Ashton, Robert H. 2012. Reliability and Consensus of Experienced Wine Judges: Expertise Within and Between?. Journal of Wine Economics, Vol. 7, Issue. 01, p. 70.


An Examination of Judge Reliability at a major U.S. Wine Competition*

  • Robert T. Hodgson (a1)
  • DOI:
  • Published online: 01 June 2012

Wine judge performance at a major wine competition has been analyzed from 2005 to 2008 using replicate samples. Each panel of four expert judges received a flight of 30 wines imbedded with triplicate samples poured from the same bottle. Between 65 and 70 judges were tested each year. About 10 percent of the judges were able to replicate their score within a single medal group. Another 10 percent, on occasion, scored the same wine Bronze to Gold. Judges tend to be more consistent in what they don't like than what they do. An analysis of variance covering every panel over the study period indicates only about half of the panels presented awards based solely on wine quality. (JEL Classification: Q13, Q19)

Linked references
Hide All

This list contains references from the content that can be linked to their source. For a full set of references and notes please see the PDF or HTML where available.

O. Ashenfelter and R.E. Quandt . (1999). Analyzing wine tasting statistically. Chance, 12, 1620.

J.J. Bartko (1966). The intraclass correlation coefficient as a measure of reliability. Psychological Reports, 19, 311.

D.V. Cicchetti (2004a). Who won the 1976 wine tasting of French Bordeaux and U.S. cabernets? Parametrics to the rescue. Journal of Wine Research, 15, 211220.

D.V. Cicchetti (2004b). On designing experiments and analyzing data to assess the reliability and accuracy of blind wine tastings. Journal of Wine Research, 15, 221226.

M.A. Cliff and M.C. King (1996). A proposed approach for evaluating expert judge performance using descriptive statistics. Journal of Wine Research, 7, 8390.

M.A. Cliff and M.C. King (1997). The evaluation of judges at wine competitions: The application of eggshell plots. Journal of Wine Research, 8(2), 7580.

P.E. Shrout and J.L. Fleiss . (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86(2), 420428.

Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Journal of Wine Economics
  • ISSN: 1931-4361
  • EISSN: 1931-437X
  • URL: /core/journals/journal-of-wine-economics
Please enter your name
Please enter a valid email address
Who would you like to send this to? *