Skip to main content
    • Aa
    • Aa

An Examination of Judge Reliability at a major U.S. Wine Competition*

  • Robert T. Hodgson (a1)

Wine judge performance at a major wine competition has been analyzed from 2005 to 2008 using replicate samples. Each panel of four expert judges received a flight of 30 wines imbedded with triplicate samples poured from the same bottle. Between 65 and 70 judges were tested each year. About 10 percent of the judges were able to replicate their score within a single medal group. Another 10 percent, on occasion, scored the same wine Bronze to Gold. Judges tend to be more consistent in what they don't like than what they do. An analysis of variance covering every panel over the study period indicates only about half of the panels presented awards based solely on wine quality. (JEL Classification: Q13, Q19)

Linked references
Hide All

This list contains references from the content that can be linked to their source. For a full set of references and notes please see the PDF or HTML where available.

O. Ashenfelter and R.E. Quandt . (1999). Analyzing wine tasting statistically. Chance, 12, 1620.

J.J. Bartko (1966). The intraclass correlation coefficient as a measure of reliability. Psychological Reports, 19, 311.

D.V. Cicchetti (2004a). Who won the 1976 wine tasting of French Bordeaux and U.S. cabernets? Parametrics to the rescue. Journal of Wine Research, 15, 211220.

D.V. Cicchetti (2004b). On designing experiments and analyzing data to assess the reliability and accuracy of blind wine tastings. Journal of Wine Research, 15, 221226.

M.A. Cliff and M.C. King (1996). A proposed approach for evaluating expert judge performance using descriptive statistics. Journal of Wine Research, 7, 8390.

M.A. Cliff and M.C. King (1997). The evaluation of judges at wine competitions: The application of eggshell plots. Journal of Wine Research, 8(2), 7580.

P.E. Shrout and J.L. Fleiss . (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86(2), 420428.

Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Journal of Wine Economics
  • ISSN: 1931-4361
  • EISSN: 1931-437X
  • URL: /core/journals/journal-of-wine-economics
Please enter your name
Please enter a valid email address
Who would you like to send this to? *


Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 83 *
Loading metrics...

Abstract views

Total abstract views: 2300 *
Loading metrics...

* Views captured on Cambridge Core between September 2016 - 29th June 2017. This data will be updated every 24 hours.