Hostname: page-component-8448b6f56d-sxzjt Total loading time: 0 Render date: 2024-04-16T18:24:38.465Z Has data issue: false hasContentIssue false
  • English
  • Français

Improving Experts' Wine Quality Judgments: Two Heads Are Better than One*

Published online by Cambridge University Press:  08 June 2012

Robert H. Ashton
Robert H. Ashton
Affiliation:
Fuqua School of Business, Duke University, 100 Fuqua Drive, Box 90120, Durham, NC 27708, email: robert.ashton@duke.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

This paper reports an analysis of data from the “Judgment of Paris,” the 1976 blind tasting of California and French wines that revolutionized the wine world. Using both empirical and analytical methods, I demonstrate that the wine quality judgments of the renowned experts who participated in the Paris tasting would have been improved simply by averaging the quality ratings of two or more of the judges. Moreover, I explore both how many of the Paris judges should be included in the average and which ones they should be. The results have implications for the practical issue of choosing judges to include in tasting panels that award prizes or provide expert advice to consumers, as well as for better understanding the variability in the price-quality association across hedonic wine pricing studies. (JEL Classification: C93)

Type
Articles
Information
Journal of Wine Economics , Volume 6 , Issue 2 , Autumn 2011 , pp. 160 - 178
Copyright
Copyright © American Association of Wine Economists 2011

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Armstrong, J.S. (2001). Combining forecasts. In Armstrong, J. S. (Ed.), Principles of Forecasting: A Handbook for Researchers and Practitioners. Norwell, MA: Kluwer Academic Publishers.CrossRefGoogle Scholar
Ashenfelter, O. and Quandt, R. (1999). Analyzing a wine tasting statistically. Chance, 12, 16–20.CrossRefGoogle Scholar
Ashton, R.H. (1986). Combining the judgments of experts: How many and which ones? Organizational Behavior and Human Decision Processes, 38, 405–414.CrossRefGoogle Scholar
Ashton, A.H. and Ashton, R.H. (1985). Aggregating subjective forecasts: Some empirical results. Management Science, 31, 1499–1508.CrossRefGoogle Scholar
Bates, J.M. and Granger, C.W.J. (1969). The combination of forecasts. Operational Research Quarterly, 20, 451–468.CrossRefGoogle Scholar
Benjamin, B.A. and Podolny, J.M. (1999). Status, quality, and social order in the California wine industry. Administrative Science Quarterly, 44, 563–589.CrossRefGoogle Scholar
Broomell, S.B. and Budescu, D.V. (2009). Why are experts correlated? Decomposing correlations between judges. Psychometrika, 74, 531–553.CrossRefGoogle Scholar
Budescu, D.V. and Yu, H.T. (2007). Aggregation of opinions based on correlated cues and advisors. Journal of Behavioral Decision Making, 20, 153–177.CrossRefGoogle Scholar
Cicchetti, D.V. (2004a). Who won the 1976 blind tasting of French Bordeaux and U.S. cabernets? Parametrics to the rescue. Journal of Wine Research, 15, 211–220.CrossRefGoogle Scholar
Cicchetti, D.V. (2004b). On designing experiments and analysing data to assess the reliability and accuracy of blind wine tastings. Journal of Wine Research, 15, 221–226.CrossRefGoogle Scholar
Cicchetti, D.V. (2006). The Paris 1976 wine tastings revisited once more: comparing ratings of consistent and inconsistent tasters. Journal of Wine Economics, 1, 125–140.CrossRefGoogle Scholar
Clemen, R.T. (1989). Combining forecasts: a review and annotated bibliography. International Journal of Forecasting, 5, 559–583.CrossRefGoogle Scholar
Clemen, R.T. and Winkler, R.L. (1985). Limits for the precision and value of information from dependent sources. Operations Research, 33, 427–442.CrossRefGoogle Scholar
Combris, P., Lecocq, S. and Visser, M. (1997). Estimation of a hedonic price equation for Bordeaux wine: Does quality matter? The Economic Journal, 107, 390–402.CrossRefGoogle Scholar
Combris, P., Lecocq, S. and Visser, M. (2000). Estimation of a hedonic price equation for Burgundy wine. Applied Economics, 32, 961–967.CrossRefGoogle Scholar
Dawes, R.M. (1970). An inequality concerning correlation of composites vs. composites of correlations. Oregon Research Institute Methodological Note, Vol. 1., No. 1.Google Scholar
Dawes, R.M. and Corrigan, B. (1974). Linear models in decision making. Psychological Bulletin, 81, 95–106.CrossRefGoogle Scholar
Di Vittorio, A. and Ginsburgh, V. (1996). Pricing red wines of Medoc vintages from 1949 to 1989 at Christie's auctions. Journal de la Société de Statistique de Paris, 137, 19–49.Google Scholar
Einhorn, H.J. and Hogarth, R.M. (1975). Unit weighting schemes for decision making. Organizational Behavior and Human Performance, 13, 171–192.CrossRefGoogle Scholar
Einhorn, H.J., Hogarth, R.M. and Klempner, E. (1977). Quality of group judgment. Psychological Bulletin, 84, 158–172.CrossRefGoogle Scholar
Ferrell, W.R. (1985). Combining individual judgments. In Wright, G. (Ed.), Behavioral Decision Making. New York: Plenum Publishing Corporation.Google Scholar
Flores, B.E. and White, E.M. (1988). A framework for the combination of forecasts. Journal of the Academy of Marketing Science, 16, 95–103.CrossRefGoogle Scholar
Gawel, R. and Gooden, P.W. (2008). Evaluation of the consistency of wine quality assessments from expert wine tasters. Australian Journal of Grape and Wine Research, 14, 1–8.CrossRefGoogle Scholar
Ghiselli, E.E. (1964). Theory of Psychological Measurement. New York: McGraw-Hill.Google Scholar
Gibbs, M., Tapia, M. and Warzynski, F. (2009). Globalization, superstars, and reputation: theory & evidence from the wine industry. Journal of Wine Economics, 4, 46–61.CrossRefGoogle Scholar
Gigone, D. and Hastie, R. (1997). Proper analysis of the accuracy of group judgments. Psychological Bulletin, 121, 149–167.CrossRefGoogle Scholar
Goldwyn, C. and Lawless, H. (1991). How to taste wine (for fun and profit). ASTM Standardization News, 19, 32–37.Google Scholar
Hadj Ali, H., Lecocq, S. and Visser, M. (2008). The impact of gurus: Parker grades and en primeur wine prices. The Economic Journal, 118, 158–173.Google Scholar
Haeger, J.W. and Storchmann, K. (2006). Prices of American pinot noir wines: climate, craftsmanship, critics. Agricultural Economics, 35, 67–78.CrossRefGoogle Scholar
Hodgson, R.T. (2008). An examination of judge reliability at a major U.S. wine competition. Journal of Wine Economics, 3, 105–113.CrossRefGoogle Scholar
Hogarth, R.M. (1978). A note on aggregating opinions. Organizational Behavior and Human Performance, 21, 40–46.CrossRefGoogle Scholar
Hogarth, R.M. and Karelaia, N. (2007). Heuristic and linear models of judgment: Matching rules and environments. Psychological Review, 114, 733–758.CrossRefGoogle ScholarPubMed
Hulkower, N. (2009). The judgment of Paris according to Borda, Journal of Wine Research, 20, 171–182.CrossRefGoogle Scholar
Jones, G.V. and Storchmann, K. (2001). Wine market prices and investment under uncertainty: an econometric model for Bordeaux crus classés. Agricultural Economics, 26, 115–133.CrossRefGoogle Scholar
Larrick, R.P. and Soll, J.B. (2006). Intuitions about combining opinions: Misappreciation of the averaging principle. Management Science, 52, 111–127.CrossRefGoogle Scholar
Lawless, H., Liu, Y. and Goldwyn, C. (1997). Evaluation of wine quality using a small-panel hedonic scaling method. Journal of Sensory Studies, 12, 317–332.CrossRefGoogle Scholar
Lecocq, S. and Visser, M. (2006). What determines wine prices: objective vs. sensory characteristics. Journal of Wine Economics, 1, 42–56.CrossRefGoogle Scholar
Libby, R. and Blashfield, R. K. (1978). Performance of a composite as a function of the number of judges. Organizational Behavior and Human Performance, 21, 121–129.CrossRefGoogle Scholar
Lindley, D. V. (2006). Analysis of a wine tasting. Journal of Wine Economics, 1, 33–41.CrossRefGoogle Scholar
Makridakis, S. and Winkler, R.L. (1983). Averages of forecasts: Some empirical results. Management Science, 29, 987–996.CrossRefGoogle Scholar
Oczkowski, E. (1994). A hedonic price function for Australian premium table wine. Australian Journal of Agricultural Economics, 38, 93–110.CrossRefGoogle Scholar
Oczkowski, E. (2001). Hedonic wine price functions and measurement error. The Economic Record, 77, 374–382.CrossRefGoogle Scholar
Quandt, R.E. (2006). Measurement and inference in wine tasting. Journal of Wine Economics, 1, 7–30.CrossRefGoogle Scholar
Quandt, R.E. (2007). A note on a test for the sum of ranksums. Journal of Wine Economics, 2, 98–102.CrossRefGoogle Scholar
Roberts, P.W. and Reagans, R. (2007). Critical exposure and price-quality relationships for new world wines in the U.S. market. Journal of Wine Economics, 2, 56–69.CrossRefGoogle Scholar
Rudin, W. (1987). Real and Complex Analysis. New York: McGraw-Hill.Google Scholar
San Martin, G.J., Troncoso, J.L. and Brümmer, B. (2008). Determinants of Argentinean wine prices in the U.S. Journal of Wine Economics, 3, 72–84.CrossRefGoogle Scholar
Schamel, G. and Anderson, K. (2003). Wine quality and varietal, regional and winery reputations: hedonic prices for Australia and New Zealand. The Economic Record, 79, 357–369.CrossRefGoogle Scholar
Schnabel, H. and Storchmann, K. (2010). Prices as quality signals: evidence from the wine market. Journal of Agricultural & Food Industrial Organization, 8, 1–21.CrossRefGoogle Scholar
Taber, G.M. (2006). Judgment of Paris: California vs. France and the Historic 1976 Paris Tasting that Revolutionized Wine. New York: Scribner.Google Scholar
Tellis, G.J. and Wernerfelt, B. (1987). Competitive price and quality under asymmetric information. Marketing Science, 6, 240–253.CrossRefGoogle Scholar
Troncoso, J.L. and Aguirre, M. (2006). Price determinants of Chilean wines in the U. S. market: A hedonic approach. Spanish Journal of Agricultural Research, 4, 124–129.CrossRefGoogle Scholar
Unwin, T. (1999). Hedonic price indexes and the qualities of wines. Journal of Wine Research, 10, 95–104.CrossRefGoogle Scholar
Wallsten, T.S., Budescu, D.V., Erev, I. and Diederich, A. (1997). Evaluating and combining subjective probability estimates. Journal of Behavioral Decision Making, 10, 243–268.3.0.CO;2-M>CrossRefGoogle Scholar
Winkler, R.L. and Makridakis, S. (1983). The combination of forecasts. Journal of the Royal Statistical Society, Series A, 146, 150–157.CrossRefGoogle Scholar