2009. A Kullback-Leibler view of linear and log-linear pools. Decision Analysis 6, 25–37.
1995. Risk. UCL Press, London.
1995. Indigenous and scientific knowledge: some critical comments. Indigenous Knowledge and Development Monitor 3, 3–6.
2012. Combining expert opinions in prior elicitation. Bayesian Analysis 7, 503–532., , , , and
2007. Age differences in the value of statistical life: revealed preference evidence. Review of Environmental Economics and Policy 1, 241–260. and
ALRC. 1985. Evidence. Australian Law Reform Commission Report 26. Volume 1. Australian Government Publishing Service, Canberra.
ALRC. 2000. Managing justice: a review of the Federal Civil Justice system. Australian Law Reform Commission Report 89. Australian Government Publishing Service, Canberra.
1998. Embracing uncertainty: the interface of Bayesian statistics and cognitive psychology. Conservation Ecology 2, 2.
2012. The half-life of facts. Why everything we know has an expiration date. Penguin, New York.
Politics. Volume 3. Translated by , 1995. Oxford World's Classics..
1980. The seer-sucker theory: the value of experts in forecasting. Technology Review 82, 16–24.
2001. Principles of forecasting: a handbook for researchers and practitioners. Kluwer Academic Publishers, Dordrecht.
2006. Findings from evidence-based forecasting: methods for reducing forecast error. International Journal of Forecasting 22, 583–598.
2000. A review and analysis of research on the test–retest reliability of professional judgment. Journal of Behavioral Decision Making 13, 277–294.
2010. A route to more tractable expert advice. Nature 463, 294–295.
2011. The value of feedback in forecasting competitions. International Journal of Forecasting 27, 845–849. and
2001. Elicitation of expert opinions for uncertainty and risks. CRC Press, Boca Raton, Florida.
2009. Judges, courts, the legal profession and public risk. Risk and Regulation Advisory Council, UK., and
1980. The base-rate fallacy in probability judgments. Acta Psychologica 44, 211–233.
2001. Boys will be boys: Gender, overconfidence, and common stock investment. Quarterly Journal of Economics 116, 261–292. and
2006. Contracting: a new form of professional practice. Academy of Management Perspectives 20, 45–66. and
1992. Risk society: towards a new modernity. Sage Publications, London.
2001. Probabilistic risk analysis: foundations and methods. Cambridge University Press. and
2006. Expert elicitation for reliable system design. Statistical Science 21, 428–450., and
1998. On the use of demographic models of population viability in endangered species management. Journal of Wildlife Management 62, 821–841. and
2002. How to avoid train wrecks when using science in environmental problem solving. Bioscience 52, 1127–1136., , , , , , and
2006. Information-gap decision theory: decisions under severe uncertainty. Academic Press, San Diego.
1982. An investigation of motivational bias in subjective predictive probability distributions. Decision Sciences 13, 225–239. and
1988. Research methods in cultural anthropology. Sage Publications, London.
2008. Overconfidence as a cause of diagnostic error in medicine. American Journal of Medicine 121, S2–S23. and
1996. Against the Gods: the remarkable story of risk. Wiley, New York.
1985. An elementary approach to thinking under uncertainty. Lawrence Erlbaum Associates, Hillside, New Jersey., , and
1999a. Chinese and Americans agree on what is fair, but disagree on what is best in societal decisions affecting health and safety risks. Risk Analysis 19, 439–452. and
1999b. Patterns of fairness judgments in North America and the People's Republic of China. Journal of Consumer Psychology 8, 301–320. and
2004. Implications of the research on expert overconfidence and dependence. Reliability Engineering and System Safety 85, 321–329.
1999. A survey of approaches for assessing and managing the risk of extremes. Risk Analysis 19, 83–94., , , and
2011. Does the Delphi process lead to increased accuracy in group-based judgmental forecasts or does it simply induce consensus amongst judgmental forecasters?Technological Forecasting and Social Change 78, 1671–1680., , and
2003. An integrated approach to TMDL development for the Neuse River Estuary using a Bayesian probability network model (Neu-BERN).Journal of Water Resources Planning and Management 129, 271–282., and
2002. Propagation of individual bias through group judgment: error in the treatment of asymmetrically informative signals. Journal of Risk and Uncertainty 25, 147–163., and
1950. Verification of forecasts expressed in terms of probability. Monthly Weather Review 78, 1–3.
2006. Understanding popular science. Open University Press, Maidenhead, England.
2007. Can you believe the weather?New Scientist issue 2588, 27 January 2007, 32–35.
2005. Risks and decisions for conservation and environmental management. Cambridge University Press.
1993. Risk assessment in conservation biology. Chapman and Hall, London., and
2000. Inferring threat from scientific collections: power tests and application to Western Australian Acacia species. In and , editors. Quantitative methods for conservation biology. Springer-Verlag, New York., , , , and
2011a. Expert status and performance. PLoS ONE 6, e22998., , , , , , , and
2011b. Redefining expertise and improving ecological judgement. Conservation Letters 4, 81–87., , , , , and
1991. The process-performance paradox in expert judgment: how can experts know so much and predict so badly? In and , editors. Towards a general theory of expertise: prospects and limits. Cambridge University Press, New York. and
2002. Science and sustainable use: views of marine turtle conservation experts. Ecological Applications 12, 1229–1246.
1976. The difficulty of assessing uncertainty. Journal of Petroleum Technology 28, 843–850.
2009. Strategies to improve the communication of probability information in risk analyses. International Journal of Risk Assessment and Management 12, 380–395.
2001. Risk based characterization of contaminated industrial site using multivariate and geostatistical tools. Environmental Pollution 111, 417–427., , and
1999. Ecological and social dynamics in simple models of ecosystem management. Social Systems Research Institute, University of Wisconsin., and
2004. Why do we all need community science?Society and Natural Resources 17, 1–9.
1993. Graphical elicitation of a prior distribution for a clinical trial. The Statistician 42, 341–353., , and
1890. The method of multiple working hypotheses. Reprinted in Science 148, 754–759.
2003. Statistical thought: a perspective and history. Oxford University Press.
2002. Information aggregation mechanisms: concepts, design and implementation for a sales forecasting problem. Social Science Working Paper 1131, California Institute of Technology. and
1978. Predator–prey theory and variability. Annual Review of Ecology and Systematics 9, 323–347.
2006. Two approaches to the study of experts’ characteristics. In , , and , editors. The Cambridge handbook of expertise and expert performance. Cambridge University Press.
1981. Physicians use of probabilistic information in a real clinical setting. Journal of Experimental Psychology: Human Perception and Performance 7, 928–935. and
1999. Combining probability distributions from experts in risk analysis. Risk Analysis 19, 187–203. and
1978. Prediction in psychiatry: an example of misplaced confidence in experts. Social Problems 25, 265–276. and
2007. Rethinking expertise. University of Chicago Press. and
2001. Is probability the only coherent approach to uncertainty?Risk Analysis 28, 645–652.
1991. Experts in uncertainty: opinion and subjective probability in science. Oxford University Press.
2008. On the performance of social network and likelihood based expert weighting schemes. Reliability Engineering and System Safety, Special Issue 93, 745–756., and
2000. Procedures guide for structured expert judgement in accident consequence modelling. Radiation Protection and Dosimetry 90, 303–309. and
2000. Processing expert judgements in accident consequence modelling. Radiation Protection Dosimetry 90, 311–315. and
2004. On the use of imprecise probabilities in reliability. Quality and Reliability Engineering International 20, 193–202.
2011. The Polymath Projects: lessons from an experiment in large-scale online collaboration in mathematics. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 1865–1874. ACM. and
1999. Risk-based environmental decisions: method and culture. Kluwer Academic Publishers, Boston.
2006. Accuracy of physician self-assessment compared with observed measures of competence: a systematic review. Journal of the American Medical Association 296, 1094–1102., , , , ,
1979. The robust beauty of improper linear models in decision making. American Psychologist 34, 571–582.
1994. House of cards: psychology and psychotherapy built on myth. Simon and Schuster, New York.
1995. Coherent combination of experts’ opinions. Test 4, 263–313., , , , , , , , and
1974. Theory of probability. Wiley, New York.
1980. Eyewitness accuracy and confidence. Law and Human Behavior 4, 243–260.
2003. Why people fail to recognize their own incompetence. Current Directions in Psychological Science 12, 83–87., , and
2010. The effects of presenting imprecise probabilities in intelligence forecasts. Risk Analysis 30, 987–1001., and
1999. Hazard identification of agricultural biotechnology: finding relevant questions. International Books, Utrecht.
2010. Using expert opinion surveys to rank threats to endangered species: a case study with sea turtles. Conservation Biology 24, 1586–1595., , and
2005. When emotions get the better of us: the effect of contextual top-down processing on matching fingerprints. Applied Cognitive Psychology 19, 799–809., , and
2008. Meta-analytically quantifying the reliability and biasability of fingerprint experts’ decision making. Journal of Forensic Sciences 53, 900–903. and
1976. An actual application of collective choice theory to the selection of trajectories for the Mariner Jupiter/Saturn 1977 project. Operations Research 24, 220–244. and
1978. Confidence in judgment: persistence of the illusion of validity. Psychological Review 85, 395–416. and
2010. A Delphi method expert survey to derive standards for flood damage data collection. Risk Analysis 30, 107–124., , and
1995. Clinical reasoning in medicine. In: , editor. Clinical Reasoning in the Health Professions. Butterworth-Heinemann Limited, Oxford, England.
1986. Patterns of controversy and closure: the interplay of knowledge, values, and political forces. In and , editors. Scientific controversies: case studies in the resolution and closure of disputes in science and technology. Cambridge University Press, New York. and
2009. Expert financial advice neurobiologically “offloads” financial decision-making under risk. PLoS ONE 4, e4957., , and
2009. Moody experts – how mood and expertise influence judgmental anchoring. Judgment and Decision Making 4, 41–50. and
2006. The Cambridge Handbook of Expertise and Expert Performance. Cambridge University Press, New York., , and , editors.
European Food Safety Authority 2011. Guidance on expert knowledge elicitation in food and feed safety risk assessment. European Food Safety Authority (EFSA), Parma, Italy.
2008. Dual-processing accounts of reasoning, judgment, and social cognition. Annual Review of Psychology 59, 255–278.
2006. Professionalization, scientific expertise, and elitism: a sociological perspective. In , , and , editors. The Cambridge handbook of expertise and expert performance. Cambridge University Press., and
1990. The descent of Icarus: science and the transformation of contemporary democracy. Harvard University Press, Cambridge, Massachusetts.
2007. Integrating science and local knowledge in environmental risk management: a decision-focused approach. Ecological Economics 64, 47–70., and
FAO / WHO. 2001. Call for experts for the Joint FAO / WHO risk assessment activities in the areas of Campylobacter in broilers and Vibrio in seafood. Joint Expert Consultations on Risk Assessment of Microbiological Hazards in Food. Food and Nutrition Division, Food and Agriculture Organization of the United Nations, Rome, Italy.
2003. Constructing probability boxes and Dempster–Shafer structures. SAND Report, SAND2002-4015. Sandia National Laboratories, Albuquerque, New Mexico., , , and
1986. Appendix F – Personal observations on the reliability of the Shuttle. In Hearing on the Space Shuttle Accident and the Rogers Commission Report. Report of the Presidential Commission on the Space Shuttle Challenger Accident. U.S. Senate Committee on Commerce, Science and Transportation, Subcommittee on Science, Technology and Space, Washington, DC.
1979. Subjective information and market efficiency in a betting market. Journal of Political Economy 87, 75–88.
1996. Comparing risks thoughtfully. Risk, 7, 325.
2000. Citizens, experts, and the environment. Duke University Press, Durham.
1994. Acceptable risk: a conceptual proposal. Risk: Health, Safety and Environment 1, 1–28.
1995. Risk perception and communication unplugged: twenty years of progress. Risk Analysis 15, 137–145.
1982. Lay foibles and expert fables in judgements about risk. American Statistician 36, 240–255., and
1998. Environmental science and environmental law. Journal of Environmental Law 10, 3–8.
2012. Facilitated expert judgment of environmental risks: acquiring and analysing imprecise data. International Journal of Risk Assessment and Management 16, 199–212.., , and
2005. The quality of expert judgment: an interdisciplinary investigation. PhD Thesis. Department of Mechanical Engineering, University of Maryland. URL: http://hdl.handle.net/1903/3267.
2008. Evaluating extreme risks in invasion ecology: learning from banking compliance. Diversity and Distributions 14, 581–591., , and
1995. The challenge of junk psychiatry, psychology and science: the evolving role of the forensic expert. In , editor. Tomorrow's law. Federation Press, Sydney.
2011. Aggregating expert judgement. Revista De La Real Academia De Ciencias Exactas Fisicas Y Naturales Serie a-Matematicas 105, 181–206.
2012. Expert judgment, meta-analysis and participatory risk analysis. Decision Analysis 9, 119–127.
1992. Heuristics, biases, and the not-so-general publics: expertise and error in the assessment of risks. In and , editors. Social theories of risk. Praeger Publishing, Westport, Connecticut.
1996. Risky thinking: irrational fears about risk and society. Annals of the American Academy of Political and Social Science 545, 44–53.
1999. Tools for understanding the socioeconomic and political settings for environmental decision making. In and , editors. Tools to aid environmental decision making. Springer, New York.
1996. Media coverage of hazard events: analyzing the assumptions. Risk Analysis 16, 31–42., , and
2009. System and estimator variables in eyewitness identification: a review (Chapter 3). In and , editors. Psychological expertise in court. Ashgate Publishing, Farnham.
1907. Vox populi. Nature 75, 450–451.
2007. Subjective probability distribution elicitation in cost risk analysis. United States Airforce Report, RAND Corporation, Santa Monica.
2004. Australian principles of evidence. Routledge Cavendish, Sydney. and
2005. Statistical methods for eliciting probability distributions. Journal of the American Statistical Association 100, 680–700., and
2002. Calculated risks: how to know when numbers deceive you. Simon and Schuster, New York.
2007. Gut feelings: the intelligence of the unconscious. Viking Press, New York.
2005. A 30% chance of rain tomorrow: How does the public understand probabilistic weather forecasts?Risk Analysis 25, 623–629., , and
1999. Simple heuristics that make us smart. Oxford University Press, New York., and the ABC Research Group.
1997. Proper analysis of the accuracy of group judgments. Psychological Bulletin 121, 149–167. and
2002. Heuristics and biases: the psychology of intuitive judgement. Cambridge University Press., , and
2004. Neuroeconomics: the consilience of brain and decision. Science 306, 447–452. and .
1965. Diagnosticians vs. diagnostic signs: the diagnosis of psychosis vs. neurosis from the MMPI. Psychological Monographs 79 (Whole No. 602).
2003. An introduction to statistical ‘evidence’. Australian Bar Review 23, 239–262.
1959. Kinds of probability. Science 129, 443–447.
2010. The limits of forecasting methods in anticipating rare events. Technological Forecasting and Social Change 77, 355–368. and
2001. Expert judgement elicitation in risk assessment. In and , editors. Assessment and management of environmental risks. Kluwer Academic Publishers, Dordrecht. and
1924. Group judgments in the field of lifted weights. Journal of Experimental Psychology 7, 398–400.
2009. Massively collaborative mathematics: the Polymath Project. Nature 461, 879–881. and
2011. Comparing face-to-face meetings, nominal groups, Delphi and prediction markets on an estimation task. International Journal of Forecasting 27, 183–195. and
1996. Refining the CRA framework. In , editor. Comparing environmental risks. Resources for the Future, Washington, DC. and
2012. Structured decision making. Wiley-Blackwell, Chichester., , , , and
2006. Some pitfalls of an overemphasis on science in environmental risk management decisions. Journal of Risk Research 9, 717–735., , and
1993. The role of past states in determining reference points for policy decisions. Organisational Behavior and Human Decision Processes 55, 195–206., and
1998. Science in public: communication, culture, and credibility. Plenum Press, New York. and
2007. Global warming: forecasts by scientists versus scientific forecasting. Energy and Environment 18, 997–1021. and
1958. Experience of clinicians and speech characteristics and statements of clients as variables in clinical judgment. Journal of Consulting Psychology 22, 315–319.
1996. Comparative efficiency of informal (subjective, impressionistic) and formal (mechanical, algorithmic) prediction procedures: the clinical-statistical controversy. Psychology, Public Policy, and Law 2, 293–323. and
2000. Clinical vs. mechanical prediction: a meta-analysis. Psychological Assessment 12, 19–30., , , and
2000. The precautionary principle in Australia: policy, law and potential precautionary EIAs. Risk: Health, Safety and Environment 11, 93–124.
1958. Statistics of extremes. Columbia University Press, New York.
1973. A comparative study of differences in subjective likelihood estimates made by individuals, interacting groups, Delphi groups, and nominal groups. Organizational Behavior and Human Performance 9, 280–291., , and
2013. Integrating expert judgment in veterinary epidemiology: example guidance for disease freedom surveillance. Preventive Veterinary Medicine 109, 1–9., , and
2001. An epistemologist in the bramble-bush: at the Supreme Court with Mr. Joiner. Journal of Health Politics Policy and Law 26, 217–248.
1975. The emergence of probability: a philosophical study of early ideas about probability, induction and statistical inference. Cambridge University Press, London.
2011. Scientists on trial: at fault?Nature 477, 264–269.
2006. Designing real terrorism futures. Public Choice 128, 257–274.
1986. Knowledge acquisition for expert systems. McGraw-Hill, New York.
2000. Traffic accident statistics and risk perceptions in Japan and the United States. Accident Analysis and Prevention 32, 827–835., and
1986. Assessing uncertainty in physical constants. American Journal of Physics 54, 791–798. and
2012. Tapping into the wisdom of the crowd – with confidence. Science 36, 303–304.
2009. The wisdom of many in one mind. Psychological Science 20, 231–237. and
2005. Limits of intelligence analysis. Orbis 49, 75–94.
1987. Living with uncertainty in resource management. North American Journal of Fisheries Management 7, 1–5.
1997. The ecological detective: confronting models with data. Monographs in Population Biology 28. Princeton University Press, New Jersey. and
1990. The dominant view of popularization: conceptual problems, political uses. Social Studies of Science 20, 519–539.
2008. Informing risk-mitigation priorities using uncertainty measures derived from heterogeneous expert panels: a demonstration using foodborne pathogens. Reliability Engineering and System Safety 93, 687–698., , and
2002. Representation facilitates reasoning: what natural frequencies are and what they are not. Cognition 84, 343–352., , and
2001. Educating intuition. University of Chicago Press.
2005. Deciding analytically or trusting your intuition? The advantages and disadvantages of analytic and intuitive thought. In and , editors. The routines of decision making. Lawrence Erlbaum Associates, Mahwah, New Jersey.
2004. Groups of diverse problem solvers can outperform groups of high-ability problem solvers. Proceedings of the National Academy of Sciences of the USA 101, 16385–16389. and
2011. The impact of gender diversity on the performance of business teams: evidence from a field experiment. Tinbergen Institute Discussion Paper TI 2011–074/3. Amsterdam School of Economics, University of Amsterdam., and
1992. Acquisition of expert judgment: examples from risk assessment. Journal of Energy Engineering 118, 136–148.
2007. Eliciting probabilities from experts (Chapter 8). In , and , editors. Advances in decision analysis. Cambridge University Press.
1998. Introduction of contagious animal diseases into the Netherlands: elicitation of expert opinions. Livestock Production Science 53, 253–264., , and
1993. The Daubert decision: Frye is dead: long live the Federal Rules of Evidence. Trial 29, 60–65.
2008. Six pillars: futures thinking for transforming. Foresight 10, 4–21.
International Monetary Fund. 2006. Global Financial Stability Report. World Economic and Financial Surveys, Washington.
2005a. Contradicted and initially stronger effects in highly cited clinical research. Journal of the American Medical Association 294, 218–228.
2005b. Why most published research findings are false. PLoS Medicine 2, e124.
2008. Why most discovered true associations are inflated. Epidemiology. 19, 640–648.
2001. Constructing the scientific citizen: science and democracy in the biosciences. Public Understanding of Science 10, 1–18.
2011. How to build the global mathematics brain. New Scientist issue 2811, 4 May 2011, 10–11.
2010. Elicitator: an expert elicitation tool for regression in ecology. Environmental Modelling and Software 25, 129–145., and
2006. Does telling people what they have been doing change what they do? A systematic review of the effects of audit and feedback. Quality and Safety in Health Care 15, 433–436., , , and
Groupthink: Psychological studies of policy decisions and fiascos. Houghton Mifflin, Boston.
2006. Transparency in public science: purposes, reasons, limits. Law and Contemporary Problems 69, 21.
2007. Technologies of humility. Nature 450, 33.
1976. Confidence intervals vs Bayesian intervals. In and , editors. Foundations of probability theory, statistical inference, and statistical theories of science. Volume 2. D. Reidel Publishing, Dordrecht, Holland.
2002. On the definition and age-dependency of the value of a statistical life. Journal of Risk and Uncertainty 25, 251–263.
2011. The evolution of overconfidence. Nature 477, 317–320. and
2001. Calibration of subjective probability judgments in a naturalistic setting. Organizational Behavior and Human Decision Processes 85, 265–290. and
1983. Logical self-defense, 2nd edition. McGraw-Hill Ryerson, Toronto. and
1980. Interactive elicitation of opinion for a normal linear model. Journal of the American Statistical Association 75, 815–885., , , and
2013. Ideology, motivated reasoning, and cognitive reflection. Judgment and Decision Making 8, 407–424.
2015. Climate science communication and the measurement problem. Advances in Political Psychology 36, 1–43.
1967. The year 2000: a framework for speculation. Macmillan, New York. and
2011. Thinking, fast and slow. Farrar, Straus, and Giroux, New York.
1979. Prospect theory: an analysis of decision under risk. Econometrica 47, 263–291. and
1984. Choices, values, and frames. American Psychologist 39, 342–347. and
2002. Software glitch threw off mortality estimates. Science 296, 1945–1946.
2002. Race, gender and environmentalism: the atypical values and beliefs of white men. Race, Gender and Class 9, 1–19., , and
1999. Should we risk it? Exploring environmental, health, and technological problem solving. Princeton University Press, New Jersey. and
1992. ‘Expert opinion’ versus ‘expert opinions.’ Another approach to the problem of eliciting/combining/using expert opinion in PRA. Reliability Engineering and System Safety 35, 61–72.
1981. On the quantitative definition of risk. Risk Analysis 1, 11–27. and
2006. When should consumers and managers trust their intuition?Journal of Consumer Psychology 16, 20–24.
1991. Eliciting probabilities from experts in complex technical problems. IEEE Transactions on Engineering Management 38, 191–201. and
1996. A new way to ask the experts: rating radioactive waste risks. Science 274, 913–914.
2004. Group performance and decision making.Annual Review of Psychology 55, 623–655. and
2011. Crystal structure of a monomeric retroviral protease solved by protein folding game players. Nature Structural and Molecular Biology 18, 1175–1177., , Foldit Contenders Group, Foldit Void Crushers Group, , , , , , , , , and
1995. Daubert v. Merrell Dow Pharmaceuticals: active judicial scrutiny of scientific evidence. Food and Drug Law Journal 50, 213–234.
2006. Subjective confidence and the sampling of knowledge. In K. Fiedler and P. Juslin, editors. Information sampling and adaptive cognition. Cambridge University Press, New York, pp. 153–182., , and
1997. Types and measures of uncertainty. In , and , editors. Consensus under fuzziness. Kluwer Academic Publishers, Boston. and
1998. Uncertainty-based information: elements of generalized information theory. Physica-Verlag, Heidelberg. and
2007. Uncertainty communication issues and good practice. Copernicus Institute for Sustainable Development and Innovation. Report NWS-E-2007–199. Netherlands Environmental Assessment Agency, Utrecht., and
2010. The use of expert elicitation in environmental health impact assessment: a seven step procedure. Environmental Health 9, 19., , and
2003. Bayesian artificial intelligence. CRC Press, Boca Raton, Florida. and
2012. When two heads are better than one and why?Science 336, 360–362.
2012. Crowd IQ: measuring the intelligence of crowdsourcing platforms. In Proceedings of the 3rd Annual ACM Web Science Conference, 151–160. ACM., , , and
1993. Earthquake probability in engineering – Part 1: the use and misuse of expert opinion. Engineering Geology 33, 257–288.
2009. Unskilled and unaware: how difficulties in recognizing one's own incompetence lead to inflated self-assessments. Psychology 1, 30–46. and
1962. The structure of scientific revolutions. University of Chicago Press.
1990. The case for motivated reasoning. Psychological Bulletin 108, 480–498.
1976. Proofs and refutations. Cambridge University Press.
1986. Demonstrability and social combination processes on mathematical intellective tasks. Journal of Experimental Social Psychology 22, 177–189. and
2006. Judgmental forecasting: a review of progress over the last 25 years. International Journal of Forecasting 22, 493–518., , and
1900. The law of expert and opinion evidence, 2nd edition. T. H. Flood, Chicago.
2003. Amateurs: a 21st-century remake – social commentator Charles Leadbeater looks at the growing number of people in Britain taking a serious approach to pastime pursuits. RSA Journal 150, 22–25.
2012. Groupthink: the brainstorming myth. The New Yorker 30, 12.
2006. A general, analytic method for generating robust strategies and narrative scenarios. Management Science 52, 514–528., , and
2000. Knowledge partitioning: context-dependent use of expertise. Memory and Cognition 28, 295–305. and
1987. Member variation, recognition of expertise, and group performance. Journal of Applied Psychology 72, 81–87., and
1982. Calibration of probabilities: the state of the art to 1980. In , and , editors. Judgment under uncertainty: heuristics and biases. Cambridge University Press, New York., and
2008. A study of expert overconfidence. Reliability Engineering and System Safety 93, 711–721. and
2002. Comparison of treadmill scores with physician estimates of diagnosis and prognosis in patients with coronary artery disease. American Heart Journal 143, 650–658., , , , , , and
2009. Elicitation by design in ecology: using expert opinion to inform priors for Bayesian statistical models. Ecology 90, 265–277., and
1950. Implicit hypotheses and clinical predictions. Journal of Abnormal and Social Psychology 5, 756–760.
2008. Collective problem solving: are the many smarter than the few?Durham Anthropology Journal 15, 23–58.
2008. Explosion probability of unexploded ordnance: expert beliefs. Risk Analysis 28, 825–841., and
2001. Decomposition for judgmental forecasting and estimation. In , editor. Principles of forecasting: a handbook for researchers and practitioners. Kluwer Academic Publishers, Boston.
2006. The Delphi process – an expert-based approach to ecological modelling in data-poor environments. Animal Conservation 9, 11–19. and
2005. Can behavioral decision theory explain risk-averse fire management decisions?Forest Ecology and Management 211, 47–58. and
2012. An expert panel approach to assessing potential effects of bull trout reintroduction on federally listed salmonids in the Clackamas River, Oregon. North American Journal of Fisheries Management 32, 450–465., , , and
2004. Competing approaches to predicting supreme court decision making. Perspectives on Politics 2, 761–767., , and
2010. Guidance note for lead authors of the IPCC Fifth Assessment Report on consistent treatment of uncertainties. Intergovernmental Panel on Climate Change., , , , , , , , , , , and
2012a. Structured elicitation of expert judgments for threatened species assessment: a case study on a continental scale using email. Methods in Ecology and Evolution 3, 906–920., , , , , , , , , , and
2012b. Evaluating the accuracy and calibration of expert predictions under uncertainty: predicting the outcomes of ecological research. Diversity and Distributions 18, 782–794., and
2012. Representing vague opinion. Principia 16, 341–344. and
1954. Clinical versus statistical prediction: a theoretical analysis and review of the evidence. University of Minnesota Press, Minneapolis.
1986. Causes and effects of my disturbing little book. Journal of Personality Assessment 50, 370–375.
2001. Using comparative risk to set pollution prevention priorities in New York State: a formula for inaction: a critical analysis of the New York State Department of Environmental Conservation Comparative Risk Project. A report from the New York State Assembly Legislative Commission on Toxic Substances and Hazardous Wastes. Assemblyman Steve Englebright, Chair. New York.
2015. Identifying and cultivating superforecasters as a method of improving probabilistic predictions. Perspectives on Psychological Science 10, 267–281., , et al.
1990. Eliciting and analyzing expert judgment: a practical guide. Office of Nuclear Regulatory Research, Division of Systems Research, US Nuclear Regulatory Commission, Washington, DC. and
2013. When is consensus knowledge based? Distinguishing shared knowledge from mere agreement. Synthese, 190, 1293–1316.
2008. The trouble with overconfidence. Psychological Review 115, 502–517. and
2010. Conflict of interest and the intrusion of bias. Judgment and Decision Making 5, 37–53., and
1993. Risk analysis and management. Scientific American 269, 32–42.
2014. Use (and abuse) of expert elicitation in support of decision making for public policy. Proceedings of the National Academy of Sciences of the USA 111, 7176–7184.
1996. A proposal for ranking risk within Federal agencies. In , editor. Comparing environmental risks. Resources for the Future, Washington, DC., , and
1990. Uncertainty: a guide to dealing with uncertainty in quantitative risk and policy analysis. Cambridge University Press. and
2012. The market for financial advice: an audit study. National Bureau of Economic Research Working Paper No. 17929. Cambridge, Massachusetts., and
2004. Whom to trust? Public concerns, late modern risks, and expert trustworthiness. Journal of Agricultural and Environmental Ethics 17, 113–130.
1993. What is a good forecast? An essay on the nature of goodness in weather forecasting. American Meterological Society 8, 281–293.
1977. Can weather forecasters formulate reliable probability forecasts of precipitation and temperature?National Weather Digest 2, 2–9. and
1984. Probability forecasting in meteorology. Journal of the American Statistical Association 79, 489–500. and
2009. How useful is expert opinion for predicting the distribution of a species within and beyond the region of expertise? A case study using brush-tailed rockwallabiesPetrogale penicillata. Journal of Applied Ecology 46, 842–851., , , , and
2013. Science, policy, and the public discourse of shark ‘attack’: a proposal for reclassifying human-shark interactions. Journal of Environmental Studies and Sciences 3, 65–73. and
2001. Improving decision making by means of dissent. Journal of Applied Social Psychology 31, 48–58., , and
2000. Making better environmental decisions: an alternative to risk assessment. MIT Press, Cambridge, Massachusetts.
2012. Probabilistic uncertainty specification: overview, elaboration techniques and their application to a mechanistic model of carbon flux. Environmental Modelling and Software 36, 35–48.
2006. Uncertain judgements: eliciting experts’ probabilities. Wiley, Hoboken, New Jersey., , , , , , and
2004. Probability is perfect, but we can't elicit it perfectly. Reliability Engineering and System Safety 85, 239–248. and
2009. Comparison of three expert elicitation methods for logistic regression on predicting the presence of the threatened brush-tailed rock-wallabyPetrogale penicillata. Environmetrics 20, 379–398., , , , , and
2008. Using expert knowledge to assess uncertainties in future polar bear populations under climate change. Journal of Applied Ecology 45, 1649–1659., , , and
2003. Professional vs. amateur judgment accuracy: the case of foreign exchange rates. Organizational Behavior and Human Decision Processes 91, 169–185., , and
2013. Impact of plain packaging of tobacco products on smoking in adults and children: an elicitation of international experts’ estimates. BMC Public Health 13, 18., and
2005. The precautionary principle in practice: environmental decision-making and scientific uncertainty. Federation Press, Annandale, New South Wales.
2004. The role of self-rated ability in the accuracy of confidence judgements in eyewitness memory and general knowledge. Applied Cognitive Psychology 18, 157–168.
1998. Comparative risk assessment: where does the public fit in?Science, Technology, and Human Values 23, 221–241.
2008. Deliberating the risks of nanotechnologies for energy and health applications in the United States and United Kingdom. Nature Nanotechnology 4, 95–98., , and
1993. The psychology of judgment and decision making. McGraw-Hill, New York.
2003. River flows and water wars: emerging science for environmental decision making. Frontiers in Ecology and the Environment 1, 298–306., , , , , , , and
1999. Subjective versus statistical model assessment of mortality risk in open heart surgical procedures. Annals of Thoracic Surgery 67, 635–640., , , , and
2006. Measuring observer performance in chest radiology: some experiences. Journal of the American College of Radiology 3, 423–432.
2002. Truth survival in clinical research: an evidence-based requiem. Annals of Internal Medicine 136, 888–895., , , , , and
2003. Science and the law: evaluating evidentiary reliability. Australian Bar Review 23, 263–295.
2007. 60% Proof. Australasian Journal of Logic 5, 89–100.. and
1997. Fisheries stock assessment and decision analysis: the Bayesian approach. Reviews in Fish Biology and Fisheries 7, 35–63. and
2000. A proposal for fuzzy IUCN categories and criteria. Biological Conservation 92, 101–108., and
2002. A taxonomy and treatment of uncertainty for ecology and conservation biology. Ecological Applications 12, 618–628., and
2001. The currency and tempo of extinction. American Naturalist 157, 1–10., , and
2002. Critical care cardiovascular nurse expert and novice diagnostic cue utilization. Journal of Advanced Nursing 39, 24–34.
2009. Assessing authority. Journal of the American Medical Association 301, 1819–1821.. and
1994. Risk perception of different societal groups: Australian findings and cross-national comparisons. Australian Journal of Psychology 46, 150–163.
1998. The risk notion: epistemological and empirical considerations. In and , editors. Integrated risk assessment. Balkema, Rotterdam.
2010. Future declines of the binational Laurentian Great Lakes fisheries: the importance of environmental and cultural change. Frontiers in Ecology and the Environment 8, 239–244., , and
2001. Expert opinions in forecasting: the role of the Delphi technique. In , editor. Principles of forecasting: a handbook for researchers and practitioners. Kluwer Academic Publishers, Boston. and
2002. The Pacific Salmon wars: what science brings to the challenge of recovering species. Annual Review of Ecology and Systematics 33, 665–706., , and
2004. The Supreme Court forecasting project: legal and political science approaches to predicting Supreme Court decisionmaking. Columbia Law Review 104, 1150–1209., , and
1992. Managing overconfidence. Sloan Management Review 33, 7–17. and
1972. The foundations of statistics. Dover Publications, New York.
2006. Group decision making in hidden profile situations: dissent as a facilitator for decision quality. Journal of Personality and Social Psychology 91, 1080–1093., , , and
2012. Why groups perform better than individuals at quantitative judgment tasks: group-to-individual transfer as an alternative to differential weighting. Organizational Behavior and Human Decision Processes 118, 24–36., and
2007. Assessing the competence and credibility of human sources of intelligence evidence: contributions from law and probability. Law, Probability and Risk 6, 247–274. and
2002. Combination of evidence in Dempster-Shafer theory. SAND Report, SAND2002-0835. Sandia National Laboratories, Albuquerque, New Mexico. and
1987. Minimum viable populations: coping with uncertainty. In , editor. Viable populations for conservation.Cambridge University Press.
1992a. Competence in experts: the role of task characteristics. Organizational Behavior and Human Decision Processes 53, 252–266.
1992b. How much information does an expert use? Is it relevant?Acta Psychologica 81, 75–86.
2000. Cause and correlation in biology. Cambridge University Press.
1996a. Methodological rules for four classes of scientific uncertainty. In , editor. Scientific uncertainty and environmental problem solving. Blackwell, Cambridge, Massachusetts.
1996b. Value judgments in verifying and validating risk assessment models. In , editor. Handbook for environmental risk decision making: values, perceptions and ethics. CRC Lewis Publishers, Boca Raton.
1959. Theories for decision-making in economic and behavioral science. The American Economic Review 49, 253–283.
1999. Trust, emotion, sex, politics, and science: surveying the risk-assessment battlefield. Risk Analysis 19, 689–701.
1984. Perception and acceptability of risk from energy systems. In and , editors. Public reactions to nuclear power: are there critical masses?AAAS/Westview, Boulder, Colorado., and
2000. Violence risk assessment and risk communication: the effects of using actual cases, providing instruction, and employing probability versus frequency formats. Law and Human Behavior 24, 271–296., and
2004. Overconfidence in interval estimates. Journal of Experimental Psychology: Learning, Memory, and Cognition 30, 299. and
2007. Climate Change 2007: The Physical Science Basis. Contribution of Working Group I to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change. Cambridge University Press, Cambridge and New York., , , , , , and
2010. Reducing overconfidence in the interval judgments of experts. Risk Analysis 30, 512–523., , , , and
1975. Probability encoding in decision analysis. Management Science 22, 340–358. and
1994. Empirical evaluation of prior beliefs about frequencies: methodology and a case study in congenital heart disease. Journal of the American Statistical Association 89, 435–443., and
2014. Roles of scientists as policy advisers on complex issues: a literature review. Environmental Science and Policy 40, 16–25., , , , and
1985. Pooling of unshared information in group decision making: biased information sampling during discussion.Journal of Personality and Social Psychology 48, 1467–1478. and
1993. Deriving parameter probability density functions. Reliability Engineering and System Safety 42, 271–291., and
1996. Understanding risk: informing decisions in a democratic society. National Academy Press, Washington, DC. and , editors.
1993. Practical intelligence: the nature and role of tacit knowledge in work and at school. In and , editors. Mechanisms of everyday cognition. Lawrence Erlbaum Associates, Hillside, New Jersey., and
1997. Probabilistic risk assessment of engineering systems. Chapman and Hall, London. and
2007. A general framework for analysing diversity in science, technology and society. Journal of the Royal Society Interface 4, 707–719.
2003. Enhancing causal assessment of estuarine fishkills using graphical models. Ecosystems 6, 11–19. and
2005. The wisdom of crowds: why the many are smarter than the few and how collective wisdom shapes business, economies, societies and nations. Doubleday, New York.
1988. Illusion and well-being: a social psychological perspective on mental health. Psychological Bulletin 103, 193–210. and
2005. When 90% confidence intervals are 50% certain: on the credibility of credible intervals. Applied Cognitive Psychology 19, 455–475. and
2005. Expert political opinion, how good is it? How can we know?Princeton University Press, New Jersey.
2011. Intelligent management of intelligence agencies. American Psychologist 66, 542–554. and
1991. Quasi-rational economics. Russell Sage Foundation, New York.
2007. Environments that make us smart. Current Directions in Psychological Science 16, 167–171. and
1958. The uses of argument. Cambridge University Press.
1997. Visual explanations: images and quantities, evidence and narrative. Graphics Press, Cheshire, Connecticut.
1971. Belief in the law of small numbers. Psychological Bulletin 76, 105–110.. and
1974. Judgement under uncertainty: heuristics and biases. Science 185, 1124–1131. and
1985. The framing of decisions and the psychology of choice. In G. Wright, editor. Behavioral decision making, pp. 25–41. Springer, New York. and
1982a. Belief in the law of small numbers. In , and , editors. Judgement under uncertainty: heuristics and biases. Cambridge University Press, pp. 23–30. and
1982b. Causal schemata in judgments under uncertainty. In , , , editors. Judgment under uncertainty: heuristics and biases. Cambridge University Press. and
2011. Accuracy and reliability of forensic latent fingerprint decisions. Proceedings of the National Academy of Sciences of the USA 108, 7733–7738., , and
2001. Expert judgment resolution in technically-intensive policy disputes. In and , editors. Assessment and management of environmental risks. Kluwer Academic Publishers, Dordrecht.
1996. Scenarios: the art of strategic conversation. Wiley, Chichester.
1984. Belief and the will. Journal of Philosophy 81, 235–256.
2013. Argument mapping. In H. Pashler, editor. Encyclopedia of the mind. Sage, Thousand Oaks, CA.
1999. Minimizing ambiguity and paraconsistency. Logique et analyse 165, 39–160.
1991. Consensus of expertise: the role of consensus of experts in formulating public policy and estimating facts. Journal of Medicine and Philosophy 16, 427–445.
2002. Transferring strategies of land management: indigenous land owners and environmental scientists. Research in Science and Technology Studies, Knowledge and Society 13, 155–181.
1996. Quantitative risk analysis: a guide to Monte Carlo simulation modelling. Wiley, Chichester.
2000. Bayesian methods in conservation biology. Conservation Biology 14, 1308–1316.
1986. Does the expert know? The reliability of predictions and confidence ratings of experts. In , and , editors. Intelligent decision support in process environments. Springer-Verlag, Berlin. and
1991. Statistical reasoning with imprecise probabilities. Chapman and Hall, London.
2000. Towards a unified theory of imprecise probability. International Journal of Approximate Reasoning 24, 125–148.
2001. A behavioral model for linguistic uncertainty. Information Sciences 134, 1–37. and
1983. Encoding subjective probabilities: a psychological and psychometric review. Management Science 29, 151–173. and
1997. Evaluating and combining subjective probability estimates. Journal of Behavioral Decision Making 10, 243–268., , and
1997. Appeal to expert opinion: arguments from authority. Pennsylvania State University Press, University Park.
2002. A domain-specific risk-attitude scale: measuring risk perceptions and risk behaviors. Journal of Behavioral Decision Making 15, 263–290., , and
1991. A novel approach to reducing uncertainty: the Delphi group. Technological Forecasting and Social Change 39, 253–263., , and
2012. Decloaking the privileged expert. Journal of Management and Organization 18, 300–310. and
2009. The complex relationship between pediatric cardiac surgical case volumes and mortality rates in a national clinical database. Journal of Thoracic and Cardiovascular Surgery 137, 1133–1140., , , , and
2002. Judging the accuracy of a likelihood judgment: the case of smoking risk. Journal of Behavioral Decision Making 15, 19–35.
1993. Evaluating and combining physicians’ probabilities of survival in an intensive care unit. Management Science 39, 1526–1543. and
2013. Making Decisions When Estimates Conflict: Improving Judgements in Environmental Science. PhD Thesis, University of Melbourne.
2012. The intelligence game: assessing Delphi groups and structured question formats. In , and , editors. Proceedings of the 5th Australian Security and Intelligence Conference. pp. 14–26. Security Research Institute, Edith Cowan University., , , , , , , , and
2013b. The trial of scientists at L'Aquila: Lessons for risk communication. Risk Analysis (in review)., and
1976. The accuracy of technological forecasts, 1890–1940. Futures 8, 411–419.
WMD Commission. 2005. The commission on the intelligence capabilities of the United States regarding weapons of mass destruction. March 31, 2005. US Government 374. In Wheaton, K. and Chido, D. 2007. Evaluating Intelligence. Competitive Intelligence Magazine 10, 2–19.
2002. Witness ‘conferencing’. Arbitration International 18, 47–58.
2010. Evidence for a collective intelligence factor in the performance of human groups. Science 330, 686–688., , , and
2002. An empirical test of the relative validity of expert and lay judgments of risk. Risk Analysis 22, 1107–1122., and
1996. May the sheep safely graze? A reflexive view of the expert-lay knowledge divide. In , and , editors. Risk, environment and modernity: towards a new ecology. Sage Publications, London.
2000. Making systematic sense of public discontents with expert knowledge: two analytical approaches and a case study. Public Understanding of Science 9, 105–122.
1986. A simple view of the Dempster–Shafer theory of evidence and its implication for the rule of combination. The AI Magazine 1986, 85–90.