1 Introduction
How much of human behavior (including judgments and decisions) is due to the “person” versus the “situation”? This question dates back to ancient Greece (e.g., Aristotle’s tabula rasa vs. Plato’s divinely preformed mind). Today the debate continues amid increasing evidence that the answer is neither one (the person, e.g., Reference AllportAllport, 1937; Reference DigmanDigman, 1990) nor the other (the situation, e.g., Reference MilgramMilgram, 1974; Reference ZimbardoZimbardo, 2004), but rather the two in combination (e.g., Reference MischelMischel, 1968, 2004). Further evidence against simple, one-or-the-other approaches comes from the emerging field of epigenetics, which documents biochemical mechanisms through which environmental conditions regulate gene expression (e.g., Reference HymanHyman, 2009). Nevertheless individual differences continue to be widely used as explanatory variables, in everything from risk aversion in economics (Reference WeberWeber, 2001) to animal personality in biology (e.g., Reference Herborn, Macleod, Miles, Schofield, Alexander and ArnoldHerborn et al., 2010).
We argue that the persistent emphasis on a large range of individual differences as main effects in the field of judgment and decision making (JDM) is outdated. Thus, we propose four guidelines for the more productive pursuit of individual differences research within JDM: a more systematic approach, a shift toward theoretically relevant measures, a greater emphasis on interactions, and more extensive communication of results. We offer our Decision Making Individual Differences Inventory (DMIDI; http://www.sjdm.org/dmidi), a free online database, as a tool to help accomplish these aims. Before elaborating upon our guidelines, we will present an overview of common individual difference measures in JDM research.
1.1 Decision making by individuals
The decisions made by individuals are widely recognized as being affected by three sets of factors—decision features, situational factors, and individual differences (Reference EinhornEinhorn, 1970; Reference Hunt, Krzystofiak, Meindl and YousryHunt et al., 1989). Of these three, decision features, which are characteristics of the decision itself, are probably understood best. A wealth of research has demonstrated the impact of decision features such as the framing of choice options (see Reference KühbergerKühberger, 1998, and Reference Levin, Schneider and GaethLevin et al., 1998, for reviews), the ordering of choice options (e.g., Reference Davis, Tindale, Nagao, Hinsz and RobertsonDavis et al., 1984; Reference Krosnick, Miller, Tichy, Crigler, Just and McCafferyKrosnick et al., 2004; Reference Nadler, Irwin, Davis, Au, Zarnoth and RantillaNadler et al., 2001), and the requirement of choice justification (see Reference Lerner and TetlockLerner & Tetlock, 1999, for a review). Additionally, a consensus has emerged regarding the effects of many situational factors—characteristics of the situation in which the decision is faced—including time pressure (e.g., Reference Dror, Busemeyer and BasolaDror et al., 1999; Reference VerplankenVerplanken, 1993), cognitive load (e.g., Reference Drolet and LuceDrolet & Luce, 2004; Reference EbertEbert, 2001), and social context (e.g., Reference Nadler, Irwin, Davis, Au, Zarnoth and RantillaNadler et al., 2001). In contrast, even though there has been a fair amount of research about the effects of individual differences—characteristics of the decision maker—on decision making, it is not clear that we as a field fully understand them. As a result, theory in judgment and decision making has focused on the construction of preferences as determined by decision features and situational factors (e.g., Reference Lichtenstein and SlovicLichtenstein & Slovic, 2006; Reference Weber and JohnsonWeber & Johnson, 2009), and not on the influence of chronic individual, group, or cultural differences, as noted by Reference Weber and MorrisWeber and Morris (2010).
1.2 Individual differences and decision making
There are frequent calls to study the effects of individual differences on decision processes and outcomes in order to rectify what has been seen as an overemphasis on decision features and situation factors (e.g., Reference LevinLevin, 1999; Reference Mohammed and SchwallMohammed & Schwall, 2009; Reference Scott and BruceScott & Bruce, 1995; Reference Shiloh, Koren and ZakayShiloh et al., 2001). Contrary to what these appeals suggest, there actually is a considerable amount of JDM research on the effects of individual differences. What is lacking, however, is consensus about the interpretation and significance of existing results and, thus, about the role of individual differences in decision making. Even a cursory review reveals a constellation of confusing and often contradictory results for many individual differences (for example, see Reference Levin, Gaeth, Schreiber and LauriolaLevin et al., 2002, and Reference Shiloh, Salton and SharabiShiloh et al., 2002, for contradictory results regarding cognitive style and framing effects; see Reference Mohammed and SchwallMohammed and Schwall, 2009, for a review).
There are multiple ways to improve this picture. A recent review by Reference Mohammed and SchwallMohammed and Schwall (2009) urges decision researchers to explore the topic in more detail (e.g., including more pre- and post-decision variables) and with more appropriate tools, such as experimental designs that minimize the power of the situation, which can overwhelm any impact of individual differences. We suggest a different and more comprehensive set of guidelines that addresses individual differences research from the study design stage through the publication stage. We believe that a change of approach can better our understanding of individual differences in JDM. To that end, we hope that our guidelines spur discussion about the importance of individual differences in JDM and encourage a more systematic approach to the topic. We also hope that the DMIDI can aid in the efforts toward a more standardized and cumulative analysis of individual differences. Before we offer our recommendations, we turn to our overview.
2 Overview of individual difference measures
“Individual differences” is a broad term, covering any variable that differs between people, from decision style to cognitive ability to personality. Our overview highlights the most common categories of individual difference measures used in judgment and decision-making research. Because common measures change over time and also differ between subfields, our overview is representative rather than comprehensive. Information on other measures and their effects on judgment and choice can be found in the DMIDI, our extensive and continuously evolving online database, which we hope will serve as a dynamic forum for a more complete and cumulative analysis and discussion of individual difference measures.
We divide measures into seven categories: decision-making measures, risk attitude measures, cognitive ability measures, motivation measures, personality inventories, personality construct measures, and miscellaneous measures. Recognizing that there are probably as many categorization schemes as there are measures, we based ours initially on that of Reference Mohammed and SchwallMohammed and Schwall (2009) for consistency and then extended it based on conversations with other individual differences researchers. Other difficulties that we ran into when categorizing measures were fuzzy boundaries between constructs (e.g., cognitive style measures are often used as measures of decision style) and measures belonging to multiple categories (e.g., epistemic motivation measures which assess both motivation and cognition). In our overview and on the DMIDI, we have attempted to indicate the gray areas and to cross-list measures so that they can be found under any of their member categories.
2.1 Decision-making measures
Measures of individual differences in decision making can be divided into measures of style, approach, and competence. Under style measures, we include both decision style measures, such as General Decision-Making Style (GDMS; Reference Scott and BruceScott & Bruce, 1995), and cognitive style measures, such as the Rational-Experiential Inventory (REI; Reference Epstein, Pacini, Raj and HeierEpstein et al., 1996; Reference Norris, Pacini and EpsteinNorris et al., 1998). Although there is some disagreement as to whether decision style and cognitive style represent the same construct or not (e.g., Reference Schwartz, Ward, Monterosso, Lyubomirsky, White and LehmanMohammed et al., 2007; Reference Schwartz, Woloshin, Black and WelchMohammed & Schwall, 2009; Reference ThunholmThunholm, 2004), they can both be said to assess individuals’ methods of making decisions, or thinking more generally, and the extent to which they use a certain strategy or style (e.g., rational or intuitive). We also include here measures of epistemic motivation. As measures of motivated cognition (e.g., information processing, thinking, and judgment), they appear under both decision style and motivation. Measures of epistemic motivation include the Need for Cognitive Closure Scale (NFCS; Webster & Kruglanski, 1994) and the Need for Cognition Scale (NFC; short form by Reference Cacioppo, Petty and KaoCacioppo et al., 1984).
Measures of decision approach assess various aspects of individuals’ management of decision making, both pre- and post-decision, and include such constructs as indecision (e.g., the Indecisiveness Scale by Reference Frost and ShowsFrost & Shows, 1993), decision conflict (e.g., the Melbourne Decision Making Questionnaire by Reference Mann, Burnett, Radford and FordMann et al., 1997), and regret (e.g., the Regret Scale by Reference Schwartz, Ward, Monterosso, Lyubomirsky, White and LehmanSchwartz et al., 2002).
Decision making competence refers to the ability or set of skills needed to make good decisions, based on normative models of decision making (see Reference Bruine de Bruin, Parker and FischhoffBruine de Bruin et al., 2007, and Parker & Fischhoff, 2005, for more details). Decision competence measures, such as Adult Decision Making Competence (A-DMC; Reference Bruine de Bruin, Parker and FischhoffBruine de Bruin et al., 2007) and the Decision Outcome Inventory (DOI; Reference Bruine de Bruin, Parker and FischhoffBruine de Bruin et al., 2007), assess how well individuals make decisions and whether they tend to reach satisfactory outcomes. Measures of specific abilities, such as numerical ability (e.g., numeracy by Reference Peters, Dieckmann, Dixon, Hibbard and MertzPeters et al., 2007), contribute to decision making competence and are cross-listed here. Relatively new to the scene, decision competence measures are promising individual difference measures for JDM because of their ability to predict real-world decision performance (e.g., Reference Bruine de Bruin, Parker and FischhoffBruine de Bruin et al., 2007; Reference Parker, de Bruin and FischhoffParker et al., 2007). There is also evidence linking specific cognitive control abilities with specific dimensions of decision making competence (Reference Del Missier, Mäntylä and Bruine de BruinDel Missier et al., 2010). We will return to the potential utility of decision making competence measures in our guidelines.
2.2 Risk attitude measures
In economics, risk attitude is typically modeled as the shape of a decision maker’s utility function. Other frameworks, including that of finance, model risk attitude as the tradeoff between perceived risks and returns (e.g., Reference Weber, Blais and BetzWeber et al., 2002). Across frameworks, measures of risk attitude generally assess decision makers’ preferred levels of risk. Measures of risk attitude fall into three categories (see Weber & Johnson, 2008, for a review).
In one category are behavioral measures of risk where an individual’s risk preferences are determined from actual choices made in games or scenarios, both real and hypothetical. The Balloon Analog Risk Task (BART; Reference Lejuez, Read, Kahler, Richards, Ramsey and StuartLejuez et al., 2002), Columbia Card Task (CCT; Reference Figner, Mackinlay, Wilkening and WeberFigner et al., 2009), Cups Task (Reference Levin and HartLevin & Hart, 2003; Reference Levin, Weller, Pederson and HarshmanLevin et al., 2007), and Iowa Gambling Task (Reference Bechara, Damasio, Damasio and AndersonBechara et al., 1994) are examples of behavioral measures of risk.
A second category assesses risk attitude using self-report questionnaires, such as the Choice Dilemmas Questionnaire (CDQ; Kogan & Wallach, 1964) and Risk-taking Propensity (Reference Jackson, Hourany and VidmarJackson et al., 1972), which directly question an individual about risky situations. Included in this category are measures that also assess decision makers’ perceptions of risks and benefits in order to infer their preferred levels of risk (e.g., the Domain Specific Risk Task (DOSPERT), Reference Weber, Blais and BetzWeber et al., 2002).
A third category approaches risk attitude through individuals’ self-reports of personality traits related to risk-taking and aversion. Because these measures assess relevant personality traits, such as impulsivity, some of them are also included as personality construct measures (e.g., Eysenck’s Impulsivity Inventory by Eysenck & Reference Eysenck and EysenckEysenck, 1978) or represent a subset of a larger personality inventory (e.g., the Zuckerman-Kuhlman Personality Questionnaire (ZKPQ) by Reference Zuckerman, Kuhlman, Joireman, Teta and KraftZuckerman et al., 1993). Some measures are also listed under motivation (e.g., Need for Arousal by Reference Figner, Mackinlay, Wilkening and WeberFigner et al., 2009).
Closely related to risk attitude measures are ambiguity attitude measures. Ambiguity can be conceptualized as “uncertainty about uncertainty” (Reference Lauriola, Levin and HartLauriola et al., 2007; e.g., the Ambiguity-Probability Tradeoff Task by Reference Lauriola and LevinLauriola & Levin, 2001) or, more broadly, as a lack of sufficient probability information (e.g., Multiple Stimulus Types Ambiguity Tolerance by Reference McLainMcLain, 1993).
2.3 Cognitive ability measures
Measures of cognitive ability assess decision makers’ intelligence and/or capabilities. Cognitive ability measures can be divided into measures of global ability and measures of specific abilities or skills. Global ability, or overall intelligence, measures assess fluid intelligence and include Raven’s Standard Progressive Matrices (SPM; Raven et al., 2003) and the Wechsler scales (e.g., Wechsler Adult Intelligence Scale (WAIS) by Reference WechslerWechsler, 1955, Reference Wechsler1997).
Measures of specific abilities assess specific skills or competency areas, such as reading comprehension (e.g., the Nelson-Denny Reading Test by Reference Brown, Fishco and HannaBrown et al., 1993) and numerical ability (e.g., objective numeracy by Reference Peters, Dieckmann, Dixon, Hibbard and MertzPeters et al., 2007). Some specific abilities, such as numeracy, contribute to decision making competence and are cross-listed there. Also included under specific measures of ability are subjective measures of ability, such as the Subjective Numeracy Scale (SNS; Reference Fagerlin, Fisher, Ubel, Jankovic, Derry and SmithFagerlin et al., 2007). While these measures rely on self-report and are not objective tests of ability, they are often used to complement objective measures.
2.4 Motivation measures
Individual differences in motivation are differences in the drives to engage or not engage in various behaviors. Motivation measures vary greatly in their target constructs. They can be partitioned into four groups based on what they assess. (1) Measures of motivated self-presentation, also known as social desirability, assess how individuals present themselves to others and include the Balanced Inventory of Desirable Responding (BIDR; Reference PaulhusPaulhus, 1991) and the Marlowe-Crowne Social Desirability Scale (MC-SDS; Reference Crowne and MarloweCrowne & Marlowe, 1960). (2) Measures of motivated self-regulation during goal pursuit assess individuals’ orientations to their goals (e.g., the Behavioral Inhibition Scale and Behavioral Activation Scale (BIS/BAS) by Carver & White, 1994, the Regulatory Focus Questionnaire (RFQ) by Reference Higgins, Friedman, Harlow, Idson, Ayduk and TaylorHiggins et al., 2001, and the Regulatory Mode Questionnaire (RMQ) by Reference Kruglanski, Thompson, Higgins, Atash, Pierro and ShahKruglanski et al., 2000). (3) Measures of interpersonal motivation assess the drives underlying individuals’ interactions with others and include such measures as the Ring Measure of Social Values (Liebrand & McClintock, 1988) and Self-Report Altruism (Reference Rushton, Chrisjohn and FekkenRushton et al., 1981). (4) Lastly, there are measures assessing psychological needs and fears as motivations; these include Fear of Negative Evaluation (FNE; Reference LearyLeary, 1983), Need to Evaluate (Jarvis & Petty, 1996), and Consumers’ Need for Uniqueness (short form by Reference Ruvio, Shoham and BrenčičRuvio et al., 2008). Some of these measures are also used to assess risk attitude (e.g., Need for Arousal by Reference Figner, Mackinlay, Wilkening and WeberFigner et al., 2009). Within this fourth category (as well as under measures of decision/cognitive style), we place measures of epistemic motivation, which assess motivated cognition (e.g., information processing, thinking, and judgment) and include the Need for Cognitive Closure Scale (NFCS; Webster & Kruglanski, 1994) and the Need for Cognition Scale (NFC; short form by Reference Cacioppo, Petty and KaoCacioppo et al., 1984).
2.5 Personality inventories
Personality is another umbrella term. Personality refers to individuals’ traits, or characteristics that are stable over time—although there is debate about whether traits are stable across situations (the trait model; e.g., Reference AllportAllport, 1937; Reference DigmanDigman, 1990) or only within situations (the interactionist model; e.g., Reference MischelMischel, 1968, 2004). We will return to this debate in our guidelines. We divide personality measures into inventories, which assess constellations of traits, and construct measures, which assess single traits.
Inventories can be distinguished by their underlying theory of personality. The most common of such theories is the Five Factor Theory of Personality (“the Big Five”), which posits five dimensions of personality: openness (also called culture or intellect), characterized by originality and curiosity; conscientiousness (sometimes called dependability), characterized by orderliness and responsibility; extraversion (also called surgency), characterized by talkativeness and assertiveness; agreeableness, characterized by trust and being good-natured; and neuroticism (often reverse-scored and labeled emotional stability), characterized by being easily upset (Reference DigmanDigman, 1990; Reference GoldbergGoldberg, 1992; John & Srivastava, 1999). Common Big Five measures include the Ten Item Personal ity Inventory (TIPI; Reference Gosling, Rentfrow and SwannGosling et al., 2003), Trait Descriptor Adjectives (TDA; Reference GoldbergGoldberg, 1992), and the NEO Personal Inventory-Revised (NEO-PI-R; Reference Costa and McCraeCosta & McCrae, 1992). There are also several inventories, such as the California Personality Inventory (CPI; Reference GoughGough, 1987) and the Minnesota Multiphasic Personality Inventory (MMPI; Reference Hathaway and McKinleyHathaway & McKinley, 1943), that stem from other theories that describe subsets, supersets, or non-overlapping sets of traits from those of the Big Five. Subsets of some of these inventories are also used to assess risk attitude (e.g., the Zuckerman-Kuhlman Personality Questionnaire (ZKPQ) by Reference Zuckerman, Kuhlman, Joireman, Teta and KraftZuckerman et al., 1993).
2.6 Personality constructs
Personality constructs abound, but JDM research tends to focus on a subset. These construct measures can be categorized into six groups: (1) measures relating to facets of the self, including self-esteem (e.g., the Coopersmith Self-Esteem Inventory by Reference CoopersmithCoopersmith, 1967, 1981, and the Rosenberg Self-Esteem Scale by Reference RosenbergRosenberg, 1965) and self-consciousness (e.g., the revised Self-Consciousness Scale by Scheier & Carver, 1985); (2) interpersonal measures assessing how individuals perceive and act toward others and including constructs such as empathy (e.g., the Interpersonal Reactivity Index (IRI) by Reference DavisDavis, 1980) and trust (e.g., the Trust Inventory by Reference Couch, Adams and JonesCouch et al., 1996); (3) measures of impulsiveness (i.e., the tendency to act without forethought), such as the Barratt Impulsiveness Scale (BIS; Reference Patton, Stanford and BarrattPatton et al., 1995), some of which are also used to assess risk attitude; (4) measures of cultural differences, which assess dimensions on which cultures are assumed to vary, such as individualism versus collectivism, power distance, and masculinity versus femininity (e.g., the Values Survey Module (VSM) by Reference HofstedeHofstede, 2001); (5) measures of time orientation, which assess perceptions of time (e.g., Consideration of Future Consequences (CFC) by Reference Strathman, Gleicher, Boninger and EdwardsStrathman et al., 1994, and Future Time Orientation (FTO) by Reference GjesmeGjesme, 1975); and (6) measures of perceived control which distinguish between perceptions of control as internal versus external (e.g., Spheres of Control by Reference PaulhusPaulhus, 1983).
2.7 Miscellaneous measures
Finally, there are measures that do not fall squarely into any of our major categories. These measures assess a wide array of individual differences, including attitudes (e.g., New Environmental Paradigm-Revised (NEP-R) by Reference Dunlap, Liere, Mertig and JonesDunlap et al., 2000) and emotions and moods (e.g., the Positive and Negative Affect Scale (PANAS) by Reference Watson, Clark and TellegenWatson et al., 1988). While some or all of these constructs may evolve into their own categories over time, they are currently situated within our miscellaneous category.
3 Guidelines for future research
As our overview demonstrates, individual differences vary greatly in their theoretical underpinnings and their target constructs. Given this, one should not expect that all categories of individual differences would be important and significant main effects predictors in all domains, nor are they (see Reference Bazerman, Curhan, Moore and ValleyBazerman et al., 2000; Reference Kassarjian, Sheffet, Kassarjian and RobertsonKassarjian & Sheffet, 1991; Reference MischelMischel, 2004; and Reference Mohammed and SchwallMohammed & Schwall, 2009, for reviews in the domains of negotiation, consumer behavior, personality, and JDM, respectively). Thus, the persistent use of such a wide range of individual difference measures within the domain of JDM may not be advisable. Instead, we suggest amending the ways in which individual difference effects are investigated in JDM. Specifically, we offer four recommendations that address the pursuit of individual differences research from measure selection through publication: (1) a more systematic approach to individual differences; (2) a shift toward theoretically relevant measures; (3) a shift from a search for direct effects of individual differences to an examination of individual differences in interaction with decision features, situational factors, and other individual differences; and (4) more comprehensive sharing of a wider range of results. We elaborate on each of these recommendations below.
3.1 A systematic approach to individual differences
The study of individual differences in JDM has been unsystematic, with different studies using different measures of the same individual difference construct or using the same measure but adapting it for their own needs. There is currently no standardized set of measures for use in JDM research. For each construct, such as decision style, there are a number of different measures that have been used in various contexts, and different effects have been found with different measures (see Reference Mohammed and SchwallMohammed and Schwall, 2009, for a review). Additionally, researchers often modify existing measures by selecting a handful of questions or altering question wording. We recognize that it is sometimes necessary to develop a new measure or to modify an existing measure to fit the needs of a particular study. However, in many cases there exists an appropriate, validated measure of which the researchers are simply unaware because of the difficulties inherent in searching various literatures for suitable measures. In cases like these, we advocate using existing measures in their standard form (and we offer the DMIDI to facilitate the search process) because frequent creation and modification of measures leads to few studies using the same measures as predictors to investigate the same decision phenomena in similar experimental settings, which in turn means that a meta-analysis is impossible.
We argue that more standardization is required to allow results to accumulate and for a better understanding of the effects of individual measures on decision-making processes and outcomes. The selection and repeated use of a standard battery of measures (where appropriate) would provide data on the abilities and limitations of different scales. It would also allow cross-study comparisons (and even meta-analyses) to better establish when and how individual difference measures affect decision processes and outcomes.
We realize that scale selection can be difficult and time-consuming. Therefore, as a companion piece to this review we offer our DMIDI database as a free, public resource designed to encourage standardization of individual difference measures. By categorizing and describing the most common individual difference measures used in JDM research, the DMIDI can help the judgment and decision-making community share information about individual difference measures. The DMIDI can also host discussions about the relative merits of various individual difference measures and allow a consensus to build about the best measures for use in JDM. To jump start this process, each measure’s entry in the DMIDI contains a brief description of the measure, a link to the original paper introducing the measure, and links to examples of published research using the measure. Where we have been given permission by measure authors, the entry also has the measure itself available for download. The DMIDI is intended to be a collaborative wiki-style endeavor. Consequently, its success is dependent upon the JDM community. We encourage researchers to support the DMIDI by adding additional individual difference measures as well as information on their experiences with measures. We also encourage researchers to contribute to discussions about various individual difference measures or even entire constructs.
3.2 Theory-based selection of individual difference measures
Beyond a more systematic approach to the study of individual differences, we also need to think carefully as a field about which individual difference measures are worth pursuing as predictors of decision making variables. It is all too common to add a long list of individual difference measures to a study in order to see what has predictive ability (known as the “kitchen sink” or “see what sticks” approach). Personality measures (e.g., the Big Five) are often included in questionnaire batteries as a matter of course rather than to test a priori predictions. This approach explains the heterogeneous array of individual difference measures that have been used, mostly with limited success, in JDM research. In lieu of the kitchen sink approach, we recommend that, for each study, researchers carefully select a limited number of measures that have clear theoretical relevance for the paradigm. The inclusion of measures with theory-driven hypotheses provides better tests of measures than random inclusion.
If any individual difference measures are going to bear fruit, measures with clear theoretical ties and proven domain relevance are the most likely to do so. For example, for research investigating decision-making outcomes, the direct theoretical ties with decision-related measures, such as decision making competence, cognitive ability, and risk attitude, suggest that these measures may hold the most promise for this research. In this area, Reference Bruine de Bruin, Parker and FischhoffBruine de Bruin et al.’s (2007) A-DMC measure of decision competence is particularly promising because of the reliability and external validity of its components and because of evidence showing a link between executive functioning and elements of decision making competence (i.e., applying decision rules and consistency in risk perception) (Reference Del Missier, Mäntylä and Bruine de BruinDel Missier et al., 2010). High A-DMC scores are associated with more effective decision-making styles as well as better decision outcomes as measured by the Decision Outcomes Inventory (DOI) (Reference Bruine de Bruin, Parker and FischhoffBruine de Bruin et al., 2007).
Another individual difference variable with some initial promise in predicting decision outcomes is numeracy, which has been shown to affect decision variables from susceptibility to framing (Reference Peters, Västfjäll, Slovic, Mertz, Mazzocco and DickertPeters et al., 2006) to understanding health risks (e.g., Reference Black, Nease and TostesonBlack et al., 1995; Reference Schwartz, Woloshin, Black and WelchSchwartz et al., 1997; although see Reference Reyna, Nelson, Han and DieckmannReyna et al., 2009, for a discussion of some shortcomings of recent research on numeracy and decision making). Such evidence from A-DMC and numeracy studies helps explain individual differences in certain decision tasks and points to ways in which people could be trained or decision presentations simplified to boost decision competence (for example, see Reference Peters, Dieckmann, Dixon, Hibbard and MertzPeters et al., 2007, for differences in how high- and low-numerate people respond to different information displays in a hospital quality judgment task).
Going a step beyond general domain relevance, within a domain such as decision making, different dependent variables (e.g., decision outcomes, decision experience, and judgments) are explained by different underlying theories. Thus, different individual differences can be expected to drive effects for outcomes versus experience versus judgments. It is therefore important to select individual differences that are theoretically relevant to the dependent variable of interest specifically as well as the domain broadly. Once again, the DMIDI can be of service. By sharing information about what does and does not work and by fostering discussion about what should and should not be expected to work, the JDM community can create and continually refine sets of relevant individual difference measures. To whit, Reference Reyna, Nelson, Han and DieckmannReyna and colleagues (2009) point out that the decision competence construct of numeracy would benefit from a more refined definition and improved measure(s) to better account for its relationship with decision making outcomes.
3.3 Individual differences in interaction with other factors
The effects of individual difference measures are often contextual; measures are significant for one decision-making phenomenon and not for another. Thus, like others before us, we advocate a person-by-decision-and/or-situation interaction approach that examines how individual differences interact with other individual differences, with decision features, and with situational factors to influence behavior in a given context. The person × situation approach has been fruitfully applied to other domains and its use has been previously advocated in consumer research (Kassarjian & Sheffet, 1991), in psychology generally (e.g., Reference CronbachCronbach, 1957, 1975; Reference LewinLewin, 1943; Reference Magnusson and EndlerMagnusson & Endler, 1977; Reference MischelMischel, 1968, Reference Mischel2004; Reference Ross and NisbettRoss & Nisbett, 1991), and in decision research specifically (Reference Blais and WeberBlais & Weber, 2006; Reference Mohammed and SchwallMohammed & Schwall, 2009). Reference MischelMischel (2004) proposed that consistency arises across time within certain types of situations, which suggests that the interactions, rather than the direct effects, may be stable. In other words, a certain individual difference in the context of certain task features will have a reliable effect on decision-making behavior, but the effect depends on both the individual difference and the task features. Where possible, studies should be designed and analyzed with such interactions in mind.
Of course, one obstacle to investigating interactions with between-subject designs is the larger sample sizes required for adequate statistical power. Fortunately, within-subject designs offer a way to explore interactions with small sample sizes by looking for effects within individuals (see Reference BaronBaron, 2010, for a discussion of appropriate methods for such analyses). Researchers can also explore methodologies used to identify interactions in other domains, such as clinical psychology where it is common practice to look for individual differences in the difference between performance on two tasks (e.g., a control task and an experimental task) (see Reference Baron and TreimanBaron & Treiman, 1980, for a discussion of how to overcome some of the difficulties inherent in this approach). We believe that, regardless of how they are pursued, interactions will be a real contribution of individual differences to JDM research. This argues further for standardization and the use of a repository like the DMIDI: If studies use the same measures and results are accessible, JDM can build cumulatively toward an understanding of individual differences in interaction with one another, with task features, and with situational factors.
3.4 More extensive communication of results
A final piece of the puzzle is the importance of reporting all results, whether significant or not. We believe that individual differences research currently suffers from a “file drawer problem” (Reference RosenthalRosenthal, 1979, p. 638), meaning that reported results are only a fraction of the actual results (Reference Bradley and GuptaBradley & Gupta, 1997; Reference Howard, Lau, Maxwell, Venter, Lundy and SweenyHoward et al., 2009; Reference PautassoPautasso, 2010; Reference RosenthalRosenthal, 1979). Although this criticism can be justly applied to many fields, it may be particularly glaring for individual differences: For various reasons, researchers frequently employ a wide range of individual difference measures in a study, but report only those that are significant. At the same time, journals, with reason, are often reluctant to publish non-significant results. The result is that studies that find no significant relationships often do not get published. Studies that are well-designed and have adequate statistical power but nonetheless find non-significant results are not only worth reporting, they are a necessary part of a complete picture of individual differences.
The DMIDI can help fill this important gap. Reporting non-significant results online will help alleviate the file drawer problem and also reveal the real state of individual differences in JDM research. As a repository of information on the uses of individual difference measures and their effects on JDM variables, the DMIDI will centralize results and increase their accessibility. This will allow the JDM community to more easily assess the state of various measures in JDM research and to continually evaluate the utility of their pursuit. Thus, researchers are encouraged to share results (whether significant or non-significant, published or unpublished) as well as relevant reviews or meta-analyses for inclusion in the DMIDI.
4 Conclusion
Individual differences have long been a topic of interest in psychology generally as well as in JDM specifically, as evidenced by the wealth of individual difference measures commonly used. We suggest that this persistent interest would be better served by a change in approach—namely, a more systematic investigation that is more extensively communicated and that emphasizes both the theoretical selection of measures and the interactions between individual differences and task features, situational factors, and other individual differences. We believe that, by following these suggested prescriptions, we can better our understanding of individual differences in JDM. It is our hope that this overview and the DMIDI can serve together as first steps toward a more fruitful future for individual differences in judgment and decision-making research.
Open access
