1 Introduction
Goldstein and Gigerenzer (Reference Goldstein, Gigerenzer, Gigerenzer and Todd1999) proposed the recognition heuristic as a mechanism to describe inferential judgments. Briefly, this heuristic asserts that an inference can be made merely on the basis of the presence or absence of information in memory: “if one of two objects is recognized and the other is not, then infer that the recognized object has the higher value” (Goldstein & Gigerenzer, Reference Goldstein and Gigerenzer2002, p. 76). The elegance of the recognition heuristic as a decision rule lies in its simplicity — not only does it purport that inferences can be achieved through a simple rule, but it is also appears to be an intuitive process model.
The development of the recognition heuristic by Goldstein and Gigerenzer (Reference Goldstein, Gigerenzer, Gigerenzer and Todd1999, Reference Goldstein and Gigerenzer2002; see also Gigerenzer & Goldstein, Reference Gigerenzer and Goldstein1996) has fueled over a decade of research within behavioral decision-making. While the recognition heuristic has been well received by many researchers (e.g., Borges, Goldstein, Ortmann, & Gigerenzer, Reference Borges, Goldstein, Ortmann, Gigerenzer, Gigerenzer and Todd1999; Hertwig, Herzog, Schooler, & Reimer, Reference Hertwig, Herzog, Schooler and Reimer2008; Pachur, Reference Pachur2010; Pachur & Hertwig, Reference Pachur and Hertwig2006; Pachur & Biele, Reference Pachur and Biele2007; Pachur, Mata & Schooler, Reference Pachur, Mata and Schooler2009; Serwe & Frings, Reference Serwe and Frings2006; Snook & Cullen, Reference Snook and Cullen2006; Volz et al., Reference Volz, Schooler, Schubotz, Raab, Gigerenzer and Cramon2006), others have been rather critical of its accuracy in describing the underlying psychological processes (e.g., Dougherty, Franco-Watkins, & Thomas, Reference Dougherty, Franco-Watkins and Thomas2008; Hilbig & Pohl, Reference Hilbig and Pohl2008; Bröder & Eichler, Reference Bröder and Eichler2006; B. R. Newell & Fernandez, Reference Newell and Fernandez2006; Oppenheimer, Reference Oppenheimer2003; Pohl, Reference Pohl2006; Richter & Späth, Reference Richter and Späth2006). These criticisms notwithstanding, the recognition heuristic has led to a focus on recognition-based inference and, much like other heuristics, such as representativeness, has managed a permanent place within the judgment and decision making vernacular (Tversky & Kahneman, Reference Tversky and Kahneman1973; Tversky & Kahneman, Reference Tversky and Kahneman1974). While the construct of representativeness proved transformational to judgment and decision making research, it has arguably run its course as a useful theoretical construct (Gigerenzer, 1996). The question, however, is whether the recognition heuristic will follow a similar path.
In this article, we present four closely related challenges for researchers investigating the recognition heuristic, as well as a call for a fundamental shift in research strategy. We argue that these challenges should be met and our call for a strategy shift be taken up if theoretical progress in research on the recognition heuristic is to move forward.
Before going into detail, a few comments are warranted. First, our four challenges and our call for a research strategy shift are not intended as criticisms, but rather as proposals for future research directions. Second, while we frame these challenges and our call for strategy shift with respect to the recognition heuristic, they may apply to related models of inference, including classics such as the availability heuristic. Importantly, it is likely that meeting the challenges will require extending and revising the recognition heuristic; whether this heuristic will then still be named recognition heuristic is immaterial to our points, because, as Hintzman (Reference Hintzman1990) observed, “the explanatory burden is carried by … the proposed mechanism …, not by what they are called” (p. 121).
Finally, some of our challenges are intended to consider the full import of memory processes for judgment and decision making. The literature on memory theory provides fertile ground for enriching and extending theories of judgment and decision making. While recent work in decision theory has emphasized the adaptive role that recognition may play in facilitating accurate judgment, errors and biases are commonplace amongst memory phenomena, including such effects as false-memories (Roediger & McDermot, Reference Roediger and McDermott1995), misinformation (Loftus, Reference Loftus2005), imagination inflation (Garry, Manning, Loftus, & Sherman, Reference Garry, Manning, Loftus and Sherman1996), and reality and source monitoring errors (M. Johnson, Hashtroudi, & Lindsay, Reference Johnson, Hashtroudi and Lindsay1993), amongst many others (see Koriat, Goldsmith, & Pansky, Reference Koriat, Goldsmith and Pansky2000). Inasmuch as memory processes are exploited for the purposes of judgment and decision making, then understanding the full range of effects associated with common usage of memory seems relevant for understanding judgment and decision making. In a similar vein, inasmuch as there are multiple strategies that may be deployed for guiding judgment and decision making—as is assumed by the framework in which the recognition heuristic has been developed—there is a need for developing cognitive architectures that both accommodate multiple strategies and model how people select amongst them. For example, it is not sufficient to postulate that some choices are made by relying on recognition memory; one must model as precisely as possible how the decision maker has come to utilize recognition, as opposed to some other process. Thus, the challenges posed below are embedded in a far greater challenge—a meta-challenge, of sorts—which is to build integrative models that respect the empirical and theoretical foundations of both memory and judgment and decision making.
2 Challenge 1: Define and model the basis of recognition as a function of memory
The recognition heuristic, as specified by Goldstein and Gigerenzer (Reference Goldstein, Gigerenzer, Gigerenzer and Todd1999, Reference Goldstein and Gigerenzer2002), operates on the output of memory, and therefore does not directly model the processes of recognition memory. On the one hand, modeling inferences as a function of the output of memory seems perfectly appropriate—much can be learned by starting where memory leaves off. However, we suggest that much richer theoretical insights can be made by integrating models of memory with models of choice, and that a full account of recognition-based inference as assumed by the recognition heuristic necessitates that we bridge the gap between the memory processes underlying recognition and the rules that operate on the output of memory.
Why is it necessary to understand the memorial basis of recognition in order to study recognition-based inference? Perhaps the most compelling reason for integrating memory theory with research on the recognition heuristic is that the underlying basis of the recognition heuristic is in fact memory. Thus, in order to understand inference behavior that is assumed to operate on the output of memory, it seems important to understand the processes that enable one to recognize elements within the choice set, that is, within the set of objects about which an inference is to be made.
The second reason is that the assumptions one makes about memory can have important consequences for understanding the factors that affect the recognition decision, and how likely it is that a person making inferences in line with the recognition heuristic will be accurate. Similar arguments have been made by Wixted (Reference Wixted2007) in comparing the unequal variance signal detection theory and dual-process models in accounting for recognition memory data. This point was also demonstrated by Schooler and Hertwig (Reference Schooler and Hertwig2005) in their implementation of the recognition heuristic in the ACT-R cognitive architecture (e.g., Anderson et al., Reference Anderson, Bothell, Byrne, Douglass, Lebiere and Qin2004) and in a signal detection model, and Pleskac (Reference Pleskac2007) in his application of signal detection theory to the recognition heuristic. As this work illustrates, depending on the assumptions one makes regarding the underlying memory processes, different predictions regarding inference behavior and the accuracy of a person’s inferences will be realized. For example, Schooler and Hertwig (Reference Schooler and Hertwig2005) showed that the forgetting of memory contents over time will systematically influence the accuracy of decisions that can be made with the recognition heuristic.
In short, the challenge for researchers moving forward is to better describe the memorial processes that underpin the recognition decision. In our view, if one were to accept the assumption that recognition-based inference is dependent on the particular assumptions made regarding the memorial processes, then specifying these memory processes must take precedence over research on the recognition heuristic itself. Thus, the study of recognition-based inference should take place within the context of well-specified models of memory. First steps in this direction, albeit without actually implementing well-specified memory models, have already been taken a couple of years ago. For instance, B. R. Newell and Fernandez (Reference Newell and Fernandez2006) experimentally examined whether people make inferences based on a binary or a continuous recognition trace, and Pachur and Hertwig (Reference Pachur and Hertwig2006) ran two experiments to investigate whether the recognition of an alternative is assessed faster than knowledge about the alternative being retrieved from memory. How these and other questions can be addressed by using a well-specified model of memory has recently been shown by Marewski and Schooler (Reference Marewski and Schooler2010). They use the ACT-R architecture to predict quantitatively the retrieval of knowledge beyond recognition, the binary recognition judgment, and the underlying continuous memory activation. Using a much different modeling architecture, Thomas, Dougherty, Sprenger, Harbison’s (Reference Thomas, Dougherty, Sprenger and Harbison2008) HyGene model is able to account for a host of phenomena within both the memory and judgment and decision making literature. Within this model, the same processes required for recognizing a colleague at a conference or a word in a memory experiment are used as the basis of inference and probability judgment. Finally, as of writing this article, Erdfelder, Küpper-Tetzel, and Mattern (Reference Erdfelder, Küpper-Tetzel and Mattern2011) studied the recognition heuristic from the perspective of a two-high-threshold model of recognition memory (Bredenkamp & Erdfelder, Reference Bredenkamp and Erdfelder1996; Snodgrass & Corwin, Reference Snodgrass and Corwin1988) that belongs to the class of multinomial processing tree models (Batchelder & Riefer, Reference Batchelder and Riefer1990; Erdfelder et al., Reference Erdfelder, Auer, Hilbig, Aßfalg, Moshagen and Nadarevic2009). The idea that memory theory is important beyond mere recognition-based inference motivates our second challenge.
3 Challenge 2: Study decision tasks beyond two-alternative forced choice
The recognition heuristic, as originally formulated by Goldstein and Gigerenzer (Reference Goldstein, Gigerenzer, Gigerenzer and Todd1999, Reference Goldstein and Gigerenzer2002), is a model for a situation in which a decision maker sees two names, one recognized and the other not, and has to decide which of the two options possesses a higher score on a given criterion. While people may face this type of task in some situations, people routinely utilize recognition processes in a variety of other contexts. We suspect that much can be learned about recognition-based inference by studying such tasks. Let us point to a couple of examples.
Multi-alternative inference. Recent work has begun to extend research on recognition-based inferences beyond the choice between two names (e.g., Frosch, Beaman, & McCloy, Reference Frosch, Beaman and McCloy2007). For example, in the marketing literature, many theories of choice assume a two-stage process: When we are evaluating multiple options, such as which of 20 jams to buy, or which food to choose in a restaurant, a smaller set of relevant options is formed first, then a choice is made after more detailed examinations of the options in this consideration set (Alba & Chattopadhyay, Reference Alba and Chattopadhyay1985; Hauser & Wernerfelt, Reference Hauser and Wernerfelt1990; Howard & Sheth, Reference Howard and Sheth1969). Marewski, Gaissmaier, Schooler, Goldstein, and Gigerenzer (Reference Marewski, Gaissmaier, Schooler, Goldstein and Gigerenzer2010) have proposed that the recognition heuristic allows a person to create such consideration sets, consisting of options with recognized names. In a second stage, recognized options can be ranked with heuristics that use alternatives’ attributes as cues—say, knowledge about a jam’s ingredients, or about a meal’s taste. Unrecognized options can be put aside. By ignoring the unheard-of and unrecognized, the recognition heuristic reduces the complexities of choosing among many options—much like Tversky’s (Reference Tversky1972) classic elimination by aspects model does on the basis of other probabilistically selected criteria. However, the process specified by Marewski, Gaissmaier, et al. applies only in those cases where the participant has been provided the k options, say when choosing among jams in supermarket shelves, or when choosing among meals in a restaurant’s menu. Thus, this process does not address the case where the options are generated from memory in the first place.
Option generation. In many real-world inference tasks, it is incumbent on the decision maker to generate the set of options to be considered. For example, before considering where to eat dinner, the decision maker must generate a set of options (e.g., Mexican, Japanese, or German food); prior to nominating a student for an academic award, the set of viable contenders must be defined. In such cases, the choice set is not laid out in front of the decision maker; instead the choice set must be defined internally; one must consult his or her memory to determine the set of viable dining options, or to identify particularly meritorious students. For making a choice among such options, the recognition heuristic would scarcely be applicable, since, by definition, all retrieved options will necessarily be recognized.Footnote 1
The determination of the choice set constitutes what Gettys and Fisher (Reference Gettys and Fisher1979) termed pre-decision processes—processes that take place prior to the implementation of a decision rule. According to Gettys and Fisher (Reference Gettys and Fisher1979), these processes are foundational to the decision process: Decision and judgment can take place only after the choice set has been defined. Moreover, inasmuch as the memory processes responsible for generating the choice set are prone to errors or biases, these errors in memory can easily cascade into errors or biases in judgment (Dougherty & Hunter, Reference Dougherty and Hunter2003; Dougherty & Sprenger, Reference Dougherty and Sprenger2006; Dougherty, Thomas, & Lange, Reference Dougherty, Thomas and Lange2010; Thomas, et al. Reference Thomas, Dougherty, Sprenger and Harbison2008).
The HyGene model of Thomas et al. (Reference Thomas, Dougherty, Sprenger and Harbison2008) attempts to bridge the gap between memory and judgment by implementing decision rules within the context of a well-specified model of memory (see also Dougherty, Gettys, & Ogden, Reference Dougherty, Gettys and Ogden1999; Juslin, & Persson, Reference Juslin and Persson2002; Reyna, Lloyd, & Brainerd, Reference Reyna, Lloyd and Brainerd2003; Schooler & Hertwig, Reference Schooler and Hertwig2005). Thus, HyGene models the process by which choice sets are generated, by explicitly linking the generation process to memory retrieval. Within HyGene options are recalled from memory using contextual or evidence-based information and maintained in a capacity limited set of leading contenders (e.g., the set of best options), which then are input into an decision rule that allows individuals to choose the best option. The decision rule in HyGene is based on memory activation, which is akin to a recognition signal in models of memory. In as much as the choice set is determined by the recall of options from memory, it will affect later decision making (see Dougherty, Thomas, & Lange, Reference Dougherty, Thomas and Lange2010). This directly leads us to our next challenge for the recognition heuristic.
4 Challenge 3: Describe how people choose to use recognition as opposed to other inference processes, where applicable
The thesis that people possess a repertoire of strategies to choose from has been formulated in many areas, including choice (Einhorn, Reference Einhorn1970; Fishburn, Reference Fishburn1980; Payne, Reference Payne1976; Payne, Bettman, & Johnson, Reference Payne, Bettman and Johnson1988, Reference Payne, Bettman and Johnson1993; Rapoport & Wallsten, Reference Rapoport and Wallsten1972), social interactions (Erev & Roth, Reference Erev, Roth, Gigerenzer and Selten2001; Stahl, Reference Stahl1999), mathematical skill (Siegler, Reference Siegler1988), word recognition (Eisenberg & Becker, Reference Eisenberg and Becker1982), and question answering (Reder, Reference Reder1987), to name a few. In a similar manner, the recognition heuristic is realized as one of a repertoire of heuristics—an adaptive toolbox—with each heuristic being adapted to different decision contexts or ecologies. Assuming a set of heuristics to describe behavior may well be useful for understanding the complexities of human decision making. Yet, for such a multiple-strategy approach to drive long-term theoretical advances, work must be done to integrate the heuristics within a common theoretical framework that allows one to predict when one heuristic is chosen over another (e.g., Dougherty et al., Reference Dougherty, Franco-Watkins and Thomas2008; Gigerenzer, Hoffrage, & Goldstein, Reference Gigerenzer, Hoffrage and Goldstein2008; Glöckner, Betsch, & Schindler, Reference Glöckner, Betsch and Schindler2010; Marewski, Reference Marewski2010; Marewski, Schooler, Gigerenzer, Reference Marewski, Schooler and Gigerenzer2010).
In fact, much work has tackled the question of how people choose between the different processes, decision strategies, operators, routines, and production rules available to them (e.g., Lovett & Anderson, Reference Lovett and Anderson1996, Beach & Mitchell, Reference Beach and Mitchell1978; Busemeyer & Myung, Reference Busemyer and Myung1992; Rieskamp & Otto, Reference Rieskamp and Otto2006, Payne, Bettman, & E. J. Johnson, Reference Payne, Bettman and Johnson1988, Reference Payne, Bettman and Johnson1993), and some of this work has also addressed the issue of strategy selection for the recognition heuristic (e.g., Pachur & Hertwig, Reference Pachur and Hertwig2006; Marewski, Gaissmaier, et al., Reference Marewski, Gaissmaier, Schooler, Goldstein, Gigerenzer, Taatgen and van Rijn2009, Reference Marewski, Gaissmaier, Schooler, Goldstein and Gigerenzer2010; Volz et al., Reference Volz, Schooler, Schubotz, Raab, Gigerenzer and Cramon2006; Hertwig et al., Reference Hertwig, Herzog, Schooler and Reimer2008). However, except for Marewski and Schooler (Reference Marewski, Schooler and Gigerenzer2010), strategy selection for the recognition heuristic has not been cast into a detailed architectural and quantitative model. Hence, the challenge here is to develop modeling approaches that allow one to capture behavior both at the heuristic level, and at the strategy selection level.
Various theoretical pathways can be taken to tackle this modeling challenge, with various perspectives disagreeing on which path is most productive. Dougherty and colleagues (Dougherty et al., Reference Dougherty, Franco-Watkins and Thomas2008; Thomas et al., Reference Thomas, Dougherty, Sprenger and Harbison2008; Dougherty et al., Reference Dougherty, Thomas and Lange2010) have argued that judgment and decision making is best modeled within the context of a highly constrained cognitive architecture. In their view, theoretical advances require that the multiple-strategy approach provide a unifying theory that describes how individual strategies are selected, and which places constraints on the growth of the proposed repertoire of heuristics. In their view, without such constraints, the repertoire of heuristics runs the risk of continuing to grow, with a new heuristic identified for each unique context, environment, or task in which a decision is made.
An example of a constrained system is exemplified by HyGene. In this model, the requirement that the model retain its ability to capture and model memory phenomena in addition to judgment phenomena imposes tight constraints on making modifications to accommodate new experimental findings. Within the HyGene model judgment and decision making is assumed to be based on (a) recall of options from long-term memory, (b) maintenance of these options in working memory, and (c) a small-set of rules that guide choice, enable the formulation of probability judgments, and inform information search (e.g., hypothesis testing). The recognition heuristic, and other recognition-based decision mechanisms are special cases of more general memory retrieval processes (Dougherty et al., Reference Dougherty, Gettys and Ogden1999), which also serve as input into algorithms for computing subjective probability and selecting information for hypothesis testing. By assuming a set of algorithms for probability judgment and hypothesis testing, HyGene technically qualifies as a multiple-strategy model; however, the operation of these rules is constrained by the retrieval processes that precede their implementation (see Dougherty et al., Reference Dougherty, Thomas and Lange2010). Moreover, the algorithms are conceived of as fairly flexible and general, that is, applicable across different decision problems.
An alternative theoretical framework for modeling strategy selection is the ACT-R cognitive architecture, which has been used to implement the recognition heuristic, the fluency heuristic, and other decision strategies within a unifying quantitative theory (e.g., Schooler & Hertwig, Reference Schooler and Hertwig2005; Marewski & Schooler, Reference Marewski and Schooler2010). In ACT-R, decision strategies can be realized as sets of production rules (i.e., if-then rules) that operate on motor, perceptual, memory, intentional, and other processes, which in turn are implemented as a set of modules and information-processing bottlenecks (called buffers). Being perhaps the most detailed quantitative theory of cognition developed to date, ACT-R—much like HyGene—imposes strong constraints on accommodating new empirical findings.
However, in contrast to Dougherty et al. (Reference Dougherty, Thomas and Lange2010) who have argued for a small set of general cognitive processes, the heuristics that have been implemented in ACT-R are typically considered to form part of a larger repertoire. According to this view, (e.g., Gigerenzer, Todd, & the ABC Research Group , Reference Gigerenzer and Todd1999; Marewski, Gaissmaier, & Gigerenzer, Reference Marewski, Gaissmaier and Gigerenzer2010-a, Reference Marewski, Gaissmaier and Gigerenzer2010-b) human judgment can be error-prone; yet, across many tasks the ability to make accurate and fast inferences is fundamental to adaptive behavior. As a number of computer simulation studies, mathematical analyses, and experiments shows, this goal may be better served by a repertoire of specialized heuristics than by more general, less specialized tools (e.g., Brighton, Reference Brighton, Lebiere and Wray2006; Czerlinski, Gigerenzer, Goldstein, Reference Czerlinski, Gigerenzer, Goldstein, Gigerenzer and Todd1999; Gigerenzer & Brighton, Reference Gigerenzer and Brighton2009; Gigerenzer & Goldstein, Reference Gigerenzer and Goldstein1996).
Finally, a very different position than those embraced by Dougherty, Gigerenzer, Marewski, and others, is taken by Glöckner et al. (Reference Glöckner, Betsch and Schindler2010), who suggest replacing the notion of multiple decision strategies with an all-purpose mechanism that applies to all tasks, using just a single decision rule. Metaphorically, the distinction between the various theoretical views can be stated thusly: do people have a collection of specialized tools in their mental toolbox (e.g., a wrench, a hammer, screwdriver, etc.), or do they have a small number of general purpose tools—or possibly even just one—akin to an adjustable wrench (B. R. Newell , Reference Newell2005)? As a matter of theory development, the explanatory power of theories arguably rests, at least in part, on its generality. Within the multiple-strategy approaches, such as those advocated by Gigerenzer, Marewski, and others, this generality is captured by both the architectural theory of strategy selection and the specific tools—regardless of the number of tools. Within the single-strategy approach taken by Glöckner and colleagues, the generality is captured by the individual tool. The approach taken by Dougherty and colleagues is a compromise between these two positions, wherein the decision maker possesses a limited number of general purpose tools.
While it is easy to point out the distinction between the various multiple and single-strategy approaches outlined above, they are not necessarily at odds with one another. For example, specific heuristic mechanisms (as postulated by Gigerenzer and colleagues, for example) might be best conceptualized as being embedded within the context of a small set of more general processes (as postulated by Dougherty and colleagues), which oversees resource allocation, strategy selection, and option generation (amongst other things). Such a hierarchical system would be analogous to contemporary models of working memory, which assume that the working memory system consists of a collection of more specific processes (set shifting, updating, inhibition, maintenance, search, etc.; Baddeley, Reference Baddeley2003; Friedman & Miyake, Reference Friedman and Miyake2004; Unsworth & Engle, Reference Unsworth and Engle2007).
Contemporary cognitive architectures provide corresponding hierarchical systems. For instance, as mentioned above, ACT-R consists of a smaller number of modules that are coordinated through a production system. As well, HyGene is conceived of as a cognitive architecture consisting of a small-set of general processes. In principle, both of these architectures enable one to model the processes underlying heuristic mechanisms, including the recognition heuristic, within a single, overarching quantitative theory. Within the context of these hierarchical models, the distinction between the adjustable-wrench and tool-kit metaphors is largely dissolved, as they are merely different levels of the same hierarchy.
In short, in developing models of strategy selection for the recognition heuristic and other mechanisms of decision-making, it may be useful to consider how individual strategies are related to more general information processing mechanisms. This would enable researchers to predict when the recognition processes will be employed and when other decision mechanisms will come into play. As we will argue next, such predictions are important to test the recognition heuristic.
5 Challenge 4: Development of multiple methods for examining recognition use in decision making: Towards tests of competing models
There are two potential reasons that a decision strategy does not predict behavior. One is that the strategy is not used because people choose not to use it in a particular situation. A second is that the decision strategy is generally not a good model of behavior. The question of when people select to use the recognition heuristic or any other mechanism of inference is thus directly related to the need for developing methods to evaluate how good the recognition heuristic is as a model of behavior (see Marewski, Schooler, et al., Reference Marewski, Schooler and Gigerenzer2010, for a discussion).
Past evaluations of the recognition heuristic’s descriptive adequacy have largely focused on one central assumption of this model, namely that further knowledge about alternatives’ attributes (i.e., about cues) is not integrated into inference. Rather, decisions are based solely on recognition. Different approaches have been taken to test this assumption. These can be roughly categorized in evaluations in absolute and evaluations in relative terms.
Past evaluations in absolute terms: Accordance rates. The most common approach entails reporting the proportion of inferences that are made consistent with the recognition decision; often referred to as accordance or adherence rate (Bröder & Eichler, Reference Bröder and Eichler2006; Goldstein & Gigerenzer, Reference Goldstein and Gigerenzer2002; Hilbig & Pohl, Reference Hilbig and Pohl2008; B. R. Newell & Fernandez, Reference Newell and Fernandez2006; B. R. Newell & Shanks, Reference Newell and Shanks2004; Oppenheimer, Reference Oppenheimer2003; Pachur, Bröder, & Marewski, Reference Pachur, Bröder and Marewski2008; Pohl, Reference Pohl2006). To illustrate this point, Richter and Späth (Reference Richter and Späth2006) ran a series of studies and—observing that a smaller proportion of inferences were consistent with the recognition heuristic when knowledge that contradicted recognition were available—concluded that there was no evidence for the decision processes assumed by the recognition heuristic. Similar absolute evaluations of the recognition heuristic based on such accordance rates have also been made by many others, including Goldstein and Gigerenzer (Reference Goldstein and Gigerenzer2002), who, however, came to opposite conclusions than Richter and Späth, suggesting that further knowledge about alternatives’ attributes is not integrated into the inference.
Past evaluations in absolute terms: Measurement tools. Besides the proportion of inferences consistent with the recognition heuristic, there are a number of other absolute measures available, including d´ and related indices based on signal detection theory (Pachur & Hertwig, Reference Pachur and Hertwig2006; Pachur et al., Reference Pachur, Mata and Schooler2009), the discrimination index (Hilbig & Pohl, Reference Hilbig and Pohl2008), and a multinomial processing tree model (Hilbig, Erdfelder, & Pohl, Reference Hilbig, Erdfelder and Pohl2010). To illustrate this point, the multinomial processing tree model attempts to separate out inferences based solely on recognition from those in which people may rely on other information in addition to recognition, thereby providing an indicator of what one may deem recognition heuristic use. Based on this indicator, Hilbig, et al. argued that the rate of recognition heuristic usage tends to be overestimated by the proportion of people making decisions consistent with the recognition decision.Footnote 2
Past evaluations in relative terms: Formal comparisons of competing models. Only a few studies (i.e., Marewski, Gaissmaier et al., Reference Marewski, Gaissmaier and Gigerenzer2009, Reference Marewski, Gaissmaier, Schooler, Goldstein and Gigerenzer2010; Pachur & Biele, Reference Pachur and Biele2007; see also Bröder & Glöckner, Reference Glöckner and Bröder2011, for decisions in a non-memory based paradigm) have evaluated the recognition heuristic in relative terms, which entails comparing the match of the recognition heuristics predictions and observed data to the match between other models’ predictions and observed data. As has been argued repeatedly in the model selection literature, formal model comparisons establish yardsticks for evaluating the descriptive adequacy of competing models, with the models being each other’s benchmarks in model evaluation (on some of the merits and complications of modeling, see Fum, Del Missier, & Stocco, Reference Fum, Del Missier and Stocco2007; Hintzman, Reference Hintzman, Hockley and Lewandowsky1991; Lewandowsky, Reference Lewandowsky1993; Marewski & Olsson, Reference Marewski and Olsson2009; Pitt, Myung, & Zhang, Reference Pitt, Myung and Zhang2002). When just one model is tested—and thus evaluated in absolute terms—a seemingly large discrepancy between the model’s predictions and the observed data might lead a researcher to reject that model. In contrast, with a comparison, the researcher may find that all models suffer, enabling her to find out which model suffers least. Moreover, with a comparison a researcher does not have to specify a threshold (or other criterion) in order to decide what counts as good or bad performance of a model, as has been done (implicitly or explicitly) in the absolute evaluations of the heuristic hitherto conducted.
In terms of theory testing, both absolute and relative model evaluations can be useful, but at different times in the evolution of a theory. Early on, absolute model testing may be useful for establishing, rejecting, or motivating the modification of a theory. However, once alternative theories have been developed, then (ideally) absolute model testing should give way to comparative model evaluation. Given developments on recognition-based inference over the past several years, the time is ripe to begin more vigorous comparative modelling evaluation, where various instantiations of the recognition heuristic and competing models are pitted against one another, rather than tested in isolation. This way, researchers will not only (a) learn how well the recognition heuristic really predicts behavior in comparison to alternative models, but also have (b) the chance of identifying a model that accounts for behavior better than the recognition heuristic, as well as (c) the chance of building a unifying model that can account for both, inference processes based on mere recognition, and inference processes based on knowledge, recognition, and/or other information. We also argue that such comparisons should use multiple approaches to model evaluation, perhaps not limited to those discussed above.
For example, another potentially useful methodology that has not yet been employed is state-trace analysis (Bamber, 1979). State trace analysis permits the researcher to test hypotheses about whether the underlying cognitive processes are best described by a single- versus multi-process model. This analysis would enable researchers to test whether a single process, such as recognition, best accounts for recognition-based inference or whether a dual process with recognition and knowledge best accounts for the data. For example, state-trace theory has been applied to meta-cognitive judgments, such as remember-know judgments (Dunn, Reference Dunn2008). For remember-know judgments participants are asked to judge whether they recognize a word from a previous testing because they explicitly remember seeing the word (i.e., they have a specific recollection of the specific item) or they know the word was presented because it is so familiar (i.e., absent recollection, one my have a feeling of knowing based on a high familiarity). Dunn (Reference Dunn2008) applied state-trace theory to these judgments and found that a single-process model of “strength of evidence” explained the remember-know task better than a dual-process model of recollection and familiarity.
Similarly, state-trace analysis may be applied to recognition-based inference to assess if one (recognition) or two (recognition and knowledge) processes best explain the data. While state-trace analysis does not allow one to assess the relative contribution of multiple processes (in this case recognition and knowledge), it may provide a useful framework for determining if the data are best represented by a single versus dual process model, which may be a useful step in the development of formal models of decisions from recognition and knowledge.
6 Call for a research strategy shift
Research on the recognition heuristic has steadily grown over the last 15 years. However, is the recognition heuristic, and more generally, recognition-based decision processes, really as simple as they may seem to be? In detailing the four challenges in this paper, our original goal was to provide a roadmap of sorts for researchers interested pursing work on the recognition heuristic. However, while writing this article, we came to the view that truly transformative work on the recognition heuristic, and other judgment and decision making phenomena for that matter, will require addressing a number of very complex problems. While the recognition heuristic is intuitively simple as an explanation, its implementation in any given context requires the coordination of a number of more complex underlying cognitive (and indeed neural) mechanisms. For example, as we have pointed out, inasmuch as the recognition heuristic is based on underlying memory processes, then models of the recognition heuristic ought to stay true the fundamental properties of memory. That is, any model of recognition-based inference should retain its ability to account for recognition memory phenomena.
So how should researchers proceed in addressing our challenges? There are two possibilities. On the one hand, researchers could proceed as they have over the past 15 years: by carving a specific research problem (i.e., challenge) and tackling it in isolation; by building and testing verbal hypotheses; and by theorizing in terms of simple dichotomies, such as whether the memory process underlying recognition judgments are continuous or binary (e.g., B. R. Newell & Fernandez, Reference Newell and Fernandez2006), whether recognition is used with our without combing it with additional knowledge (e.g., Richter & Späth, Reference Richter and Späth2006); or whether decision making is best described by repertoire of decision strategies or by a single all-purpose mechanism (e.g., B. R. Newell, Reference Newell2005).
Alternatively, researchers could strive to meet all of our challenges in concert, by developing models that transcend each challenge and connect them through a unified theoretical framework—a path advocated by the present authors and others (A. Newell, Reference Newell and Chase1973; Nikolić, Reference Nikolić2009). It is not sufficient to meet one, two, or even all four of the challenges, for a truly cumulative account of judgment and decision-making requires that all of the pieces to the puzzle be connected and firmly in place.
Open access
