3.1 Response times

Response time (RT), also often called “reaction time”, is the time that it takes an individual to make their choice after being presented with the alternatives. The study of RT has a long history in the behavioral sciences (Luce Reference Luce1986). While RT is a seemingly simple unidimensional measure, its interpretation is far from straightforward. On one hand, the well-known speed–accuracy trade-off implies that an increase in deliberation time will yield more accurate and consistent decisions. Therefore, long RT has often been associated with deliberate, calculated decision-making, while short RT has often been associated with intuitive and error-prone decision-making. On the other hand, the difficulty in discriminating between the values of different choice options, i.e., the strength of preference, also has a substantial impact on RT. The weaker the preference or the more complex the alternatives, the longer the RT. Thus, long RT is also often associated with inaccurate, unpredictable decision-making.

What distinguishes one regime from the other is still not well understood, but the literature suggests that problems that require multiple steps of thinking or the use of logic/algorithms tend to benefit most from additional time, and thus demonstrate a positive relationship between RT and accuracy (Rubinstein Reference Rubinstein2016). Meanwhile, people seem not to differ much in their ability to solve simple preference elicitation problems, and so in these settings, time is less a measure of individual effort and more a measure of the proximity to indifference between the alternatives available in that particular choice problem. Thus, in such settings, we tend to observe a negative relationship between RT and accuracy (Alós-Ferrer et al. Reference Alós-Ferrer, Granić, Kern and Wagner2016; Chabris et al. Reference Chabris, Morris, Taubinsky, Laibson and Schuldt2009; Echenique and Saito Reference Echenique and Saito2017; Konovalov and Krajbich Reference Konovalov and Krajbich2016; Moffatt Reference Moffatt2005; Mosteller and Nogee Reference Mosteller and Nogee1951).

Guiding much of the recent research on RT in economic choice is the literature on sequential sampling models (SSM), such as the drift–diffusion model (Clithero Reference Clithero2018; Dickhaut et al. Reference Dickhaut, Rustichini and Smith2009; Fehr and Rangel Reference Fehr and Rangel2011; Fudenberg et al. Reference Fudenberg, Strack and Strzalecki2018; Krajbich et al. Reference Krajbich, Oud and Fehr2014; Webb Reference Webb2018; Woodford Reference Woodford2014). These models conceptualize the decision process as noisy information gathering, akin to a Bayesian framework. Decision-makers observe a sequence of noisy signals about how much they value each alternative, and simply decide when to stop and pick the winning option. These models have been employed in the study of cognition and perception for decades, and are increasingly being applied to decision-making behavior. They provide an elegant mathematical framework linking stochastic choice to RT that is based on statistically optimal procedures (Wald Reference Wald1945) and has much support from neuroscience (Shadlen and Shohamy Reference Shadlen and Shohamy2016). Moreover, they can easily explain both positive and negative correlations between RT and accuracy, depending on whether variability is dominated by the incoming signals or by the evidence thresholds set by the decision-makers.

Several articles in this special issue discuss SSMs in detail, with connections to random utility theory (Smith et al. Reference Smith, Krajbich and Webb2019), risky decision-making (Leuker et al. Reference Leuker, Pachur, Hertwig and Pleskac2019), and distinguishing between effort and ability (Alekseev Reference Alekseev2019). Another article (Gawryluk and Krawczyk Reference Gawryluk and Krawczyk2019) investigates changes of mind over time. We discuss these articles in more detail below.

3.2 Eye movements

Eye-tracking technology records a decision-maker’s gaze direction as well as their pupil dilation. Gaze direction is generally used to measure an individual’s center of attention (i.e., what they are looking at). Attention patterns dictate how information is extracted from the environment. Research examining gaze direction tends to focus either on what information is or is not examined, how gaze transitions between pieces of information, or how long gaze lingers on pieces of information (dwell time).

There have recently been a number of papers examining eye movements in normal-form games (Devetag et al. Reference Devetag, Di Guida and Polonio2016; Knoepfle et al. Reference Knoepfle, Wang and Camerer2009; Polonio et al. Reference Polonio, Di Guida and Coricelli2014). Some papers in this literature have revealed that certain individuals simply do not look at the other players’ payoffs. Unsurprisingly, these individuals’ choices are inconsistent with Nash equilibrium. Other individuals do look at others’ payoffs, as well as their own, but exhibit gaze patterns that differ depending on whether they play cooperatively, competitively, or simply make a best response to Level 1 players. One paper in this special issue (Zonca et al. Reference Zonca, Coricelli and Polonio2019) contributes to this line of inquiry.

There have been recent efforts to incorporate attention into the SSM framework described above. This research argues that gaze amplifies the value of the attended information, biasing the choice process in that direction (Krajbich Reference Krajbich2019). This work has provided researchers with a precise mathematical tool for explaining the relationship between dwell time, value, RT, and choice. In this special issue, Smith et al. (Reference Smith, Krajbich and Webb2019) provide a simple method for estimating this effect of gaze on choice, using standard logistic regression. Another article examines similar SSM gaze effects in risky choice (Leuker et al. Reference Leuker, Pachur, Hertwig and Pleskac2019). A third article also examines dwell-time effects in risky choice, albeit within a different modeling framework (Harrison and Swarthout Reference Harrison and Swarthout2019).

Finally, pupil dilation has become a hot topic of late in other social sciences, although not yet in economics. Pupil dilation has been shown to reflect cognitive processing such as mental effort (Kahneman and Beatty Reference Kahneman and Beatty1966), arousal (Hochman and Yechiam Reference Hochman and Yechiam2011), deception (Wang et al. Reference Wang, Spezio and Camerer2010), learning rate (Nassar et al. Reference Nassar, Rumsey, Wilson, Parikh, Heasly and Gold2012), and breadth of focus (Eldar et al. Reference Eldar, Niv and Cohen2016). It has also been shown to closely track the variance of outcomes (Preuschoff et al. Reference Preuschoff, ’t Hart and Einhauser2011). Thus, while pupil dilation appears to reflect many things economists are interested in, we still lack a clear way to interpret it.

3.3 Mouselab/tracking

Mouselab is also a method to track subjects’ information collection. In general, pieces of information (e.g., payoffs) are concealed behind ‘boxes’, until the subject moves the mouse over to those boxes and clicks to open them (Camerer and Johnson Reference Camerer, Johnson, Carillo and Brocas2004; Reeck et al. Reference Reeck, Wall and Johnson2017). Mouse-tracking data are often discussed interchangeably with eye-tracking data, though some work has indicated that mouse-tracking can alter the way people search for information (Lohse and Johnson Reference Lohse and Johnson1996). One paper in this special issue compares eye-tracking and mouse-tracking data, finding that mouse-tracking does a better job predicting real purchasing decisions (Imai et al. Reference Imai, Kang and Camerer2019).

More recently, several studies have unobtrusively recorded subjects’ mouse movements as they make their selections, to study internal conflict during the choice process. These studies argue that a less direct mouse path from start to finish is indicative of changes of mind mid-decision, or at least, different factors entering into consideration along the way (Koop and Johnson Reference Koop and Johnson2011; Stillman et al. Reference Stillman, Shen and Ferguson2018; Sullivan et al. Reference Sullivan, Hutcherson, Harris and Rangel2015). A recent article also indicated that click position can reveal which attribute a subject was focused on at the time of choice, and thus likely considered more important (Chen and Fischbacher Reference Chen and Fischbacher2016).

3.4 Facial expressions

A large body of evidence indicates that emotional state influences economic decision-making, and that emotions react to economic outcomes. A number of models of the interplay between decisions and emotions have been influential in management and psychology, including the Affect Infusion Model (Forgas Reference Forgas1995), The Affect-as-Information model (Schwarz and Clore Reference Schwarz and Clore1983), and the Appraisal Tendency Framework (Lerner and Keltner Reference Lerner and Keltner2000, Reference Lerner and Keltner2001). The traditional method for gathering data to measure emotional state when testing these models, validating emotion induction protocols, or conducting exploratory research, has been to use questionnaires such as the Positive and Negative Affect Schedule (PANAS) survey (Watson et al. Reference Watson, Clark and Harkness1994).

In recent years, however, software has been developed that allows the emotional content of facial expressions to be measured from photos or videos. This has permitted the tracking of emotional states in nearly continuous time during an experimental session. Recent studies that have used Face Reading include the work of Breaban and Noussair (Reference Breaban and Noussair2018), who consider the relationship between emotional state and trading in financial markets, and that of van Leeuwen et al. (Reference van Leeuwen, Noussair, Offerman, Suetens, van Veelen and van de Ven2017), who study the connection between emotions and behavior in the ultimatum game. They investigate whether observers can beneficially use emotion information from photographs and videos to predict whether a responder in the game will accept or reject an offer that she has received.

The article in this issue by Doyle and Schindler (Reference Doyle and Schindler2019) is a substantial contribution to the methodology of emotion measurement. It introduces MuCap, a software package that allows the merger of the output from Noldus Facereader, the most commonly used tool to track emotional state in experimental economics, with output from Z-Tree (Fischbacher Reference Fischbacher2007), the most widely employed platform to conduct and acquire data from economic experiments. MuCap allows emotion and decision data to be linked with a very precise synchronization of timing.

3.5 Chat

Economists have long considered communication important, and many prominent theories incorporate communication as a central element of a game (Cho and Kreps Reference Cho and Kreps1987; Crawford and Sobel Reference Crawford and Sobel1982). Moreover, experimental evidence has indicated that allowing communication can substantially alter behavior (Cooper et al. Reference Cooper, DeJong, Forsythe and Ross1989, Reference Cooper, DeJong, Forsythe and Ross1992; Isaac and Walker Reference Isaac and Walker1988). Unlike psychologists, who long ago recognized the potential value of quantifying communication records (Guetzkow Reference Guetzkow1950), economists have been far slower to recognize that the content of communication, specifically free-form communication, can be a valuable source of process data, illuminating how decisions are made.

This is particularly true for group decisions, where the discussions between teammates are a rich source of process data. Communication is a natural part of making a group decision, and is unlikely to strike subjects as artificial or intrusive. In the process of trying to persuade their teammates to follow a course of action, team members often directly reveal the thought process underlying their preferred action. For example, Cooper and Kagel (Reference Cooper and Kagel2005) find that teams play surprisingly strategically, not only out-performing individuals but beating the demanding truth-wins benchmark proposed by Lorge and Solomon (Reference Lorge and Solomon1955). Analysis of the chat between teammates indicates that a key element in the strong performance of teams is learning to think about the game from others’ point of view. Burchardi and Penczynski (Reference Burchardi and Penczynski2014) provide another good example of using chat to gain insights into players’ thought processes. The large literature on the level-k model typically uses some form of structural estimation to infer the actions of naïve (level-0) players and the beliefs of more sophisticated individuals about said actions. Burchardi and Penczynski use chat data to directly examine these actions and beliefs. They find that the actions of level-0 individuals are not uniformly distributed and that level-1 types successfully anticipate this.

While the analysis of message content has become increasingly common in economics, its adoption is held back by methodological issues such as the best method of quantifying the content of chat (Houser and Xiao Reference Houser and Xiao2011; Penczynski Reference Penczynski2019), costs associated with the use of teams (using two person teams implies a doubling of costs), and concern that teams may make inherently different decisions than individuals. Advice can also serve as a useful source of process data that addresses the latter issue, as the process of advising somebody reveals what the advisor feels is important without input from another individual (Chaudhuri et al. Reference Chaudhuri, Schotter and Sopher2009; Cooper and Kagel Reference Cooper and Kagel2016). In this special issue, Capra (Reference Capra2019) addresses some of these concerns through the use of talk aloud protocols (TAP). She shows that TAPs do not affect the decision-making of individuals and establishes the value of TAPs as an easily obtained form of process data.

3.6 Other process measures

There are a number of other important methodologies that have contributed greatly to our understanding of decision processes. Measuring skin conductance response is a valuable way to track the level of stress or arousal that a participant exhibits over an experimental session. In this special issue, Ring and Schmidt (Reference Ring and Schmidt2019) use skin conductance to study gains vs. losses. Similarly, registering heart rate is also a valuable methodology for stress measurement. The use of brain activation data, typically measured with EEG or fMRI technologies, is also a very large and productive area. Due to space limitations, we do not discuss these other methodologies here, but see (Glimcher and Fehr Reference Glimcher and Fehr2013), for a thorough treatment of the field.

4.1 Papers with traditional models

Zonca et al. use eye-tracking data in an innovative fashion in their article “Does exposure to alternative decision rules change gaze patterns and behavioral strategies in games?” Previous studies have shown a relationship between classification in a level-k model and look-up patterns for payoffs. Zonca et al. show that these look-up patterns and, to a lesser extent, choices are malleable. Stage 1 of the experiment consists of subjects playing a series of matrix games without feedback. The second stage of the experiment has subjects to implement the decision rules of level-1, level-2, and cooperative types, respectively. For example, to implement level-1 play, subjects are told to maximize their payoff assuming their computerized opponent randomly chooses a strategy. Subjects’ Stage 1 decisions can then be classified as level-1, level-2, or cooperative based on comparing their look-up pattern in Stage 1 to those in Stage 2 when the induced type is known. Stage 3 repeats the games from Stage 1. The main result is striking; after exposure to the three decision rules in Stage 2, subjects classified as level-1 in Stage 1 shift towards level-2 in Stage 3. This contrasts with subjects classified as cooperative in Stage 1, who do not shift away from being cooperative in Stage 3. One interesting interpretation of these results is that subjects’ types in a level-k model are malleable rather than reflecting deep-seated cognitive limitations. Minimal training can induce a level-1 “type” to reason more deeply about games. Being a cooperative type is more stable, presumably reflecting preferences rather than bounded rationality. This paper provides interesting insights for the study of bounded rationality of games that would be difficult to obtain without using choice-process data.

The paper “Eye-tracking and economic theories of choice under risk” by Harrison and Swarthout uses eye-tracking to study expected utility (EUT) and rank-dependent utility (RDU) in choices between multi-outcome gambles. The authors estimate structural utility functions for each subject and find that they display a mix of EUT and RUD preferences in their choices (approximately 50–50). They then use regression analysis to study how dwell time on probabilities affects the estimates of each parameter (utility function curvature, probability-weighting parameters, temperature parameter, and the mix between RDU and EUT). They find that more dwell time on probabilities correlates with more use of EUT (relative to RDU) and more pessimistic beliefs within RDU.

One popular physiological variable to measure is Skin Conductance Response (SCR), which registers the level of arousal that an individual displays. The study reported in the paper “Skin conductance responses in anticipation of gains and losses”, by Ring and Schmidt, uses this type of process data to study loss aversion. Traditionally, loss aversion, a greater sensitivity to financial losses than to gains of equal magnitude, has been thought of in terms of decisions (for example, an individual would be indifferent between accepting or declining a lottery that would yield $20 or − $10 with equal probability, indicating that a loss is offset by a gain that is twice its magnitude). This paper asks whether the skin conductance responses of individuals, when a lottery is displayed to them, exhibit the same type of differential sensitivity to gains and to losses. The principal result from the study is that the level of arousal, as measured with SCR, is similar for gains and losses of similar magnitude. However, the level of arousal is increasing in the magnitude of the potential change in income, regardless of whether the change is positive or negative.

The study “Additional deliberation reduces pessimism: evidence from the double-response method”, by Gawryluk and Krawczyk, focuses on the time profile of the decision process. It is conventional for a participant in an experiment to be permitted a certain length of time to make a decision. Then, once the decision is made, it cannot be changed or recalled. In the procedure proposed and implemented in the study of Gawryluk and Krawczyk, called the double-response method, initial decisions may be changed at any time during a fixed time interval (here 60 s). However, any moment (any second) during that interval may be chosen as the time at which the decision currently in effect is binding for the determination of the individual’s earnings. Thus, there is an incentive to make some decision as quickly as possible, but also to revise it as one thinks more carefully about the decision problem and determines that improvement is possible. The method provides a clever way to distinguish between system 1 (spontaneous and rapid) and system 2 (deliberate and slow) thinking (Stanovich and West Reference Stanovich and West2000). The application in the paper is the elicitation of certainty equivalents of lotteries to measure the extent of probability weighting. The main result is that when the individual has an ability to reflect further and change her earlier decision, certainty equivalents increase on average, that is, are changed in the direction of lower pessimism.

4.2 Papers with process models

The paper “Using response times to measure ability on a cognitive task” by Alekseev uses a single-alternative SSM to estimate ability separately from effort. It focuses on performance in a task designed to measure cognitive ability. Ability on such tasks is often measured using simple performance measures like number of problems solved, and the author argues that these measures may confound ability with effort. He uses his model to explain subjects’ data on the task and then finds that the pre-existing method is biased, such that subjects are incorrectly ordered based on their ability.

“Do people exploit risk–reward structures to simplify information processing in risky choice?” by Leuker et al., investigates whether people exploit correlations between risks and rewards in their environment to simplify the decision-making process by inferring rather than inspecting some information. Cleverly, they embed the same decisions in otherwise correlated or uncorrelated environments and compare behavior only on those matched trials. They find that in correlated environments, subjects made faster and worse choices, as judged by expected value, and spent more time looking at payoffs rather than probabilities, compared to the uncorrelated environments. Using gaze-weighted SSMs, the authors go on to show that in the correlated environments, subjects are less cautious and more influenced by gaze in their decisions, compared to the uncorrelated environments.

The paper “Estimating the dynamic role of attention via random utility”, by Smith et al., focuses the role of attention, another important element of the decision process. An elegant manner of measuring attention in laboratory experiments is to track eye movements. The paper deals with the issue of how to incorporate this type of attention data into the estimation of models that assume a drift–diffusion process of decision-making. Under this type of model, an agent facing a decision does not know the payoff of each option. However, it is assumed that there is sequential sampling of information, and as new information arrives, it may increase or decrease the likelihood of making a certain decision. When a critical threshold of evidence in favor of a particular alternative is reached, a decision in favor of that alternative is taken. Currently, the parameters of the model are found by elaborate simulation and grid search methods. The paper in this volume shows that the parameters can simply be estimated using a random utility regression framework. The paper compares the two techniques regarding whether they are able to recover the underlying parameters for a computer-generated data set, and whether they estimate the same parameters for the actual data set from an experimental study. The authors conclude that using a random utility regression framework is the better empirical strategy.

4.3 Methodological advances

Imai et al. study hypothetical bias in their paper “When the Eyes Say Buy: Visual Fixations during Hypothetical Consumer Choice Improve Prediction of Actual Purchases”. In comparisons of hypothetical and real choices such as decisions to purchase products, hypothetical choices overstate the probability of positive actions (e.g., buying a product). Imai et al. explore the use of mouse-tracking and eye-tracking data to improve the prediction of real choices based on hypothetical choices. They find that use of mouse-tracking, but not eye-tracking, data improves the ability to predict real choices from hypothetical choices. Specifically, subjects who were slow (looked at prices longer and took longer to transition from looking at prices to making a decision) were also more likely to switch from a hypothetical choice to purchase a product to a real choice to not purchase. The magnitude of the effect is small, but Imai et al. show that process data aew a potentially valuable tool for improving the predictive power of hypothetical studies.

Capra’s paper “Understanding decision processes in guessing games: A protocol analysis approach” explores the use of think aloud protocols (TAP) as a method for gathering data about the thought processes that underlie experimental subjects’ decisions. Major concerns with TAP include the possibility that use of TAP will affect the distribution of choices or that TAP does not accurately reflect the actual thought processes being used. As a proof of concept, Capra gathers TAP within a guessing game (Nagel Reference Nagel1995). She finds that the distribution of choices with the TAP is virtually identical to the distribution in control treatments without any verbal protocol. Capra also shows that changes in content of the TAP predict changes in observed behavior across populations. This indicates that TAP captures important aspects of subjects’ underlying thought processes, rather than reflecting their desire to please the experimenter. TAP shows promise as a simple and inexpensive method for gaining insight about subjects’ thought processes.

An important influence on many decisions is the emotional state of the decision-maker. The paper “MuCap: Connecting FaceReader™ to Z-Tree”, by Doyle and Schindler, introduces a software package, MuCap, that greatly facilitates the analysis of the emotional state of participants in an experiment. The software allows the synchronization of timing between Z-Tree (Fischbacher Reference Fischbacher2007), the most commonly used software platform for implementing economic experiments, and Noldus Facereader™ software. Noldus Facereader tracks facial expressions in nearly real time (up to 30 frames per second) on a video recorded during an experiment with a webcam. It then analyzes the emotions that the face expresses in each frame. This allows the measurement of emotional reactions to stimuli, as well as the emotional correlates of specific decisions. Before the development of MuCap, researchers had difficulties in synchronizing the timing between software that recorded decisions and programs that registered facial expressions. They had to resort to various ad hoc manual techniques to link Facereader and Z-tree output, and to laboriously edit out useless footage. This article explains the way that MuCap operates and describes how to employ it in an experiment.