1 Introduction
Many everyday choices involve options that pose a conflict between immediate, but small gains, and delayed, but larger or more beneficial gains. This conflict occurs on many time scales. For example, you might wonder whether to enjoy spending your money now or saving it for a pension. Or you might be seduced to take the hearty burger – which is immediately very tasty - instead of the light salad – which might be better for your figure in the long-term. In such intertemporal choices (for a review, see Frederick, Loewenstein & O’Donoghue, 2002), humans discount the offered gain by the delay of delivery. This delay discounting is well described by utility discounting models which assume that the greater the delay in delivery of a reward, the more the utility of a reward is discounted. Hence, these discounting models represent the subjective value of a reward as a function of its delay (see Doyle, 2013 for an overview). While these models offer a good description of the average outcome of the decision process – the final choice – they mostly leave open how the decision process itself unfolds in time. Dissecting this process, however, is necessary in order to fully understand the way decisions are made as well as the sources of failures and deviations from the average discounting model (Reference Lempert and PhelpsLempert & Phelps, 2016). To fill this gap, recent developments aim to uncover the process dynamics leading to a final decision in delay discounting (Reference Dai and BusemeyerDai & Busemeyer, 2014; Reference Rodriguez, Turner and McClureRodriguez, Turner & McClure, 2014).
Here, we will combine the static description of decisions offered by delay discounting models with the process dynamics as described by attractor models. Such attractor models have been used successfully to study perceptual decisions (see Reference Tuller, Case, Ding and KelsoTuller, Case, Ding & Kelso, 1994 for the model also sketched out here), and recently also higher level decisions, e.g., risky decisions (Reference van Rooij, Favela, Malone and Richardsonvan Rooij, Favela, Malone & Richardson, 2013) or the development of preferences (Reference O’Hora, Dale, Piiroinen and ConnollyO’Hora, Dale, Piiroinen & Connolly, 2013). Applying an attractor model to delay discounting decisions enriches the mere uncovering of discounting curves by adding a level of description for the processes leading to a final choice and, thus, allowing for intuitive, heuristic theorizing about this process and possible failures resulting from it. With its simple mathematics and metaphorical heuristic power, an attractor model offers a bridge between the simple yet static models of intertemporal decisions incorporated by discounting curves and other more dynamic yet also more complex models of decision processes, e.g., neural network models. In the remainder of this paper, we will first derive our attractor model of decision making in delay discounting. Based on this model, we will then derive qualitative predictions about the dynamics of the decision process within single choices as well as across sequences of choices, and about the resulting deviations in the final decision. Finally, we will test these predictions with a newly developed discounting paradigm in four behavioral experiments.
1.1 Deriving an attractor model of decision making in delay discounting
Discounting models are based on the common ground of a psychometric perspective (Reference Takahashi, Oono and RadfordTakahashi et al., 2008) that follows psychophysical reasoning (Reference FechnerFechner, 1860): Their common aim is to derive the best function that describes choice outcomes. Such a function provides a clear decision criterion determining an individual’s choice of the sooner-smaller (SS) option over the later-larger (LL) option and vice versa for a given combination of times and values. The function does this by specifying indifference points on a graph of subjective value as a function of delay, that is, points at which the LL option has the same subjective value as the SS option. A decision involving combinations of values and delays that lie far away from these indifference points should be relatively easy (i.e., induce low levels of decision conflict) because the two options are very different with regard to their subjective value. In contrast, a decision involving combinations of values and delays that lie close to indifference will be relatively difficult (i.e., induce more conflict).
In this regard, high-conflict decisions are comparable to perceptual decisions when observing ambiguous figures like the Necker Cube. The Necker Cube offers two equal perceptual perspectives – an upwards and a downwards one – between which an observer’s perception alternates continuously. Similar to the visual system choosing between the two possible perspectives on the ambiguous Necker Cube, an intertemporal decision can be interpreted as an ambiguous state where the cognitive system chooses one of two conflicting delayed options. In the perceptual decision making literature, attractor models have proven themselves as a useful tool to understand the processes underlying choice variability in the perception of ambiguous stimuli (Reference Hock, Schöner and GieseHock, Schöner & Giese, 2003; Reference Noest, Ee and WezelNoest, Ee, Nijs & Wezel, 2007). This is because attractor models provide a high-level, abstract description of the otherwise complex, non-linear dynamics lying at the core of the different classes of more complex models (Reference Onnis and SpiveyOnnis & Spivey, 2012) that have been used in the past to study ambiguous perception such interactive activation and competition networks (Reference Rumelhart and McClellandRumelhart & McClelland, 1986) or parallel constraint satisfaction networks (Reference FeldmanFeldman, 1981). Because attractor models provide a simple yet elegant mathematical description of the dynamics of a perceptual decision process, they are also well-suited to reason about and to model value-based decision processes. Value-based decisions have also been studied based on network models, e.g., parallel constraint satisfaction networks (Reference Glöckner and BetschGlöckner & Betsch, 2008; Reference Glöckner, Hilbig and JekelGlöckner, Hilbig & Jekel, 2014), suggesting that the attractor model could again provide a parsimonious high-level description of the underlying decision processes. In order to test this assumption for delay discounting decisions, in the following we will first apply such an attractor modelFootnote 1 (Reference Tuller, Case, Ding and KelsoTuller et al., 1994) to delay discounting from which we will then derive and test empirical hypotheses about observable behavior.
In such an attractor model, the two options are represented by two attractors, that is, two valleys in a potential- or energy-landscape (Figure 1). The deeper a point in such an energy landscape, the more stable is the respective potential state of the system and the higher is the certainty that the system makes its decision. In comparison to the stable end-states representing the two options, the starting state of the system is unstable and lies in the middle between the two valleys. In this indecisive state, no decision has been made by the system and both choices are still possible. To make this abstract description more concrete, we can apply this attractor-model to the activation of neural representations of the SS and the LL option. The two attractor-states reflect the exclusive activation of the SS or the LL representation, indicating complete certainty of the system that this option is the one to choose. The states in between these stable attractors, e.g., the start state, reflect varying amounts of concurrent activation in which the system has not settled into a decision yet (see the appendix for a formal implementation of such a neural system).
Figure 1: Sketch of the attractor model for decisions with two possible choices representing a sooner smaller (SS) and a later larger (LL) option. The potential-landscape defines a one-dimensional state-space of the system representing all potential states of decisiveness for/against a certain option. The depth of the potential wells (as shown in the five insets on the left) defines the stability/attractiveness of each of these states. The relative depth of the well is represented by the control parameter c. This parameter depends on the relative difference in subjective value (attractiveness) of the options for a subject and hence configures the system for each potential combination of SS and LL options (as indicated by the continuous variation of c on the right): An increase in attractiveness for the SS option (e.g., because the SS option’s value becomes higher while both options’ delays are held constant) results in a negative control parameter which, in turn, increases the depth of the attractor representing the SS option. In contrast, an increase in attractiveness for the LL option (e.g., because the LL option’s delay is reduced while both option’s values are held constant) results in a positive control parameter which, in turn, increases the depth for the attractor representing the LL option. Hence, the control parameter c is primarily dependent on the values and delays of the presented options, but also on a subject’s tendency to discount, as indicated in Figure 2. Within this potential landscape, the current system state (marked by a red dot) tends to move to the bottom of the potential wells and travels through all intermediate states on its way to a stable final choice. The deeper a potential well of an option (compared to the alternative), the more probable, direct, and quicker is the movement of the current system state into this potential well.
In an attractor-model, the depth of the attractors and hence the stability of its end-states varies with the properties of the system’s environment. In our case, the crucial environmental properties are the value and the delay of the two options which – in combination – determine each option’s relative attractivenessFootnote 2, that is, its subjective (discounted) value. The difference in relative attractiveness between the SS and LL option (which elegantly combines all choice-relevant environmental information about the options’ values and delays) is represented by the control parameter, which we will call c. In the example of neural representations for the SS and the LL option, c reflects the relative difference in the input to the neural representations. Accordingly, the special case where c = 0 (Figure 1 middle panel) represents a decision where both options receive an identical input and are thus equally attractive. In such high conflict situations, a neutral starting state would keep the system indifferent until slight differences in input (or random noise) tips the system to one side or the other, resulting in a more or less arbitrary decision. For c < 0, the SS option is more attractive, for example, because the SS option’s reward is almost as large as the LL option’s (Figure 1, left panel). In contrast, for c > 0, the LL option is more attractive, for example, because the LL option’s delay is almost as low as the SS option’s (Figure 1, right panel). Low conflict decision situations with large differences in relative attractiveness and values of c ≪ 0 and c ≫ 0 result in unambiguous decision situations with no decision conflict at all.
The connecting element between our attractor model and discounting curves is the attractor layout where c = 0 (Figure 2). At this point, both the attractors representing the SS and the LL option are equally deep and, hence, equally attractive – a state to which we refer as the indifference point (Figures 1 and 2). By determining the precise location of these indifference points for the different attractor landscapes resulting from changes in the environment (such as the time interval between the two options in an intertemporal decision, see Figure 2), we arrive at the discounting curve over all delays.
Figure 2: Illustrative delay discounting curve (in blue) and the respective underlying attractor layouts (grey-scaled with darker shades of grey representing deeper attractors) of subjects in a previous experiment by Reference Scherbaum, Dshemuchadse, Leiberg and GoschkeScherbaum, Dshemuchadse, Leiberg & Goschke (2013). Blue circles mark indifference points, that is, the subjective value of an immediate SS option that has the same subjective value as an LL option that is delayed by a given interval. (Only the time intervals 1, 4, and 7 are depicted here.) For a time interval of 4, for example, the indifference point indicates that an SS option has to yield a valueSS = 0.6 × valueLL in order to be (subjectively) equally attractive as the discounted LL option. The range of attractor configurations defined by the control parameter c (the grey scaled insets, see also Figure 1) is aligned so that the attractor layout for two equally attractive options (control parameter c = 0) is located at the indifference point. For combinations of options lying above the indifference point (e.g., interval 4 and valueSS = 0.8 × valueLL,) c is smaller than 0 and the SS option is more attractive than the LL option. For combinations of options below the indifference point (e.g., interval 4 and valueSS = 0.4 × valueLL,) c is larger than 0 and the SS option is less attractive than the LL option.
Importantly, indifference points vary between individuals which means that people differ with regard to how much they discount a given value due to the delay of its delivery. These individual differences are also reflected in the attractor model. The precise configuration of the attractor landscape for a given combination of options depends on a person’s individual level of discounting, because this level of discounting defines the subjective values of the options. Hence, the attractor model is aligned at each interval to a subject’s individual discounting curve and in that way the control parameter c integrates the properties of the environment (the options’ relative values) and individual differences (the subjective value of the options’ discounted values as determined by the discounting curve). The attractor model thus integrates the amount of discounting as indicated by discounting functions and the overall process description of a choice, as indicated by a given attractor configuration which defines how the system’s current state develops over time until a final choice is made.
The temporal extension that the attractor model offers also provides a means to study the temporal dynamics of intertemporal decisions on two time-scales. One, already mentioned, is an intra-trial time-scale that focuses on how a final choice comes about over the course of one decision. The other is the inter-trial time-scale that focuses on how consecutive decisions may influence one another. Classic discounting-curves are blind to both kinds of time-scales. The former extension that pertains to the intra-trial dynamics of decision making focuses on how the system state gradually moves into one of the two attractors over the course of a trial until it makes a final choice.
However, this more process-oriented perspective on intertemporal decision making is not a new one. For example, sequential sampling models (Reference Dai and BusemeyerDai & Busemeyer, 2014; Reference Rodriguez, Turner and McClureRodriguez et al., 2014) also describe how decisions come about by describing how information about the offered options accumulates gradually until one of them hits a threshold, thus, eliciting a decision. The major step forward offered by the attractor model over sequential sampling models is the extension to the inter-trial time-scale: While sequential sampling models focus on intra-trial dynamics exclusively, the attractor model makes strong predictions about the inter-trials dynamics occurring over the course of multiple decisions. It inherently assumes that the attractors are formed by the offered options and, hence, these attractors are not present between trials. This leads the system state to stay in the area where it ended up previously – in the vicinity of the vanished attractor representing the recent choice – and to relax only slowly to the neutral start state under no input. For example, if the model choses the SS option in a first decision trial, it would remain in the vicinity of the SS attractor in the inter trial interval. In a second decision trial, it would hence start the decision with a bias to the SS decision, even if this trial comprises of a more attractive LL option (Figure 3).
Figure 3: Inter-trial dynamics in the attractor model. Choosing SS in a first trial leads to a bias in a second trial due to slow relaxation of the system state during the inter trial interval..
In sum, the attractor model implies that the previous system state automatically carries over to the next trial, thus making the system’s behavior in a given trial inherently dependent on the history of its previous decisions (Reference Scherbaum, Dshemuchadse and KalisScherbaum, Dshemuchadse & Kalis, 2008; Reference Townsend and BusemeyerTownsend & Busemeyer, 1989).
In the following, we will use the attractor model to derive predictions about intra-trial and inter-trial dynamics in real decision-makers. The simplicity of the model allows us to derive these predictions in a purely qualitative, argumentative manner. However, Appendix I shows that the exact same hypotheses can be derived by means of computational simulation based on a competitive neural-network.
1.2 Predictions from the attractor model
The intra-trial process dynamics focus on the movement of the system state into the attractor of the final choice. Here, we derive two predictions about the decision process. The first prediction concerns decisions in which both options are equally attractive (i.e., high conflict decisions) compared to decisions in which one option is clearly more attractive than the other one (i.e., low conflict decisions): For high conflict decisions, the system should settle only slowly into one of the two equally deep attractors compared to a quicker movement into the deeper attractor for low conflict decisions.
The second prediction concerns only low conflict decisions: When the system chooses the more attractive option – the deeper attractor – it should settle into this attractor quickly, compared to choosing the less attractive option. Note that choosing the less attractive option is possible if the system already is in the vicinity of this option’s attractor. This could happen because of previous choices of this option leading the system to stay within the vicinity of an option’s attractor (compare Figure 3). As explained above, this is a typical phenomenon for inter-trial dynamics as explained in the following.
The inter-trial process dynamics focus on the stability of choice patterns across consecutive decisions, and hence, on staying within the vicinity of one attractor or switching from one attractor to the other one as induced by changes of attractiveness and stability. Here, we derive a third prediction for decisions with either one or two potentially attractive options. For no conflict decisions with only one attractive option we expect the system to always choose the same option, irrespective of previous decisions. In contrast, for high conflict decisions with two potentially attractive, but not necessarily equal options we expect the system’s choice to depend not only on the attractiveness of the currently presented options but also on contextual factors, that is, the system’s history of previous choices – a phenomenon that is also known as path-dependence or order-effects.
For example, a choice of the SS option in one trial should bias the starting state of the system for the subsequent trial which can result in a renewed choice of the SS option – even when its attractiveness has been slightly reduced (Reference Noest, Ee and WezelNoest et al., 2007). In its most extreme form this phenomenon is also known as hysteresis (Reference Tuller, Case, Ding and KelsoTuller et al., 1994): If a sequence of choices starts with a very attractive SS option, this option will be chosen initially; decreasing the attractiveness of the SS option systematically (decreasing c) will only then lead to a switch to the LL option when the SS option’s attractiveness approaches zero. This is because the system state stays in the vicinity of the initially chosen SS option and leaves this attractor only when it has vanished completely (Figure 4, blue arrows). In contrast, if a sequence of choices starts with an attractive LL option and then the attractiveness of the LL option is systematically decreased (increasing c), the switch to the SS option happens only after the LL option’s attractiveness has reached (almost) zero, because the system stays in the vicinity of the attractor of the LL option until it vanishes completely (Figure 4, red arrows). Simply put, this means that the outcome of a given choice depends not only on the current properties of the options but also on the history of previous choices.
Figure 4: Changes in the attractor model as a function of the control parameter c. Brightness reflects the attractor layout as depicted in Figure 1 (black = deep attractor). Arrows mark the final choice of the system; arrows’ directions indicate the direction of change of c from two different starting states. If a sequence of choices starts with an attractive SS option (blue), the system will stay with the SS option despite changes in c making the SS option less and less attractive from trial to trial. Only when the SS option becomes very unattractive, the system switches to choose the LL option. If the sequence starts with an attractive LL option (red), the system stays with the LL option despite the changes in c. In the area near c = 0 (the indifference point with two equally stable attractors), the system has two stable states: In this range, whether the system chooses the SS or the LL option depends on the parameter’s history. Hence, the switch point between the two states depends on the state the system initially settled into. At c ≪ 0 and c ≫ 0, the weaker attractor completely loses stability and hence, only one stable state exists and the system reliably chooses only one option.
If we find such predicted behavior in real decision makers, it will stress why it is important to extend discounting functions with a compatible process account: While discounting functions readily describe the average outcome of multiple decision processes, they do not take the system’s choice history into account and, hence, cannot predict such a pattern (but see Reference Scholten and ReadScholten & Read, 2006, 2010). While an attractor delay discounting model parallels discounting functions in the way that it also defines a clear decision criterion, namely, through the layout of the attractor, it extends them through the idea that the system’s actual behavior is produced by the interaction of this attractor layout with the current state (and, hence, history and potentially other influences) of the system which can potentially lead to final choices that deviate from the discounting model. In the following, we will test the attractor model and the predictions for the process dynamics of intertemporal choice derived from it empirically. Note that, while we applied a metaphorical approach to derive our predictions in the previous paragraphs, Appendix I describes a formalized derivation of these predictions from a computational model.
1.3 A dynamic investigation of delay discounting
To validate the predictions of the attractor model, we needed a delay discounting setup that could provide us with markers for the expected phenomena. Concerning the predictions on the intra-trial process dynamics, a fruitful approach to study the temporal dynamics of decision processes behaviorally is mouse tracking (Reference Scherbaum, Dshemuchadse, Fischer and GoschkeScherbaum, Dshemuchadse, Fischer & Goschke, 2010; Reference Freeman, Ambady, Rule and JohnsonFreeman, Ambady, Rule & Johnson, 2008; Reference Dale, Kehoe and SpiveyDale, Kehoe & Spivey, 2007; Reference Kieslich and HilbigKieslich & Hilbig, 2014; Reference Koop and JohnsonKoop & Johnson, 2011). In this methodology, subjects make binary choices by moving a computer mouse into the left or right corner of the computer screen instead of pressing left or right buttons. Importantly, the movement trajectories recorded with this method not only provide the final outcomes but also capture features of the decision process. In particular, for several cognitive tasks (Reference Scherbaum, Dshemuchadse, Fischer and GoschkeScherbaum et al., 2010; Reference Song and NakayamaSong & Nakayama, 2009; Reference Spivey, Grosjean and KnoblichSpivey, Grosjean & Knoblich, 2005) as well as for a standard intertemporal choice task (Reference Dshemuchadse, Scherbaum and GoschkeDshemuchadse, Scherbaum & Goschke, 2012), strong competition between alternative responses produces less direct movement trajectories. Hence, regarding prediction 1 we expect more direct movements to the final choice in low conflict decisions and less direct movements to the final choice in high conflict decisions. Regarding prediction 2, we expect choices of the attractive option to be more direct than choices of the unattractive option.
Concerning the prediction of hysteresis for the inter-trial process dynamics, standard intertemporal choice tasks (for example, as used in Soman et al., 2005) pose a problem, since the options’ values and times are presented verbally and explicitly on a screen. This makes sequential manipulation of values or times simply too obvious (as indicated by the results of Reference Robles and VargasRobles & Vargas, 2008). Hence, for our investigation we use a recently developed, non-verbal delay discounting task (Reference Scherbaum, Dshemuchadse, Leiberg and GoschkeScherbaum et al., 2013). In this task, subjects collect coins of different sizes with an avatar which they move on a checkered playing field by clicking with the computer mouse (Figure 5). The playing field stayed constant across trials — except the options which changed from trial to trial — and the avatar started each trial from the position of the previously chosen option. The goal was to collect as much reward as possible in the allotted amount of time. In each trial of the task, subjects have to choose between two reward options of different magnitude (small vs. large) at different distances (near vs. far fields). So unlike standard intertemporal choice tasks, this paradigm allows us to observe intra-trial dynamics through mouse movements (Experiment 1) and investigate inter-trial effects of hysteresis by manipulating (without any notification) the distance between the two options for sequences of trials (Experiments 2–4).
Figure 5: The dynamic delay discounting paradigm. Subjects moved an avatar across a playing field by clicking with the mouse into horizontally and vertically adjacent movement fields (white border). They had to collect rewards (red border), with one reward being near but small (small coin) and one reward being far but large (large coin). The remaining time (“Zeit”) within one block and the collected credits (“Gewinn”) were presented next to the playing field. Zoomed Inset: Mouse movements were measured from the position of the avatar to the first click into a movement field. Movement deviations in the direction of the unchosen option were used as a measure of the attractor layout underlying the decision.
2 Experiment 1
Experiment 1 aims to validate the predictions for low vs. high conflict trials and for attractive vs. unattractive choices. By investigating these two predictions of intra-trial dynamics, it offers a fine-grained validation of the model. For Experiment 1, we report newly analyzed data from a study using the dynamic delay discounting paradigm described above (Reference Scherbaum, Dshemuchadse, Leiberg and GoschkeScherbaum et al., 2013).
2.1 Methods
2.1.1 Subjects
25 students (15 female, mean age = 23.04 years) of the Technische Universität Dresden took part in the experiment. All subjects had normal or corrected to normal vision. They gave informed consent to the study and received a 3 € show-up fee and the money they collected within the experiment.
2.1.2 Apparatus and stimuli
The experiment was presented on a 17-inch screen (1280 x 1024 pixels, 75 Hz). As presentation software, we used Psychophysics Toolbox 3 (Reference BrainardBrainard, 1997; Reference PelliPelli, 1997) in Matlab 2006b (the Mathworks Inc.), running on a Windows XP SP2 personal computer. Responses were carried out by moving a high precision computer mouse (Logitech Laser Mouse USB). Mouse movement trajectories were sampled with a frequency of 92 Hz. Recording started at presentation of the choice options and lasted until the first click into a movement field (see procedure).
Subjects’ moved an avatar on a playing field divided into 20 x 20 fields (Figure 5). To move the avatar, subjects clicked with the mouse in one of four horizontally or vertically adjacent movement fields, as signaled by a white border surrounding the fields. On each trial two reward options were presented as coins on fields marked with a red border: One reward was near but small, the other reward was far but large. The two options’ positions were always chosen so that the first move into one direction decreased the distance to one option but increased the distance to the other option. This way, the first move of the avatar already represented a clear preliminary decision for one option and against the other option.
For both options, the size of the coin represented the reward valueand the horizontal and vertical distance of the reward field to the field of the avatar represented the distance of the option. Reward values ranged from one to ten credits and distance ranged from two to fifteen fields. For better comprehensiveness in the context of intertemporal choice, we maintain in the following the standard description of the time dimension using “soon”, “late”, “delay”, and “interval”, although in our scenario time delay is represented by spatial distance.
Next to the playing field (Figure 5) subjects could see the remaining time within one block and the collected credits in Euro (1 credit = 1/10 € cent).
2.1.3 Procedure
Subjects’ task was to collect as much reward as possible within the allotted time limit. In each trial, they had to choose between two reward options (one soon but small, one late but large; see design). They collected the selected reward by moving their avatar with the mouse across the playing field. Moving over longer distances took more time, and thus was more costly with respect to the limited time.
A trial started with an inter trial interval (ITI) of 1.3 seconds. Within this interval, the mouse cursor was locked in the center of the field containing the avatar. After the ITI, the two options were presented. As soon as the two options appeared, subjects could click on the adjacent movement fields to move their avatar towards the chosen option. When the avatar reached one option, both options disappeared, the value of the selected option was shown to the subject, and the next trial started.
The experiment consisted of three blocks, with one block lasting eight minutes. This amount of time allowed subjects to work through the complete design matrix of trials (see design) at least one time. Between blocks, subjects were informed about the credits collected and were instructed to rest briefly before the self-paced start of the next block.
Before the start of the experimental blocks, subjects worked through a test block of one minute to get used to the virtual environment and handling of the mouse.
2.1.4 Design
Reward values ranged from one to ten credits, with the reward values of the two options in each trial adding up to eleven credits to keep the overall value of each trial constant (smaller/larger reward pairs were: 1/10, 2/9, 3/8, 4/7, 5/6). Distances ranged from one to fourteen fields with the nearer option being at a distance (D S) of 1, 3, or 7 fields and the additional interval to the farer option (D L - D S) being 1, 2, 4, or 7 fields. Reward values, smaller distance, and interval to the larger distance were varied orthogonally with a random order of trials. Importantly, the reward/distance combinations were chosen such that consistently choosing only one option, either the SS or the LL option, yielded worse results than a dynamic strategy that optimizes each choice by the current options’ value/time ratio.
The combination of 5 value combinations, 3 distance of the SS options and 4 intervals between the SS and the LL option yielded a complete sequence of 60 trials. We generated 8 such sequences, with a randomized order of trials within each sequence. This resulted in 480 trials that subjects could potentially complete in the whole experiment. From pretests, we knew that subjects would run into the time limit of the experiment (3 blocks, 8 minutes each) before completing all potential trials.
2.1.5 Data preprocessing
Choice categorization.
To distinguish low conflict decisions from high conflict decisions (prediction 1) we categorized trials as low vs. high conflict trials in three steps. First, we determined the indifference point for each interval, that is, the subjective (discounted) value of a delayed option that is equivalent to the value of a (hypothetical) immediate option. As an estimate of the indifference point, we determined the point of inflection of a logistic function fitted to the individual choices (SS vs. LL) as a function of increasing value differences (Reference Ballard and KnutsonBallard & Knutson, 2009; Reference Dshemuchadse, Scherbaum and GoschkeDshemuchadse et al., 2012). Second, we took these indifference points and calculated the distance in subjective value of each specific trial to this indifference point – that is, the difference between the indifference point and the small/large reward ratio of the trial. We then categorized all trials by performing a median split on their distance relative to the indifference point: trials with a low distance to a given subject’s indifference points were categorized as high conflict and trials with a large distance to a given subject’s indifference point were categorized as low conflict (Reference Robles and VargasRobles & Vargas, 2008; Reference Scherbaum, Dshemuchadse, Leiberg and GoschkeScherbaum et al., 2013). Hence, high conflict trials overall yielded a low distance to the indifference point (M= 0.14, SD = 0.01) whereas low conflict trials overall yielded a high distance to the indifference point (M = 0.47, SD = 0.02).
To distinguish choices of the attractive option from choices of the unattractive option (prediction 2), we calculated the ratio of costs (invested time) and benefits (received credits) for both alternatives in each trial, choosing the SS or the LL option (Reference Scherbaum, Dshemuchadse, Leiberg and GoschkeScherbaum et al., 2013). We inferred costs from the number of fields necessary to reach the option and from the movement-speed of the respective subject. If subjects chose the option with the better benefit/cost ratio, we defined that trial as attractive-chosen; if subjects chose the option with the worse benefit/cost ratio, we defined that trial as unattractive-chosen.Footnote 3
Mouse movements.
Subjects’ movements were recorded from the center of the avatar’s current field until the click into the respective movement field. For analysis we followed the procedure of the original study (Reference Scherbaum, Dshemuchadse, Leiberg and GoschkeScherbaum et al., 2013): Trajectories were time-normalized in length to 33 time slices per trial and aligned to common starting and end positions such that an ideal movement would be a straight line and less direct movements would show a deflection of the trajectory from this straight line into the direction of the alternative option. With regard to statistical testing, this allows to analyze the mean deflection – the area under the curve – in a trial as a marker for less direct movements (Reference Dshemuchadse, Scherbaum and GoschkeDshemuchadse et al., 2012; Reference Scherbaum, Dshemuchadse, Fischer and GoschkeScherbaum et al., 2010; Reference Spivey, Grosjean and KnoblichSpivey et al., 2005).
2.2 Results
On average, subjects completed 351 out of the possible 480 trials (SE = 7). Hence, subjects completed each of the 60 possible combinations of delays and values about 6 times. In the following, we will present the results of the reanalysis of existing data with a focus on the dynamic predictions derived from the attractor model. Analyses on choice behavior and discounting functions can be found in the report of the original analyses (Reference Scherbaum, Dshemuchadse, Leiberg and GoschkeScherbaum et al., 2013).
The first prediction derived from the model was that high conflict trials in which the two options are equally attractive should show more heavily deflected movements than low conflict trials in which one of the options is clearly more attractive. Accordingly, a one-sided t-test on mean deviation for low conflict vs. high conflict trials revealed the expected larger deviations for high conflict trials (M = 1.89 px, SE = 0.22 px) compared to low conflict trials (M = 1.43 px, SE = 0.19 px), t(24) = 2.05, p = 0.025, d = 0.41 see Figure 6, left).
Figure 6: Results of Experiment 1. Mouse movements from the starting position (center of the position of the avatar) to the position of the first click (in the first movement field). A straight line along the Y-axis would indicate a direct movement while deviations to the right indicate a deflection of the movement to the unchosen alternative option and hence conflict in the decision process. Shaded areas represent standard errors. Left: Movements for low vs. high conflict decisions. Right: Movements for low conflict decisions in which the attractive option was chosen or the unattractive option was chosen.
The second prediction derived from our attractor model of delay discounting was that, in low conflict trials in which one of the options is clearly more attractive than the other one, choosing the unattractive option should show more heavily deflected movements than choosing the attractive option. Accordingly, a one-sided t-test on mean deviation for low conflict attractive-chosen vs. unattractive-chosen trials revealed the expected larger deviations for unattractive-chosen trials (M = 2.75 px, SE = 0.45 px) compared to attractive-chosen trials (M = 1.43 px, SE = 0.19 px), t(24) = 3.27, p = 0.002, d = 0.65, see Figure 6, right).
2.3 Discussion
In Experiment 1, we reanalyzed data from a previous study (Reference Scherbaum, Dshemuchadse, Leiberg and GoschkeScherbaum et al., 2013) to validate our predictions from the attractor model of delay discounting. Low conflict choices showed more direct movements than high conflict choices and choices of the attractive option showed more direct movements than choices of the unattractive option. This indicates that for intra-trial processes, the attractor model correctly describes the process dynamics. The next two experiments will focus on the inter-trial dynamics and explore the prediction of hysteresis by systematically varying the relative attractiveness of the two options across trials, which would provide further evidence for the validity of the model.
3 Experiment 2
Experiment 1 corroborated the predictions of the model for intra-trial dynamics. This leaves untested the decisive extension to inter-trial dynamics, for which the model predicted hysteresis. Experiment 2, will test this prediction by slowly varying the interval between the SS and the LL option. If hysteresis was present, this should lead to subject’s choices showing stronger discounting behavior for sequences with a decreasing delay to the LL option in comparison to those with an ascending delay.
3.1 Methods
3.1.1 Subjects
20 students (11 female, mean age = 24.4 years) of the Technische Universität Dresden took part in the experiment. All subjects had normal or corrected to normal vision. They gave informed consent to the study and received a 3 € show-up fee and the money they collected within the experiment.
3.1.2 Apparatus, Stimuli and Procedure
The setup followed the setup of Experiment 1. Subjects’ task was again to collect as much reward as possible within the allotted amount of time. In each trial, they had to choose between two reward options
3.1.3 Design
To study choice sequences, we consecutively increased or decreased the delay of the LL option to the avatar while keeping all other factors constant within the sequence. Hence, we orthogonally varied the difference in delay between the SS and the LL option (D L – D S = 1 to 12 fields, manipulated sequentially), the direction of these sequences (direction = ascending or descending), and the delay to the SS option (D S = 2 or 3 fields). This resulted in four possible sequences (direction x D S) of 12 trials (D L). Orthogonally to these combinations, we varied the value V of the options, (VS = 1, 2, 3, 4, and 5 credits; VL = 10, 9, 8, 7, and 6 credits), following the original experiment (Reference Scherbaum, Dshemuchadse, Leiberg and GoschkeScherbaum et al., 2013). Hence, a set of trials consisting of all combinations included 5 (values) x 2 (SS delay) x 2 (sequence direction) x 12 (delay difference) = 240 trials. We created 2 such sets, with each set’s order of sequences being randomized individually, resulting in 480 trials. Similar to the original experiment, subjects did not finish all of these trials due to the time limit, but worked on each sequence of trials at least once.
3.2 Results
On average, subjects completed 379 out of 480 possible trials (79 %, SD= 11.4 %), with the SS option being chosen in 44.3 % (SD = 13.6 %) of these trials. We hence successfully acquired each complete combination of trials (240 trials) at least once in each subjects’ dataset. As the order of sequences was randomized across subjects, repeated trials from partially completed sequences were also included into the analyses.
We checked for the effects of the sequence manipulation by performing an ANOVA on discounting curves with the factors interval and direction (ascending/descending; see Figure 7, top right). As expected, we found a main effect of the factor interval (F(11,209) = 102.87, P < 0.001, η 2= 0.84), indicating discounting, a main effect for the factor direction (F(1,19) = 18.34, P < 0.001,η 2= 0.49), indicating hysteresis, and an interaction interval x direction (F(11,209) = 5.06, P < 0.001, η 2= 0.21), indicating different amounts of hysteresis for different intervals. In line with the model’s predictions, we found hysteresis in responses for intermediate intervals and intermediate value differences (Figure 7, left), with stronger hysteresis for shorter intervals. Importantly, our data showed that hysteresis is a general effect that is not only driven by a minority of extreme subjects: When checking individual subjects for hysteresis (Figure 7, bottom right) we found that 16 out of 20 subjects showed the hysteresis effect (i.e., a hyperbolic k-value that was lower for ascending sequences than for descending sequences), while only 4 subjects showed no or a slightly reversed effect.
Figure 7: Results of Experiment 2. Left: Median response patterns across intervals for different ratios of small and large values. Hysteresis is most prominent for intermediate intervals and value ratios. Top right: Grand average discounting curves indicating the subjective value of a delayed value as a function of interval/delay and order of presentation. Shaded areas (left) and error bars (right) indicate standard errors. Bottom right: Discounting as indicated by hyperbolic k-values of each subject for sequences with ascending and descending order of presentation.
3.3 Discussion
As predicted by the model, we found hysteresis in Experiment 2: When the delay between the two options increased from trial to trial, subjects stuck to their choice of the LL option. In contrast, when the delay between the two options decreased from trial to trial subjects stuck to their choice of the SS option. Hence, in for intermediate intervals, the choice depended not only on the current values and delays of the SS and the LL option but also on the history of previous choices. This lead to a shift of the discounting curve indicating less discounting for increasing intervals compared to stronger discounting for decreasing intervals. Concerning their choice behavior within a sequence, subjects perseverated in their choices: They tended to stick to the option chosen at the beginning of a sequence and had difficulties to release it even when its objective attractiveness decreased. This rather unintuitive finding validates the attractor model, but stands in stark contrast to psychometric models of delay discounting which assume that a system’s decision in a given moment is exclusively determined by the two options‘ values and their delay. The results of Experiment 2 imply that a systems’ choice history is a third factor contributing to the outcome of the decision process.
Experiment 2 also leaves a couple of questions unanswered. For example, while our ascending vs. descending sequence of presentation manipulation did indeed shift discounting curves, it did so only for the lower part of intervals. This result might indicate that higher intervals exhibit only a small area where two attractors are present at the same time. Moreover, one might argue that the found perseverative choice behavior may be related to a lack of precision in the presentation of the options’ values: Whereas the variation in values were presented as relatively small differences in the size of the coins, the variation in delays was represented by the readily observed distances of the options. To rule out such an alternative explanation for the found hysteresis as a perceptual effect (Reference KelsoKelso, 1997; Reference Tuller, Case, Ding and KelsoTuller et al., 1994) and to provide further evidence for its existence, we performed Experiment 3 where we used a different format of presenting the options’ values.
4 Experiment 3
Experiment 3 was identical to Experiment 2 except for the way in which the options’ values were presented: Instead of representing the value of each option implicitly by the size of a coin, the value was now indicated explicitly by a number printed on a coin with fixed size. We expected to replicate Experiment 2. Hence, we hypothesized that subjects would, again, show hysteresis, i.e., stronger discounting behavior for sequences with a decreasing delay to the LL option in comparison to sequences with an ascending delay.
4.1 Methods
4.1.1 Subjects
20 students (12 female, mean age = 25.0) of the Technische Universität Dresden took part in the experiment. All subjects had normal or corrected to normal vision. They gave informed consent to the study. As in Experiment 1 and two, they received a 3 € show-up fee and the money they collected within the experiment.
4.1.2 Setup and Design
The setup and design were the same as in Experiment 2, with one exception: Instead of representing the value of each option by the size of a coin, we used red numbers printed on a coin of constant size to explicitly show the reward value of the option (1 up to 10 credits).
4.2 Results
On average, subjects completed 372 out of 480 possible trials (77.4%, SD = 10%), with the SS option being chosen in 49.4% (SD = 12%) of these trials. Again, this meant that we were successful in acquiring at least one complete sequence of trials for each subject.
We checked for the effects of the sequence manipulation by performing a repeated measures ANOVA on discounting curves with the factors interval and direction(ascending/descending; see Figure 8, top right). As expected and similar to Experiment 2, we found a main effect of the factor interval (F(11,209) = 20.28, P < 0.001, η 2= 0.52), a main effect for the factor direction (F(1,19) = 163.01, P < 0.001, η 2= 0.9) and an interaction interval x direction (F(11,209) = 7.28, P < 0.001, η 2= 0.28). Checking individual subjects for hysteresis indicated that 18 out of 20 subjects showed a hysteresis effect (i.e., a higher hyperbolic k-value for ascending than for descending sequences), while only 2 subjects showed a slightly reversed effect (Figure 8, top right).
Figure 8: Results of Experiment 3. Left: Median response patterns across intervals for different ratios of small and large values. Hysteresis is most prominent for intermediate intervals and value ratios. Right top: Grand average discounting curves indicating the subjective value of a delayed value as a function of interval/delay and order of presentation. Shaded areas (left) and error bars (right) indicate standard errors. Right bottom: Discounting as indicated by hyperbolic k-values of each subject for ascending and descending order of presentation.
To check for differences between the graphical presentation of rewards as coins of different size (Experiment 2) and the presentation of rewards a numbers (Experiment 3), we compared Experiments 2 and 3 statistically by performing an ANOVA with the within-subject factors interval(1-4) and direction (ascending/descending) and the between-subject factor presentation(size of coin vs. numerical). As expected, we found a main effect of the factor interval (F(11,418) = 250.1, P < 0.001, η 2= 0.87), a main effect for the factor direction (F(1,38) = 37.61, P < 0.001, η 2= 0.5) and an interaction interval x direction (F(11,478) = 11.54, P < 0.001, η 2= 0.23). There was no main effect of the factor presentation (F(1,38) = 2.34, P = 0.14, η 2= 0.06) and no significant interaction with this factor (all P > 0.18).
4.3 Discussion
Experiment 3 replicated Experiment 2 by finding the expected hysteresis effect for intermediate intervals and value ratios. We found no relevant difference to Experiment 2, where setup and design were similar except for the way in which the values of the options were presented. The asymmetry in the shifting of discounting curves was also present in the second dataset, indicating that the highest intervals lead only to small areas in which two attractors are present at the same time.
While Experiments 2 and 3 both validated the inter-trial process dynamics predicted by our attractor model of delay discounting, they left one important question regarding one of its core assumptions unanswered. In the attractor model, the observed hysteresis effect results from the model’s persistence at an abstract level of integration that represents the options’ integrated attractiveness or their subjective utility. However, one might argue that the exact same dynamics could also result from persistence at earlier levels of the decision process. For example, more complex models of intertemporal choice (for example connectionist models of delay discounting as described in Reference Scherbaum, Dshemuchadse and GoschkeScherbaum, Dshemuchadse & Goschke, 2012) not only contain levels that represent the abstract attractiveness of the two options but also include subordinate levels that represent preceding steps of information processing, for example the perception and representation of the delay and value features of each option. From this point of view, it would be possible that the persisting choice behavior we observed in experiments two and three did not actually result from persistence at the superordinate level of the SS vs. LL choice. Instead the same behavioral pattern could also be caused by a persisting activation of the previously chosen option’s features at a lower level of processing (i.e., the representation of its value or delay). As this would prime the perception of the features of the following options towards the previously chosen ones, this mechanism would also lead to a preference for the more similar option and hence to persisting choice behavior. As our model explicitly predicts hysteresis to take place at the more abstract, choice-based level of representation, this alternative explanation needs to be excluded.
While we could exclude simple response priming, since options were rearranged in every trial, we conducted Experiment 4 to exclude feature priming at lower levels as an alternative explanation.
5 Experiment 4
Experiment 4 was designed to locate the level of the found trial-by-trial persistence in choice behavior: A feature priming explanation which attributes choice persistence to the mere priming of concrete values and delays at lower levels of representation; and the more abstract explanation of our attractor model which attributes choice persistence to a prevailing tendency to choose the SS or the LL option on an aggregate level representing the overall attractiveness of both options. To this end, we implemented pairs of trials into our paradigm that consisted of a prime trial and a target trial. Prime trials were designed such that they contained one clearly attractive option, e.g., the SS option. In the subsequent target trials, the features – value and distance - of the previously primed option (i.e. SS), were applied to the alternative option (i.e., LL). Hence, we transferred the features of one option (SS) in prime trials to the opposite option (i.e., LL) in the target trial. This feature repeating option (i.e., LL) was then paired with a new alternative option (i.e. SS; note that the opposite scheme was applied to trial pairs where the prime was favoring LL choices).
From a feature priming account, one could expect a repetitive choice of the option exhibiting the previously chosen features. Hence, in our paradigm feature priming predicts a switch of the chosen option from trial to trial, i.e., from SS to LL or from LL to SS, since features are transferred from one option in the prime trial to the opposite option in the target trial. In contrast, choice persistence at an integrated attractiveness level predicts that subjects stick to their choice of the previously chosen option – even though this option’s features would have changed. This would lead to a repetition of the chosen option, i.e., SS to SS or LL to LL. Importantly, the attractor model predicts that this effect would be present for the whole range of controlparameter c in which both options show some attractiveness – hence, two attractors exist. Hence we also slightly varied the relative attractiveness of the two options in the target trial.
An important prerequisite for this prime-target scheme was to precisely manipulate subjects’ choices. Hence, we first needed to estimate each subject’s overall level of discounting to later fit the presented options to these individual subjective values accordingly.
5.1 Methods
5.1.1 Subjects
33 students (26 female, mean age = 23.93) of the Technische Universität Dresden took part in the experiment. All subjects had normal or corrected to normal vision. They gave informed consent to the study. As in Experiment 2, they received a 3 € show-up fee and the money they collected within the experiment. After checking our manipulation in the prime trials, we had to exclude 6 subjects from subsequent analyses since they showed a reversed effect of the bias manipulation during prime trials and, hence, lacked a sufficient number of valid prime-target trial pairs for further analysis. Hence, we performed all subsequent analyses on 27 subjects (observed effects did not change qualitatively when the 6 additional subjects were included).
5.1.2 Setup and Design
The setup was similar to Experiment 3, with two modifications. First, the experiment now consisted of four blocks (8 minutes each) where the first block served to estimate the individual rate of discounting of each subject. The subsequent three blocks then implemented the paired trial scheme (i.e., combinations of prime and target trials) to test whether the repetition bias observed in experiments two and three was due to feature or to choice repetitions. Second, while each option’s reward value was again depicted explicitly through red numbers printed on coins of equal size, we now used a credit system ranging from 10 to 99 credits instead of 1 to 10 credits in experiments two and three. This allowed us to match the values presented in the prime trials as precisely as possible to the individual subjective values of each subject that were measured during the first block.
To estimate individual discounting in the first block, we independently varied the delay of the SS option (D S= 2 or 8 fields) and the difference in delay between the SS and the LL option (D L - D S = 3 or 6 fields). The value of the LL option (VL = 50 to 99 credits) was chosen randomly, the value of the SS option was then derived systematically by setting it to VS = 20, 50, 70, 80, 85, 89, 93, 97% of VL (Reference Dshemuchadse, Scherbaum and GoschkeDshemuchadse, Scherbaum & Goschke, 2012). This resulted in 32 different types of trials. Each trial could be repeated 6 times, leading to a pool of 192 trials in the first block of the experiment.
In the subsequent three blocks, we used pairs of prime and target trials where the features of the option chosen in a given prime trial (i.e., its’ value and relative distance) were transferred to the opposite option in the subsequent target trial.
With regard to options’ delays, in prime trials, we varied the delay of the SS option (D S= 2 or 8 fields) and the difference in delay between the SS and the LL option (D L – D S = 3 or 6 fields). In target trials, we varied the delay of the SS option (D S = 2, 5, or 8 fields) and the distance to the LL option (D L= 8 or 14 fields).
With regard to options’ values, we combined the individual discounting functions defining the subjective value SVas measured in block 1 with an additional bias SV bias = 0.15 or -0.15, in favor of the SS or the LL option. In prime trials, this yielded values of the SS option (10 < VS < 50) and the LL option (40 < V L < 99), with VS < V L. In target trials, the value of the favored option of the prime trial was then used as the value of the opposite option and the value of the remaining option was determined by SV biasto favor the SS (SV bias= 0.15) option, no option (SV bias= 0), or the LL option (SV bias= - 0.15). This manipulation in the target trial served to create both, high conflict and less conflict trials. Hence, it allows us to investigate that the priming effect/choice persistence not only affects indifferent/high conflict decisions, but also decisions with an expected choice outcome.
This procedure resulted in 48 different combinations of prime and target trials, with 10 possible repetitions leading to a pool of 480 trials (see Appendix II and Table A1 for more details).
5.2 Results
On average, subjects completed 408 out of 480 possible pairs of prime and target trials (85.05%, SD = 10.47%).
To check for the effects of priming on target trials, we excluded all prime-target trial pairs where subjects did not choose the option favored by the biasing manipulation. For the remaining trials (65%), we checked whether subjects stayed with the option chosen in the prime trial or whether they stayed with the features chosen in the prime trial and hence, switched options. To this end, we performed an ANOVA with the factors Bias prime (SS vs LL) and Bias target(SS, none, LL) on the amount of choices of the SS option in target trials (Figure 9). This yielded a significant effect of Bias prime (F(1,26) = 6.15, P < 0.05, η 2= 0.19) and of Bias target (F(2,52) = 40.94, P< 0.001, η 2= 0.61), but no significant interaction (F(2,52) = 1.72, P = 0.19, η 2= 0.06). After choosing SS in the prime trial, choosing the SS option in the target trial was more likely (M = 48.59%, SD = 3.11%) than after choosing the LL option in the prime trial (M= 37.60%, SD = 0.34%). This result indicates that the current choice was influenced by a persisting bias of the previous choice on the option level and by the current attractiveness of an option, and that feature priming did not influence subjects’ choices.
Figure 9: Results of Experiment 4. Mean probability of choosing the SS option as a function of priming in the previous trial and relative attractiveness/dominance in the current trial. Dominance showed the expectable main effect, while priming an option shows the effect as predicted by the attractor model. In primed SS trials the priming trial successfully led to increased choices of the soon option. In primed LL trials, the priming trial successfully led to increased choices of the late option. Error bars denote standard errors.
5.3 Discussion
Experiment 4 provides evidence that choice persistence can be attributed to persistence on the level of the representation of choice options – and not to a mere persistence on the level of the representation of option features. Instead, current choices tend seem to be drawn towards the option chosen previously (SS or LL), even if this option’s features (i.e., its value and distance) have switched to the opposite option (from SS to LL or vice versa). This finding supports the assumption that our attractor model targets the correct level of representation with attractors representing the SS and the LL choice at an abstract, integrative level of representation instead of in terms of their features at conceptually lower levels of representation.
6 General discussion
In this article, we aimed to integrate a process-oriented attractor model into the outcome-based, psychometric modelling of delay discounting decisions. Through this integration, we opened up a window on the process dynamics leading to a final choice and the dynamics across consecutive choices. While these dynamics only recently came into the focus of research, they nevertheless must be studied in order to better understand both consistent decision making as well as decision failures and deviations from the average discounting model (Reference Lempert and PhelpsLempert & Phelps, 2016). We presented the attractor model for delay discounting (Reference Scherbaum, Dshemuchadse and KalisScherbaum et al., 2008; see also Reference Svyantek, Deshon and SilerSvyantek, Deshon & Siler, 1991; Reference Townsend and BusemeyerTownsend & Busemeyer, 1989) that is based on models of perceptual decision making (Reference Hock, Schöner and GieseHock et al., 2003; Reference Noest, Ee and WezelNoest et al., 2007; Reference Tuller, Case, Ding and KelsoTuller et al., 1994). From this model, we derived qualitative predictions regarding both the intra-trial and the inter-trial process dynamics underlying choice behavior and validated these predictions in four experiments based on a delay discounting game (Reference Scherbaum, Dshemuchadse, Leiberg and GoschkeScherbaum et al., 2013): While Experiment 1 used mouse movement trajectories to provide evidence for the validity of the model’s predictions regarding intra-trial dynamics, experiments two and three demonstrated the importance to consider inter-trial dynamics by detecting hysteresis in intertemporal choice, that is, a persisting influence of previous on current choices. Finally, Experiment 4 showed that the model’s level of abstraction – representing the SS and the LL option as attractors – matches the phenomena of interest.
In the following, we will first evaluate the model and then continue with an evaluation of the empirical findings.
6.1 The attractor model of delay discounting
The attractor model represents the two possible choices and their attractiveness by two attractor basins of varying depth. We show, that this approach is fully compatible with discounting functions: While the latter merely define the configuration where the two options possess the same subjective value – as measured by equal final choice outcomes for the SS and the LL option – the attractor model additionally adds the stability of decision states – the depth of the attractors – and hence provides the dynamics leading to the final choice outcomes. Thereby, it offers a heuristic intuition about the process dynamics leading to a final choice (Reference Duran and DaleDuran & Dale, 2014; Reference Frank, Richardson, Lopresti-Goodman and TurveyFrank, Richardson, Lopresti-Goodman & Turvey, 2009; Reference van Rooij and Bongersvan Rooij & Bongers, 2000; Reference van Rooij, Favela, Malone and Richardsonvan Rooij et al., 2013). The model’s simplicity and abstractness supports theorizing without the demand of complex technical details (Reference Onnis and SpiveyOnnis & Spivey, 2012). Nevertheless, a formalized implementation of the model is possible, and in Appendix I, we demonstrate that it leads to the exact same predictions as described above. Moreover, the formalized implementation illustrates not only that the attractor model is compatible with discounting curves but also that it builds a bridge to more complex models of decision making which are equivalent on a conceptual level: The attractor dynamics can be seen as an abstraction of the neural interaction dynamics within more complex neural networks (Reference Onnis and SpiveyOnnis & Spivey, 2012) which have, in turn, been linked to the stability of firing patterns of neural assemblies representing the various options and their properties in the brain (Reference Meyer-Lindenberg, Ziemann, Hajak, Cohen and BermanMeyer-Lindenberg, Ziemann, Hajak, Cohen & Berman, 2002; Reference Scherbaum, Dshemuchadse and KalisScherbaum et al., 2008). Specifically, the dynamics of the attractor model describe the dynamics that arise from interactive activation and competition networks (Reference Rumelhart and McClellandRumelhart & McClelland, 1986) as they are applied in judgement and decision making, especially in the form of parallel constraint satisfaction networks models (Reference Freeman and AmbadyFreeman & Ambady, 2011; Reference Glöckner and BetschGlöckner & Betsch, 2008). While the latter models allow for the description of the decision process on a fine-grained level, the attractor model provides a more abstract (and mathematically less complex) description of the dynamics of these processes while still allowing to predict and study phenomena like hysteresis which are typically associated with more complex dynamical system.
The simplicity of the attractor model is a feature that is shared with other recent models of delay discounting, i.e., drift diffusion models and linear ballistic accumulator models (Reference Dai and BusemeyerDai & Busemeyer, 2014; Reference Milosavljevic, Malmaud, Huth, Koch and RangelMilosavljevic, Malmaud, Huth, Koch & Rangel, 2010; e.g., Reference Ratcliff and SmithRatcliff & Smith, 2004; Reference Rodriguez, Turner and McClureRodriguez et al., 2014). Concerning the intra-trial dynamics, the attractor model is comparable to these models (Reference WangWang, 2008). However, it adds the non-linear dynamics which lead to hysteresis, which diffusion and linear accumulator models do not explain without additional assumptions. Therefore, the attractor model can be seen as an instrument to describe and integrate the behavioral dynamics of decision making on both the intra-trial and the inter-trial time-scale, thus, building a bridge between simple yet static discounting curves and dynamic yet complex neural process models.
6.2 Mouse movement deflections in delay discounting
The data on the intra-trial choice dynamics as measured by mouse movements matched the predictions of the model. First, we found greater mouse movement deflections in high conflict compared to low conflict trials, which matches previous findings on process dynamics in intertemporal choice (Reference Dshemuchadse, Scherbaum and GoschkeDshemuchadse et al., 2012). Second, we found greater mouse movement deflections for low conflict trials in which the unattractive option was chosen compared to low conflict trials in which the attractive option was chosen. While the choice of the unattractive option is an anomaly for discounting-models, it is perfectly compatible when the attractor dynamics are taken into account: a slight bias at the start of trial suffices to lead the system state into the weaker attractor. Such an initial bias could be induced by noise or – as the other experiments suggest – by the choice in the previous trial leading to choice perseveration (compare Scherbaum et al., 2013).
6.3 Sequential effects in choices
Choice perseveration in sequences of decisions are best illustrated in their strongest manifestation as hysteresis as observed in Experiments 2 and 3. While perseverative choice patterns have been observed previously in studies focusing on value based decision making (Reference Coulson and NunnCoulson & Nunn, 1999; Reference Svyantek, Deshon and SilerSvyantek et al., 1991) or social perspective taking (Reference Duran and DaleDuran & Dale, 2014), to the best of our knowledge this is the first report of hysteresis in delay discounting. It is likely that earlier investigations have not observed this effect due to the classic paradigm (for example as used by Reference Scholten and ReadScholten & Read, 2010) employed in these studies where the explicit presentation of options’ attributes makes sequential manipulations obvious. In fact, this issue might even have led to opposite effects (Reference Robles and VargasRobles & Vargas, 2008) similarly to what has been found for perceptual decisions (so-called enhanced contrast due to repeated exposure, see Tuller et al., 1994). In contrast to the classic paradigm, our delay discounting game impeded the detection of sequences as it not only varied the delay (i.e., distance) of two consecutive options but also their positions on the x-y-plane. Importantly, we found hysteresis in two experiments with different option presentations, attesting to the robustness and validity of the finding.
This finding of hysteresis adds to the importance of complementing the outcome-based, psychophysical approach to delay discounting (Reference Takahashi, Oono and RadfordTakahashi, Oono & Radford, 2008) with a more process-oriented approach that not only focuses on the outcome of the decision process but also takes into account how the process unfolds over time. Such a process-oriented approach allows to integrate different features of the choice situation and contextual factors influencing the final decision – which have led to disconcertion in the research community (Reference Lempert and PhelpsLempert & Phelps, 2016) and a vast multitude of different discounting models (Reference DoyleDoyle, 2013) – into a single theoretical framework (Reference Scholten and ReadScholten & Read, 2010; Reference Townsend and BusemeyerTownsend & Busemeyer, 1995).
6.4 Limitations
6.4.1 Generalizability of the model and the evidence from the discounting game
With regard to our experimental approach, the advantages inherent to our discounting game come at the cost of the question whether or not the attractor model and the evidence for it presented here are generalizable to other, more standard intertemporal choice tasks and to intertemporal decisions in general. Classic intertemporal choice tasks present subjects with a number of hypothetical or partly factual intertemporal decisions between sooner/smaller and later/larger options with values and delays presented in writing and with ranges varying from days (e.g., Reference Kirby, Petry and BickelKirby, Petry & Bickel, 1999) to years (e.g., Reference Read, Frederick, Orsel and RahmanRead, Frederick, Orsel & Rahman, 2005). Our paradigm is very similar in that we ask subjects to decide between nearer/smaller and farer/larger options, with distances and values clearly visible to the subject and distances reflecting the time to reach the options and hence representing the delays of the options. Three differences between the classical and our paradigm are apparent, however. First, the values and times used in our paradigm are on a smaller scale than in most delay discounting tasks. Second, the values and times are presented implicitly (as the size and distance of coins) while they are typically presented explicitly (as amounts of money and delays in days, weeks or months) in classic intertemporal choice tasks. Third, time is limited in our paradigm and it hence poses an objective constraint on how to spend time by reaching an option. This is in contrast to classical intertemporal choices where the value of time is per se a subjective factor.
In response to these concerns, we suggest that the paradigm used in this set of studies is indeed a valid instrument to study intertemporal decision making. First, with regard to the small and accumulative amounts of money we used as rewards in our task, the discounting game joins a wide range of delay discounting paradigms employing various types of gains ranging from primary rewards such as food or drinks (Reference McClure, Ericson, Laibson, Loewenstein and CohenMcClure, Ericson, Laibson, Loewenstein & Cohen, 2007), one-out-of-many-choices-rewards (Reference Ballard and KnutsonBallard & Knutson, 2009; Reference McClure, Laibson, Loewenstein and CohenMcClure, Laibson, Loewenstein & Cohen, 2004), or hypothetical rewards (Reference Green, Fristoe and MyersonGreen, Fristoe & Myerson, 1994; Reference Robles and VargasRobles & Vargas, 2008; Reference Takahashi, Oono and RadfordTakahashi et al., 2008). In this diverse collection, we see our approach as reasonable in so far as it implements real rewards that the subject receives in total by the end of the experiment. The comparatively small temporal frame the discounting game operates on would indeed be problematic had this resulted in our subjects not discounting at all. However, our original study indicated that the discounting behavior observed in the discounting game is similar to the behaviour observed in more standard paradigms of delay discounting. For example, we showed that subjects completing the delay discounting computer game also weighed delays and reward attributes for each option which was reflected in a bias towards sooner/smaller choices (Reference Scherbaum, Dshemuchadse, Leiberg and GoschkeScherbaum et al., 2013). This is also reflected in the fact that disadvantageous choices of the SS option show significantly lower distances between the avatar and the SS option than advantageous ones. This finding indicates that also in the discounting game subjects are seduced by immediate or very short-term options (Reference Kalenscher and PennartzKalenscher & Pennartz, 2008). Of course, it is out of question that certain timescale-dependent effects (Reference Soman, Ainslie, Frederick, Li, Lynch and MoreauSoman et al., 2005) might not be captured within the short temporal horizon of the discounting, but such effects were not in the focus of the current studies. Nevertheless, the general comparability of the weighing of both, the small values and time intervals in the discounting game and the larger values and intervals in more classical tasks is also reflected in a high correlation of r = 0.64 between the k-values measured in our discounting game and a commonly used, paper pencil intertemporal choice questionnaire (Reference Kirby, Petry and BickelKirby et al., 1999) that we observed in our previous study (Reference Scherbaum, Dshemuchadse, Leiberg and GoschkeScherbaum et al., 2013).
Second, concerning the format of presentation of options’ values and times in terms of sizes and distances, it is indeed well known that different formats of presentation can influence the decision process, as is evident e.g., in the so called date-delay effect (Reference Read, Frederick, Orsel and RahmanRead et al., 2005; Reference Scherbaum, Dshemuchadse and GoschkeScherbaum et al., 2012). However, we tried to rule out at least some of the concerns resulting from this observation by comparing the effects of presenting the two options’ values implicitly (as the size of the coins) vs. explicitly (as written numbers on same-sized coins). Having found no difference in discounting behaviour and hysteresis between these two modes of presentation makes us confident that these effects are robust with respect to the mode of presentation.
Third, the limited time frame that we used may have encouraged subjects to use certain strategies like always choosing the SS option in order to make as many choices as possible. As this kind of strategic behaviour would stop subjects from making deliberate, considerate choices based on the careful weighing of the two options’ values and delays, this would render the decisions made in the discounting game incomparable to classical intertemporal decisions. Crucially, the choice patterns we observed stand against this possibility. We designed the combination of options presented to our subjects during the discounting game in such a way that persistently choosing one (e.g., the SS) option led to suboptimal results whereas a mixed, trial-dependent strategy resulted in the highest overall gain (for further details, please see the methods section of experiment 1). As the choice patterns we observed empirically closely resembled this optimal behaviour, we can conclude that subjects did not rigidly stick to one or the other option throughout the experiment but that they carefully weighed both options’ values and delays on a trial-by-trial basis instead.
Over and above these rather technical points illustrating the comparability of our dynamic and more classic measures of intertemporal decision making, we have gathered recent evidence that behaviour in the discounting game is linked to a problematic behaviour in the real world that has been consistently linked to impairments in delay discounting in the past: drug abuse. In this study (Scherbaum, Haber, Morley, Underhill & Moustafa, under review) we had a group of heroin addicts and a group of matched controls complete the discounting game. Consistent with previous evidence linking classic measures of intertemporal choice to addiction (Reference Kirby, Petry and BickelKirby et al., 1999), we found increased discounting for heroin addicts as compared to the control group, thus, providing further evidence for the validity of our paradigm.
All of these arguments combined make us confident that the informative value of the work presented here – the attractor model and the behavioural patterns we observed in support of it — are not restricted to the novel paradigm we employed, but that they are also valid and applicable to more classical investigations of intertemporal choices. What is more, the paradigm used here has a number of advantages compared with more classical intertemporal choice tasks. First, the it allows us to study phenomena that are difficult to study in the classical paradigm: While the sequential manipulation of delays that is necessary to study hysteresis is quite easy to see through in classic versions of the intertemporal choice paradigm, the possibility to rotate the options presented on a trial by trial basis makes these types of sequential manipulations much harder to detect in the discounting game. Second, whereas classical tasks are based on hypothetical choices or on partially realized choices (often, one trial is selected by a lottery), every trial counts for our subjects, thus, motivating them to stay focussed and on task throughout the experimental session. Third and last, but not least, the paradigm is built around an immersive, playful environment and subjects report that they were so engaged by the game that they almost forgot about participating in an experiment.
6.4.2 Level of generality of the attractor model
The attractor model builds a bridge from discounting functions to more complex models of delay discounting, for example accumulation models with different timescales (Reference Scherbaum, Dshemuchadse and GoschkeScherbaum et al., 2012), parallel constraint satisfaction networks (Reference Glöckner and BetschGlöckner & Betsch, 2008), or decision field models including several transformations (Reference Dai and BusemeyerDai & Busemeyer, 2014). The compatibility of the attractor model with more complex implementations of the decision process is illustrated further by the neural network implementation of the attractor model in Appendix I. Importantly, this implementation describes the decision dynamics on an aggregate level of detail. In particular, this model does not give any indication of a number of details of the decision process that are a matter of debates in the literature, for example, whether the attractiveness of an option is derived by calculating the subjective value of each option first and then performing alternative-wise comparisons of both options (Reference Dai and BusemeyerDai & Busemeyer, 2014; Reference MazurMazur, 1987; Reference SamuelsonSamuelson, 1937) or by attribute-wise comparisons of both options’ delays and values (Reference Scholten and ReadScholten & Read, 2010). However, elucidating these open questions is beyond the scope of our attractor model. Instead, the value of the abstract model presented here is that it provides an intuitive and easy-to-work-with instrument for theorizing about the intra- and inter-trial dynamics of the decision process. Of course, the merit of a model at this level hinges on additional empirical tests of its validity of the sort we have reported here.
6.5 Conclusion and prospects for research on delay discounting and decision making
The model and paradigm presented here led us to predictions and findings of behavioral patterns that could not be expected from the discrete models and behavioral methods typically used in intertemporal choice research. Instead, the predictions we tested in this article are typical patterns produced by dynamic systems which can be readily described with attractor models. Hence, our application of the attractor model to delay discounting (as initially sketched in Scherbaum et al., 2008) corroborates other applications of dynamic systems theory to value-based decision making in general (Reference Coulson and NunnCoulson & Nunn, 1999; Reference Koop and JohnsonKoop & Johnson, 2011; Reference McKinstry, Dale and SpiveyMcKinstry, Dale & Spivey, 2008; Reference Oullier and KelsoOullier & Kelso, 2006; Reference Oullier, Kirman and KelsoOullier, Kirman & Kelso, 2008; Reference Roe, Busemeyer and TownsendRoe, Busemeyer & Townsend, 2001; Reference van Rooij and Bongersvan Rooij & Bongers, 2000; Reference van Rooij, Favela, Malone and Richardsonvan Rooij et al., 2013; Reference Svyantek, Deshon and SilerSvyantek et al., 1991; Reference Townsend and BusemeyerTownsend & Busemeyer, 1995). In their combination, this growing body of work showcases that opening a window into the processes dynamics underlying value-based decisions is an important and fruitful endeavor. Therefore, the value of the dynamic approach presented here goes far beyond our findings for delay discounting and that it offers prospects for both the theoretical discussion and the empirical investigation of delay discounting and value-based decisions in general.
On a more abstract, theoretical level, our approach extends the focus of decision research from discounting curves and decision outcomes to dynamic properties of the decision process unfolding both within and across trials (as asked for e.g., in Reference Townsend and BusemeyerTownsend & Busemeyer, 1995). Within trials, deviations from the ideal choice line are a sensitive measure (Reference Frisch, Dshemuchadse, Görner, Goschke and ScherbaumFrisch, Dshemuchadse, Görner, Goschke & Scherbaum, 2015; Reference Scherbaum, Gottschalk, Dshemuchadse and FischerScherbaum, Gottschalk, Dshemuchadse & Fischer, 2015) that can provide information about the overall stability of decision states. Similarly, the strength of the hysteresis effect is an indicator of the overall stability of the attractor layout. Considering these process dynamics could hence provide explanations of dysfunctional behavior, e.g., too stable attractors in obsessive-compulsive disorders (Reference Rolls, Loh and DecoRolls, Loh & Deco, 2008) or relatively flat and hence volatile attractor layouts in impulsive decision makers (Kräplin et al., 2014, 2015; Reference Wittmann and PaulusWittmann & Paulus, 2007).
For empirical investigations, the presented paradigm offers an immersive way to study behavioral choice patterns both within trials (i.e., via mouse tracking) and across trials (e.g., by allowing unobtrusive sequential manipulations). Such measures could uncover differences between populations or conditions that might be occluded when only measuring choice outcomes or focusing on the outcome of (averaged) single choices without considering sequential effects (see Reference Spencer, Smith and ThelenSpencer, Smith & Thelen, 2001 for a comparable approach to the study of development of spatial memory). For example, changes in the hysteresis effect could be indicative of developmental differences in the attractor layout of younger and older populations (Reference Li, Lindenberger and SikströmLi, Lindenberger & Sikström, 2001) which would be occluded when only looking at discounting curves without considering sequential effects.
In summary, the combination of an attractor model with a dynamic behavioral paradigm integrates process dynamics into discounting functions and thereby supports the understanding of decision failures and deviations from static descriptions. The way how we make delay discounting decisions necessarily tells more than the final result of this process.
Appendix I: Simulations of dynamics in delay discounting based on a formalized model
Architecture of the formalized model
The formalized model used for the simulations below built on models which have been used to study the neural activation dynamics of perceptual decisions in the past (Reference Hock, Schöner and GieseHock et al., 2003; Reference Noest, Ee and WezelNoest et al., 2007). It comprised of two neural units with a non-linear activation function. These two units inhibit each other (Figure A1, left) which opens a natural mapping of these two units to the architecture of more complex models, i.e., parallel constraint satisfaction networks (e.g., Glöckner & Betsch, 2008). The two units in the model here could be seen as the two options in parallel constraint satisfaction network that are fed by informative cues from lower network levels. By interactive activation and competition both systems lead to a settled decision state in the end of the decision process.
Figure A1: Left: The formalized version of the attractor model is based on the non-linear neural activation dynamics of two competing neural units representing the two choice options. The input to the two units reflects the attractiveness of each option. Right: The activation of the two units mapped onto a two-dimensional state space in which each unit’s activation spans one dimension. Arrows in the vector field indicate the potential trajectories of this two-dimensional system under equal input. The mutual inhibition results in two potential stable activation states (indicated by dots compared to arrows in the vector-field). The dynamics of this two-dimensional system can be mapped to the one-dimensional model consisting of two attractors that we explain in greater detail in the introduction.
The activation of both units can be mapped as a two-dimensional state space in which each units’ activation spans one dimension. The mutual inhibition of the two units results in two potential stable states of activation. These two stable states can, in turn, be mapped to the attractor valleys (Figure A1 right) of the model sketched in the main part of the text (Figure 1): In state one, unit one is active and unit two is inhibited (representing e.g., a SS choice) while in state two, unit two is active and unit one is inhibited (representing e.g., a LL choice).
Similar to the depth of the attractors in the conceptual model explained in the main part of the text, the input to the units is defined by their attractiveness, i.e., their value/time ratio as used in the advantageous choice model or more complex metrics as the discounted value according to hyperbolic models (e.g., Green et al., 1994).
Mathematical model description
The dynamics of the model are defined by two coupled differential equations (one for each neural unit) representing non-linear neural activation dynamics (Reference Hock, Schöner and GieseHock et al., 2003; Reference Noest, Ee and WezelNoest et al., 2007; see also Amari, 1977).
Here, τ denotes the timescale (defining the step size of the Euler solution), h denotes the resting level, w i the (inhibitory) coupling strength of the two equations, and wr the recurrent feedback, I SS and I LL the input representing the attractiveness of the two options, and σ a sigmoid non-linearity, mirroring non-linear neural population dynamics:
Hence, interactions between the two units happen only to the extent that the activation u exceeds a soft threshold (Reference Cohen, Servan-Schreiber and McClellandCohen, Servan-Schreiber & McClelland, 1992; Reference Erlhagen and SchönerErlhagen & Schöner, 2002). Note that the β-parameter is also called the gain-parameter, modulating the discreteness of neural activation states (for hypotheses concerning β and aging, see e.g., Li et al., 2001). For a list of the chosen parameter values, see Table A1.
Table A1: Basic parameters of the model.
The two coupled differential equations constitute a neural system with two units inhibiting each other so that only one unit can win the competition and shape to the final choice. This competition unfolds over time.
The dynamics of the system are modulated by a control-parameter, c, representing the relative attractiveness of the two options via the strength of the two Inputs I SS and I LL relative to a general input strength I = 6 (c < 0 favoring the LL option and c > 0 favoring the SS option): I SS = I + c/2 and I LL = I − c/2.
Hence, the lower c, the lower the input to the SS option and the higher the input to the LL option, leading to u LL winning the competition over u SS. Hence, c determines the final choice when both units are in the same starting state, i.e., an activation U SS = u LL = 0. In contrast, if the starting state differs, i.e., u SS ≫ 0 and u LL = 0, u SS might win the competition due to its initial advantage.
The equation of motion that defines the dynamics of this system used for simulation is derived by differentiation. We simulated the behavior of the derived dynamical system by numerical integration with each trial having a maximum length of 200 time steps. Within each trial of 200 time steps, the input for the choice options was switched on after 50 time steps. Results were obtained using Matlab 2010b running under Windows 7.
Simulation 1: Intra-trial dynamics of delay discounting
The aim of Simulation 1 was to support the qualitative predictions of intra-trial dynamics with data from a formalized model.
Methods
For reasons of comparability, we will use the same labels as in Experiment 1, namely low vs. high conflict decisions and attractive-chosen vs. unattractive-chosen decisions. Low conflict decisions were operationalized by c = 0.25, high conflict ones by c = 0.05.
Unattractive-chosen decisions were produced by preceding the current trial with a previous trial where the current unattractive option had previously been the attractive option and hence been chosen. This lead to a bias in the start state of the current trial, favouring the unattractive option. In parallel to this, an attractive-chosen decision was produced by a bias of the start state into the other direction (notably, this would not be necessary since the system tends to choose advantageously because of the deeper attractor/the stronger activation of the advantageous unit).
With this setup, we simulated the activation trajectories of the system from its start state (zero activation of both units) to its end state (full activation of the chosen option’s unit).
We assume that the mouse movements measured in Experiment 1 represent exactly this competition for activation between the two units.
Results
As shown in Figure A2, Simulation 1 yielded modeled trajectories mirroring the qualitative predictions.
Figure A2: Results of Simulation 1. The activation of the chosen option’s unit (Y-Axis) is plotted against the activation of the unchosen option’s unit. Left: Activations for low vs. high conflict decisions. Right: Activations for low conflict decisions in which the attractive option was chosen vs. the unattractive option was chosen.
The data indicate a relatively direct activation of the chosen option’s unit for low conflict decisions compared to high conflict decisions. For low conflict decisions, such a direct activation was present for attractive-chosen compared unattractive-chosen decisions.
Simulation 2: inter-trial dynamics of delay discounting
We performed Simulation 2 to support the qualitative predictions about choice persistence in inter-trial behavior, that is, hysteresis in choice sequences. We expected effects similar to previous studies, e.g. on the perception of ambiguous figures (Reference Hock, Schöner and GieseHock et al., 2003), ambiguous auditory patterns (Reference Tuller, Case, Ding and KelsoTuller et al., 1994), or social perspective taking (Reference Duran and DaleDuran & Dale, 2014).
Methods
In Simulation 2, one unit represented the SS and the other unit the LL option. Again, we defined the control parameter c such that c = 0 represented options of equal attractiveness for a subject. Consequently, c < 0 represented a more attractive LL choice and c > 0 represented a more attractive SS choice. Since the parameter c represented the relative attractiveness of the two options, we reasoned that c should vary, first across different intervals between the SS and the LL option, and second, across different value differences between the small and the large option. We assumed multi-stability of the system for intermediate intervals between options and mono-stability for extreme intervals between options. Hence, we overall varied c within a large parameter window of [–1.5, 1.5]. Since we aimed to derive discounting curves from the models’ choices, we varied intervals and differences independently (similar to the experimental designs). Hence, the variance in c could be decomposed into variance due to differences in intervals and values: For intervals, we varied c in 12 steps between [–1,1] and for value differences, we varied c from –0.5 to 0.5. Hence, for every one of 5 value differences, we built ascending and descending sequences of 12 intervals between the two options. Within one sequence, we simulated trials continuously, so that activation from the previous trial could carry over to the next trial. Similar to Simulation 1, this led to incomplete relaxation to the starting state of the system and hence, the repetition priming found in the original study. To completely reset the system state between sequences, we inserted empty trials allowing the model to completely relax back to the neutral starting state. We expected Simulation 2 to yield different switch points between SS and LL choices depending on previous choice history, i.e., hysteresis.
Results
Simulation 2 predicts hysteresis effects for intermediate intervals and value differences, as can be seen for responses split up by the different interval conditions (Figure A3, left). The effect is also visible in the discounting curves marking the indifference points for each interval in ascending and descending sequences of intervals (Figure A3, right).
Figure A3: Results of Simulation 2. Left: Choice patterns across intervals for different ratios of small and large values (as used to determine discounting curves). Hysteresis is predicted for intermediate intervals and value ratios. Right: Discounting function, indicating subjective value of an option at the respective interval.
Hence, the decision of the system for an option was dependent on the history of previous decisions through the parameter range of the system.
Appendix II: Prime-Target manipulation of Experiment 3
Experiment 4 consisted of four blocks: in the first block, we estimated the individual discounting, resulting in individual measures of each subjects’ subjective value SV across intervals. The following three blocks then used the paired trial scheme, with a prime and a target trial, to test at which level of the processing hierarchy (feature vs. option level) choices persist (see Table A2 for an overview of the prime target scheme).
Table A2: The prime target scheme for testing feature- and option-level repetition priming. Individual subjective value SV was estimated from the trial in the first block of the experiment. (constant features shown in grey background, switching features in white background).
For option delays, we used the following scheme:
In prime trials favoring the SS option, we set D Sto 8 fields; in prime trials favoring the LL option, we set D Sto 2 fields. In both cases, D L was derived by the interval between the SS and the LL option (D L - D S = 3 or 6 fields).
In target trials following SS prime trials, D Lwas transferred from the previous D S (8 fields). D Swas then derived from the same intervals as in the prime trial (D L - D S = 3 or 6 fields). Similarly, in target trials following LL prime trials, DSwas transferred from the previous DL (5 or 8 fields). Again, D Swas then derived from the same intervals as in the prime trial (D L – D S = 3 or 6 fields).
For option values, we used a scheme depending on the subjects’ subjective value: We combined the individual discounting functions defining the subjective value SVas measured in block 1 with an additional bias SV bias = 0.15 in favor of the SS or the LL option.
In prime trials favoring the SS choice, we used a constrained random V Lso that V L x (SV + SV bias) > 40 credits. Hence, VS was guaranteed to stay above 40 credits, high enough to become the LL in the following target trial. Similarly, in prime trials favoring the LL choice, we used a constrained random V Swith V S / (SV – SV bias) < 50 credits. Hence, V L was guaranteed to stay below 50 credits, low enough to become the SS choice in the following target trial.
In target trials, the value of the favored option of the prime trial was then used to determine the value of the opposite option. However, we independently varied the SVbias in the target trial to favor either the SS (SV bias = 0.15) option, no option (SVbias = 0), or the LL option (SV bias = –0.15).
Open access
