1. Introduction
Imagine being at the train station, waiting for your commute home after a day at work. You are standing next to the kiosk, making it tempting to indulge in a quick snack during your wait. However, you had just set yourself the goal to stop snacking in order to finally reach your target weight at the end of the year. What would you do? This situation confronts you with a so-called intertemporal choice, requiring you to make a trade-off between the magnitude of the possible rewards and their delay until delivery. In everyday life, such choices between sooner-smaller (SS; e.g., the immediate pleasure that is experienced while eating the snack) and later-larger (LL; e.g., reaching a weight goal in the future) rewards are ubiquitous. In these types of situations, we may tend to forego the LL in favor of the SS (Berns et al., Reference Berns, Laibson and Loewenstein2007), which we refer to as impatient intertemporal choice behavior (also known as temporal impulsivity; Fenneman et al., Reference Fenneman, Frankenhuis and Todd2022). This choice tendency has been found to be particularly strong when the SS is available immediately (the now effect, also known as the present bias or immediacy effect; Benhabib et al., Reference Benhabib, Bisin and Schotter2010; Figner et al., Reference Figner, Knoch, Johnson, Krosch, Lisanby, Fehr and Weber2010; McClure et al., Reference McClure, Laibson, Loewenstein and Cohen2004, Reference McClure, Ericson, Laibson, Loewenstein and Cohen2007; O’Donoghue and Rabin, Reference O’Donoghue and Rabin1999).
1.1. Existing theories of intertemporal choice
Impatient intertemporal choice behavior has been consistently reported in the literature, and various models have been proposed to describe how people trade-off reward magnitudes and delays (e.g., Ainslie, Reference Ainslie1991; Amasino et al., Reference Amasino, Sullivan, Kranton and Huettel2019; Dai and Busemeyer, Reference Dai and Busemeyer2014; Laibson, Reference Laibson1997; McClure et al., Reference McClure, Laibson, Loewenstein and Cohen2004; Roelofsma and Read, Reference Roelofsma and Read2000; Samuelson, Reference Samuelson1937). Moreover, intertemporal choice has been shown to be implicated in various maladaptive behaviors, such as unhealthy food choice, maladaptive financial decisions, and environmentally unsustainable behavior (Amlung et al., Reference Amlung, Petker, Jackson, Balodis and Mackillop2016; Barlow et al., Reference Barlow, Reeves, McKee, Galea and Stuckler2016; Hardisty and Weber, Reference Hardisty and Weber2009; Hirsh et al., Reference Hirsh, Costello and Fuqua2015; Meier and Sprenger, Reference Meier and Sprenger2010), and psychological disorders, such as addictive disorders, ADHD, and bipolar disorder (Amlung et al., Reference Amlung, Marsden, Holshausen, Morris, Patel, Vedelago, Naish, Reed and McCabe2019; Lempert et al., Reference Lempert, Steinglass, Pinto, Kable and Simpson2019; Levitt et al., Reference Levitt, Oshri, Amlung, Ray, Sanchez-Roige, Palmer and MacKillop2022). In order to understand (and potentially change) impatient intertemporal decisions and their associated outcomes, research has directed attention to the psychological mechanisms that contribute to, and could therefore explain our decisions. Among existing accounts that aim to elucidate these mechanisms are those that focus on how we assign value to immediate and delayed rewards (e.g., exponential, hyperbolic or quasi-hyperbolic discounting models; Ainslie, Reference Ainslie1991; Laibson, Reference Laibson1997; McClure et al., Reference McClure, Laibson, Loewenstein and Cohen2004; Samuelson, Reference Samuelson1937), the aversive feelings associated with waiting for delayed rewards (Sonuga-Barke et al., Reference Sonuga-Barke, Taylor, Sembi and Smith1992), subjective time perception (Zauberman et al., Reference Zauberman, Kim, Malkoc and Bettman2009), the competition between multiple selves (Ainslie, Reference Ainslie1992; Thaler and Shefrin, Reference Thaler and Shefrin1981) or between affective versus deliberative (or hot versus cool) systems (Loewenstein and O’Donoghue, Reference Loewenstein and O’Donoghue2004; Metcalfe and Mischel, Reference Metcalfe and Mischel1999), and the role of self-control in overriding the prepotent response of choosing immediate rewards (Figner et al., Reference Figner, Knoch, Johnson, Krosch, Lisanby, Fehr and Weber2010; Luo et al., Reference Luo, Ainslie, Giragosian and Monterosso2009). Importantly, these accounts commonly assume immediate rewards to be particularly attractive or tempting, and although some focus on what makes delayed rewards less tempting, or how we are capable of overriding immediate temptations, they do not explain which psychological mechanisms underlie the pronounced attractiveness of immediate rewards. More insight into this question could improve our understanding of impatient intertemporal decisions, and, more specifically, could shed more light on our particular impatience in the face of a sooner-smaller reward that is available immediately.
1.2. Pavlovian-to-instrumental transfer (PIT)
One proposed theoretical framework that addresses this question is grounded in reinforcement learning theory, and revolves around the interaction between Pavlovian and instrumental control of behavior (Dayan et al., Reference Dayan, Niv, Seymour and Daw2006). Pavlovian control relies on responses to reinforcers or cues predicting reinforcers (Boureau and Dayan, Reference Boureau and Dayan2011; Dayan and Niv, Reference Dayan and Niv2008). These Pavlovian responses can be broadly categorized as approach toward rewards and reward-predictive cues and withdrawal from punishment and punishment-predictive cues (Glickman and Schiff, Reference Glickman and Schiff1967). Pavlovian responses are acquired through repeated pairing of an initially neutral stimulus with a reinforcer (i.e., through Pavlovian conditioning), creating cue–outcome associations. In the example above, the sight of the train station kiosk, eliciting the thought of a snack, would be sufficient to increase salivary flow rate and activate an approach tendency toward a snack. Instrumental or goal-directed control, in contrast, is established through repeated stimulus–response–outcome pairing, known as instrumental conditioning. By trial and error, the agent acquires knowledge about the stimulus–response–outcome contingencies, and resultingly learns which actions to perform in order to optimize outcomes, i.e., to maximize reward and minimize punishment, or, more generally, to achieve goals (Dorfman and Gershman, Reference Dorfman and Gershman2019; Guitart-Masip et al., Reference Guitart-Masip, Duzel, Dolan and Dayan2014). In the example above, inhibiting oneself would contribute to achieving the long-term weight goal.
Crucially, the Pavlovian and instrumental control systems do not merely coexist in parallel, but have been shown to interact in a number of ways, thereby potentially biasing decision-making (Dayan et al., Reference Dayan, Niv, Seymour and Daw2006; Dorfman and Gershman, Reference Dorfman and Gershman2019; Guitart-Masip et al., Reference Guitart-Masip, Duzel, Dolan and Dayan2014). One such interaction is known as Pavlovian-to-instrumental transfer (PIT), which refers to the effect of Pavlovian cues on instrumental responding (Cartoni et al., Reference Cartoni, Balleine and Baldassarre2016). PIT effects are commonly assessed using a three-phase paradigm. In the first phase of this paradigm, the instrumental conditioning phase, participants learn to perform certain actions to win rewards and/or avoid punishments. In the second phase, the Pavlovian conditioning phase, participants learn the associations between Pavlovian cues and outcomes. In the third and final phase, termed the transfer phase, participants again perform the instrumental conditioning task, but this time in the presence of the Pavlovian cues. Importantly, these cues provide no information on the instrumentally optimal action, and are therefore irrelevant to the task at hand. Nevertheless, by eliciting Pavlovian approach or withdrawal responses, the cues may intensify or interfere with instrumental actions, resulting in so-called transfer effects. Research with human and non-human animal participants (for reviews, see Cartoni et al., Reference Cartoni, Balleine and Baldassarre2016; Holmes et al., Reference Holmes, Marchand and Coutureau2010) has provided support for this interaction between Pavlovian and instrumental control, by showing that the presence of Pavlovian cues associated with rewards promotes instrumental approach and inhibits instrumental avoidance behavior (appetitive PIT). Pavlovian cues associated with punishments, in contrast, have been found to inhibit instrumental approach and facilitate instrumental avoidance behavior (aversive PIT). Furthermore, the value of the Pavlovian outcome has been found to influence transfer effects in a parametric manner, with higher absolute Pavlovian outcome values resulting in stronger transfer effects compared to lower absolute outcome values (Chen et al., Reference Chen, Schlagenhauf, Sebold, Kuitunen-Paul, Chen, Huys, Heinz, Smolka, Zimmermann and Garbusow2023; Garbusow et al., Reference Garbusow, Schad, Sebold, Friedel, Bernhardt, Koch, Steinacher, Kathmann, Geurts, Sommer, Müller, Nebe, Paul, Wittchen, Zimmermann, Walter, Smolka, Sterzer, Rapp, Huys, Schlagenhauf and Heinz2016, Reference Garbusow, Nebe, Sommer, Kuitunen-Paul, Sebold, Schad, Friedel, Veer, Wittchen, Rapp, Ripke, Walter, Huys, Schlagenhauf, Smolka and Heinz2019; Huys et al., Reference Huys, Cools, Gölzer, Friedel, Heinz, Dolan and Dayan2011; Schad et al., Reference Schad, Rapp, Garbusow, Nebe, Sebold, Obst, Sommer, Deserno, Rabovsky, Friedel, Romanczuk-Seiferth, Wittchen, Zimmermann, Walter, Sterzer, Smolka, Schlagenhauf, Heinz, Dayan and Huys2020; Sommer et al., Reference Sommer, Garbusow, Jünger, Pooseh, Bernhardt, Birkenstock, Schad, Jabs, Glöckler, Huys, Heinz, Smolka and Zimmermann2017, Reference Sommer, Birkenstock, Garbusow, Obst, Schad, Bernhardt, Huys, Wurst, Weinmann, Heinz, Smolka and Zimmermann2020).
1.3. Intertemporal impatience from a PIT perspective
Transfer effects can thus lead to suboptimal instrumental behavior when the Pavlovian influence conflicts with instrumental requirements. Dayan et al. (Reference Dayan, Niv, Seymour and Daw2006) theorized that impatient intertemporal decisions may arise from a similar interaction between Pavlovian and instrumental control. That is, they proposed that cues associated with immediate gratification (e.g., the sight of the train station kiosk) may trigger a Pavlovian approach response (toward eating a snack), thereby interfering with instrumental withdrawal behavior (dieting) directed toward a long-term (weight) goal. This theory forms an alternative to traditional dual-system theories of intertemporal choice that have been criticized in the past (for some of these criticisms, see e.g., Gladwin et al., Reference Gladwin, Figner, Crone and Wiers2011; Gladwin and Figner, Reference Gladwin, Figner, Wilhems and Reyna2014). Although the account proposed by Dayan et al. (Reference Dayan, Niv, Seymour and Daw2006) parallels traditional dual-system theories by proposing two types of behavioral control (i.e., Pavlovian and instrumental), they differ on important points. First, the Pavlovian–instrumental model does not rely on a division between motivational and control processes, such as the binary division between a ‘hot’ versus ‘cool’ (Metcalfe and Mischel, Reference Metcalfe and Mischel1999), or ‘affective’ versus ‘deliberative’ (Loewenstein and O’Donoghue, Reference Loewenstein and O’Donoghue2004) system. In addition to introducing relatively loose theoretical terms, the separation of motivational and control processes that is central to these dual-system theories has been argued to cause a motivational homunculus problem, since the decision to exert control must be accompanied by a motivational explanation for why to exert control (i.e., what is the expected value; Gladwin et al., Reference Gladwin, Figner, Crone and Wiers2011; Gladwin and Figner, Reference Gladwin, Figner, Wilhems and Reyna2014). The Pavlovian–instrumental model, in contrast, interweaves motivation and control by having both Pavlovian and instrumental control revolve around expected outcome value. Importantly, however, whereas instrumental control learns expected outcome values based on stimuli and responses, Pavlovian control learns expected outcome values based on stimuli only (Dorfman and Gershman, Reference Dorfman and Gershman2019). As a result, Pavlovian control is less flexible compared to instrumental control, because it relies on the relatively rigid approach behavior in response to reward-predictive cues and avoidance behavior in response to punishment-predictive cues (Dayan et al., Reference Dayan, Niv, Seymour and Daw2006). It is the lack of flexibility of these Pavlovian responses, which are performed regardless of their effect on instrumental outcomes, that may give them an impulsive or short-sighted character. Second, in contrast to traditional dual-system models, the Pavlovian–instrumental control model is strongly grounded in the animal reinforcement learning research (and has been highly influential in this field since then), obviating the need to postulate uniquely human functions.
Providing indirect support for the role of PIT in intertemporal choice, research has found PIT to be implicated in behavioral anomalies of impulsivity, focusing mostly on addictive behaviors (see Garbusow et al., Reference Garbusow, Ebrahimi, Riemerschmid, Daldrup, Rothkirch, Chen, Chen, Belanger, Hentschel, Smolka, Heinz, Pilhatsch and Rapp2022, for a review). Furthermore, exposure to appetitive cues has been found to increase impatient intertemporal choice (e.g., Kim and Zauberman, Reference Kim and Zauberman2013; Li, Reference Li2008; Mathar et al., Reference Mathar, Wiebe, Tuzsus, Knauth and Peters2022; Van et al., Reference Van Den Bergh, Dewitte and Warlop2008; Wilson and Daly, Reference Wilson and Daly2004), which has been interpreted as resulting from a cue-elicited increase in approach behavior (Van et al., Reference Van Den Bergh, Dewitte and Warlop2008). It should be noted, however, that these effects were mostly limited to studies that presented the appetitive cues before instead of during the intertemporal choice task; results of studies presenting the cues during the intertemporal choice task, in a trial-wise manner, have been more mixed in the direction and statistical significance of effects (Guan et al., Reference Guan, Cheng, Fan and Li2015; Knauth and Peters, Reference Knauth and Peters2022; Luo et al., Reference Luo, Ainslie and Monterosso2014; Simmank et al., Reference Simmank, Murawski, Bode and Horstmann2015; Sohn et al., Reference Sohn, Kim, Sohn, Seok, Choi and Watanuki2015). Moreover, while these studies examined the effect of generally appetitive or aversive cues, we were specifically interested in testing the Pavlovian properties of cues signaling immediate (versus delayed) gratification on goal-directed actions. Finally, a study by Luo et al. (Reference Luo, Ainslie, Giragosian and Monterosso2009) found that cues signaling the availability of immediate rewards invigorated responding in a monetary incentive delay task compared to cues signaling the availability of preference-matched delayed rewards. The authors discuss the possibility that their findings reflect a conditioned response,Footnote 1 with cues signaling immediacy potentiating action,Footnote 2 which would be in line with the proposed role of PIT in intertemporal choice. Despite these first empirical indications, no research has, to the best of our knowledge, directly tested the role of Pavlovian–instrumental interactions in intertemporal choice. Therefore, the goal of the present study was to provide an empirical test of this theoretical account.
1.4. The present study
We developed an intertemporal variant of a classic three-phase PIT paradigm (based on Huys et al., Reference Huys, Cools, Gölzer, Friedel, Heinz, Dolan and Dayan2011), extending it to investigate the Pavlovian properties of rewards and delays commonly used in intertemporal choice tasks. In the first phase of this task, participants were trained to make instrumental go versus no-go responses in order to win monetary rewards and/or avoid monetary losses. In the second phase, the Pavlovian conditioning phase, participants learned the associations between six colored squares and six intertemporal monetary rewards. Reflecting the trade-off that is inherent to intertemporal rewards, these Pavlovian rewards varied systematically in their amounts (small/medium/large) and delays (immediate/delayed). The exact reward amounts used for each participant were derived on an individual basis prior to the PIT task, using a titration procedure. Finally, in the third phase of the task, participants again performed the instrumental go/no-go task. This time, however, they did not receive feedback on the correctness of their response in order to prevent further instrumental learning, and, most crucially, the Pavlovian cues were presented in the background.
Our research tested six hypotheses, which can be ordered hierarchically. We first examined whether the Pavlovian conditioning procedure successfully manipulated the Pavlovian value participants assigned to the cues. More specifically, we tested whether these Pavlovian values varied as a function of the reward amount and delay associated with the cues. The values were obtained by asking participants to rate how much they liked each of the Pavlovian cues, which is a commonly used approach in the PIT literature (e.g., Hogarth et al., Reference Hogarth, Dickinson, Wright, Kouvaraki and Duka2007; Talmi et al., Reference Talmi, Seymour, Dayan and Dolan2008; Trick et al., Reference Trick, Hogarth and Duka2011). We expected that after the PIT task, cues associated with large rewards would have a higher Pavlovian value, i.e., would be evaluated more positively, compared to cues associated with smaller rewards (H1: Pavlovian conditioning hypothesis—Amount effect). In addition, we expected that Pavlovian cues associated with immediate rewards would acquire a higher Pavlovian value compared to cues associated with delayed rewards (H2: Pavlovian conditioning hypothesis—Delay effect).
These Pavlovian conditioning hypotheses (H1-2) formed prerequisites for our next two hypotheses, in which we tested whether the Pavlovian values associated with the cues exerted transfer effects by biasing instrumental behavior. In line with previous findings showing transfer effects to be parametrically influenced by reward value (Chen et al., Reference Chen, Schlagenhauf, Sebold, Kuitunen-Paul, Chen, Huys, Heinz, Smolka, Zimmermann and Garbusow2023; Garbusow et al., Reference Garbusow, Schad, Sebold, Friedel, Bernhardt, Koch, Steinacher, Kathmann, Geurts, Sommer, Müller, Nebe, Paul, Wittchen, Zimmermann, Walter, Smolka, Sterzer, Rapp, Huys, Schlagenhauf and Heinz2016, Reference Garbusow, Nebe, Sommer, Kuitunen-Paul, Sebold, Schad, Friedel, Veer, Wittchen, Rapp, Ripke, Walter, Huys, Schlagenhauf, Smolka and Heinz2019; Huys et al., Reference Huys, Cools, Gölzer, Friedel, Heinz, Dolan and Dayan2011; Schad et al., Reference Schad, Rapp, Garbusow, Nebe, Sebold, Obst, Sommer, Deserno, Rabovsky, Friedel, Romanczuk-Seiferth, Wittchen, Zimmermann, Walter, Sterzer, Smolka, Schlagenhauf, Heinz, Dayan and Huys2020; Sommer et al., Reference Sommer, Garbusow, Jünger, Pooseh, Bernhardt, Birkenstock, Schad, Jabs, Glöckler, Huys, Heinz, Smolka and Zimmermann2017, Reference Sommer, Birkenstock, Garbusow, Obst, Schad, Bernhardt, Huys, Wurst, Weinmann, Heinz, Smolka and Zimmermann2020), we hypothesized that in the transfer phase, the presence of cues associated with larger rewards would result in increased go-responding compared to the presence of cues associated with smaller rewards (H3: Transfer hypothesis—Amount effect). Similarly, we hypothesized that the presence of cues associated with immediate rewards would result in increased go-responding compared to the presence of cues associated with delayed rewards (H4: Transfer hypothesis—Delay effect).
The hypotheses described thus far tested the Pavlovian and transfer effects of the reward amount (H1 and H3) and reward delay (H2 and H4) separately. Our final two hypotheses integrate these effects by addressing the trade-off between reward amount and delay. This allows us to investigate the effect of immediacy while controlling for the subjectively discounted value of the rewards associated with the cues. To this end, the six intertemporal rewards derived individually prior to the PIT task encompassed two indifference pairs: pairs of preference-matched sooner-smaller and later-larger rewards. The first indifference pair was formed by a delayed large versus an immediate medium reward, and the second pair by a delayed medium versus an immediate small reward. Given the choice indifference between members of such pairs, we expected there to be no significant difference in Pavlovian value of cues associated with members of an indifference pair (H5: Pavlovian indifference hypothesis). With respect to the transfer effect of these cues, we expected one of two outcomes. First, if the Pavlovian indifference hypothesis (H5) is assumed to be true, it could be expected that they do not differentially affect go-responding in the transfer phase (H6a: Transfer indifference hypothesis). However, given the research by Luo et al. (Reference Luo, Ainslie, Giragosian and Monterosso2009) that found cues signaling immediate rewards to invigorate responding compared to preference-matched delayed rewards, and the possibility that this reflects a conditioned response bias rather than a direct consequence of valuation processes, we may alternatively expect that Pavlovian cues associated with the immediate member of an indifference pair evoke stronger go-responding compared to cues associated with the delayed member of the indifference pair (H6b: Transfer now effect hypothesis). This would be consistent with the idea that Pavlovian biasing might be a factor underlying the now effect. Given the hierarchical ordering of our hypotheses, it should, however, be noted that the indifference pair hypotheses (H5 and H6) are mainly of interest under the condition that we find the hypothesized effects of reward amount and delay separately, as this would subsequently allow us to informatively examine the trade-off between these two effects.
To test these hypotheses, we performed two experiments. In Experiment 1, we developed and used the intertemporal PIT paradigm to test our hypotheses. In Experiment 2, we adjusted our study paradigm to address a potential alternative explanation of our results arising from our research design in Experiment 1. Finally, given the similarities in study designs across experiments 1 and 2, we also tested our hypotheses after pooling the data from both experiments. These pooled data analyses served to investigate the robustness of our results, and to test whether the patterns of results found in both experiments were significantly different.
2. Experiment 1
2.1. Methods
2.1.1. Participants
Fifty participants took part in Experiment 1 (38 female, 12 male, M age = 22.68, SD age = 3.93). The sample size was determined by the available budgetary and time constraints at the time of running the study (in 2016), while considering the sample sizes of existing studies reporting PIT effects (median N = 39 and mean N = 46Footnote 3 ). Participants were recruited via the local research participation system, and were included if they indicated to have good color vision. The study was conducted in accordance with the ethical guidelines from the local institutional review board, and all participants provided written informed consent. Payments consisted of vouchers worth €15 or 1.5 course credits, plus an additional performance-contingent bonus (up to €7.70).
2.1.2. Materials and procedure
2.1.2.1. General procedure
The complete experiment consisted of a 1.5-hour laboratory session. The experiment was presented on a 24-inch computer running Windows 7, with the display resolution set to 1920x1080. After receiving general information on the experiment and providing informed consent, participants completed the tasks in the same order as they are described below. All tasks were implemented in Python 2.7, with the main components executed in PsychoPy (Peirce, Reference Peirce2007). For each task, detailed instructions were provided on the computer screen. Participants additionally completed several tasks and questionnaires that were not analyzed for the current paper, which are listed in Supplementary material S1. At the end of the experiment, participants were debriefed on the research purpose, paid, and thanked for their participation.
2.1.2.2. Pre-PIT delay discounting task
The goal of this titration procedure was to derive six intertemporal rewards, including two indifference pairs, which were later implemented as Pavlovian rewards in the PIT task. The discounting task (see Supplementary material S2 for a visual depiction of the procedure) consisted of two blocks. In each trial of the first block, participants were presented with a choice between an immediate reward varying between 0 and 20 cents (sooner-smaller reward; SS), versus a constant reward of 20 cents delayed by 10 seconds (later-larger reward; LL). The ranges of the reward amounts and delays were based on previous research reporting temporal discounting effects with similar amounts and delays (Gregorios-Pippas et al., Reference Gregorios-Pippas, Tobler and Schultz2009; for a review, see Scheres et al., Reference Scheres, de Water and Mies2013). As an illustration, a choice trial could present a choice between ‘12 cents now’ and ‘20 cents later’ on the screen. If participants chose the SS, the outcome of their choice was presented in the middle of the screen (e.g., ‘You chose 12 cents’). In contrast, if they chose the LL, they first experienced a delay of 10 seconds during which the screen remained blank, followed by the presentation of their choice outcome (e.g., ‘You chose 20 cents’). No explicit information was provided on the duration of the delay. Each binary choice was presented twice, and all 42 trials were presented in a fixed order. The rewards were hypothetical, but participants were instructed to make their choices as if the rewards were real. After completing this first task block, the indifference value was derived by fitting a generalized linear model with choice (SS vs LL) as dependent variable and the reward amount of the SS as predictor (ranging between 0 and 20). The SS for which the predicted probability of choosing the LL was closest to 0.5 (i.e., closest to indifference) was selected as the indifference value, and formed an indifference pair with an LL of 20 cents in 10s. The second task block was similar to the first block, except that the reward amount of the immediate member of the indifference pair derived in the first block (e.g., 14 cents) was now taken as the constant amount of the delayed reward in the second block. Correspondingly, in each trial of this second block, participants were presented with a choice between an immediate small reward (e.g., ranging between 0 and 14 cents) and a delayed larger reward (e.g., 14 cents in 10s). Trials were presented in random order. The maximum SS reward amount, and hence the number of trials, depended on the LL reward amount, since all amounts up to and including the LL amount were presented as SS. Again, a generalized linear model was used to derive the SS amount for which the predicted probability of choosing the LL was closest to indifference.
The delay discounting task thus resulted in two indifference pairs: one formed by a delayed large (20 cents in 10s) versus an immediate medium (e.g., 14 cents now) reward, and one formed by a delayed medium (e.g., 14 cents in 10s) versus an immediate small (e.g., 6 cents now) reward. In order to have a fully balanced design, allowing us to assess the effect of reward amount and delay separately, two additional rewards were added: an immediate large reward (20 cents now) and a delayed small reward (e.g., 6 cents in 10s). In the Pavlovian conditioning phase of the PIT task, each of the six intertemporal rewards was randomly assigned to one of six colored squares, resulting in six unique Pavlovian cue–outcome associations.
2.1.2.3. Pre-PIT liking ratings
In this task, participants were presented with each of the six squares in random order, and asked to indicate how much they liked them on a visual analog scale presented as a slider, ranging from 0 (not at all) to 100 (very much). These ratings served to assess whether there were any systematic differences in subjective evaluation of the colored squares before Pavlovian conditioning.
2.1.2.4. PIT task
The PIT task (Figure 1A) consisted of three phases: an instrumental conditioning phase, a Pavlovian conditioning phase, and a transfer phase.
PIT tasks used in Experiment 1 (panel A) and 2 (panel B), displayed alongside each other to facilitate comparison of the two experiments. Panel A: PIT task used in Experiment 1. In the instrumental conditioning phase, participants learned to respond to good mushrooms (go trials) and not respond to bad mushrooms (no-go trials). Correct responses resulted in a 5 cents win, while incorrect responses resulted in a 5 cents loss. In the Pavlovian conditioning phase, participants learned cue–outcome associations between squares of different colors (the Pavlovian cues) and rewards varying in amount and delay. Each trial started with the presentation of the square, either followed directly by the presentation of the reward amount (immediate reward trial), or followed by a 10-s blank screen delay before the reward amount was presented (delayed reward trial). In the transfer phase, participants again performed the instrumental mushroom task, but with one of the Pavlovian cues tiling the background. This phase was performed in nominal extinction, i.e., no outcome feedback was presented. Panel B: The PIT task used in Experiment 2 was similar to that used in Experiment 1, except that in the Pavlovian phase, the reward delays were no longer experiential. Instead, presentation of the square was always immediately followed by presentation of reward amount and delay information. In addition, we used larger rewards (€0-20) and longer delays (now/in 50 days).
2.1.2.4.1. Instrumental conditioning phase
The goal of this phase was for participants to learn to give go or no-go responses to each of six instrumental stimuli. In the transfer phase, this instrumental behavior would become subject to the presence of Pavlovian cues. The instrumental conditioning task was framed in terms of collecting mushrooms (the instrumental stimuli), whereby good mushrooms were to be collected (go response) and bad mushrooms were to be left behind (no-go response). Each trial started with participants clicking on a small central box (bringing their attention and cursor to the center of the screen), upon which the box disappeared and a mushroom was presented on the left or right side of the screen. The mushroom could be collected by clicking on it within a 1,500-ms response window. Alternatively, the mushroom could be left behind by not clicking on it. If participants collected a mushroom, a rectangular border around the mushroom changed from grey to orange to provide participants with a visual confirmation that they had collected the mushroom; the screen otherwise remained unchanged until the end of the response window. The six mushrooms were randomly assigned to be either good (three mushrooms) or bad (three mushrooms). Participants had to find out which mushrooms were good and bad by trial and error. After the response window, feedback was presented for 1,500 ms in the form of +5 cents (correct response) or -5 cents (incorrect response). If participants clicked outside the frame in which the mushroom was presented, the response was coded as invalid and participants were reminded to click inside the frame. Invalid responses were coded as missing values for the analyses. Feedback was probabilistic, as on only 75% of the trials, the presented feedback (+5/-5) corresponded with the correctness of the response (correct/incorrect). However, regardless of the feedback that participants received on each trial, the actual responses (correct/incorrect) were stored and used for payment. Participants were instructed that whether a mushroom was good or bad did not change throughout the task, and did not depend on the location (left/right) where the mushroom was presented. Each mushroom was presented 12 times in random order and location, resulting in a total of 72 trials. After the complete experiment, participants received the total amount of cents won during this phase.
2.1.2.4.2. Pavlovian conditioning phase
The goal of the Pavlovian conditioning phase was for participants to learn the cue–outcome associations between the six colored squares and the six intertemporal rewards derived in the delay discounting tasks. For each participant, the colored squares and intertemporal rewards were randomly paired. Each trial of this phase started with the presentation of a fixation cross in the center of the screen for 800 ms, after which one of the six colored squares was presented for 1,500 ms. The associated monetary outcome was presented either immediately afterward, or after a blank screen delay of 10 seconds. The outcome was presented in the center of the screen for 1,500 ms, stating the reward amount associated with the cue (e.g., ‘With this picture, you win 20 cents’). No explicit information was provided on the duration of the delay. Each square–reward pair was presented 20 times in random order, resulting in a total of 120 trials. Participants were instructed to pay attention to the information that corresponded to each picture. After each fifth trial, a query trial was presented, in which two of the colored squares were presented alongside each other, and participants were asked to indicate which of these they preferred or whether they had no preference. Which squares were presented and on which side of the screen (left/right) was randomly determined. The purpose of these query trials was to keep participants’ attention on the task, which was otherwise passive in nature. Responses on the query trials were not analyzed.
2.1.2.4.3. Transfer phase
This final phase served to test whether the Pavlovian cues exerted a transfer effect on instrumental go/no-go behavior. Participants were instructed to continue collecting mushrooms as they had done in the instrumental conditioning phase. Which mushrooms were good and bad had not changed since the instrumental conditioning phase. However, the task was performed in nominal extinction, meaning that no feedback (+5/-5) was provided about the in- /correctness of their responses, serving to prevent further instrumental learning. Nevertheless, participants were informed beforehand that their responses would still be recorded and that upon completion of the experiment, they would receive the total amount of cents won. Furthermore, and most crucially, during each transfer trial, one of the Pavlovian squares tiled the background of the screen. Participants were not provided with any instructions regarding these backgrounds, in line with Huys et al. (Reference Huys, Cools, Gölzer, Friedel, Heinz, Dolan and Dayan2011). Each mushroom was presented in combination with each background three times, resulting in a total of 108 trials.
2.1.2.5. Post-PIT liking ratings
This task served to assess the effects of Pavlovian conditioning on the value that participants assigned to the Pavlovian cues. Participants were asked to rate each of the colored squares, in the same way as they had done before the PIT task.
2.1.2.6. Pavlovian contingency test
Following previous PIT studies (Jeffs and Duka, Reference Jeffs and Duka2017; Talmi et al., Reference Talmi, Seymour, Dayan and Dolan2008; Trick et al., Reference Trick, Hogarth and Duka2011), we explored the role of participants’ awareness of the Pavlovian cue–outcome contingencies on the main Pavlovian and transfer results. We tested participants’ contingency awareness through the use of multiple-choice questions. On each trial of this test, one of the colored squares was presented, and participants were asked ‘How many cents belong to this picture?’ (with the small, medium, and large amounts as answer options), and ‘Is the outcome that belongs to this picture immediate or after a delay?’ (with ‘immediate’ and ‘after a delay’ as answer options).
2.1.2.7. Post-PIT delay discounting task
Finally, we assessed whether the indifference pairs (as derived in the pre-PIT delay discounting task) remained stable throughout the experiment, by asking participants to complete a shortened version of the delay discounting task. The choices presented during this shortened task were identical to those presented in the pre-PIT delay discounting task; however, each choice was presented only once. Moreover, since the amount of the delayed medium reward was determined a priori (i.e., it was identical to that used in the pre-PIT task), the choices between delayed large versus immediate medium rewards on the one hand (presented in the first block of the pre-PIT delay discounting task), and between delayed medium versus immediate small rewards on the other hand (presented in the second block of the pre-PIT delay discounting task), were presented in one block, in an intermixed manner.
2.1.3. Data analysis
Here, we provide a general overview of the statistical approach adopted throughout this paper. Given the large number of analyses conducted, the statistical models used for each analysis are specified in the results section and, in more detail, in Supplementary material S3. We tested our main hypotheses using Bayesian mixed-effects models, using the package brms (version 2.14.4; Bürkner, Reference Bürkner2018) in R (version 3.6.3; R Core Team, 2020). We used weakly informative default priors for all analyses. As recommended by Barr et al. (Reference Barr, Levy, Scheepers and Tily2013) and Yarkoni (Reference Yarkoni2020), we accounted for by-participant and by-stimulus random variation using a maximal random-effects structure. That is, where appropriate we included participant identity, instrumental stimulus identity (mushroom), and Pavlovian stimulus identity (square) as grouping variables, and included all possible random intercepts, slopes, and covariances varying over these grouping variables. Categorical predictors were coded using sum-to-zero contrasts, and continuous predictors were z-standardized. Models with a binary outcome variable (e.g., go/no-go, correct/incorrect) were analyzed using a Bernoulli distribution, discount rates were analyzed using a lognormal distribution, and all other models were analyzed using a Gaussian distribution. Please note that for models with a Bernoulli distribution, parameter estimates are reported on the log-odds scale, whereas model-based means are reported on the probability scale. For variables modeled with a lognormal distribution, parameter estimates are reported on the lognormal scale, and medians and interquartile ranges are reported in addition to means and standard deviations to account for the distributions’ skewness.
We directly tested our hypotheses through planned comparisons, following the advocated superiority of this approach over the use of omnibus tests (Baguley, Reference Baguley2012), using the package emmeans (version 1.5.0; Lenth, Reference Lenth2019). Despite using a Bayesian statistics package to run our models, we adopted a frequentist approach to statistical inference: Effects were denoted as significant when the 95% credible interval (CI), more specifically, the 95% highest density interval (HDI), did not include 0. For each set of planned comparisons corresponding to a specific hypothesis, the CI levels used for these significance tests were adjusted for multiple comparisons using the Benjamini–Hochberg procedure (Benjamini and Hochberg, Reference Benjamini and Hochberg1995), with a false discovery rate of 5%. For descriptive purposes and to facilitate interpretation of the results, the uncorrected 95% CIs are reported throughout this paper. In case the corrected and uncorrected CI resulted in different conclusions regarding statistical significance, both CIs are reported. Visualizations of the results were created using the packages brms, ggplot2 (version 3.3.2; Wickham, Reference Wickham2016) and forestplot (version 3.3.1; Gordon and Lumley, Reference Gordon and Lumley2022).
2.2. Results
Figure 2 provides an overview of the main study results from Experiment 1, Experiment 2, and the analyses conducted on data pooled across both experiments. The purpose of this figure is to facilitate comparison of these results. In the current results section, we report the results of Experiment 1 in detail; the results of Experiment 2 and on the pooled data analyses are discussed further below.
Summary of the main model estimates and their 95% credible intervals (CIs) as observed in the Pavlovian conditioning phase (panel A) and the transfer phase (panel B), across Experiment 1, Experiment 2, and the pooled data analyses. The panels display the planned contrasts for the effects of reward amount, reward delay, and the indifference pairs on the post-PIT liking ratings (panel A) and the probability of making a go response in the transfer phase (panel B). For the effects of reward amount (hypotheses 1 and 3), estimates represent the differences between large versus small (L-S), large versus medium (L-M), and medium versus small (M-S) rewards. For the effects of delay (hypotheses 2 and 4), estimates represent the difference between delayed versus immediate (D-I) rewards. For the indifference pair hypotheses (hypotheses 5, 6a, and 6b), estimates represent the difference between the delayed large versus immediate medium reward (DL-IM), and between the delayed medium versus immediate small reward (DM-IS). Effects are denoted as significant if their 95% CI does not include zero, which is displayed in the figure by the 95% CIs not crossing the vertical zero line.
2.2.1. Pre-PIT delay discounting task
The indifference values derived from the delay discounting task showed substantial individual differences, as commonly observed in intertemporal choice tasks (Scheres et al., Reference Scheres, de Water and Mies2013). Immediate medium rewards that were subjectively matched with the delayed large reward of 20 cents in 10 seconds ranged between 8 and 20 cents (M = 13.96, SD = 3.34, Md = 14.0, IQR = 4.0). Immediate small rewards that were subjectively matched with a delayed medium reward—the latter of which was identical in amount to the immediate medium reward, but had a 10-s delay—ranged between 1 and 16 cents (M = 9.50, SD = 4.12, Md = 9.0, IQR = 6.0).
2.2.1.1. Deriving Pavlovian rewards
For the majority of participants, the indifference values derived from the delay discounting task were directly used as rewards in the Pavlovian conditioning phase. However, seven participants showed a choice pattern in the delay discounting task we termed non-discounting. These participants exclusively selected delayed rewards in one or both of the task blocks (except, in some cases, for trials in which the SS and LL amounts were equal, e.g., 20 cents now versus 20 cents later). Directly using their choice-derived indifference values as Pavlovian reward values in the PIT task would have resulted in indifference pairs with equal reward amounts, not allowing for a differentiation in reward amount associated with Pavlovian cues. To circumvent this issue, these seven participants were assigned predefined Pavlovian reward values, namely 20, 16, and 12 cents as large, medium, and small rewards, respectively. These values were based on the average medium and small reward values derived from a pilot experiment. Moreover, for seven additional participants, the Pavlovian reward values deviated slightly (between 1 and 4 cents) from their choice-derived indifference values due to a technical error. As a robustness check, we reran our main Pavlovian and transfer models with a subsample excluding these 14 participants. Results were largely similar to the full sample results; differences and their interpretation are discussed in detail in S4.
2.2.2. Instrumental conditioning
With respect to the PIT results, we first examined whether participants learned to perform the correct instrumental go and no-go actions over the course of the instrumental conditioning phase. Figure 3A shows the average instrumental learning trajectory, separately for go (good mushrooms) and no-go (bad mushrooms) trials. Instrumental learning was assessed using a model with response accuracy (correct/incorrect) as dependent variable, and trial type (go/no-go trial), trial number (continuous predictor), and their interaction as predictors. This model showed a significant effect of trial number (b = 0.35, 95% CI [0.09, 0.59]), with accuracy improving across trials. Consistent with Figure 3A, which suggests a stronger learning effect for no-go trials compared to go trials, we found a significant interaction between trial number and trial type (b TrialNr*GovsNo-Go = -0.40, 95% CI [-0.77, -0.03]), with accuracy improving significantly for no-go trials (b = 0.55, 95% CI [0.23, 0.85]) but not for go trials (b = 0.14, 95% CI [-0.18, 0.44]). This finding can be explained by a general tendency to give go responses at the start of the instrumental phase (a go bias), when participants were still unaware which mushrooms had to be collected or left behind. As the task progressed, participants learned to leave bad mushrooms behind, thus improving accuracy for the no-go trials, while accuracy was already high at the start for go trials and remained so over the course of the task. Figure 3A indeed shows that accuracy on both go and no-go trials converged to satisfactory levels toward the end of the instrumental phase, and there was no significant difference in accuracy between trial types during the last 10 trials of this phase (estimated M p_correct go trials = 0.81; estimated M p_correct no-go trials = 0.76; b GovsNo-Go = 0.30, 95% CI [-0.75, 1.35]). Please note that average accuracy was likely restricted around 75% because correct feedback was provided with a 75% probability (a phenomenon known as probability matching; Herrnstein, Reference Herrnstein1961; Vulkan, Reference Vulkan2000). Thus, it can be concluded that on average, participants learned to perform the more rewarded action over the course of the instrumental conditioning phase for both go and no-go trials. We did observe inter-individual differences in instrumental accuracy during the final 10 trials. As reported in detail in Supplementary material S5, we tested whether instrumental accuracy moderated the hypothesized transfer effects, hereby exploring the idea that external, task-irrelevant Pavlovian cues may exert stronger transfer effects on participants who are more uncertain about the appropriate instrumental response. However, we found no evidence for such moderation effects.
Results of the PIT task in Experiment 1. Panel A, C: Observed trial-by-trial probability (± 95% CI) of giving the correct instrumental response for go and no-go trials during the instrumental conditioning phase (A) and the transfer phase (C). Panel B: Post-PIT liking ratings of the Pavlovian cues (colored squares) as a function of the amount and delay associated with the cues. Panel D: Probability of giving a go response, p(go), aggregated over go and no-go trials, as a function of the amount and delay associated with Pavlovian cues presented in the background. See Supplementary Figure S6.1 for the effects of amount and delay separated by trial type (go and no-go trials). Panels B and D display estimated marginal means and 95% CIs.
2.2.3. Pavlovian conditioning
We assessed Pavlovian conditioning effects using a model with post-PIT liking ratings as dependent variable, and amount (small/medium/large), delay (immediate/delayed), and their interaction as predictors. Figure 3B displays the liking ratings as a function of amount and delay. Planned comparisons showed that all three levels of amount (hereafter referred to as large, medium, and small cues) differed significantly from each other in the expected direction: large cues (M L = 65.90) were rated higher than medium (M M = 50.90; b LvsM = 15.10, 95% CI [6.58, 22.40]) and small cues (M S = 40.80; b LvsS = 25.10, 95% CI [13.97, 36.50]), and medium cues were rated higher than small cues (b MvsS = 10.00, 95% CI [0.83, 19.70]). In contrast, we found no significant difference between cues associated with delayed (M D = 49.20) and immediate rewards (M I = 55.90; b DvsI = -6.70, 95% CI [-14.10, 0.96]), hereafter referred to as delayed and immediate cues, although the ratings differed in the expected direction. There was no significant interaction between amount and delay (b LvsM*DvsI = -0.58, 95% CI [-15.30, 15.30]; b LvsS*DvsI = 3.12, 95% CI [-17.70, 23.10]; b MvsS*DvsI = 3.70, 95% CI [-14.90, 24.20]). See Supplementary Table S7.1 for estimates of all planned comparisons. The Pavlovian conditioning procedure thus resulted in the hypothesized significant effect of reward amount, but not of reward delay, on the Pavlovian value assigned to cues.
Moreover, as predicted, the indifference pairs were reflected in the rating data, as there was no significant difference between ratings of cues associated with the delayed large (M DL = 63.00) versus immediate medium (M IM = 53.50) reward (indifference pair 1; b DLvsIM = 9.51, 95% CI [-3.17; 22.61]), nor between ratings of the delayed medium (M DM = 48.20) versus immediate small (M IS = 45.30) reward (indifference pair 2; b DMvsIS = 2.91, 95% CI [-9.40, 16.33]). It should be noted that although these results indicate no significant difference in liking ratings between these cues, this does not provide statistical support for equivalence of the liking ratings. We further explored this equivalence using non-preregistered Region of Practical Equivalence (ROPE) tests. The ROPE test forms a Bayesian analog to the frequentist equivalence test (Kruschke, Reference Kruschke2018; Lakens, Reference Lakens2017), as it involves defining a region of parameter values that we consider practically equivalent to the null value (e.g., practically equivalent to a difference in post-PIT liking ratings of 0). We subsequently evaluated how much of the posterior distribution of the effect of interest (i.e., the difference in post-PIT liking ratings between the two indifference pair cues) fell inside the ROPE. This allowed us to quantify the evidence for the null value, reflecting the certainty with which we can conclude that there is no meaningful difference. One way to evaluate this proportion is to accept the null value only if the 95% highest density interval (HDI) falls entirely within the ROPE (Kruschke, Reference Kruschke2018). A visual representation of this index can be found in Supplementary material S8. An alternative, more quantitative index implies evaluating the proportion of the whole posterior distribution that falls inside the ROPE, with a higher proportion indicating more support for equivalence (Makowski et al., Reference Makowski, Ben-Shachar, Chen and Lüdecke2019). As we consider such a quantitative index an informative metric, we report these results here (and in Supplementary material S8).
Our reasons for using the ROPE to quantify the evidence for the null value, instead of, for instance, the more commonly reported Bayes factors, are twofold. First, the ROPE test uses the posterior distribution to evaluate the evidence for a parameter value (i.e., the null value), which is generally robust to the choice of prior distribution (Kruschke, Reference Kruschke2013; Wagenmakers et al., Reference Wagenmakers, Lodewyckx, Kuriyal and Grasman2010). Bayes factors, in contrast, do not rely on the posterior distribution, but reflect the ratio of the marginal likelihoods of the null and alternative model. These marginal likelihoods are extremely sensitive to the choice of prior distribution, and using a weakly informative prior to convey a state of minimal prior knowledge will result in Bayes factors, but not posterior distributions, that are biased toward the null model (Kruschke and Liddell, Reference Kruschke and Liddell2018; Rouder et al., Reference Rouder, Morey, Speckman and Province2012; Schad et al., Reference Schad, Nicenboim, Bürkner, Betancourt and Vasishth2022; Tendeiro and Kiers, Reference Tendeiro and Kiers2019; Wagenmakers et al., Reference Wagenmakers, Lodewyckx, Kuriyal and Grasman2010). Since we did not have a strong theory regarding our prior distributions, we favored an approach that was more robust to the choice of prior distribution, and that allowed for the use of a weakly informative prior (as recommended by e.g., Kruschke and Liddell, Reference Kruschke and Liddell2018; Liao et al., Reference Liao, Midya and Berg2021; Makowski et al., Reference Makowski, Ben-Shachar, Chen and Lüdecke2019). Second, by visualizing the posterior distribution in relation to the ROPE, the ROPE test has been argued to convey information about the uncertainty of the parameter estimate (reflected by the width of the posterior distribution) in a more explicit and transparent manner compared to Bayes factors (Kruschke, Reference Kruschke2013).
We acknowledge that although the ROPE test is robust to the choice of prior distribution, it does strongly depend on the choice of ROPE limits. Evaluating the ROPE limits on the raw response scale (e.g., the probability of go responses or the post-PIT liking ratings) facilitated a careful consideration of these limits. For the preregistered ROPE test on go-responding that we report in the pooled data analyses, we used a conventional ROPE radius of Cohen’s d = 0.10, following Kruschke (Reference Kruschke2018). This ROPE radius translates to a ROPE of 1 ± 0.119 in odds ratios or 0.5 ± 0.045 on the probability scale, which we considered to be appropriate. In contrast, for the liking ratings, this conventional ROPE radius resulted in extremely narrow ROPEs, translating to an unstandardized difference in ratings of 2-3 points on a scale from 0 to 100.Footnote 4 As participants were unable to see the exact value of their rating, and because the observed ratings covered the entire scale (and were thus not restricted to a small proportion of the scale), we deemed this radius to be an overly strict equivalence criterion. Therefore, for the liking ratings, we slightly increased the ROPE width by using a Cohen’s d value of 0.15, resulting in ROPEs of 0 ± 4-5 points on the rating scale. To communicate the sensitivity of our results to the choice of ROPE limits, we show in Supplementary material S8 how the proportion of the posterior distribution that fell inside the ROPE varies as a function of varying ROPE radii. With a ROPE radius of Cohen’s d = 0.15, 18% of the posterior distribution of the first indifference pair, and 47% of the posterior distribution of the second indifference pair fell inside the ROPE. Thus, we did not find strong support for the equivalence in post-PIT liking ratings of either of the two indifference pairs, although more support was found for the equivalence of the second indifference pair.
2.2.4. Transfer test
To test our main transfer hypotheses, we ran a model with response (go/no-go) as dependent variable, and amount (small/medium/large), delay (immediate/delayed), trial type (go/no-go), and their interactions as predictors. Unsurprisingly, we found a significant effect of trial type, such that the probability of giving a go response was higher in go (M GoTrials = 0.83) than in no-go trials (M No-GoTrials = 0.17; b GovsNo-Go = 3.18, 95% CI [1.69, 4.50]). Figure 3D displays the probability of go responses as a function of amount and delay. In line with our hypothesis, the presence of large cues (M L = 0.57) increased the probability of giving a go response compared to medium cues (M M = 0.46; b LvsM = 0.45, 95% CI [0.11, 0.78]). However, the difference between large and small cues (M S = 0.47) and between medium and small cues was not significant (b LvsS = 0.44, 95% CI [-0.07, 0.96]; b MvsS = -0.01, 95% CI [-0.42, 0.43]). There was no significant interaction between amount and trial type, indicating that the effect of amount was not significantly different between go and no-go trials (b LvsM*GovsNo-Go = -0.04, 95% CI [-0.62, 0.49]; b LvsS*GovsNo-Go = 0.13, 95% CI [-0.62, 0.87]; b MvsS*GovsNo-Go = 0.17, 95% CI [-0.54, 0.91]). In contrast to the transfer effect of amount, we found no significant difference in go-responding between delayed (M D = 0.51) and immediate cues (M I = 0.49; b DvsI = 0.05, 95% CI [-0.41, 0.49]). This did not interact with trial type (b DvsI*GovsNo-Go = 0.03, 95% CI [-0.60, 0.62]). Furthermore, there was no significant interaction between amount and delay (b LvsM*DvsI = 0.06, 95% CI [-0.63, 0.69]; b LvsS*DvsI = 0.07, 95% CI [-0.80, 1.04]; b MvsS*DvsI = 0.01, 95% CI [-0.85, 0.91]). See Supplementary Table S7.4 for estimates of all planned comparisons (varying over levels of amount, delay, and trial type).
Finally, we found no significant difference between cues associated with members of an indifference pair (IP1: b DLvsIM = 0.51, 95% CI [-0.11, 1.15]; IP2: b DMvsIS = 0.02, 95% CI [-0.62, 0.68]). Again, we performed a ROPE test to further investigate the equivalence in go-responding between indifference pair cues. Following the preregistered ROPE test reported in the pooled data analyses, we used a ROPE radius based on a Cohen’s d value of 0.10, resulting in a ROPE of 1 ± 0.119 in odds ratios, and 0.5 ± 0.045 on the probability scale. As reported in detail in Supplementary material S8 and in line with the Pavlovian conditioning results, neither pair showed strong support for equivalence, although we found more support for the second indifference pair, with 11% and 45% of the posteriors of indifference pair 1 and 2, respectively, falling inside the ROPE.
In summary, our transfer results follow the Pavlovian conditioning results reported above, as we found support for the predicted effect of reward amount (albeit for only one out of three planned comparisons) but not for reward delay, and we found no significant differences between indifference pair cues.
2.2.5. Secondary analyses
In addition to the analyses and results described above, which we used to test our main hypotheses, we performed several secondary analyses. These analyses involved examining the pre-PIT liking ratings of the cues, the instrumental accuracy at the start of the transfer phase (also see Figure 3C displaying this), the possible extinction of Pavlovian cue–outcome associations over the course of the transfer phase, and participants’ choice behavior in the post-PIT delay discounting task. None of the results of these analyses raised any major concerns for the interpretation of our primary results; they are described in detail in Supplementary materials S9–S11.
We do discuss here the results of the Pavlovian contingency test, which showed a discrepancy between awareness of the cue–amount contingencies and the cue–delay contingencies. That is, average performance on the questions that assessed cue–amount contingency awareness was 85.33% (SD = 24.20%), with six out of 50 participants scoring at or below chance level. In contrast, average performance on the questions that assessed cue–delay contingency awareness was only 72.33% (SD = 30.04%), with eighteen out of 50 participants scored at or below chance level. Excluding participants who scored at or below chance level on these questions from our main Pavlovian and transfer analyses resulted in findings that were largely consistent with the full sample results reported here, although the transfer effect of amount was non-significant; a detailed reported can be found in Supplementary material S12.
2.3. Discussion experiment 1
Experiment 1 resulted in several core findings. First, as hypothesized, we observed that Pavlovian cues associated with large rewards acquired a higher Pavlovian value (assessed with post-PIT liking ratings) compared to cues associated with medium, and smaller cues. Second, in line with these differences in Pavlovian value, we found support for our transfer hypothesis that cues associated with large rewards increase instrumental go-responding compared to cues associated with medium rewards. Third, however, we did not find the hypothesized Pavlovian conditioning and transfer effects of the delay associated with Pavlovian cues, as cues associated with immediate (versus delayed) rewards did not acquire a significantly higher Pavlovian value and did not increase instrumental go-responding. Fourth and finally, with respect to the indifference pairs, we found no significant difference in Pavlovian value of cues associated with members of indifference pairs, and these cues did not differentially influence instrumental go-responding. However, as described earlier, the lack of a main Pavlovian and transfer effect of delay prevents us from drawing strong conclusions regarding the indifference pairs, which revolve around the trade-off between the effects of reward amount and reward delay.
The results of this experiment could be interpreted as evidence that only the amount, but not the delay of the reward associated with a Pavlovian cue influenced instrumental action. Our Pavlovian conditioning procedure, however, points toward a potential alternative explanation for these results. During our Pavlovian conditioning procedure, the reward amounts associated with the cues were presented in a descriptive and explicit manner, as the reward amount was presented in text on the screen. In contrast, the delays associated with the Pavlovian cues were experiential, i.e., participants experienced a 10-s blank screen delay without any written information about the delay duration. However, the lack of explicit delay duration information may have reduced the salience of the cue–delay association compared to the cue–amount association. Consistent with this explanation, participants showed substantially lower cue–delay than cue–amount contingency awareness. Several studies have found transfer effects to be dependent on Pavlovian cue–outcome contingency awareness (e.g., Hogarth et al., Reference Hogarth, Dickinson, Wright, Kouvaraki and Duka2007; Jeffs and Duka, Reference Jeffs and Duka2017; Talmi et al., Reference Talmi, Seymour, Dayan and Dolan2008; Trick et al., Reference Trick, Hogarth and Duka2011). Hence, the asymmetry in presentation, and the resultingly relatively low salience of the cue–delay association, may explain why we observed the hypothesized effects of the amount, but not the delay associated with the Pavlovian cues. In a second, preregistered experiment, we therefore aimed to increase the salience of the cue–delay association by making two adjustments to the Pavlovian conditioning procedure. First, we removed the asymmetry in presentation by presenting both the amount and delay associated with the Pavlovian cues in an explicit (i.e., written) manner. Second, we used longer delays (50 days) and larger rewards (up to €20), as these have been more typically used in studies reporting temporal discounting effects (da Matta et al., Reference da Matta, Gonçalves and Bizarro2012; Scheres et al., Reference Scheres, de Water and Mies2013).
3. Experiment 2
3.1. Method
The sampling strategy, design, hypotheses, and analyses of this experiment were preregistered at Open Science Framework. The preregistration is available at https://osf.io/rz6uf/.
3.1.1. Participants
Seventy-two participants took part in Experiment 2, one of whom was excluded from the analyses because no indifference pairs could be derived from the pre-PIT delay discounting task (see below). The final sample thus consisted of 71 participants (45 female, 26 male, M age = 24.39, SD age = 6.77). We preregistered to recruit a minimum of 50 participants (i.e., the sample size of Experiment 1) and to subsequently continue collecting data using a Bayesian precision-based stopping rule (Kruschke, Reference Kruschke2015). This stopping rule takes as stopping criterion the precision with which the effects of interests can be estimated (i.e., the width of the 95% credible interval). Thus, it is the precision, not the size, direction, or significance of the point estimate, that determines the decision to stop sampling. The advantage of such a precision-based stopping rule over the more traditional power analyses is that it does not depend on the a priori specification of an effect size. Experiment 1 found no significant transfer effect of delay, and the goal of Experiment 2 was to design it in a way to increase the magnitude of this potential effect. As a consequence, we did not have a relevant effect size estimate to use as a basis for an informative a priori power analysis. Therefore, we adopted a sampling strategy that is based on estimation precision rather than the probability of detecting a certain effect. The precision criterion was determined a priori, by adopting the average CI width (0.45) obtained in Experiment 1 for the transfer effects of amount and delay.Footnote 5 We thus planned to test a minimum of 50 participants and continue testing until our precision criterion was reached, unless financial or time constraints would be exceeded before reaching this criterion (see preregistration for details). Eventually, our precision criterion was reached after testing 72 participants.
Similar to Experiment 1, participants were recruited via the local research participation system. Inclusion criteria were a sufficient proficiency in English, normal or corrected-to-normal vision, good color vision, and the ability to use a computer keyboard and mouse (as indicated by self-report). The study fell under a research line that received ethical approval from the local institutional review board prior to data collection (number ECSW-2019-153), and written informed consent was obtained from all participants. Participation was compensated with €10 or 1 course credit. In addition, participants could win a performance-contingent bonus (up to €4.50) and a lottery prize (up to €20).
3.1.2. Materials and procedure
3.1.2.1. General procedure
The experimental setup and general procedure were similar to Experiment 1, except that the delay discounting task was implemented in Qualtrics (version February 2020; https://www.qualtrics.com). The remaining tasks were implemented in Python 3.6, with the main components executed in PsychoPy (Peirce, Reference Peirce2007). The complete experiment took approximately one hour. The majority of the tasks in Experiment 2 were identical to those used in Experiment 1. Below, we only report the tasks that were different.
3.1.2.2. Pre-PIT delay discounting task
As the delay discounting task used in Experiment 1 led to technical issues for some participants, we used a slightly different procedure for Experiment 2. Instead of presenting choices one by one, we adopted a titration procedure in which choice options were presented in rows (all simultaneously presented on the screen), with each row representing one binary choice (see Supplementary material S2 for a visual depiction of the procedure). The task again consisted of two blocks, each serving to derive one indifference pair. In the first block, participants were presented with a series of choices between an immediate reward (SS) ranging from €2 to €20, increasing by steps of €2, versus a constant delayed reward of €20 in 50 days (LL). The indifference value was computed by taking the mean of the smallest SS reward that the participant preferred over the LL reward, and the SS reward that preceded this SS (i.e., that was presented on the preceding row). Thus, if a participant first chose the SS when it reached €14, the indifference value would be €13, i.e., the mean of €14 and €12. This procedure resulted in a first indifference pair formed by a delayed large (€20 in 50 days) versus immediate medium (e.g., €13 now) reward. Similar to Experiment 1, the reward amount of the immediate member of this indifference pair was subsequently taken as delayed reward amount in the second block. This resulted in a second indifference pair formed by a delayed medium (e.g., €13 in 50 days) versus an immediate small (e.g., €6 now) reward. Again, an immediate large (€20 now) and a delayed small (e.g., €6 in 50 days) reward were added, resulting in a total of six intertemporal rewards to be assigned to Pavlovian cues in the PIT task.
In order to make the task incentive-compatible, we used potentially real rewards (da Matta et al., Reference da Matta, Gonçalves and Bizarro2012; Scheres et al., Reference Scheres, de Water and Mies2013). Participants were instructed that at the end of the experiment, they would be entered into a lottery where they had a 1 in 50 chance of winning the reward chosen by them in one of the randomly selected trials of the pre- and post-PIT delay discounting task. If they had chosen an immediate reward on the selected choice trial, the monetary amount would be transferred to their bank account immediately, whereas if they had chosen a delayed reward, the monetary amount would be transferred after 50 days. One participant won the lottery and received €18 immediately.
3.1.2.3. PIT Task—Pavlovian conditioning phase
Figure 1B presents a graphic illustration of the PIT task used in Experiment 2. Each trial started with the presentation of a fixation cross in the center of the screen (1,000 ms), after which one of the six colored squares was presented for 1,500 ms. Afterward, the corresponding outcome, including amount and delay information (e.g., ‘20 euros in 50 days’, or ‘20 euros now’) was presented on the screen for 2,500 ms. We increased the duration of outcome presentation compared to Experiment 1 (going from 1,500 to 2,500 ms) to account for the additional information (about the reward delay) that was presented. Participants were instructed to pay attention to the amount and delay that corresponded to each picture. Identical to Experiment 1, each cue–reward association was presented 20 times, resulting in a total of 120 trials. Again, after each fifth trial, a query trial was presented in order to keep participants’ attention on the task.
3.1.2.4. Post-PIT delay discounting task
This task was identical in all aspects to the pre-PIT delay discounting task, except that the rewards had been increased by €1 to reduce memory effects and prevent participants from simply repeating their choices from the pre-PIT discounting task.
3.1.3. Data analysis
All data were analyzed using the same statistical models as in Experiment 1.
3.2. Results
3.2.1. Pre-PIT delay discounting task
Immediate medium rewards that were subjectively matched with a delayed large reward of €20 in 50 days (indifference pair 1) ranged between €7 and €19 (M = 14.82, SD = 3.08, Md = 15.0, IQR = 4.0), and immediate small rewards that were subjectively matched with a delayed medium reward that was identical in amount to the individualized immediate medium reward (indifference pair 2) ranged between €2 and €18 (M = 11.19, SD = 4.16, Md = 12.0, IQR = 6.0).
3.2.1.1. Deriving Pavlovian rewards
As described above, one participant was excluded from the analyses. This participant repeatedly switched between immediate and delayed rewards during the pre-PIT delay discounting, such that we could not derive any indifference values, nor assign any Pavlovian reward values that approximated the participant’s true preferences. As preregistered, we therefore excluded this participant from our analyses. Furthermore, nine participants were classified as non-discounters because they exclusively selected delayed rewards (except for trials in which the LL and SS amounts were equal, e.g., €20 now versus €20 in 50 days). In Experiment 1, we had assigned predefined Pavlovian reward values to such participants. In Experiment 2, in contrast, we directly used the indifference values derived from the delay discounting task as Pavlovian rewards, with the goal of approximating these participants’ choice-derived indifference values more closely.Footnote 6 A consequence of this approach, however, was that these participants had Pavlovian reward values that were close in magnitude (i.e., a small, medium, and large rewards of €18, €19, and €20, respectively). Furthermore, two other participants exclusively selected the sooner-smaller reward on all trials of the second task block, due to which we could not directly derive a small Pavlovian reward from their choices. These participants were assigned a small reward value of €2 (i.e., the smallest value among all the presented choice options).
As a robustness check, we reran our main Pavlovian and transfer analyses with a subsample excluding these 11 participants. As reported in the detail in Supplementary material S4, the results remained unchanged compared to the full sample results reported here.
3.2.2. Instrumental conditioning
Figure 4A displays the average instrumental learning trajectory during the instrumental conditioning phase, separately for go (good mushrooms) and no-go (bad mushrooms) trials. Similar to Experiment 1, our instrumental learning model showed instrumental accuracy to improve over the course of the phase (b = 0.27, 95% CI [0.12, 0.40]). In addition, Figure 4A seems to show a similar initial go bias and subsequently improved no-go performance. Supporting this observation, we found the effect of trial number to be significant for no-go trials (b = 0.36, 95% CI [0.13, 0.57]), but not for go trials (b = 0.17, 95% CI [-0.04, 0.39]). However, in contrast to Experiment 1, this difference in improvement over time between go and no-go trials was not significant, as indicated by a non-significant interaction between trial number and trial type (bTrialNr*GovsNo-Go = -0.19, 95% CI [-0.52, 0.14]). A potential explanation for this non-significant interaction is that accuracy on no-go trials improved relatively quickly, preventing a significant interaction when assessing accuracy across the entire phase. Consistent with Experiment 1, accuracy on both go and no-go trials converged to similar levels as the phase progressed. Correspondingly, there was no significant difference in accuracy between go and no-go trials during the last 10 trials (estimated M p_correct go trials = 0.78; estimated M p_correct no-go trials = 0.74; (b GovsNo-Go = 0.23, 95% CI [-0.75, 1.19]). Also consistent with Experiment 1, we observed inter-individual differences in instrumental accuracy during the last 10 trials of the instrumental conditioning phase, but these did not moderate the transfer results (see Supplementary material S5 for a detailed report).
Results of the PIT task in Experiment 2. Panel A, C: Observed trial-by-trial probability (± 95% CI) of giving the correct instrumental response for go and no-go trials during the instrumental conditioning phase (A) and the transfer phase (C). Panel B: Post-PIT liking ratings of the Pavlovian cues (colored squares) as a function of the amount and delay associated with the cues. Panel D: Probability of giving a go response, p(go), aggregated over go and no-go trials, as a function of the amount and delay associated with Pavlovian cues presented in the background. See Supplementary Figure S6.2 for the effects of amount and delay separated by trial type (go and no-go trials). Panels B and D display estimated marginal means and 95% CIs.
3.2.3. Pavlovian conditioning
Figure 4B displays post-PIT liking ratings of the Pavlovian cues as a function of the reward amount and delay associated with the cues. As hypothesized, and in line with the results from Experiment 1, large cues (M L = 68.80) were rated significantly higher than small cues (M S = 50.00; b LvsS = 18.81, 95% CI [11.06, 27.30]), and medium cues (M M = 63.70) were rated higher than small cues (b MvsS = 13.78, 95% CI [6.42, 22.00]). The difference between large and medium cues was, however, not significant (b LvsM = 5.04, 95% CI [-1.20, 11.00]). In contrast to Experiment 1, we additionally found the hypothesized effect of delay, as immediate cues (M I = 64.60) were rated significantly higher than delayed cues (M D = 57.00; b DvsI = -7.63, 95% CI [-13.00, -1.59]). There was no significant interaction between amount and delay (b LvsM*DvsI = -3.39, 95% CI [-14.70, 8.51]; b LvsS*DvsI = -2.28, 95% CI [-16.30, 13.62]; b MvsS*DvsI = 1.11, 95% CI [-14.40, 17.25]). See Supplementary Table S7.2 for estimates of the amount effect at all levels of delay, and vice versa.
Similar to Experiment 1, the indifference pairs were reflected in the post-PIT liking ratings, as we found no significant differences between liking ratings of cues associated with the delayed large (M DL = 64.00) versus immediate medium (M IM = 66.80) reward (indifference pair 1; b DLvsIM = -2.79, 95% CI [-12.75; 6.06 ]), nor between ratings of the delayed medium (M DM = 60.70) versus immediate small (M IS = 53.60) reward (indifference pair 2; b DMvsIS = 7.09, 95% CI [-2.32, 17.57]). A ROPE test showed moderate support for the subjective equivalence of cues associated with the first indifference pair, as 55% of the posterior distribution fell inside the ROPE, but weak support for the second pair, for which 24% of the posterior fell inside the ROPE (see Supplementary material S8 for details). This contrasts with Experiment 1, in which we observed stronger support for subjective equivalence of the second indifference pair.
3.2.4. Transfer test
First, we again observed a significant effect of trial type, with a higher probability of giving a go response in the go trials (M GoTrials = 0.75) than no-go trials (M No-GoTrials = 0.19; b GovsNo-Go = 2.57, 95% CI [1.72, 3.41]). Figure 4D displays the probability of go responses as a function of the reward amount and delay associated with the Pavlovian cues presented in the background. Contrasting with our hypothesis and with the results of Experiment 1, we found no effect of amount, neither for large versus small (M L = 0.46, M S = 0.44; b LvsS = 0.15, 95% CI [-0.24, 0.53]), large versus medium (M M = 0.45; b LvsM = 0.09, 95% CI [-0.17, 0.34]) nor medium versus small cues (b MvsS = 0.06, 95% CI [-0.26, 0.39]). We found no significant interaction between amount and trial type (b LvsM*GovsNo-Go = -0.09, 95% CI [-0.37, 0.53]; b LvsS*GovsNo-Go = -0.07, 95% CI [-0.77, 0.60]; b MvsS*GovsNo-Go = -0.15, 95% CI [-0.72, 0.42]).
Consistent with Experiment 1, we also found no significant effect of delay (M D = 0.45, M I = 0.45; b DvsI = -0.003, 95% CI [-0.36, 0.31]). This did not interact with trial type (b DvsI*GovsNo-Go = 0.03, 95% CI [-0.35, 0.43]). Moreover, there was no significant interaction between amount and delay (b LvsM*DvsI = -0.13, 95% CI [-0.63, 0.44]; b LvsS*DvsI = -0.02, 95% CI [-0.83, 0.77]; b MvsS*DvsI = 0.10, 95% CI [-0.60, 0.84]). See Supplementary Table S7.5 for estimates of all planned comparisons (varying over levels of amount, delay, and trial type).
Finally, similar to Experiment 1, we found no significant difference between cues associated with members of an indifference pair (IP1: M DL = 0.47, M IM = 0.44, b DLvsIM = 0.10, 95% CI [-0.37, 0.60]; IP2: M DM = 0.46, M IS = 0.44, b DMvsIS = 0.09, 95% CI [-0.43, 0.58]). A ROPE test showed moderate support for the equivalence in go-responding of both indifference pairs, as 55% and 54% of the posteriors of pair 1 and 2, respectively, fell inside the ROPE (see Supplementary material S8 for details).
3.2.5. Secondary analyses
We performed the same set of secondary analyses as in Experiment 1, reported in detail in Supplementary materials S9–S11. For the sake of comparison with Experiment 1, we again discuss the results of the Pavlovian contingency test here. Average performance on the questions that assessed cue–amount contingency awareness was 87.09% (SD = 21.31%). Four out of 71 participants scored at or below chance level. Average performance on the questions that assessed cue–delay contingency awareness was 86.62% (SD = 18.61%). Nine out of 71 participants scored at or below chance level. Again, we reran our main Pavlovian and transfer models excluding participants who scored at or below chance level on the test. Results from these analyses were largely consistent with the full sample results reported here, except that excluding participants who scored at or below chance level for the cue–delay contingencies resulted in a non-significant Pavlovian conditioning effect of reward delay (see Supplementary material S12 for details).
3.3. Discussion experiment 2
The goal of Experiment 2 was to enhance the potential effect of reward delay by increasing the salience of the Pavlovian cue–delay association. To this end, we adjusted our Pavlovian conditioning procedure by providing explicit information about the delay associated with the Pavlovian cues, and by using longer delays (and larger rewards). These adjustments successfully increased average cue–delay contingency awareness from 72% in Experiment 1 to 87% in Experiment 2. In line with this increase in contingency awareness, Experiment 2 showed that the Pavlovian value of the cues increased not only as a function of reward amount (albeit only for large versus small, and medium versus small rewards), but also as a function of reward delay, with higher values for immediate compared to delayed cues. Our design adjustments thus seemed to have had the intended effect on establishing the Pavlovian cue–delay associations. It should, however, be acknowledged that the increased sample size of Experiment 2 (N = 71) compared to Experiment 1 (N = 50) may have also contributed to a higher statistical power and thus the statistical significance of the Pavlovian conditioning effect of delay in Experiment 2. This is supported by the relatively similar effect sizes across the two experiments (i.e., unstandardized effect sizes of -6.70 in Experiment 1 versus -7.63 in Experiment 2), and the increased estimation precision (i.e., narrower credible interval) in Experiment 2. Thus, perhaps the combination of our design adjustments (leading to a modest increase in the effect size and/or a decrease in between-participant variation) and the increased sample size may have resulted in the significant Pavlovian conditioning effect of delay in Experiment 2.
The Pavlovian conditioning effect of delay was, however, not accompanied by a significant transfer effect of delay. The consistency of this result across both experiments suggests that the lack of a transfer effect in Experiment 1 could at least not be fully ascribed to our specific Pavlovian conditioning procedure, thereby eliminating this alternative explanation. In addition, we did not replicate the transfer effect of amount observed in Experiment 1, suggesting that this effect might not be very robust. Potential explanations for and implications of the results as found across both experiments are further discussed in the general discussion.
4. Pooled data analyses
As depicted in Figure 2, Experiments 1 and 2 generally resulted in relatively comparable estimates. Nevertheless, some discrepancies were found in terms of the statistical significance (e.g., the Pavlovian conditioning effect of delay) or size (e.g., the effects of amount) of the effects. Therefore, we next set out to examine the robustness of the results, as well as whether any differences in results between experiments 1 and 2 were statistically significant. The strong similarity between the PIT tasks used in both experiments allowed us to investigate this by pooling the data across the experiments. We subsequently ran our primary Pavlovian and transfer analyses on the pooled data, adding Experiment (Experiment 1/ Experiment 2) as predictor in our statistical models to investigate its interaction with our effects of interest. The pooled data analyses were preregistered as an amendment to Experiment 2’s preregistration (https://osf.io/5cdah).
A second reason for pooling the data across experiments is that this reduces the probability of Type-II errors (i.e., incorrectly deciding not to reject a false null hypothesis) by increasing the sample size and thus increasing statistical power. In Experiments 1 and 2, our main focus was on investigating the existence of a specific effect (the transfer effect) and accordingly, we focused on keeping Type-I errors (i.e., incorrectly deciding to reject a correct null hypothesis) nominal. Given that we demonstrated the absence of a statistically significant transfer effect of delay in both experiments with a nominal Type-I error level, we believe scholarly diligence now requires that we shift our attention more strongly toward safeguarding our conclusions also against the possibility of inflated Type-II errors, thus trying to minimize the chance that we are overlooking potentially existing effects.
Taken together, by increasing statistical power and examining the differences between the two experiments, we believe the pooled data analyses to result in more reliable estimates compared to experiments 1 and 2 separately, hereby increasing the confidence with which we can draw our general conclusions.
4.1. Results
4.1.1. Pavlovian conditioning—post-PIT liking ratings
Figure 5A displays post-PIT liking ratings of the Pavlovian cues as a function of the reward amount and delay associated with the cues. All three levels of amount differed significantly from each other: Large cues (M L = 67.20) were rated significantly higher than medium cues (M M = 57.30; b LvsM = 9.99, 95% CI [5.76, 14.40]) and small cues (M S = 45.30; b LvsS = 21.95, 95% CI [15.91, 28.30]), and medium cues (M M = 57.30) were rated higher than small cues (b MvsS = 11.96, 95% CI [6.91, 17.20]). We found no significant interactions between amount and experiment (1/2), indicating that the patterns of results found in both experiments were not significantly different (b LvsM*1vs2 = 9.89, 95% CI [-0.76, 20.90]; b LvsS*1vs2 = 6.34, 95% CI[-8.07, 21.90]; b MvsS*1vs2 = -3.55, 95% CI[-18.37, 11.20]). These results are in line with our hypotheses, and with the results found in experiments 1 and 2 (with the exception of the non-significant difference between large and medium cues in Experiment 2).
Pavlovian and transfer results in data pooled across experiments 1 and 2. Panel A: Post-PIT liking ratings of the Pavlovian cues (colored squares) as a function of the amount and delay associated with the cues, aggregated over Experiments 1 and 2. Panel B: Probability of making a go response, p(go), as a function of the amount and delay associated with Pavlovian cues presented in the background, aggregated over go and no-go trials and Experiments 1 and 2. See Supplementary Figure S6.3 for the effects of amount and delay separated for go and no-go trials. Panel A and B display estimated marginal means and 95% CIs. Panel C: Histogram of the posterior distribution of the transfer effect of delay, i.e., the probability of making a go response in the presence of a delayed versus immediate cue. Dashed vertical lines mark the ROPE limits around the null value (0.5 ± 0.045). The bold horizontal line marks the 95% highest density interval (HDI). Panel D: Proportion of the posterior distribution falling inside the ROPE as a function of the radius (half width) of the ROPE (on the probability scale). The dashed vertical line marks the ROPE radius at which the 95% HDI falls completely within the ROPE; the dashed horizontal line indicates the proportion of the whole posterior distribution that falls within the ROPE for this radius.
Consistent with Experiment 2, we found the hypothesized effect of delay, as immediate cues (M I = 60.20) were rated significantly higher than delayed cues (M D = 53.00; b DvsI = -7.25, 95% CI [-12.90, -1.95]). The effect of delay was not significantly different between the two experiments, as reflected by a non-significant interaction (b DvsI*1vs2 = 0.76, 95% CI [-7.12, 8.92]). This finding concords with the similarity of estimates across experiments 1 and 2 (see Figure 2).
There was no significant interaction between amount and delay (b LvsM*DvsI = -2.40, 95% CI [-10.82, 5.78]; b LvsS*DvsI = 0.06, 95% CI [-9.86, 10.24]; b MvsS*DvsI = 2.46, 95% CI [-7.87, 13.22]). See Supplementary Table S7.3 for estimates of the amount effect on all levels of delay, and vice versa.
Finally, in line with Experiments 1 and 2 and as predicted, the indifference pairs were reflected in the post-PIT liking ratings, as we found no significant differences between liking ratings of cues associated with the delayed large (M DL = 63.20) versus immediate medium (M IM = 60.10) reward (indifference pair 1; b DLvsIM = 3.16, 95% CI [-4.20; 10.89]), nor between ratings of the delayed medium (M DM = 54.40) versus immediate small (M IS = 49.30) reward (indifference pair 2; b DMvsIS = 5.10, 95% CI [-2.92, 12.82]). These differences did not interact with experiment (IP1: b LDvsMI*1vs2 = 11.77, 95% CI [-3.77, 27.67]; IP2: b MDvsSI*1vs2 = -3.74, 95% CI [-21.43, 14.02]). A ROPE test showed moderate support for the subjective equivalence of cues associated with the first indifference pair, as 60% of the posterior distribution fell inside the ROPE, but weaker support for the equivalence of the second indifference pair, for which 37% of the posterior fell inside the ROPE (see Supplementary material S8 for details).
4.1.2. Transfer test
Our transfer model again showed a significant effect of trial type, such that the probability of giving a go response was higher in go than in no-go trials (M GoTrials = 0.79, M No-GoTrials = 0.18; b GovsNo-Go = 2.84, 95% CI [2.04, 3.73]). Figure 5B displays the probability of go responses as a function of amount and delay. In line with Experiment 1, we found a significant transfer effect of amount, as the presence of large cues (M L = 0.52) increased the probability of giving a go response compared to medium cues (M M = 0.45; b LvsM = 0.27, 95% CI [0.06, 0.48]). This finding in the pooled data supports the robustness of the transfer effect of amount. The difference between large and small cues (M S = 0.45) and between medium and small cues was not significant (b LvsS = 0.28, 95% CI [-0.05, 0.59]; b MvsS = 0.01, 95% CI [-0.27, 0.27]). We found no significant interaction between amount and trial type (b LvsM*GovsNo-Go = -0.02, 95% CI [-0.35, 0.38]; b LvsS*GovsNo-Go = -0.04, 95% CI [-0.51, 0.55]; b MvsS*GovsNo-Go = -0.02, 95% CI [-0.43, 0.46]), nor between amount and experiment (b LvsM*1vs2 = 0.37, 95% CI [-0.04, 0.74]; b LvsS*1vs2 = 0.27, 95% CI [-0.32, 0.86]; b MvsS*1vs2 = -0.10, 95% CI [-0.59, 0.40]).
Furthermore, consistent with Experiments 1 and 2, we found no significant difference in go-responding between delayed (M = 0.48) and immediate (M = 0.47) cues (b DvsI = 0.02, 95% CI [-0.25, 0.28]). This did not interact with trial type (b DvsI*GovsNo-Go = 0.02, 95% CI [-0.31, 0.32]) or experiment (b DvsI*1vs2 = 0.06, 95% CI [-0.54, 0.63]). Moreover, we found no significant interaction between amount and delay (b LvsM*DvsI = -0.03, 95% CI [-0.43, 0.36]; b LvsS*DvsI = 0.002, 95% CI [-0.52, 0.52]; b MvsS*DvsI = -0.49, 95% CI [-0.49, 0.61]). See Supplementary Table S7.6 for estimates of all planned comparisons (varying over levels of amount, delay, and trial type).
Finally, in line with experiments 1 and 2, there was no significant difference between cues associated with members of an indifference pair (IP1: b DLvsIM = 0.29, 95% CI [-0.07, 0.67]; IP2: b DMvsIS = 0.03, 95% CI [-0.36, 0.41]). These differences did not interact with experiment (IP1: b DLvsIM*1vs2 = 0.43, 95% CI [-0.40, 1.22]; IP2: b DMvsIS*1vs2 = -0.08, 95% CI [-0.89, 0.79]). The ROPE test showed more support for the equivalence in go-responding of the second compared to the first indifference pair, with moderate support for equivalence in the second pair, as 68% of the posterior fell inside the ROPE, but weak support for the first pair, for which 25% of the posterior fell inside the ROPE (see Supplementary material S8 for details).
4.1.2.1. ROPE test delay effect
None of the transfer analyses reported throughout this paper showed a significant transfer effect of delay. However, the mere absence of a significant effect does not provide statistical support for the null value. Therefore, we additionally conducted a preregistered ROPE test to investigate to what extent we can be confident that there is no meaningful effect. Aiming to minimize any potential bias in setting the ROPE resulting from the fact that we had already inspected the results from Experiments 1 and 2, we preregistered a conventional ROPE radius of Cohen’s d = 0.10, following Kruschke (Reference Kruschke2018). We converted this value to an odds ratio radius of 0.199 to account for the binomial distribution underlying our dependent variableFootnote 7 and subsequently converted this this to a radius of 0.045 on the probability scale to facilitate interpretation of the results. Thus, the ROPE was composed of all values between 0.5 ± 0.045.
Figure 5C displays the posterior distribution of the transfer effect of delay (b DvsI). Visual inspection of this figure shows that the 95% HDI did not fall entirely within the ROPE. Thus, according to the decision rule proposed by Kruschke (Reference Kruschke2018), one cannot accept the null value. However, Figure 5C also shows that 86% of the whole posterior distribution fell inside the ROPE, showing that the data provide relatively strong support for the absence of a meaningful transfer effect of delay. As explained above, we used a conventional ROPE radius. Whereas we believe this radius to be reasonable for our response variable, others may believe a different radius to be more appropriate (either now or in the future). Therefore, Figure 5D displays how much of the posterior falls inside various ROPEs, as a function of the respective ROPE radii. This allows readers to decide for themselves to what extent the null value is supported by the data.
5. General discussion
The present study investigated the role of Pavlovian-to-instrumental transfer (PIT) as possible mechanism contributing to impatient intertemporal choice. We developed an intertemporal variant of a classic three-phase PIT task to investigate the Pavlovian effects of the amount and delay attributes of intertemporal rewards on instrumental go/no-go behavior. We conducted two experiments to test our six study hypotheses (described below), and additionally tested the hypotheses after pooling the data of both experiments. Given the increased reliability of the pooled data estimates (compared to the single-experiment estimates), as well as the relatively similar estimates across the two experiments, we draw our main conclusions regarding the hypotheses from these pooled results. Nevertheless, any discrepancies between the two experiments, as well as potential explanations for these discrepancies, are discussed.
In the instrumental conditioning phase of the PIT task, participants learned to make go and no-go responses in order to win rewards and/or avoid punishments. We found that on average, participants showed satisfactory instrumental accuracy for both go and no-go trials by the end of this phase. Next, the Pavlovian conditioning phase served to establish associations between Pavlovian cues and rewards varying in their amount and delay. We hypothesized that as a result of Pavlovian conditioning, cues associated with large rewards would acquire higher Pavlovian values compared to cues associated with smaller rewards (H1). This hypothesis was confirmed (although Experiment 2 alone did not show a statistically significant difference between large and medium cues). Furthermore, we hypothesized that cues associated with immediate rewards would acquire higher Pavlovian values than cues associated with delayed rewards (H2). The pooled data analyses supported this hypothesis, and showed no significant differences between the effect observed in Experiments 1 and 2. Results from Experiment 1 alone, however, did not support this hypothesis, although the observed difference was in the expected direction. In order to enhance the potential effect, we increased the salience of the cue–delay association in Experiment 2 by providing explicit information on the delay duration and by using larger delays (and amounts). This resulted in the hypothesized statistically significant effect of delay on the Pavlovian value of the cues in Experiment 2. The results from the pooled data analyses, as well as the similar effect sizes across Experiments 1 and 2, suggest that the pattern of findings across the two experiments are comparable, and that pooling the data allowed us to detect effects that were present but possibly too subtle to be statistically significant in each separate experiment. Finally, in the transfer phase, we examined whether the reward and delay associated with the Pavlovian cues exerted transfer effects on instrumental go/no-go behavior. We found partial support for the hypothesis that cues associated with large rewards increase go-responding compared to cues associated with smaller rewards (H3), as we observed a significant difference between large and medium cues in the pooled data analyses and in Experiment 1. This effect did not interact with trial type (go versus no-go trials), indicating that the effect was not restricted to either enhancing approach (go responses) or interfering with withdrawal (no-go responses). The transfer effect of amount was not observed in Experiment 2, although the pooled data analyses showed no statistically significant differences between the two experiments. Our study did not provide support for the transfer hypothesis that cues associated with immediate rewards increase go-responding compared to cues associated with delayed rewards (H4). The Region of Practical Equivalence (ROPE) analysis conducted on our pooled data provides additional support against a transfer effect of delay. Together, these results suggest that, at least in the experimental task paradigm we used, the magnitude of the reward associated with a Pavlovian cue is more important than the reward delay in biasing instrumental action.
Finally, we investigated the potential role of PIT in driving the effect of immediacy while controlling for differences in subjectively discounted value. We examined this by testing the Pavlovian properties of cues associated with members of individually derived indifference pairs (i.e., indifference cues). Consistent with our Pavlovian indifference hypothesis (H5), we observed no significant difference in Pavlovian value between the indifference cues. This result is in line with the choice indifference observed during the delay discounting task, and contrasts with theories proposing that the immediate member of a choice-derived indifference pair is valued more than the delayed member of the pair when the rewards are evaluated in isolation (Figner et al., Reference Figner, Knoch, Johnson, Krosch, Lisanby, Fehr and Weber2010; Luo et al., Reference Luo, Ainslie, Giragosian and Monterosso2009). It should be noted, however, that some considerable, albeit non-significant, differences in ratings between indifference cues were found (i.e., mainly for indifference pair 1 in Experiment 1). In addition, ROPE tests provided only weak support for subjective equivalence of these cues. In these cases, however, it was the delayed instead of the immediate member of the pair that acquired a higher value, contrasting with the theory by Luo et al. (Reference Luo, Ainslie, Giragosian and Monterosso2009) that proposes the immediate member to have a higher value. In the transfer phase, we observed no significant difference in go-responding between the indifference pair cues. This latter finding would be in support of the Transfer indifference hypothesis (H6a). However, as discussed before, we would only be able to strongly interpret these findings under the condition of observing the hypothesized transfer effects of amount and delay separately, allowing us to examine the trade-off between these effects. Since we did not find a transfer effect of delay, we refrain from drawing any strong conclusions about our two indifference pair hypotheses (H6a and H6b).
5.1. Transfer effect of amount
The transfer effect of large versus smaller Pavlovian cues replicates previous research showing that Pavlovian cues with higher absolute value exert stronger transfer effects than cues with lower absolute value (Chen et al., Reference Chen, Schlagenhauf, Sebold, Kuitunen-Paul, Chen, Huys, Heinz, Smolka, Zimmermann and Garbusow2023; Garbusow et al., Reference Garbusow, Schad, Sebold, Friedel, Bernhardt, Koch, Steinacher, Kathmann, Geurts, Sommer, Müller, Nebe, Paul, Wittchen, Zimmermann, Walter, Smolka, Sterzer, Rapp, Huys, Schlagenhauf and Heinz2016, Reference Garbusow, Nebe, Sommer, Kuitunen-Paul, Sebold, Schad, Friedel, Veer, Wittchen, Rapp, Ripke, Walter, Huys, Schlagenhauf, Smolka and Heinz2019; Huys et al., Reference Huys, Cools, Gölzer, Friedel, Heinz, Dolan and Dayan2011; Schad et al., Reference Schad, Rapp, Garbusow, Nebe, Sebold, Obst, Sommer, Deserno, Rabovsky, Friedel, Romanczuk-Seiferth, Wittchen, Zimmermann, Walter, Sterzer, Smolka, Schlagenhauf, Heinz, Dayan and Huys2020; Sommer et al., Reference Sommer, Garbusow, Jünger, Pooseh, Bernhardt, Birkenstock, Schad, Jabs, Glöckler, Huys, Heinz, Smolka and Zimmermann2017, Reference Sommer, Birkenstock, Garbusow, Obst, Schad, Bernhardt, Huys, Wurst, Weinmann, Heinz, Smolka and Zimmermann2020). We found this transfer effect to be restricted to the difference between large and medium cues. Previous studies, however, only tested for an effect of amount as a linear predictor, instead of testing between all amount levels in a pairwise manner, hereby complicating the comparison with our findings. Future research could benefit from performing pairwise comparisons between reward levels (for example in addition to testing for linear effects), as this would allow for a closer examination of the observed parametric effects.
Although the transfer effect of amount was observed in Experiment 1 and in the pooled data analyses, it was not observed in Experiment 2. A potential explanation for this revolves around the fact that, to the best of our knowledge, no previous PIT studies have coupled Pavlovian cues to more than one attribute. In our study, however, each cue was coupled to an amount and delay attribute, because this combination is inherent to intertemporal outcomes. This implies that during the transfer phase, the cues were assumed to simultaneously elicit both attributes. Associative memory research has found that if a cue is associated with multiple attributes, the simultaneously elicited associations may compete and interfere with each other, known as a fan effect (Anderson, Reference Anderson1974; Anderson and Reder, Reference Anderson and Reder1999). Although fan effects have never been described in relation to PIT, it is tempting to speculate that the competing associations in our study may have attenuated each other’s effect on instrumental behavior. This could explain why we found a transfer effect in Experiment 1 but not in Experiment 2. That is, participants’ relatively low cue–delay contingency awareness in Experiment 1 may have reduced the competition between the cue–delay and cue–amount associations. In other words, the cue–amount association may have suffered from less interference by the cue–delay association, allowing it to exert a transfer effect. The increase in cue–delay contingency awareness in Experiment 2 may have increased the competition between the two associations, attenuating the transfer effect of amount. Similarly, any potential transfer effect of delay may have been attenuated as a result of its competition with amount. Whereas this alternative explanation remains speculative, future research could examine this by keeping one of the intertemporal choice attributes constant. In the current study, we refrained from this approach in order to capture the trade-off between amount and delay that is crucial to intertemporal choice. However, focusing on only one of the two attributes might allow for a maximization of any possible transfer effects of amount or delay separately.
5.2. Transfer effect of the delay
The absence of a transfer effect of delay does not support our hypothesis that cues associated with immediate rewards elicit a stronger Pavlovian approach response and hereby increase instrumental approach and interfere with instrumental responding more strongly compared to cues associated with delayed rewards. The results from Experiment 2 allow us to exclude the possibility that the absence of the transfer effect in Experiment 1 was caused by the presentation of the delay information during the Pavlovian conditioning phase (experiential versus explicit), or by the scale of the reward delays (seconds versus days) and amounts (cents versus euros) used. Our results thus provide evidence against the idea that PIT effects driven by reward delay form a crucial contributing factor in impatient intertemporal choice, suggesting that we should consider alternative theoretical frameworks that might help explain impatient intertemporal choice. Nevertheless, we wish to be careful to avoid sweeping generalizations, as alternative explanations for this null result exist that are worth discussing. These alternative explanations provide important directions for future research, as studying them would allow us to discern whether we should move our attention away from PIT as underlying mechanism, or whether our null results were possibly caused by our choice of research paradigm or other methodological aspects.
One potential explanation was discussed above, concerning the attenuation of transfer effects as a result of the competition between the cue–amount and cue–delay associations. Research by Ebert and Prelec (Reference Ebert and Prelec2007) suggests that the cue–delay association may be particularly vulnerable to attenuation in choice tasks with restricted response windows, such as the go/no-go task that we used. This would suggest that using an unlimited response window might reduce this attenuation effect. At the same time, this would likely have the undesirable side effect of decreasing any PIT effects by allowing participants to carefully consider the appropriate instrumental response before responding.
A second alternative explanation revolves around the distinction between outcome-specific and outcome-general PIT (Cartoni et al., Reference Cartoni, Balleine and Baldassarre2016). Outcome-specific PIT occurs when a Pavlovian cue only enhances instrumental responding toward a reward that is identical to the reward signaled by the Pavlovian cue. Outcome-general PIT instead refers to the ability of a Pavlovian cue to enhance (or inhibit, depending on the Pavlovian value) also instrumental behavior toward a reward different than that signaled by the Pavlovian cue. In the current study, both Pavlovian and instrumental rewards were monetary, making them similar to some extent. However, only the Pavlovian rewards were intertemporal, i.e., varied in amount and delay. Hence, our study tested whether Pavlovian cues signaling immediate rewards induced a general approach bias in instrumental behavior directed toward non-intertemporal monetary rewards (also see Watson and Mahlberg, Reference Watson and Mahlberg2023, for evidence that the type of PIT task we used assesses outcome-general PIT effects). No support was found for such effects. However, this does not exclude the possibility that cues signaling immediate rewards do have a specific effect on instrumental behavior toward intertemporal rewards. One possible approach to examining this is by adopting a PIT paradigm that uses identical intertemporal outcomes in the instrumental and Pavlovian conditioning phase of the PIT task. This could be implemented by using an instrumental go/no-go task that is similar to that used in the current study, except that some instrumental cues result in an immediate reward upon a correct response, and others result in a delayed reward. Alternatively, one could make both an immediate and delayed reward available on each trial (which can be accessed, for instance, using the left and right keys), and assess the effect of the Pavlovian cues on preference for either of the two rewards (see, e.g., Watson et al., Reference Watson, Wiers, Hommel and De Wit2014, for an example of this approach). These approaches allow one to test, for instance, whether Pavlovian cues signaling immediate rewards selectively increase go-responding or preference toward immediate rewards or not, and whether Pavlovian cues signaling delayed rewards may also increase go-responding or preference toward delayed rewards. Outcome-specific PIT effects have been more frequently reported compared to outcome-general PIT effects (da Costa et al., Reference da Costa, Furukawa, Hoefle, Moll, Tripp and Mattos2020). Moreover, such effects may more closely resemble daily-life situations in which a cue signaling an immediate reward (e.g., a fast-food sign) biases one toward intertemporally impatient behavior (e.g., eating a snack). Therefore, this may form a relevant avenue for future research. However, outcome-specific PIT effects have also been argued to result from participants’ verbalizable beliefs about which action should be performed upon presentation of the cue, rather than the associative responses that are thought the underlie Pavlovian biases (Mahlberg et al., Reference Mahlberg, Seabrooke, Weidemann, Hogarth, Mitchell and Moustafa2021).
An alternative paradigm exists that can be used to examine the role of Pavlovian influences on instrumental behavior directed toward intertemporal rewards, known as orthogonalized or motivational go/no-go learning task (Guitart-Masip et al., Reference Guitart-Masip, Duzel, Dolan and Dayan2014; Swart et al., Reference Swart, Cook, Geurts, Frank, Cools and den Ouden2017). Instead of testing for the effect of external, task-irrelevant Pavlovian cue–outcome associations taught in a separate Pavlovian conditioning phase, it is used to test for the effects of existing Pavlovian response tendencies on instrumental learning. The task consists of four trial types that orthogonalize the required instrumental action (go/no-go) and the available instrumental outcome (e.g., winning a reward / avoiding a punishment), each signaled by a unique instrumental cue. Studies adopting this paradigm have found that the valence of the instrumental outcome, as signaled by the cue, biases instrumental go/no-go learning in a manner that reflects our Pavlovian response tendencies to approach reward-predictive cues and withdraw from punishment-predictive cues. That is, the anticipation of rewards (in go-to-win-reward and no-go-to-win-reward trials) has been found to increase instrumental go-responding and interfere with instrumental no-go-responding, whereas the anticipation of punishments (in go-to-avoid-punishment and no-go-to-avoid-punishment trials) has the opposite effect (Guitart-Masip et al., Reference Guitart-Masip, Duzel, Dolan and Dayan2014). By replacing rewards and punishments with immediate and delayed rewards, future research could examine whether a similar mechanism may contribute to impatient intertemporal choice: The anticipation of immediate rewards may increase instrumental go responses and interfere with instrumental no-go responses, whereas the anticipation of delayed rewards may have the opposite effect.
5.3. Study limitations
The current study carries some limitations. First, in contrast to the instrumental rewards (of +5/-5 cents), the Pavlovian rewards presented during the Pavlovian conditioning phase were not paid out to participants. Instead, participants were instructed to pay attention to the rewards that were associated with each cue. Previous PIT studies that observed transfer effects diverged in whether they paid out all (e.g., Garbusow et al., Reference Garbusow, Schad, Sebold, Friedel, Bernhardt, Koch, Steinacher, Kathmann, Geurts, Sommer, Müller, Nebe, Paul, Wittchen, Zimmermann, Walter, Smolka, Sterzer, Rapp, Huys, Schlagenhauf and Heinz2016, Reference Garbusow, Nebe, Sommer, Kuitunen-Paul, Sebold, Schad, Friedel, Veer, Wittchen, Rapp, Ripke, Walter, Huys, Schlagenhauf, Smolka and Heinz2019; Garofalo and di Pellegrino, Reference Garofalo and di Pellegrino2015; Huys et al., Reference Huys, Cools, Gölzer, Friedel, Heinz, Dolan and Dayan2011; Jeffs and Duka, Reference Jeffs and Duka2017; Schad et al., Reference Schad, Rapp, Garbusow, Nebe, Sebold, Obst, Sommer, Deserno, Rabovsky, Friedel, Romanczuk-Seiferth, Wittchen, Zimmermann, Walter, Sterzer, Smolka, Schlagenhauf, Heinz, Dayan and Huys2020; Sebold et al., Reference Sebold, Garbusow, Cerci, Chen, Sommer, Huys, Nebe, Rapp, Veer, Zimmermann, Smolka, Walter, Heinz and Friedel2021), a selection (e.g., Verhoeven et al., Reference Verhoeven, Watson and de Wit2018; Watson et al., Reference Watson, Wiers, Hommel and De Wit2014), or none (Cartoni et al., Reference Cartoni, Moretta, Puglisi-Allegra, Cabib and Baldassarre2015; Nadler et al., Reference Nadler, Delgado and Delamater2011; Watson and Mahlberg, Reference Watson and Mahlberg2023) of the Pavlovian rewards, suggesting that paying out Pavlovian rewards is not crucial to finding transfer effects. A difference between the studies that paid out all Pavlovian outcomes and our study is that the former studies included both Pavlovian gains and losses. By presenting an equal number of gains and losses, the net Pavlovian outcome that had to be paid out was always €0. As our Pavlovian conditioning phase only included rewards, the payout would have strongly exceeded our research budget. Nevertheless, a potential consequence of not paying out Pavlovian rewards is that participants may have experienced these rewards as less rewarding compared to if the rewards had been real. This may have weakened the Pavlovian conditioning process, reducing the influence of the Pavlovian cues on instrumental responding during the transfer phase, leading participants to focus more strongly on the incentive-compatible instrumental outcomes. This explanation cannot, however, account for the fact that Pavlovian conditioning phase successfully influenced the Pavlovian value participants assigned to the cues. Moreover, Experiment 1 showed Pavlovian and transfer effects of amount but not of delay, while it was only the delay that was experienced during the Pavlovian conditioning phase. This again suggests that paying out Pavlovian rewards was not crucial to finding the hypothesized effects. Nevertheless, it would be interesting to investigate whether the Pavlovian and transfer effect would be stronger if the Pavlovian rewards were paid out. Considering a limited research budget, this could be implemented by paying out a random selection of Pavlovian conditioning trials, or by including a lottery as we did for the delay discounting task in Experiment 2.
A second limitation is that, in contrast to earlier PIT studies (Chen et al., Reference Chen, Schlagenhauf, Sebold, Kuitunen-Paul, Chen, Huys, Heinz, Smolka, Zimmermann and Garbusow2023; Garbusow et al., Reference Garbusow, Schad, Sebold, Friedel, Bernhardt, Koch, Steinacher, Kathmann, Geurts, Sommer, Müller, Nebe, Paul, Wittchen, Zimmermann, Walter, Smolka, Sterzer, Rapp, Huys, Schlagenhauf and Heinz2016, Reference Garbusow, Nebe, Sommer, Kuitunen-Paul, Sebold, Schad, Friedel, Veer, Wittchen, Rapp, Ripke, Walter, Huys, Schlagenhauf, Smolka and Heinz2019; Huys et al., Reference Huys, Cools, Gölzer, Friedel, Heinz, Dolan and Dayan2011; Schad et al., Reference Schad, Rapp, Garbusow, Nebe, Sebold, Obst, Sommer, Deserno, Rabovsky, Friedel, Romanczuk-Seiferth, Wittchen, Zimmermann, Walter, Sterzer, Smolka, Schlagenhauf, Heinz, Dayan and Huys2020; Sommer et al., Reference Sommer, Garbusow, Jünger, Pooseh, Bernhardt, Birkenstock, Schad, Jabs, Glöckler, Huys, Heinz, Smolka and Zimmermann2017, Reference Sommer, Birkenstock, Garbusow, Obst, Schad, Bernhardt, Huys, Wurst, Weinmann, Heinz, Smolka and Zimmermann2020; Watson and Mahlberg, Reference Watson and Mahlberg2023), we did not include neutral Pavlovian cues, i.e., Pavlovian cues associated with neutral outcomes (€0). Such cues would have allowed for a comparison of instrumental responding in the presence of intertemporal reward-predictive Pavlovian cues to a baseline measure of instrumental responding, while controlling for the effects of a visual cue presentation. In our study, we observed that Pavlovian cues associated with immediate rewards did not significantly increase instrumental approach behavior compared to Pavlovian cues associated with delayed rewards. However, given that both types of cues predict rewards (that only differ in their delay until delivery), each of them may increase instrumental approach compared to baseline. Since many of the environments we find ourselves in (e.g., train stations) contain ample cues signaling the availability of immediate rewards, the Pavlovian approach responses elicited by these cues may still contribute to impatient intertemporal choice behavior. Importantly, however, these effects would not be driven by the temporal aspect of the rewards associated with these cues. Because such PIT effects were not of core interest to our study, we refrained from using neutral Pavlovian cues. We also wish to point out that because we only included cues associated with rewards, and not with losses, any Pavlovian cues associated with €0 may have been considered relatively aversive. Therefore, instrumental responding in the presence of such cues may thus not have served as an appropriate baseline measure. An alternative solution would be to include transfer trials without any Pavlovian cues, although this does not control for effects driven by the presentation of a visual cue. Forming a proxy to such a baseline measure, we compared instrumental responding during the final 10 trials of the instrumental phase (during which no Pavlovian cues were presented) and the first 10 trials of the transfer phase (during which the Pavlovian cues were presented). No difference in responding was found, suggesting that, averaged across cues and during the first 10 trials of the transfer phase, the Pavlovian cues did not influence instrumental responding compared to baseline.
5.4. Future directions
In addition to the research directions following from the discussion above, we wish to suggest two potential avenues for future research. First, orthogonalizing the required instrumental action (go/no-go) and the instrumental outcome (winning a reward / avoiding a punishment) would allow one to discern whether go and no-go responses are motivated by the prospect of winning rewards or avoiding punishments. This could be achieved with a PIT task with four instrumental trial types: go-to-win-reward, no-go-to-avoid-punishment, no-go-to-win-reward, and go-to-avoid-punishment.Footnote 8 Importantly, incorrect responses on go-to-win and no-go-to-win trials, as well as correct responses on go-to-avoid and no-go-to-avoid trials, should result in neutral outcomes. This ensures that responses are made either to win rewards, or avoid punishments, not both simultaneously. Since this task design would have substantially increased the complexity and duration of our PIT paradigm, and because the action-valence orthogonalization was not of primary interest, we adopted a simplified version of the task that we believed to be well suited for the goal of our study. Nevertheless, adopting this four-block design allows for a test of the delay aversion account of intertemporal choice (Sonuga-Barke et al., Reference Sonuga-Barke, Taylor, Sembi and Smith1992). This account holds that impatient intertemporal decisions are not so much driven by the attractiveness of immediate rewards (as we hypothesized), but by an aversion toward delays. Given that previous PIT research has found Pavlovian cues associated with aversive outcomes to increase avoidance behavior and interfere with approach behavior (Geurts et al., Reference Geurts, Huys, den Ouden and Cools2013a, Reference Geurts, Huys, den Ouden and Cools2013b; Huys et al., Reference Huys, Cools, Gölzer, Friedel, Heinz, Dolan and Dayan2011), Pavlovian cues associated with delayed rewards may exert a similar effect compared to Pavlovian cues associated with immediate rewards or neutral outcomes. Our research provides partial support against this idea, as no-go responses did not vary as a function of the delay associated with Pavlovian cues. However, an orthogonalized go/no-go paradigm would allow one to assess the effects of such cues on both passive (no-go-to-avoid-punishment) and active (go-to-avoid-punishment) avoidance. Extending our intertemporal PIT paradigm in this manner would thus allow one to investigate separate contributions of immediacy attraction and delay aversion in intertemporal choice.
A second avenue for future research could be to address individual differences in how participants respond to Pavlovian cues. A distinction can be made here between goal-tracking (GT) and sign-tracking (ST) individuals. For GT individuals, the cue merely serves as a predictor, and upon its presentation, they approach the location of the instrumental reward. In contrast, ST individuals attribute not only predictive but also incentive or motivational value to the cue (eliciting a feeling of ‘wanting’), and approach and engage with the Pavlovian cue itself before approaching the instrumental reward (Colaizzi et al., Reference Colaizzi, Flagel, Joyner, Gearhardt, Stewart and Paulus2020). Classifying participants as GT or ST individuals through eye-tracking, research has found ST individuals to be more strongly influenced by Pavlovian reward cues and thus show stronger transfer effects compared to GT individuals (Garofalo and di Pellegrino, Reference Garofalo and di Pellegrino2015; Schad et al., Reference Schad, Rapp, Garbusow, Nebe, Sebold, Obst, Sommer, Deserno, Rabovsky, Friedel, Romanczuk-Seiferth, Wittchen, Zimmermann, Walter, Sterzer, Smolka, Schlagenhauf, Heinz, Dayan and Huys2020). Moreover, individual differences have been found in the ability to suppress Pavlovian biases on instrumental behavior, despite the existence of such biases on valuation ratings (Cavanagh et al., Reference Cavanagh, Eisenberg, Guitart-Masip, Huys and Frank2013; Swart et al., Reference Swart, Frank, Määttä, Jensen, Cools and den Ouden2018). Together, this suggests that individual differences exist both in the extent to which people are initially motivated by Pavlovian cues, as well as in the ability to overwrite initial tendencies resulting from these cues. Investigating these individual differences in intertemporal PIT effects goes beyond the scope of the current study, but provides an important avenue for future research that aims to gain a fine-grained understanding of our susceptibility to PIT effects.
6. Conclusions
In conclusion, to the best of our knowledge, the present research was the first to study the Pavlovian properties of reward and delays commonly used in intertemporal choice tasks. We found that after Pavlovian conditioning, cues associated with large or immediate rewards acquired higher Pavlovian value compared to cues associated with smaller or delayed rewards. However, only the amount, but not the delay of the reward associated with the Pavlovian cue biased instrumental go/no-go behavior. Hence, we found no evidence for the hypothesis that PIT effects driven by reward delay form a psychological mechanism contributing to impatient intertemporal choice. In interpreting these findings, we discussed potential alternative explanations, and provided directions for future research aimed at unravelling the psychological mechanisms that drive intertemporal decisions.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/jdm.2023.42.
Data availability statement
The study’s preregistrations, materials, data, and analysis scripts are available at the Open Science Framework (OSF; https://osf.io/c2qfm/).
Author contribution
V.R.H., K.R., and B.F. conceptualized and designed Experiment 1. V.R.H. collected data for Experiment 1. F.B., K.R., and B.F. conceptualized and designed Experiment 2. F.B. collected data for Experiment 2. F.B. and V.R.H. analyzed the data. F.B. drafted the manuscript. F.B., V.R.H., A.S., K.R., and B.F. edited the paper. All authors approved the final version of the manuscript for submission.
Funding statement
This research received no specific grant funding from any funding agency, commercial or not-for-profit sectors. For full transparency, we do wish to report that several authors have received such financial support for other projects. KR was supported by a consolidator grant DARE2APPROACH from the European Research Council (ERC_CoG_772337). BF was supported by the Radboud-Western Collaboration Fund.
Competing interest
The authors declare none.
Open access
