In this chapter, we show how the SLG model applies to recovery from overshadowing, the interaction between overshadowing and latent inhibition, recovery from blocking, the inability of a blocked CS to become a blocker of another CS, backward blocking and recovery from backward blocking.

In the last decades, new phenomena have been presented that challenge traditional theories of classical conditioning. Among other observations, Matzel, Schachtman and Miller (1985; Kaufman & Bolles, 1981) established that extinction of the overshadowing CS results in the recovery of the response to the overshadowed CS; Blaisdell, Gunther and Miller (1999) showed that extinction of the blocking CS results in the recovery of the response to the blocked CS; Pineño, Urushihara and Miller (2005) reported that a time delay interposed between the last phase of backward blocking and testing results in the recovery of the response to the blocked CS; Grahame, Barnet, Gunther and Miller (1994) reported that extinction of the context following latent inhibition (LI) results in the recovery of the response of the target CS; De la Casa and Lubow (2000, 2002) demonstrated that a delay interposed between conditioning and testing results in an increased LI effect (super-LI); and Blaisdell et al. (1998) showed that LI and overshadowing counteract each other.

Recovery from overshadowing

In Chapter 3, we mentioned that the competitive rule in the SLG model describes overshadowing (Pavlov, 1927) and relative validity (Wagner et al., 1968). Here we describe recovery from overshadowing, a phenomenon that the model explains in attentional terms.

In this chapter, we apply the SLG model presented in Chapter 2 to the theoretical analysis of creativity. Creativity can be defined as a psychological process that produces original and appropriate ideas. A rather large number of theories have been proposed to account for this process, including Guilford's (1950) psychometric theory, Wertheimer's (1959) Gestalt theory, Mednick's (1962) and Eysenck's (1995) associative theories, Campbell's (1960) Darwinian theory, Amabile's (1983) social–psychological theory, Sternberg and Lubart's (1995) investment theory, and Martindale's (1995) cognitive theory. Other approaches, such as artificial intelligence models (Boden, 1999; Partridge & Rowe, 2002) also contribute to our understanding of creativity.

Mednick (1962) defined creative thinking as the combination of different associations. This combination might result from (a) contiguity, the accidental or planned temporal proximity between the elements of the association; (b) generalization, the sharing of common factors by the elements of the association; or (c) mediation, the simultaneous activation of both elements of the association. Mednick suggested that differences in creativity depend on the strength of the associations that enter in the combinations. In a similar vein, Eysenck's (1995, page 81) theory stipulates that (a) cognition requires associations, (b) differences in intelligence depend on the speed to build these associations, (c) differences in creativity depend on the range of associations considered in problem solving, and (d) a comparator is needed to eliminate wrong solutions.

Schmajuk, Lam and Gray (SLG, 1996; Schmajuk & Larrauri, 2006) proposed a neural network model of classical conditioning. The SLG model shares properties with other associative models described in Chapter 1, including equations that portray behavior on a moment-to-moment basis (Grossberg, 1975; Wagner, 1981), the attentional control of the formation of CS–US associations (Pearce & Hall, 1980), the competition among CSs to become associated with the US (Rescorla & Wagner, 1972) or other CSs (Schmajuk & Moore, 1988) and the combination of attention and competition (Wagner, 1979).

Important properties of the SLG model include that (a) it describes not only CS–US associations, as in the above-mentioned models, but also the formation and combination of CS–CS and CS–US associations; (b) attention to the CS is controlled by the CS–US associations (as in the Pearce & Hall, 1980, model), by context–CS (CX–CS) associations (as in the Wagner, 1979, model), and by CS–CS associations (as in the Schmajuk & Moore, 1988, model; a property that explains why latent inhibition can become context independent – see Chapter 5); and (c) retrieval of CS–US and CS–CS associations is controlled by the magnitude of the attention to the CS (as in Wagner's, 1981, SOP model).

Gray (1975; Hebb, 1949; McNaughton, 2004) suggested that in order to relate brain and behavior, one should first develop a “conceptual nervous system” to handle behavioral data, and second find out whether brain structures and neural elements carry out the operations described by the conceptual system. The present chapter shows that variables representing “neural activity” in the SLG model (see Chapter 2) are consistent to brain responses reported by Dunsmoor et al. (2007) during a human fear conditioning task. In the Dunsmoor et al. (2007) neuroimaging study, different patterns of neural activity were revealed to CSs that predicted an aversive US on all trials, half the trials or no trials. Computer simulations with the SLG model demonstrate that (a) activity in the amygdala and anterior cingulate is well characterized by the prediction of the US by the CS and the CX, BUS, (b) activity in the dorsolateral prefrontal cortex (dlPFC) and anterior insula is well described by the representation of the CS, XCS, and (c) the skin conductance response (SCR) is a nonlinear function of BUS. It is important to notice that variables BUS and XCS represent “neural activities” (see Figure 2.2), and not the strength of their related synaptic associations, VCS–US and zCS, which cannot be appreciated by functional magnetic resonance imaging (fMRI) methods.

During classical (or Pavlovian) conditioning, human and animal subjects change the magnitude and timing of their conditioned response (CR), as a result of the contingency between the conditioned stimulus (CS) and the unconditioned stimulus (US).

In this chapter we briefly describe results of a number of classical conditioning paradigms that are discussed in detail in different chapters of the book (see Schmajuk, 2008a, 2008b). Then we introduce different types of learning theories. Finally, we present a number of computational models of classical conditioning.

Classical conditioning data

Excitatory conditioning

1. Acquisition. After a number of CS–US pairings, the CS elicits a conditioned response (CR) that increases in magnitude and frequency.

2. Partial reinforcement. The US follows the CS only on some trials, and might lead to a lower conditioning asymptote.

3. Generalization. A CS2 elicits a CR when it shares some characteristics with a CS1 that has been paired with the US.

4. US- and CS-specific CR. The nature of the CR is determined not only by the US, but also by the CS.

Inhibitory conditioning

1. Conditioned inhibition. Stimulus CS2 acquires inhibitory conditioning with CS1 reinforced trials interspersed with, or followed by, CS1–CS2 nonreinforced trials.

2. Extinction of conditioned inhibition. Inhibitory conditioning is extinguished by CS2–US presentations, but not by presentations of CS2 alone.

3. Differential conditioning. Stimulus CS2 acquires inhibitory conditioning with CS1 reinforced trials interspersed with CS2 nonreinforced trials.

4. Contingency. A CS becomes inhibitory when the probability that the US will occur in the presence of the CS, p(US/CS), is smaller than the probability that the US will occur in the absence of the CS (p[US/noCS]).

This book extends the application of the neural network models described in my previous book on “Animal learning and cognition: A neural network approach” to a whole range of important classical conditioning paradigms, including recovery from overshadowing, recovery from blocking, backward blocking and recovery from backward blocking; extinction, and occasion setting, as well as the neurophysiology of some of those phenomena.

In the last decades, models of conditioning have shown increasing completeness and precision. This book describes a number of computational mechanisms (associations, attention, configuration, and timing) that first seemed necessary to explain a small number of conditioning results and then proved able to account for a large part of the extensive body of conditioning data. These computational mechanisms are implemented by artificial neural networks, which can be mapped onto different brain structures. Therefore, the approach permits to establish clear brain-behavior relationships.

The book is organized as follows. Part I presents major classical conditioning data and describes several theories proposed to explain them. Part II presents a neural network theory, which includes attentional and associative mechanisms, and applies it to the description of conditioning, latent inhibition, overshadowing and blocking, extinction, and creative processes. In addition, it examines the neurobiological bases of latent inhibition and extinction. Part III describes another neural network, which includes configural mechanisms, and applies it to the description of occasion setting. In addition, it examines the neurobiological bases of occasion setting.

In order to jointly explain latent inhibition and occasion setting, Buhusi and Schmajuk (1996) combined the attentional and associative systems of the SLG neural network (Chapter 2) with the configuration mechanisms of the SD neural network (Chapter 11). As mentioned, the SLG model describes the multiple properties of latent inhibition (Chapter 5), overshadowing and blocking (Chapter 8) and extinction (Chapter 9). On the other hand, the SD/SLH model correctly describes negative and positive patterning, as well as most of the features of occasion setting (Chapter 12). Buhusi and Schmajuk (1996) demonstrated that the attentional–configural model describes a broad range of experimental results, including: (a) acquisition and extinction series, (b) overshadowing and blocking, (c) discrimination acquisition and reversal, (d) conditioned inhibition and inhibitory conditioning, (e) simultaneous feature-positive discrimination, (f) serial feature-positive discrimination (occasion setting), (g) negative patterning, (h) positive patterning, and (i) reduced responding when context is switched. In addition, the model describes (j) sensory preconditioning, (k) latent inhibition, (l) contextual effects on LI, and (m) place and maze learning. This chapter presents a simplified version of the attentional–configural model described Buhusi and Schmajuk (1996) to address the properties of extinction cues.

Attentional and configural mechanisms in extinction

Some experiments have studied the effect of extinction cues (ECs), i.e. temporally discrete cues preceding the presentation of the CS during extinction. During testing, the presence of ECs tends to decrease responding.

This chapter introduces a configural neural network model of classical conditioning offered by Schmajuk and DiCarlo (1992) and modified by Schmajuk, Lamoureux and Holland (1998) in order to account for data on occasion setting. In Chapter 12, we evaluate the model by applying it to the experimental results in which occasion setting is observed.

Skinner (1938) suggested that a discriminative stimulus in an operant conditioning paradigm does not elicit a response, but simply sets the occasion for the response to occur. More recently, many investigators (e.g. Bouton & Nelson, 1994; Holland, 1983, 1992; Rescorla, 1985, 1992) have claimed that Pavlovian conditioning procedures can also endow stimuli with occasion setting functions, as well as simple associative functions. Whereas a simple conditioned stimulus (CS) may elicit conditioned responses (CRs) because it signals the occurrence of an unconditioned stimulus (US), an “occasion setter” (Holland, 1983, 1992) or “facilitator” (Rescorla, 1985) may instead modulate responding generated by another CS by indicating the relation between that CS and the US. Rather than signaling the delivery of the US, an occasion setter indicates whether another cue is to be reinforced (or not reinforced), setting the occasion for its reinforcement (or nonreinforcement). Extensive evidence seems to support this distinction between simple conditioning and occasion setting, and suggests that these two functions are acquired under different circumstances, manifest many different behavioral properties and often act quite independently of each other (see Holland, 1992; Swartzentruber, 1995, for comprehensive reviews.)

Following the ideas proposed in Chapters 4 and 6, this chapter applies the “conceptual nervous system” provided by the SLG model in order to establish brain–behavior relationships during extinction. We applied the SLG model to the simulation of (a) Frohardt et al.'s (2000) data showing that neurotoxic hippocampal lesions eliminate reinstatement in rats, and (b) LaBar and Phelps's (2005) study showing that reinstatement is absent in patients with hypoxic damage to the hippocampus. As in Chapter 6, we assumed that neurotoxic hippocampal lesions, which injure the hippocampus proper (HPLs), impair the formation of (and changes in) CS–CS, CS–CX, CX–CX and CX–CS associations as defined in the SLG model. We will refer to these associations as between-CS associations. The same assumption was made for hypoxic hippocampal damage which results in human amnesia.

Effects of neurotoxic hippocampal lesions

A number of studies seem to indicate that selective excitotoxic hippocampal lesions (Talk, Gandhi & Matzel, 2002; but see Ward-Robinson et al., 2001), fimbrial lesions (Port & Patterson, 1984), and kainic lesions of hippocampal CA1 (Port, Beggs & Patterson, 1987) impair the acquisition of between-CS associations. In the framework of the SLG model, Buhusi et al. (1998, see Equations [6.1a] and [6.1b] in Chapter 6) described the effect of these lesions by assuming that between-CS associations, presumably stored in cortical areas, remain zero. They also assumed that associations of the CS with itself, which produce habituation to the CS, are modified when the CS is perceived.

As described in Chapter 12, Schmajuk, Lamoreux and Holland (SLH) (1998) showed that an extension of a neural network model introduced by Schmajuk and DiCarlo (SD) (1992) characterizes many of the differences that distinguish simple conditioning from occasion setting.

In the framework of their model, Schmajuk and DiCarlo (1992) proposed that the hippocampus modulates (a) the competition among simple and complex stimuli to establish associations with the unconditioned stimulus, and (b) the configuration of simple stimuli into complex stimuli. Furthermore, Schmajuk and Blair (1993, 1995) suggested that (a) nonselective aspiration lesions of the hippocampal formation impair both configuration and competition, and (b) ibotenic acid selective lesions of the hippocampus proper impair only stimulus configuration.

These assumptions are somewhat similar to the ones made in Chapter 6 for the SLG model, in which we assumed that CS1–CS2, CS–CX, CX–CX and CX–CS associations, instead of the configural associations, were impaired by the selective hippocampal lesions (HPLs). In both models, we assume that competition is eliminated by nonselective lesions of the hippocampal areas.

In order to provide a theoretical account of hippocampal participation in occasion setting, the present chapter combines (a) Schmajuk, Lamoureux and Holland's (1998) account of occasion setting (see Chapter 12) with (b) Schmajuk and DiCarlo's (1992) and Schmajuk and Blair's (1993, 1995) assumptions regarding the effect of HPLs and HFLs on classical conditioning mechanisms. The results have implications for both the behavioral and neurophysiological aspects of occasion setting.

In this chapter, we apply the attentional–configural form of the SLG neural network model (see Chapter 15), which combines attentional, configural and associative mechanisms to the description of causal learning and inferential reasoning.

Causal learning

As described in Chapter 8, and according to associative theories (e.g. Rescorla & Wagner, 1972; the SLG model; and the SD/SLH model), blocking is the consequence of stimulus A winning the competition with X to predict the US, because the US is already predicted by A at the time of A–X–US presentations. In contrast, according to the inferential process view (see Beckers et al., 2006), blocking is the consequence of an inferential process based on the assumptions of additivity and maximality. Maximality refers to the evidence that the Outcome produced by each potential cause has not reached the maximal possible value. Additivity denotes the fact that two causes, each one independently producing a given Outcome, produce a stronger Outcome when presented together.

Supporting the inferential explanation, recent experimental results have shown that, both in humans (Lovibond et al., 2003; Beckers et al., 2005) and rats (Beckers et al., 2006), blocking was stronger if the maximal premise (the outcome of each cause does not reach the maximum possible value) and the additivity premise (the outcomes of effective causes can be added) are satisfied. In contrast, Beckers et al. (2005) showed that the Rescorla–Wagner (1972) model incorrectly predicts more blocking with an intense (maximal) than with a moderate (submaximal) outcome.

In this chapter, we apply the SLG model to try to determine the mechanisms at work during extinction. Extinction refers to the phenomenon by which nonreinforced presentations of the conditioned stimulus (CS) after conditioning reduce the strength and frequency of the conditioned response (CR) to an arbitrarily small value. A large number of theories have been proposed to account for the extensive information available on extinction of classical conditioning. Associative models (e.g. Mackintosh, 1975; Rescorla & Wagner, 1972) assume that the phenomenon involves the weakening of the association between a CS and the unconditioned stimulus (US). In contrast, other approaches propose that extinction leaves the initial CS–US association intact. For instance, Pavlov (1927, Lecture XXII; see Robbins, 1990, page 236) provided a “new interpretation” of extinction in terms of a decrease in the activation of the cells triggered by the CS (CS representations), without changes in the connecting path between the CS cells and those cells excited by the US. For Rescorla (1974), extinction is the consequence of a decrease in the representation of the US, which controls both the CR and changes in the CS–US association. Hull (1943) suggested that extinction is the result of a “reactive inhibition”; a tendency not to repeat the CR when it is produced in the absence of the US.

This chapter illustrates how the SD/SLH model (presented in Chapter 11) describes situations in which a CS behaves as a simple CS or as an occasion setter. We will analyze its performance in (a) a simultaneous FP discrimination with a strong feature and a weak target, (b) a simultaneous FP discrimination with a weak feature and a strong target, (c) a simultaneous FN discrimination, (d) a serial FP discrimination, (e) a serial FN discrimination, and (f) a contextual discrimination. As in previous chapters, in each case, we first present sample empirical data from test sessions administered after training, then show simulations of those data from the model, and finally detail the mechanisms by which the model acquires those discriminations.

It will be shown that a CS acts as a simple CS (through its direct associations with the US) or an occasion setter (through its indirect associations with the US via the hidden units), depending on the strength of its direct association with the US (a function of its intensity, duration and the CS–US interval) and the requirements of the task at hand.

Distinctions between occasion setting and simple conditioning

Investigations of the nature of learning in feature-positive (FP) and feature-negative (FN) conditional discriminations (Jenkins & Sainsbury, 1969) were the starting point for much of the research in occasion setting.