1. The virtual personalities neural network model
1.1. Neurobiological underpinnings
The science of human personality has many enigmas. How can we understand the dynamics of persons and situations and how they operate together over time to produce emergent behavior? How could the same underlying system that results in between-subject stability in differences in behavior over time and reliable personality structure across individuals (e.g., the Big Five), plausibly explain broad within-person variability in behavior across situations? How do underlying biological mechanisms and different learning histories produce enduring individual differences in response to situational cues? Earlier, we (Read, Droutman, & Miller, Reference Read, Droutman and Miller2017; Read et al., Reference Read, Monroe, Brownstein, Yang, Chopra and Miller2010; Read, Smith, Droutman, & Miller, Reference Read, Smith, Droutman and Miller2017) argued that a computational model (here implemented as a neural network model) is needed that allows us to begin to construct plausible models of such dynamics that could begin to address these questions. In the current work, after an introduction that describes our personality model and its implementation as a neural network model, we focus on what is known about the underlying neurobiological and learning mechanisms underpinning the Virtual Personalities Model. We then discuss the implications of these processes for understanding individual differences.
The Virtual Personalities Model (Read, Droutman, & Miller, Reference Read, Droutman and Miller2017; Read et al., Reference Read, Monroe, Brownstein, Yang, Chopra and Miller2010; Read et al., Reference Read, Smith, Droutman and Miller2017) is a motive-based neural network model of personality that is both a psychological model and a computational neural network implementation of that model. At a psychological level, the Virtual Personalities Model (Read & Miller, Reference Read and Miller2002; Read et al., Reference Read, Monroe, Brownstein, Yang, Chopra and Miller2010), grew out of a focus on the dynamics of motivational and cognitive structures (Miller & Read, Reference Miller and Read1991; Read, Jones, & Miller, Reference Read, Jones and Miller1990; Read & Miller, Reference Read and Miller1989). Thirty years ago, we argued that traits could be viewed as goal-based structures, where the goals of the individual were the central part of a structure consisting of goals, plans, resources, and beliefs (Miller & Read, Reference Miller and Read1987). Thus, a major basis of individual differences was differences between people in the chronic activation of their goals. The Virtual Personalities Model drew upon that earlier work to examine in more detail how personality could be understood in terms of the behavior of structured motivational systems. It draws on diverse literatures, summarized elsewhere (e.g., Read, Brown, Wang, & Miller, Reference Read, Brown, Wang and Millerin press; Read et al., Reference Read, Monroe, Brownstein, Yang, Chopra and Miller2010), including those involving the factor structure of personality measures (e.g., Eysenck, Reference Eysenck1983, Reference Eysenck1994; Lee & Ashton, Reference Lee and Ashton2004; McCrae & Costa, Reference McCrae and Costa1999; Tellegen, & Waller, Reference Tellegen and Waller2008; Wiggins & Trapnell, Reference Wiggins and Trapnell1996; Zuckerman, Reference Zuckerman2005), the lexical analysis of trait language (e.g., Digman, Reference Digman1997; Goldberg, Reference Goldberg1981), temperament and neurobiological bases of personality (e.g., Clark & Watson, Reference Clark and Watson2008; Gray, Reference Gray1987a, Reference Gray1987b; Gray & McNaughton, Reference Gray and McNaughton2000; Pickering & Gray, Reference Pickering and Gray1999; Rothbart & Bates, Reference Rothbart and Bates1998; Zuckerman, Reference Zuckerman2005), an evolutionary analysis of social tasks (e.g., Bugental, Reference Bugental2000; Fiske, Reference Fiske1992; Kenrick & Trost, Reference Kenrick and Trost1997), taxonomies of human motives (Chulef, Read, & Walsh, Reference Chulef, Read and Walsh2001; Talevich, Read, Walsh, Iyer, & Chopra, Reference Talevich, Read, Walsh, Iyer and Chopra2017), and our earlier work (e.g., Miller & Read, Reference Miller and Read1987, Reference Miller and Read1991; Read, Jones & Miller, Reference Read, Jones and Miller1990; Read & Miller, Reference Read and Miller1989) on traits as goal-based structures.
Central to the Virtual Personalities psychological model are two broad motivational systems: a Behavioral Approach System (BAS) and an avoidance system, the latter originally referred to as the Behavioral Inhibition System (BIS) by Gray (e.g., Gray, Reference Gray1987a, Reference Gray1991; see also Tellegen, Watson, & Clark, Reference Tellegen, Watson and Clark1999). Although there is very good agreement among researchers regarding the BAS, Gray and others reconceptualized the BIS. They now refer to various fear responses (e.g., flight, fight, freeze) as the Fight Flight Freeze System, or FFFS (not the BIS), and now refer to a separate system—one associated more with anxiety and goal conflict—as the BIS (see Gray & McNaughton, Reference Gray and McNaughton2000; Smillie, Pickering, & Jackson, Reference Smillie, Pickering and Jackson2006). To avoid confusion, we do not use the BAS/BIS specification here, and instead refer to the Approach and Avoidance systems associated with reward (opportunity) and punishment (threat).
These two broad motivational systems have been argued to map most closely to Extraversion (Approach system) and Neuroticism (Avoidance system) and they also are related to the two broad metatraits, Plasticity and Stability (DeYoung, Reference DeYoung2015; Digman, Reference Digman1997) that are found when one investigates higher order factors of the Big Five. Plasticity consists of Extraversion and Openness to Experience/Intellect, whereas Stability consists of Neuroticism, Agreeableness, and Conscientiousness.
Separate specific motives are nested in the Approach (e.g., dominance, social affiliation, mating) and Avoidance (e.g., avoid physical harm, avoid social rejection) systems and are part of the basis of more specific traits. Individuals differ in the baseline sensitivities of the two broad systems, as well as in the baseline activation of the specific goals that are “nested” within the broader motivational systems. Goals in the Approach and Avoidance systems are jointly activated by cues from the Environment that identify the goal affordances of the situation, and by internal cues that indicate the current Interoceptive state of a variety of bodily systems (see Figure 1). General activation of the Approach and Avoidance systems combined with activation of the specific motives then results in the transmission of activity to motor systems that guide behavior to satisfy the active motives. These behaviors result in changes in Interoceptive bodily state (Satiation) and alter the organism’s Environment (Consummation).
Basic Virtual Personalities Model: Abstract systems that make up the model and the flow of activation between them.
Figure 1 highlights this basic model, without the details of the specific neural network implementation of our Approach and Avoidance motivational systems—but elsewhere see our Virtual Personalities Model (Read et al., Reference Read, Brown, Wang and Millerin press; Read, Droutman, & Miller, Reference Read, Droutman and Miller2017; Read et al., Reference Read, Monroe, Brownstein, Yang, Chopra and Miller2010; Read et al., Reference Read, Smith, Droutman and Miller2017), as well as a detailed tutorial regarding the implementation process for a neural network model in general, and the Virtual Personalities Model, in particular (Read et al., Reference Read, Smith, Droutman and Miller2017).
We will shortly describe how these systems are implemented in neurobiological systems.
2. Neural networks: Brief introduction
Our theory of personality ties together the psychological assumptions of our model with what is increasingly known about the neurobiology of Motivation and decision making. We implement our theory as a neural network model. Implementing the theory as a computational model has the benefit of allowing us to actually simulate and run the model to test the implications of the various assumptions we make in the theory. Moreover, it has the added advantage that it makes it easier to map between the features of the model and the underlying neurobiology of the brain in which personality is embodied.
Neural network models are biologically inspired models. They are designed to capture what we know of the abstract properties of how real brains process information. This approach to modeling can capture central aspects of cognitive processing, such as image recognition and language comprehension (O’Reilly, Munakata, Frank, Hazy, & Contributors, Reference O’Reilly, Munakata, Frank and Hazy2012).
Neural network models are constructed of nodes and the weighted links between them. Nodes sum activation they receive over the weighted links from other nodes. Links between nodes can be either excitatory (positive) or inhibitory (negative). The input to a node is the sum of the activations of the different nodes sending a link to the target node, times the weight on the links to the nodes. The strength of the activation that a receiving node then sends is a function of this summed input. The output function can be linear, but is usually nonlinear, typically binary or sigmoidal (S-shaped). Neural networks with nonlinear activation functions have more powerful learning and processing capabilities.
In the architecture we use, nodes are typically arranged in layers and there is competition among the nodes within a layer. The degree of competition within a layer can be tuned to control the average activation of the layer and the number of nodes that become active in the layer.
Processing in a neural network model proceeds by sending activation through a hierarchy of nodes. One central feature of neural networks is what are called Hidden layers. Hidden layers provide powerful learning and representational abilities to neural network models, as they enable such models to learn a hierarchy of increasingly abstracted features (O’Reilly et al., Reference O’Reilly, Munakata, Frank and Hazy2012). (One way to think of this is that they enable the system to learn representations for interactions among input features.) For example, in the visual system they allow humans and other animals to start with “points” of light on the retina and build up increasingly abstract representations so that they are ultimately able to recognize that the object in front of them is a friend, or a cat sitting on a desk.
Our neural network models are constructed in a specific neural network architecture called Leabra (Aisa, Mingus, & O’Reilly, Reference Aisa, Mingus and O’Reilly2008; O’Reilly et al., Reference O’Reilly, Munakata, Frank and Hazy2012), which is a biologically inspired architecture. Several important aspects of Leabra, for our purposes, are the following. First, competition between nodes in a layer is an inherent feature of the architecture, and can be easily tuned, which makes it easy to capture competitive dynamics within a layer. Second, the activation function for nodes is biologically inspired, modeling the independent contributions of excitatory and inhibitory conductances (corresponding to the number of synaptic channels) into the node. These can be tuned as desired. Third, the architecture integrates Hebbian learning (correlational) and error-correcting learning into a single learning rule. The error-correcting component of the rule is functionally similar to the back-propagation rule widely used in error-correcting learning, but it implements such learning in a more biologically plausible way. Fourth, the architecture assumes that neurobiological systems are massively bidirectionally connected, consistent with what we know of the architecture of real brains.
Building a neural network model of our psychological theory has several advantages. First, because it is “runnable” we can explicitly vary different aspects of the model, such as the sensitivity of the Approach and Avoidance systems, the baseline importance of different goals, and learning history for associations between different cues and goals. Second, we can use the model to demonstrate things such as how structured motivational systems can result in something like the Big Five. In other work, we have shown how this neural network model can capture a number of different aspects of personality. Read, Droutman, and Miller (Reference Read, Droutman and Miller2017) have shown how such a neural network model, composed of structured motivational systems, can capture the between individual structure of personality (e.g., the Big Five). Read et al. (Reference Read, Smith, Droutman and Miller2017) have shown how the same kind of model can also capture the high level of within subject variability in trait-related behavior over time and across situations. Third, because the architecture is biologically inspired it allows us to more easily identify conceptual links to possible neural substrates for the different processes and representations we propose in our theory. For example, nodes or artificial neurons in Leabra have parameters for the number of excitatory and inhibitory conductances, which would allow one to do things like vary individual differences in the sensitivity of dopamine receptors in the nucleus accumbens.
However, we want to make it clear that simply because Leabra has some biological inspiration, we are not claiming that use of the Leabra architecture automatically allows us to say something about the detailed neurobiological organization of the different systems we are discussing. In order to have something to say about that we would have to do the detailed work of creating systems in which the neural organization was explicitly represented. Our current model building focuses more on modeling the functions of different systems and their functional organization.
3. Virtual Personalities Model: A neural network implementation
Figure 1 outlines the general structure of the neural network model. Features in the Environment activate nodes in the Environment layer, and information about bodily state activate nodes in the Interoceptive state layer. Weighted links from these nodes then transmit activation to the relevant nodes in the Approach and Avoidance layers where the resulting degree of activation is calculated independently for each type of reward. To be clear, the Approach and Avoidance layers function as two separate processing systems.
Following work by Berridge (Berridge, Reference Berridge2012; Zhang, Berridge, Tindell, Smith, & Aldridge, Reference Zhang, Berridge, Tindell, Smith and Aldridge2009) we characterize the degree of activation of a goal/reward in these two motivational systems as representing the degree to which the individual desires or WANTS that goal. According to Berridge (Berridge, Reference Berridge2007, Reference Berridge2012; Zhang et al., Reference Zhang, Berridge, Tindell, Smith and Aldridge2009) the degree of WANTING is a multiplicative function of the relevant Environment cues and Interoceptive cues. In an impressive body of work, Berridge and his colleagues (e.g., Berridge, Reference Berridge2007; Berridge & O’Doherty, Reference Berridge and O’Doherty2013; Berridge & Robinson, Reference Berridge and Robinson1998) have made a strong case for the distinction between Wanting a reward and Liking it. He has provided evidence that these are phenomenologically distinct and they arise in different neural circuits. WANTING is the strength of the need or desire for a reward, whereas LIKING is the pleasure received from consuming the reward (or sometimes from imagining its consumption).
As the process is multiplicative, the Environment and Interoceptive features can serve a gating influence for each other. That is, if Environmental cue strength is high, but Interoceptive state is low there will be little resultant activation (or WANTING). Moreover, the reverse would also be true. Nodes representing the different motives compete for activation within each layer or motivational system (Approach, Avoidance). The Approach and Avoidance layers process the two types of motives independently; they do not compete with each other. An Executive Function system can provide top-down biasing on the strength of activation of different motives by maintaining an active goal representation and sending excitatory activation to the corresponding representation in either the Approach or Avoidance systems. Motives that win the competition within each layer then send activation to the Behavior layer, where the different potential behaviors that are activated by the strength of Wanting from different motives compete with each other for activation. The enacted behavior can then change the Environment (Consummation) and/or it can change the Interoceptive state (Satiation), which then changes the inputs on the next step in the behavioral sequence.
4. Neurobiological underpinnings
Here we describe the mapping from our neural network model to its neurobiological underpinnings (see Figure 2).
Basic model with neurobiological Approach/Avoidance system neurobiological underpinnings. DLPFC=dorsolateral prefrontal cortex; NAcc=nucleus accumbens; VTA=ventral tegmental area; SNc=substantia nigra pars compacta.
4.1. Environment layer: Sensory/Cortical systems
The Environment layer represents inputs from sensory and cortical systems about cues to different objects or events in the Environment. Some of the cues may be unconditioned stimuli or primary rewards, such as food, water, other people, whereas other cues will be conditioned stimuli (CSs), which are learned cues, such as visual features of the Environment that are associated with and can activate the representation of a particular reward, such as food.
Through what is alternatively called Classical or Pavlovian Conditioning the organism learns that certain CSs (such as an arbitrary visual or auditory cue) predict the occurrence of rewards in the Environment. This occurs through a process of associative learning; when a CS and a reward are presented closely in time, the CS and reward become associated, so that presentation of the CS activates a representation of the reward. This process of associative learning is thought to occur in the amygdala (Hazy, Frank, & O’Reilly, Reference Hazy, Frank and O’Reilly2010). In the neural network model, when the CS and reward are concurrently activated, the weight of the link between the CS and reward representations are strengthened, eventually allowing the CS to activate Motivations.
One way to think about this process is that it represents how individuals learn what features of the Environment come to predict the presence of goal affordances in the Environment. Through experience, individuals learn what cues in the Environment predict the goal relevant aspects of the Environment. This learning process can capture one aspect of individual differences. As a result of different learning histories or experiences, different individuals will develop different cue-affordance associations. For example, some people may learn that social situations are likely to be rewarding, whereas others may learn that social situations are an opportunity for social rejection.
4.2. Interoceptive state: Posterior insula
In our current model, we focus on the role of the posterior insula in representing information about the current Interoceptive state of the organism. For example, are they hungry, thirsty, lonely, horny, fatigued? However, the insula is a relatively large neural region that is involved in a number of different aspects of processing. Recent reviews of the insula (Droutman, Bechara, & Read, Reference Droutman, Bechara and Read2015; Droutman, Read, & Bechara, Reference Droutman, Read and Bechara2015) suggest that subregions of the insular cortex play significant roles in four phases of the decision-making process. The first phase involves Salience processing and refocusing attention (dorsal anterior insula cortex [dAIC]). The second phase involves evaluation that involves all three subregions of the anterior insula cortex including: (1) the ventral anterior insula cortex (vAIC) that appears to track arousal variance, skew, and risk-prediction error, (2) the posterior insula cortex that is involved in urge processing and the signaling of homeostatic imbalance, and (3) the dAIC that plays a role in tracking arousal, and the magnitude of that arousal and its variance and associated risk and urge generation. The third phase of the insular cortex’s role in decision making involves action. The dAIC plays a role in the “what,” “when,” “whether to act” aspects of action while the vAIC plays a role in action inhibition. In the final phase of decision making the IC is involved in outcome processing, with the vAIC and dAIC both playing a role in error awareness and social outcomes and the dAIC also playing a role in posterror correction and harm prevention. An additional review of work on the insula and fatigue (Dantzer, Heijnen, Kavelaars, Laye, & Capuron, Reference Dantzer, Heijnen, Kavelaars, Laye and Capuron2014) suggests that peripheral inflammation may also register in the anterior insula, affecting a subjective experience of fatigue or uncertainty about an action’s usefulness (suggesting that the latter phases of decision processes in the insula may be impacted). Interestingly insula activation may also activate the fronto-striatal network resulting in diminished switching capacity from goal-directed to habit-like behavior and/or decreased Incentive Motivation (Dantzer et al., Reference Dantzer, Heijnen, Kavelaars, Laye and Capuron2014).
In the current model, the insula tracks different bodily states (e.g., related to hunger, sexual excitement, loneliness, etc.), which are typically thought to be represented in posterior insula. That information about Interoceptive state is multiplicatively combined with outcome predictions from the Sensory/Cortical systems to produce the WANTING (or Incentive Salience) for a reward.
4.3. Motivation system: Amygdala/nucleus accumbens (NAcc)
In Figure 2 we specify the underlying neural underpinnings for the Approach and Avoid systems. Central to the model is a Motivation system that represents the degree to which a particular motive is wanted. Both Approach and Avoidance type motives are represented within the Motivation system, although the two types of motives are processed independently, represented by two separate layers in our neural network model. Following Berridge and colleagues (Mahler & Berridge, Reference Mahler and Berridge2009, Reference Mahler and Berridge2012; Peciña & Berridge, Reference Peciña and Berridge2013) this Motivation system is composed of a circuit involving the amygdala and the NAcc. (The NAcc is also referred to as the ventral striatum.). Different regions of neurons in NAcc (Reynolds & Berridge, Reference Reynolds and Berridge2008) represent different types of rewards. This system multiplicatively combines input information about the strength of cues to reward from the Sensory/Cortical systems with Interoceptive information from the insula about the organism’s current Interoceptive or bodily state. The result is the current level of WANTING for the relevant motive. The impact of conditioned cues on WANTING, as represented in the Amygdala/NAcc circuit, has been termed by Berridge (Berridge, Reference Berridge2012; Zhang et al., Reference Zhang, Berridge, Tindell, Smith and Aldridge2009) as cue-triggered Wanting.
Because of the multiplicative relationship, each of the two types of cues essentially plays a gating role on the influence of the other cue. For example, if there is a strong cue to a reward, such as food or attractive people, but there is no need, then the Motivation system will not be activated. And if there is a strong need, but no cues to rewards then there will not be an activation of the Motivation system. However, to capture cases where there is strong need but no cues to the reward, we have implemented a direct connection from the Interoceptive state to possible seeking behaviors. For example, if an individual is very hungry or very lonely, but their current Environment does not contain the relevant affordances, the individual would be motivated to seek an Environment that does have those affordances.
As discussed earlier, Berridge and others (Berridge, Reference Berridge2012; Dayan & Berridge, Reference Dayan and Berridge2014; Zhang et al., Reference Zhang, Berridge, Tindell, Smith and Aldridge2009) have made a convincing case that Wanting something and Liking something are different both psychologically and neurobiologically. Neurobiologically, Wanting seems to strongly depend on the neurotransmitter dopamine and the dopaminergic circuitry in the brain, whereas Liking depends on brain opioids. Wanting is implemented in our current model, but Liking is not.
4.4. Dopamine levels: Ventral tegmental area/substantia nigra pars compacta
The NAcc is strongly innervated by dopaminergic neurons from the ventral tegmental area (Collins & Frank, Reference Collins and Frank2014). Higher dopamine inputs increase the sensitivity of the Approach system to the Environmental and Interoceptive inputs (through D1 receptors) (Collins & Frank, Reference Collins and Frank2014), whereas they decrease the sensitivity of the Avoidance system to threats (through D2 receptors) (Collins & Frank, Reference Collins and Frank2014). Thus, individual differences in Tonic levels of dopamine will influence the strength of Approach Motivations. Higher levels lead to greater Approach (and reduced Avoidance). Considerable evidence (e.g., Depue & Collins, Reference Depue and Collins1999; DeYoung & Allen, Reference DeYoung and Allenin press) suggests that individual differences in Tonic dopamine level is one major factor that underlies Extraversion, as well as Openness to Experience/Intellect.
There is a key distinction between Tonic levels of dopamine and Phasic levels(Schultz, Reference Schultz2015). Tonic levels of dopamine are the consistent baseline level and have a chronic influence on the strength of Wanting (Berridge & O’Doherty, Reference Berridge and O’Doherty2013; Berridge & Robinson, Reference Berridge and Robinson1998; Collins & Frank, Reference Collins and Frank2014). Phasic changes in dopamine level are momentary shifts in dopamine firing and are typically a response to reward prediction errors, and play an important role in learning a cue-reward association (Collins & Frank, Reference Collins and Frank2014; Schultz, Reference Schultz2015). We have modeled the impact of Tonic dopamine levels, but have not explicitly included Phasic dopamine changes.
4.5. Behavior: Motor systems
Activation from the Motivation system, as well as direct activation from the Environment system that conveys information about the availability of various resources, are fed into a Hidden layer, where they may be combined in a conjunctive representation, and go from there to the behavioral system. In the behavioral system, different possible actions compete for activation and the most strongly activated will be enacted by the Motor system.
4.6. Satiation and Consummation
An organism’s behavior has consequences for the next step in the behavioral sequence. Behaviors can change the nature of the Environment to which the organism is responding (reduce amount of food, leave a situation with people). Changes in the Environment can then influence the strength of the cues that feed into the Motivation system as the organism is considering subsequent behaviors. Behaviors can also lead to Satiation of internal needs: eating reduces hunger, hanging out with friends reduces loneliness. These changes in bodily states are then input into the network on the next time step. Thus, the behavior of the organism and its impact on internal and external Environment are major contributors to variability in trait-related behavior over time and situations. For example, an extraverted individual might end up seeking privacy after hours of partying.
4.7. Executive Function, self-regulation: Dorsolateral prefrontal cortex (DLPFC) and related systems
Our model captures the top-down influence of Executive Function on Wanting by sending activation from a sustained goal representation, in a layer that represents the function of DLPFC, to its corresponding representation in the Motivation system. This top-down influence (activation) helps maintain the activation of certain Motives in the face of competition from alternative Motives that might be activated, thus providing a degree of consistent goal focus. The implementation of Executive Function in our model is consistent with recent work in this area (Wiecki & Frank, Reference Wiecki and Frank2013).
Individual differences in self-regulation or Executive Function can be modeled in terms of differences in the ability to maintain a goal representation when faced with competing influences, and in terms of differences in the strength of top-down influence on the Motivation system. This would capture some of the more controlled aspects of Conscientiousness.
More automatic aspects of inhibitory control could be modeled in terms of general levels of inhibition within early processing layers, as discussed by Read et al. (Reference Read, Monroe, Brownstein, Yang, Chopra and Miller2010). In that paper, we showed that greater inhibition in the Motivation layer led to fewer switches to different goals, indicating that this can capture some aspects of automatic inhibitory control (Clark & Watson, Reference Clark and Watson1999, Reference Clark and Watson2008; Rothbart & Bates, Reference Rothbart and Bates1998).
5. Model instantiates classic learning phenomena
This model and its underlying neurobiology allow us to capture basic learning phenomena and show how they can be related to personality and individual differences. In particular, the model captures Pavlovian or Classical Conditioning, Instrumental Conditioning, and their combination, called Pavlovian to Instrumental Transfer (PIT) (see Figure 3). In addition, it can also model the development and representation of habitual behavior.
Neural underpinnings of learning. Broken red lines indicate which systems are involved in Pavlovian learning, Instrumental learning, and Pavlovian to Instrumental Transfer; they are not pathways. The gray line from Environment to the Hidden layer represents the Habit system (instantiated in the Putamen or dorsal lateral striatum that develops with high levels of learning of the associations for S to R. DLPFC=dorsolateral prefrontal cortex; NAcc=nucleus accumbens; VTA=ventral tegmental area; SNc=substantia nigra pars compacta.
5.1. Pavlovian (or Classical) Conditioning
The pathway from Environmental cues (both CS and unconditioned stimuli) to the amygdala in the Motivation system models Pavlovian Conditioning. This part of the model can capture individual differences in Conditioning history: the extent to which different features in the Environment activate different reward representations.
This pathway is also a critical part of what Berridge calls cue-triggered Wanting. As the amygdala and NAcc are tied together in a Motivation system, Pavlovian Conditioning is responsible for the development of connections between CS and WANTING for a particular reward.
5.2. Instrumental Conditioning
In Instrumental Conditioning an organism learns that a particular instrumental behavior leads to a particular reward or avoids a punishment. This has been shown to require involvement of the dorsal medial striatum (also called the Caudate) (Balleine & O’Doherty, Reference Balleine and O’Doherty2010; Dolan & Dayan, Reference Dolan and Dayan2013). Lesioning of the Caudate eliminates the sensitivity of behavior to the current level of Wanting (Balleine & O’Doherty, Reference Balleine and O’Doherty2010; Dolan & Dayan, Reference Dolan and Dayan2013). A number of researchers have argued and demonstrated that in Instrumental Conditioning the organism learns bidirectional connections between action and reward/punishment (Balleine & O’Doherty, Reference Balleine and O’Doherty2010), so that activation of the representation of a reward/punishment can also activate the corresponding behavior.
Again, individual differences in experience can result in individual differences in the strength of connections between behaviors and potential goals. As a result, different individuals may learn different behaviors in response to the activation of different rewards/punishments. For example, some individuals will learn that social interaction is rewarding and will actively seek it out, whereas others will learn that it is often punishing and will learn to avoid it.
5.3. Pavlovian to Instrumental Transfer (PIT)
A central part of the Virtual Personalities Model can be viewed as an instantiation of a well-known phenomenon in reward learning called PIT (Corbit & Balleine, Reference Corbit and Balleine2011; Talmi, Seymour, Dayan, & Dolan, Reference Talmi, Seymour, Dayan and Dolan2008). In PIT, an organism learns an association between a CS and a reward/punishment, so that presentation of the CS leads to activation of a representation of the reward/punishment (Talmi et al., Reference Talmi, Seymour, Dayan and Dolan2008). (Although PIT applies to both reward and punishment, the preponderance of work has focused on reward.) Separately, the organism learns the association between an Instrumental response (e.g., lever pressing) and the attainment of the same outcome (Talmi et al., Reference Talmi, Seymour, Dayan and Dolan2008). This association between Instrumental response and outcome has been shown to be bidirectional, such that activation of the reward or punishment representation can trigger the Instrumental response (Holland, Reference Holland2004). Once both of these sets of associations have been learned, if the CS is presented concurrently with the opportunity to perform the instrumental behavior, the presence of the CS increases the vigor of responding with the instrumental behavior (Talmi et al., Reference Talmi, Seymour, Dayan and Dolan2008).
We propose that this PIT process underlies a tremendous amount of human behavior. As we have proposed for a long time in our personality model (Miller & Read, Reference Miller and Read1987, Reference Miller and Read1991; Read & Miller, Reference Read and Miller1989), we can think of situations as containing sets of cues that predict the availability of affordances for the pursuit of various goals. Features of a classroom give cues to the affordances for academic achievement in that context. Features of a dance party give cues to the affordances for pursuing affiliation and mating. (Although for some a dance party affords opportunities for social rejection and potential humiliation.) Affordances can then specify behavior (e.g., completing classwork, approaching other partygoers, staying home and reading) to be implemented in pursuit of these motives, completing the link from cues to goal activation to behavior.
In our current neural network model of personality, learned and unlearned cues to reward activate Wanting for those rewards, which then bias choice of actions that can achieve those rewards. This process is essentially PIT. Consistent with our model, we suspect that PIT may help explain how personality traits emerge from an individual’s learning history. The chronic manifestation of a trait may result when a large portion of an individual’s chronic Environments cue the same set of goals. On the other hand, the trait will not be observed when individuals encounter stimuli that have not been conditioned with those goals, accounting for within-individual variability.
Interestingly, Pavlovian cues can also invigorate responses associated with a different reward, in a process known as general PIT (Corbit & Balleine, Reference Corbit and Balleine2011). When trained to associate a keyboard button with popcorn, human subjects increase a button response not only when a Pavlovian cue for popcorn is present, but also when a cue for cashew nuts is present (Watson, Wiers, Hommel, & de Wit, Reference Watson, Wiers, Hommel and de Wit2014). This suggests that a given Pavlovian cue can motivate responses generally, invigorating response for unrelated rewards. One possible way this might work is that the general Pavlovian cue could activate the dopamine system in the Motivational system, which would then potentiate Approach to all rewards.
Although research on this effect is limited, we suspect that general PIT may account for positive correlations between Approach-related traits (Elliot & Thrash, Reference Elliot and Thrash2002) If a given cue can increase instrumental behavior in a variety of domains, then we would expect that a cue for positive social experiences (i.e., Extraversion cue) can increase Motivation for novelty (Openness to Experience), resulting in a correlation between Extraversion and Openness to Experience. While we have not explicitly included this effect in our Virtual Personalities Model, we are currently developing a model to incorporate general PIT.
5.4. Habitual Behavior
Habitual behavior is directly triggered by a cue and typically develops after extensive practice with an Instrumental response(Balleine & O’Doherty, Reference Balleine and O’Doherty2010). Once a behavior becomes habitual its enactment is insensitive to the current goal state of the organism (Balleine & O’Doherty, Reference Balleine and O’Doherty2010). For example, once lever pressing for food pellets has become Habitual, a rat fed to satiety may continue to press the lever, even though they are no longer hungry. The learning and performance of habitual behavior depends on the Putamen (dorsolateral striatum) (Balleine & O’Doherty, Reference Balleine and O’Doherty2010) and lesioning of the Putamen eliminates Habitual responding and reinstates goal-based responding (Balleine & O’Doherty, Reference Balleine and O’Doherty2010). These lesion findings strongly argue that habitual behavior does not depend on the Motivation system, but instead depends on a separate circuit involving the Putamen. In line with this, in the Virtual Personalities Model, there is a link between the Environmental input and the behavior layer that bypasses the Motivation system. Individual differences in experience can lead to individual differences in whether a particular cue-behavior link becomes Habitual or remains sensitive to current goals.
6. How Individual differences can be captured in this model
6.1. Neurobiology
This model allows us to examine how underlying mechanisms and their interactions with one another may produce various individual differences. First, the model allows us to capture various aspects of the neurobiology of individual differences. One major factor, which has been identified by a number of theorists, is the role of the Approach or reward system and the role of dopamine in Approach behavior. Depue and others (Depue & Collins, Reference Depue and Collins1999; DeYoung, Reference DeYoung2015) have argued and provided evidence that Tonic dopamine levels are central to Extraversion, as well as to Openness to Experience/Intellect, with higher levels of Tonic dopamine resulting in stronger Approach-related behaviors (Extraversion). Tonic dopamine levels are explicitly modeled in our network and can be manipulated to examine the impact of this parameter on different behaviors.
More speculatively, individual differences in Avoidance behaviors may be related to Tonic levels of serotonin (Corr, DeYoung, & McNaughton, Reference Corr, DeYoung and McNaughton2013) and the role that they play in the Avoidance system. Somewhat controversially, higher levels of serotonin are argued to lead to greater activity of the Avoidance system and thus to increased Avoidance behaviors (Neuroticism). There is some indication that this might occur through downregulation of dopaminergic receptors (Corr, DeYoung, & McNaughton, Reference Corr, DeYoung and McNaughton2013), so that the organism would be less sensitive to Tonic levels of dopamine.
We can also simulate the impact of individual differences in the importance of different kinds of rewards by manipulating their baseline activations in the Approach and Avoidance systems. Research by Berridge and others (Reynolds & Berridge, Reference Reynolds and Berridge2008) suggests that there are specific groups of neurons in NAcc that represent different kinds of rewards. The sensitivities of these systems to their inputs may vary.
Individual differences in some aspects of Conscientiousness can be captured by individual differences in Executive Function, as captured by the DLPFC. As we noted earlier, DLPFC can maintain a representation of desired goals and through top-down connections can influence the degree of activation of WANTs in the amygdala–NAcc circuit.
As all of the parameters in the model can be systematically varied simultaneously, we can also examine interactions among different factors. For example, we can explicitly model some types of impulsivity as the result of the interaction between the strength of the Approach system and the Control system. A number of different researchers have argued that Impulsivity is not just the result of single cause or system, but is the result of the interaction of several different systems, such as the reward and the self-control system (e.g., Depue & Collins, Reference Depue and Collins1999; Zuckerman, Reference Zuckerman2005). The suggestion is that impulsive people are those whose Approach or reward systems can override self-control systems. Our model can capture this form of impulsivity. However, we recognize that there may be other forms of impulsivity (Revelle, Reference Revelle1997).
6.2. Learning
Because learning is central to our neural network architecture (and indeed to almost any neural network architecture), we can examine the impact of different learning experiences on the development of individual differences. Different individuals may learn different sets of cue-reward/punishment contingencies depending upon the kinds of contingencies to which they are exposed and whether those contingencies are rewarding or punishing. For instance, an individual who experiences consistent patterns of rejection at home and at school may develop strong rejection sensitivity and social anxiety, especially in those situations, because they develop a strong expectancy of being rejected in social situations.
Our model encompasses three different kinds of learning that might underlie individual differences. First, Pavlovian or Classical Conditioning plays a central role in the development of expectancies about the kinds of rewards and punishments that occur in different contexts. For different people, similar cues may become associated with very different kinds of rewards and punishments. For some people, cues to social contexts lead to expectancies of positive social outcomes, whereas for other people it leads to expectancies of negative outcomes. Second, Instrumental Conditioning is responsible for learning what is likely to happen in response to different actions, or conversely what actions to take if one desires a particular reward. Different individuals may learn different Instrumental responses to pursue the same reward. Or some individuals may learn the Instrumental responses needed to attain a particular reward or avoid a particular aversive outcome, whereas other individuals may not learn such associations or may not learn them as well.
Finally, Habits develop when there is long-term exposure to action-reward/punishment contingencies. After enacting the same action to get the same reward or avoid the same punishment over a large number of instances, the likelihood of enacting the action is no longer sensitive to the current goal state of the organism (Balleine & O’Doherty, Reference Balleine and O’Doherty2010; Dayan & Berridge, Reference Dayan and Berridge2014). Instead, it is directly triggered by the cue. Obviously, with different patterns of experience, different individuals can develop quite different Habitual responses in the same context.
7. Dynamics of personality
Above, we have laid out a neural network model of personality at an experiential and neurobiological level of scale. As we can model both individual differences in experience and individual differences in neurobiology, we can also study the interaction between learning history and underlying neurobiological differences. For example, as a number of researchers have pointed out, personality characteristics may influence what kinds of situations individuals seek out (Ickes, Snyder, & Garcia, Reference Ickes, Snyder and Garcia1997; Snyder, Reference Snyder1983), which would potentially lead to different patterns of outcomes. For instance, Extraverts may be more likely to seek out social situations and as a result of this increased exposure they may be more likely to learn about cues to social reward. In addition, other factors (e.g., chronic fatigue due to chronic inflammation) that personality psychologists normally do not attend to, might play surprising roles in personality expression (or changes over time in apparent personality) because of how they (e.g., Interoceptive signaling) affect action propensity, Incentive Motivation, and ability to switch from more goal-directed to more habit-like behavior over time (Dantzer et al., Reference Dantzer, Heijnen, Kavelaars, Laye and Capuron2014).
Examining the dynamics of persons and situations has historically been challenging. In the current work, we suggest that a neural network model of personality affords tremendous promise to allow us to build increasingly more refined models of these dynamics, leveraging emerging work on the neurobiological underpinnings that support them. For example, initial differences in propensities (e.g., dopamine functioning), wiring differences due to experiential learning, and other factors could operate together to develop and change personality over time: now, we are beginning to have tools to examine “what if” assumptions and the implications of those assumptions on behavioral outcomes. We can use such models and simulations not only to explain what we currently know (and to update them as new emerging findings suggest state of the art “plausible links”), but to suggest novel hypotheses—whose dynamics would have been hard to otherwise envision—that might be empirically examined. In this way, such models could become rich tools for the field to build a cumulative, ever more detailed and precise, model of personality dynamics.
8. Relationship to other neurobiological approaches to personality
Other neurobiologically inspired personality theories like reinforcement sensitivity theory (RST; Gray & McNaughton, Reference Gray and McNaughton2000) and Cybernetic Big Five Theory (DeYoung, Reference DeYoung2015) also draw upon the research described elsewhere in this article that pertains to Extraversion or Neuroticism, respectively. However, there are several important distinctions, especially with regard to RST’s handling of different components of Neuroticism and to its more nuanced speculations with regard to brain structures and processes that others have ascribed to Extraversion or Neuroticism, but not both.
Corr, DeYoung, and McNaughton’s figure 2 (Reference Corr, DeYoung and McNaughton2013, p. 163; adapted from McNaughton & Corr, 2004; see also Gray & McNaughton, Reference Gray and McNaughton2000, p. 276, for an earlier version of this diagram) is particularly helpful in summarizing the manner in which RST maps onto brain structures. To begin with the frontal lobes, Corr, DeYoung, and McNaughton’s (Reference Corr, DeYoung and McNaughton2013) figure suggests that the PFC’s ventral stream is associated with RST’s BIS, which governs “defensive Approach” behaviors, creates anxious affect, and may be likened to DeYoung, Quilty, and Peterson’ (Reference DeYoung, Quilty and Peterson2007) “Withdrawal” aspect of Neuroticism, whereas the PFC’s dorsal stream is associated with RST’s FFFS, which governs “defensive Avoidance” behaviors, creates fearful affect, and may be likened to DeYoung, Quilty, and Peterson’s (Reference DeYoung, Quilty and Peterson2007) “Volatility” aspect of Neuroticism. The PFC’s role in Neuroticism (and in the BIS and FFFS) may also differ by hemisphere; greater activation in the right prefrontal regions (as well as damage to left prefrontal regions) is positively correlated with Withdrawal, whereas greater activation in the left prefrontal regions is positively correlated with Volatility. Like the dorsal and ventral PFC, respectively, the cingulate cortex may be divided into two regions, anterior cingulate cortex (ACC) and posterior cingulate cortex, which Corr, DeYoung, and McNaughton (Reference Corr, DeYoung and McNaughton2013) claim are differentially associated with RST’s FFFS and BIS; these authors link the ACC to obsessive–compulsive disorder and the posterior cingulate cortex to ruminative anxiety. Given DeYoung and Allen’s (Reference DeYoung and Allenin press) statement that frontal structures are more likely to be related to Neuroticism via their effect on affect regulation than on affect generation, it is interesting to note that they also report that Neuroticism, as well as Conscientiousness, has been negatively related to ACC thickness.
DeYoung and Allen (Reference DeYoung and Allenin press) attribute the role of affect generation in Neuroticism primarily to medial temporal lobe structures, especially the septo-hippocampal system and amygdala (which ostensibly comprise the Withdrawal, or BIS-like, aspect of Neuroticism) and the midbrain’s hypothalamic-pituitary-adrenal axis (which controls Neuroticism’s Volatility, or FFFS-like, aspect).
The RST literature suggests that the amygdala is involved in both defensive Avoidance (FFFS) and defensive Approach (BIS). Specifically, Corr, DeYoung, and McNaughton (Reference Corr, DeYoung and McNaughton2013) cite Cunningham, Arbuckle, Jahn, Mowrer, and Abduljalil’s (Reference Cunningham, Arbuckle, Jahn, Mowrer and Abduljalil2010) functional magnetic resonance imaging (fMRI) study, in which Volatility predicted amygdala activity in response to stimulus valence (negative over positive), whereas Withdrawal predicted amygdala activity in response to changes in distance (Approach over Withdrawal) from stimuli, independent of valence.
Research in the RST tradition has also painted a somewhat different picture of the relationship between the hypothalamic-pituitary-adrenal axis and Neuroticism’s BIS- and FFFS-like aspects. Corr, DeYoung, and McNaughton (Reference Corr, DeYoung and McNaughton2013) maintain that the medial hypothalamus and the periaqueductal gray are capable of producing either defensive Approach or defensive Avoidance behaviors (e.g., risk assessment versus escape, defensive quiescence versus explosive panic); however, they admit that these more inferior structures are more likely to be activated in response to nearby, immediate threats than to threats that are more spatiotemporally distant. As such, they are more likely to create Avoidant behaviors characteristic of Volatility than defensive Approach behaviors characteristic of Withdrawal.
With regard to the role of various neurotransmitters, there is also much agreement between the RST literature and the basic neuroscience literature. For instance, the central role that dopamine plays in Extraversion (a positive correlate of the Behavioral Activation/Approach system, or BAS, which governs sensitivity to rewarding cues) is as undisputed among reinforcement sensitivity theorists (Pickering & Gray, Reference Pickering and Gray1999; Pickering & Pesola, Reference Pickering and Pesola2014) as it is among other neurobiologically inclined personality theorists (e.g., Depue & Collins, Reference Depue and Collins1999).
Where the RST literature differs from other lines of research is its strong emphasis on differentiating between defensive Approach and defensive Avoidance behaviors. Corr, DeYoung, and McNaughton (Reference Corr, DeYoung and McNaughton2013) suggest, for example, that higher tonic levels of serotonin do not increase one’s Stability metatrait score simply by decreasing general Neuroticism, but rather decreases scores on some facets of Neuroticism (i.e., those linked to Volatility or FFFS) while increasing others (i.e., those linked to Withdrawal or BIS). Similarly, these authors draw a distinction between cortisol reactivity and various types of aggression (reactive or defensive aggression is linked to the FFFS and to cortisol, whereas proactive or offensive aggression is not).
9. Conclusion, limitations, and future directions
A rising chorus of voices seeks a more precise neurobiologically based science of the complex dynamics underlying personality. Our goal in the current work is to contribute to that rising chorus of voices. In doing so we suggest that we can model personality at a number of different levels of scale using our best guesses based on the available literatures in personality and neuroscience to address and gain insight into what have seemed like intractable enigmas. These are exciting times. The tools available today suggest that we can dig in and really get traction on cumulatively understanding these complex personality dynamics. How can we understand the complex dynamics of persons and situations and how they operate together over time to produce emergent behavior? How could the same underlying system that results in between-subject Stability in behavior over time and reliable personality structure across individuals (e.g., the Big Five), plausibly explain broad within-person variability in behavior across situations? How do underlying biological mechanisms and different learning histories produce enduring individual differences in response to situational cues? To date we have provided computational models to explore how such enigmas might plausibly be explored. But, what we have provided thus far is more akin to plausible computational models.
It’s apparent to us that the road ahead involves a very steep slope. We must move beyond plausible computational solutions to enigmas, and use computational approaches to predict emerging patterns of individuals that we can measure (and then use those to refine our computational models). There are many challenges. First, ultimately the precision of our models of the dynamics of personality depend, in part, upon the precision of our measurement tools. For example, fMRI at present does not allow us to know the precise path of the neural circuitry signaling over time that results in various decisions, learning, and personality-related outcomes of interest. The cause-effect inferences needed for our computational models of the underlying neurobiological pathways depend on converging evidence (e.g., from animal and human studies where there can be systematic manipulation and fine-grained electrode measurements).
Second, understanding personality dynamics requires understanding how people respond in situations, including those involving others over time, while concurrently acquiring neurobiological (e.g., fMRI) data. fMRI at present is available only in the lab, so that studying more complex real-world situations and concurrently examining neurobiological circuits may require fMRI studies using more realistic gaming environments designed to be representative of real-life challenges (using intelligent agents, for example, who are modeled to be similar in their response patterns to other human partners). We have developed and used such a game in our lab, examining neural circuit patterns for men who have sex with men who are playing a virtual dating game that was designed to be representative of men’s real-life scenarios involving sexual risks. That work is quite exciting. However, the next step is a Step 1. Can we create an iterative way to computationally model men’s behavioral choices in the game and test those individual computational models against actual individual differences in men’s actual game choices and corresponding model-generated expectations regarding their changing neural patterns while playing the game? In any event, cumulatively better understanding the situations in which within and between person differences in personality dynamics over time can be more precisely assessed both Behaviorally and neurobiologically is a major challenge. So too is developing cumulatively more precise measures of those dynamics, computationally and in “real” or “virtual” time.
Financial Support:
This work was supported by the National Institute of General Medical Sciences (grant number R01GM109996).
Conflicts of Interest:
All authors have nothing to disclose.
Open access
