We agree with the authors that motivation is often viewed as a high-level construct, defined by many researchers as a causal determinant of behavior in a “black-box” fashion. We also agree that to understand motivation means to understand the “spring to action.” Beyond the conventional constructs of high-level motives, we need to define what and how this energization underwrites action selection or choice behavior.
We take this opportunity to rehearse the key arguments in Murayama & Jach, as seen through the lens of the predictive processing (a.k.a., active inference) account of motivated behavior. Active inference is sometimes portrayed as an account of sentient behavior, implying actions driven by the process of sense making. In this account, perception is formulated as a process of inference and, thereby, rests on a calculus of beliefs – sometimes referred to as Bayesian mechanics (Ramstead et al., Reference Ramstead, Sakthivadivel, Heins, Koudahl, Millidge, Da Costa and Friston2023) or self-evidencing (Hohwy, Reference Hohwy2016). Meanwhile, active inference suggests that behavior can be reflexive or planned, depending on whether it aims to minimize prediction errors or anticipates future consequences. The concept of expected surprise, derived from this framework, encompasses both uncertainty resolution and avoidance of unexpected outcomes, guiding goal-directed behavior. This explanation of motivated behavior is grounded in statistical physics and highlights both the inherent role of intrinsic value (i.e., epistemic affordance) and extrinsic value (i.e., instrumental affordance). We begin by critically examining “reward learning models of information seeking,” followed by an outline how self-organization processes can drive motivated behavior.
The paper's emphasis is on “reward learning models of information seeking behavior” suggesting that information gain has intrinsic value and serves as a source of motivation. We argue this view can be replaced – or at least be elaborated – under an active inference formulation. Arguably, the most crucial aspect is that prior preferences, which form the basis of expected value, encompass all aspects of sensory experience, and cannot be simplified to a mere reward function. In other words, these preferences function to prevent surprising outcomes that would diverge from an individual's typical expectations, maintaining consistency with their self-concept (i.e., “kind of thing that I am”). Consequently, the expected free energy can be decomposed into intrinsic value (i.e., epistemic affordance) and extrinsic value (i.e., instrumental affordance).
More explicitly, the “kind of thing I am” refers to a necessary aspect of entities capable of self-organization; specifically, those with the ability to infer the consequences of their actions. The imperative is to maximize the evidence (a.k.a., marginal likelihood) for generative models of how observations are caused. In contexts where individuals are actively making decisions, their beliefs about what actions to take are influenced by the expected free energy associated with each possible choice. Essentially, they weigh the potential consequences of each option and choose the one with the most favorable expected outcome in terms of minimizing expected surprise or uncertainty. This is in contrast with expected utility theory, in which there is merely one specific kind of outcome that is considered the most desirable, determining the utility or reward function (e.g., monetary payoff). However, in active inference, this approach is replaced with a system where preferences guide decision-making rather than a singular utility function. This means that instead of aiming to maximize a monothematic payoff, individuals prioritize choices based on their preferences among various possible outcomes. Your view of who you are determines how you encounter every aspect of an observable outcome. Imagine you are a student of cognitive neuroscience who is to be examined about the content of this BBS article. You face a trade-off between time spent reading the article and making dinner for your friends. Quickly reading through the article might be enough to pass the exam while leaving you adequate time to make a meal. Your approach may vary depending on your self-perception, such as whether you view yourself as an exceptional student or not. Ultimately, your actions are likely guided more by personal preferences and considerations than by a single reward function.
The big question is now: How do mental computational processes self-organize? Mental computational processes have the capacity to infer the consequences of actions by minimizing surprise and prediction errors. This endows generative models with a future-pointing aspect and the notion of planning (as inference). This perspective suggests that humans and animals often exhibit behavior aimed at mastering the environment, driven by a combination of intrinsic and extrinsic values. This dual aspect decomposition of affordances suggests that agents are compelled to explore their environments to maximize information gain – actively gather evidence to build and optimize world models – while being sensitive to the constraints of their preferences and goals. This interpretation aligns with the notion that humans can recognize regularities and creating mental categories from their own behaviors and subjective experiences. Formally, this can be expressed as minimizing an evidence bound called variational free energy (Winn & Bishop, Reference Winn and Bishop2005) that comprises complexity and accuracy (Ramstead et al., Reference Ramstead, Sakthivadivel, Heins, Koudahl, Millidge, Da Costa and Friston2023):
Complexity scores the divergence between prior beliefs (before seeing outcomes) and posterior beliefs (after seeing outcomes), while accuracy corresponds to the goodness of fit. In short, complexity scores the information gain or cost of changing one's mind in an information theoretic and thermodynamic sense, respectively. Through repeated interactions with the environment, the brain updates its models based on the prediction errors it encounters. This self-organized learning process allows the brain to proactively infer what actions will lead to expected outcomes, adaptively learning from mistakes and adjusting future behavior to minimize surprises. By understanding how different situations affect outcomes and how actions unfold over time, these cognitive systems can plan actions strategically to minimize surprises and errors in the long term, not just at that moment. This decomposition furnishes a complementary perspective on the complex interplay between extrinsic and intrinsic motivation.
In summary, integrating the predictive processing framework into our understanding of motivation offers promising avenues for theoretical development, shedding light on the computational processes underlying motivated behavior and providing insights into how individuals adapt to and interact with their environment.
We agree with the authors that motivation is often viewed as a high-level construct, defined by many researchers as a causal determinant of behavior in a “black-box” fashion. We also agree that to understand motivation means to understand the “spring to action.” Beyond the conventional constructs of high-level motives, we need to define what and how this energization underwrites action selection or choice behavior.
We take this opportunity to rehearse the key arguments in Murayama & Jach, as seen through the lens of the predictive processing (a.k.a., active inference) account of motivated behavior. Active inference is sometimes portrayed as an account of sentient behavior, implying actions driven by the process of sense making. In this account, perception is formulated as a process of inference and, thereby, rests on a calculus of beliefs – sometimes referred to as Bayesian mechanics (Ramstead et al., Reference Ramstead, Sakthivadivel, Heins, Koudahl, Millidge, Da Costa and Friston2023) or self-evidencing (Hohwy, Reference Hohwy2016). Meanwhile, active inference suggests that behavior can be reflexive or planned, depending on whether it aims to minimize prediction errors or anticipates future consequences. The concept of expected surprise, derived from this framework, encompasses both uncertainty resolution and avoidance of unexpected outcomes, guiding goal-directed behavior. This explanation of motivated behavior is grounded in statistical physics and highlights both the inherent role of intrinsic value (i.e., epistemic affordance) and extrinsic value (i.e., instrumental affordance). We begin by critically examining “reward learning models of information seeking,” followed by an outline how self-organization processes can drive motivated behavior.
The paper's emphasis is on “reward learning models of information seeking behavior” suggesting that information gain has intrinsic value and serves as a source of motivation. We argue this view can be replaced – or at least be elaborated – under an active inference formulation. Arguably, the most crucial aspect is that prior preferences, which form the basis of expected value, encompass all aspects of sensory experience, and cannot be simplified to a mere reward function. In other words, these preferences function to prevent surprising outcomes that would diverge from an individual's typical expectations, maintaining consistency with their self-concept (i.e., “kind of thing that I am”). Consequently, the expected free energy can be decomposed into intrinsic value (i.e., epistemic affordance) and extrinsic value (i.e., instrumental affordance).
More explicitly, the “kind of thing I am” refers to a necessary aspect of entities capable of self-organization; specifically, those with the ability to infer the consequences of their actions. The imperative is to maximize the evidence (a.k.a., marginal likelihood) for generative models of how observations are caused. In contexts where individuals are actively making decisions, their beliefs about what actions to take are influenced by the expected free energy associated with each possible choice. Essentially, they weigh the potential consequences of each option and choose the one with the most favorable expected outcome in terms of minimizing expected surprise or uncertainty. This is in contrast with expected utility theory, in which there is merely one specific kind of outcome that is considered the most desirable, determining the utility or reward function (e.g., monetary payoff). However, in active inference, this approach is replaced with a system where preferences guide decision-making rather than a singular utility function. This means that instead of aiming to maximize a monothematic payoff, individuals prioritize choices based on their preferences among various possible outcomes. Your view of who you are determines how you encounter every aspect of an observable outcome. Imagine you are a student of cognitive neuroscience who is to be examined about the content of this BBS article. You face a trade-off between time spent reading the article and making dinner for your friends. Quickly reading through the article might be enough to pass the exam while leaving you adequate time to make a meal. Your approach may vary depending on your self-perception, such as whether you view yourself as an exceptional student or not. Ultimately, your actions are likely guided more by personal preferences and considerations than by a single reward function.
The big question is now: How do mental computational processes self-organize? Mental computational processes have the capacity to infer the consequences of actions by minimizing surprise and prediction errors. This endows generative models with a future-pointing aspect and the notion of planning (as inference). This perspective suggests that humans and animals often exhibit behavior aimed at mastering the environment, driven by a combination of intrinsic and extrinsic values. This dual aspect decomposition of affordances suggests that agents are compelled to explore their environments to maximize information gain – actively gather evidence to build and optimize world models – while being sensitive to the constraints of their preferences and goals. This interpretation aligns with the notion that humans can recognize regularities and creating mental categories from their own behaviors and subjective experiences. Formally, this can be expressed as minimizing an evidence bound called variational free energy (Winn & Bishop, Reference Winn and Bishop2005) that comprises complexity and accuracy (Ramstead et al., Reference Ramstead, Sakthivadivel, Heins, Koudahl, Millidge, Da Costa and Friston2023):
Complexity scores the divergence between prior beliefs (before seeing outcomes) and posterior beliefs (after seeing outcomes), while accuracy corresponds to the goodness of fit. In short, complexity scores the information gain or cost of changing one's mind in an information theoretic and thermodynamic sense, respectively. Through repeated interactions with the environment, the brain updates its models based on the prediction errors it encounters. This self-organized learning process allows the brain to proactively infer what actions will lead to expected outcomes, adaptively learning from mistakes and adjusting future behavior to minimize surprises. By understanding how different situations affect outcomes and how actions unfold over time, these cognitive systems can plan actions strategically to minimize surprises and errors in the long term, not just at that moment. This decomposition furnishes a complementary perspective on the complex interplay between extrinsic and intrinsic motivation.
In summary, integrating the predictive processing framework into our understanding of motivation offers promising avenues for theoretical development, shedding light on the computational processes underlying motivated behavior and providing insights into how individuals adapt to and interact with their environment.
Financial support
None.
Competing interest
None.