Inference and Learning from Data Inference

The temporal learning algorithms TD(0) and TD(λ) of the previous chapter are useful procedures for state value evaluation; i.e., they permit the estimation of the state value function vπ(s) for a given target policy π(a|s) by observing actions and rewards arising from this policy (on‐policy learning) or another behavior policy (off‐policy learning).In most situations, however, we are not interested in state values but rather in determining optimal policies, denoted by π⋆(a|s) (i.e., in selecting what optimal actions an agent should follow in a Markov decision process (MDP)).