Statistical decision theory provides a general account of perceptual decision-making in a wide variety of tasks that range from simple target detection to complete identification. The fundamental assumptions are that all sensory representations are inherently noisy and that every behavior, no matter how trivial, requires a decision. Statistical decision theory is referred to as signal detection theory (SDT) when the stimuli vary on only one sensory dimension, and general recognition theory (GRT) when the stimuli vary on two or more sensory dimensions. SDT and GRT are both reviewed. The SDT review focuses on applications to the two-stimulus identification task and multiple-look experiments, and on response-time extensions of the model (e.g., the drift-diffusion model). The GRT review focuses on applications to identification and categorization experiments, and in the former case, especially on experiments in which the stimuli are constructed by factorially combining several levels of two stimulus dimensions. The basic GRT properties of perceptual separability, decisional separability, perceptual independence, and holism are described. In the case of identification experiments, the summary statistics method for testing perceptual interactions is described, and so is the model-fitting approach. Response time and neuroscience extensions of GRT are reviewed.

Vision science combines ideas from physics, biology, and psychology. The language and ideas of mathematics help scientists communicate and provide an initial framing for understanding the visual system. Mathematics combined with computational modeling adds important realism to the formulations. Together, mathematics and computational tools provide a realistic estimate of the initial signals the brain analyzes to render visual judgments (e.g., motion, depth, and color). This chapter first traces calculations from the representation of the light signal, to how that signal is transformed by the lens to the retinal image, and then how the image is converted into cone photoreceptor excitations. The central steps in the initial encoding rely heavily on linear systems theory and the mathematics of signal-dependent noise. We then describe computational methods that add more realism to the description of how light is encoded by cone excitations. Finally, we describe the mathematical formulation of the ideal observer using all the encoded information to perform a visual discrimination task, and Bayesian methods that combine prior information and sensory data to estimate the light input. These tools help us reason about the information present in the neural representation, what information is lost, and types of neural circuits for extracting information.

The idea that memory behavior relies on a gradually changing internal state has a long history in mathematical psychology. This chapter traces this line of thought from statistical learning theory in the 1950s, through distributed memory models in the latter part of the twentieth century and early part of the twenty-first century through to modern models based on a scale-invariant temporal history. We discuss the neural phenomena consistent with this form of representation and sketch the kinds of cognitive models that can be constructed and connections with formal models of various memory tasks.

Although learning was a key focus during the early years of mathematical psychology, the cognitive revolution of the 1960s caused the field to languish for several decades. Two breakthroughs in neuroscience resurrected the field. The first was the discovery of long-term potentiation and long-term depression, which served as promising models of learning at the cellular level. The second was the discovery that humans have multiple learning and memory systems that each require a qualitatively different kind of model. Currently, the field is well represented at all of Marr’s three levels of analysis. Descriptive and process models of human learning are dominated by two different, but converging, approaches – one rooted in Bayesian statistics and one based on popular machine-learning algorithms. Implementational models are in the form of neural networks that mimic known neuroanatomy and account for learning via biologically plausible models of synaptic plasticity. Models of all these types are reviewed, and advantages and disadvantages of the different approaches are considered.

The investigation of processes involved in merging information from different sensory modalities has become the subject of research in many areas, including anatomy, physiology, and behavioral sciences. This field of research termed "multisensory integration’’ is flourishing, crossing borders between psychology and neuroscience. The focus of this chapter is on measures of multisensory integration based on numerical data collected from single neurons and in behavioral paradigms:spike numbers, reaction time, frequency of correct or incorrect responses in detection, recognition, and discrimination tasks. Defining that somewhat fuzzy term, it has been observed that at least some kind of numerical measurement assessing the strength of crossmodal effects is required. On the empirical side, these measures typically serve to quantify effects of various covariates on multisensory integration like age, certain disorders, developmental conditions, training and rehabilitation, in addition to attention and learning. On the theoretical side, these measures often help to probe hypotheses about underlying integration mechanisms like optimality in combining information or inverse effectiveness, without necessarily subscribing to a specific model.