The human sensory system, at least in its early stages, consists of multiple channels for different modalities (e.g., vision, audition) and for different attributes in each modality (color, motion). Temporal congruency is a critical factor in the binding of signals across channels, but little is known about what representations and algorithms are used for matching. We first analyze this mechanism from a general theoretical point of view and then address the specific mechanisms underlying the perception of color–motion synchrony and audiovisual simultaneity. We hypothesize that judgments about cross-channel temporal relations are based on the comparison of time markers by a mid-level perceptual process. The time markers are amodal tokens that reference salient, figural features extracted from early-level sensory signals. A temporal marker should reference the time a specific event occurs in the world rather than the time the processing of the event completes in the brain.

How do human observers determine the relative timings of different events? One perspective, which I shall refer to as the brain–time theory of perception, suggests that apparent timing is related to when specific analyses are concluded within distinct and relatively independent regions of the brain. This proposal is controversial, not least because it suggests that temporal perception is error prone and subject to the rates at which analyses are concluded in different parts of the brain. One observation that may favor this perspective is that physically coincident changes in color and direction can appear asynchronous – a perceptual asynchrony. In this chapter I will review the theoretical interpretations and empirical evidence that relate to this phenomenon. I will argue that this timing illusion provides good evidence for a relationship between the time courses of sensory processing in the brain and perceived timing.

The term “saccadic chronostasis” refers to the subjective temporal lengthening of a visual stimulus perceived following a saccadic eye movement. In this chapter, we discuss our preferred account of the illusion, which posits that the onset of the postsaccadic stimulus is antedated to a moment just prior to movement initiation, and review supporting evidence that illustrates key characteristics of the illusion, including its dependency on saccade extent. We conclude with a brief discussion of other examples of biased time perception that have been linked to saccadic chronostasis.

Due to neuromuscular delays and the inertial properties of the arm people must consider where a moving object will be in the future if they want to intercept it. We previously proposed that people automatically aim ahead of moving objects they are trying to intercept because they pursue such objects with their eyes, and objects that are pursued with the eyes are mislocalized in their direction of motion. To test this hypothesis we examined whether asking subjects to fixate a static point on a moving target's path, rather than allowing them to pursue the target with their eyes, makes them try to intercept the target at a point that the target has already passed. Subjects could not see their hand during the movement and received no feedback about their performance. They did tend to cross the target's path later – with respect to when the target passed that position – when not pursuing the target with their eyes, but the effect of fixation was much smaller than we predicted, even considering that the subjects could not completely refrain from pursuing the moving target as their hand approached it. Moreover, when subjects first started to move, their hands did not aim farther ahead when pursuing the target than when trying to fixate. We conclude that pursuing the target with one's eyes may be important for interception, but not because it gives rise to localization errors that predict the target's displacement during the neuromuscular delay.

How does the visual system provide us with the perception of a continuous and stable world in the face of the spatial–temporal chaos that characterizes its input? In this chapter we summarize several programs of research that all point to a solution we refer to as object updating. We use this phrase because perceptual continuity seems to occur at an object level (as opposed to an image level or a higher conceptual level) and because our research suggests that the visual system makes a sharp distinction between the formation of new object representations versus the updating of existing object representations. We summarize the research that led us to this view in the areas of masking by object substitution, the flash-lag illusion, response priming, and an illusion of perceptual asynchrony.

Quasi-periodic or “discrete” brain processes are, in theory, susceptible to a phenomenon known in engineering as “temporal aliasing.” When the rate of occurrence of events in the world is fast enough, the perceived direction of these events may be reversed. We have recently demonstrated that, because of a quasi-periodic attentional capture of motion information, continuously moving objects are sometimes perceived to move in the wrong direction (the “continuous Wagon Wheel Illusion”). Using a simple Fourier energy model of motion perception, we established that this type of attentional capture occurs at a rate of about 13 Hz. We verified with EEG recordings that the electrophysiological correlates of this illusion are restricted to a specific frequency band around 13 Hz, over right parietal regions – known for their involvement in directing attention to temporal events. We summarize these results and discuss their implications for visual attention and awareness.

Space and time are modes by which we think and not the conditions in which we live.

The locations of stationary objects appear invariant, although saccadic eye movements shift the images of physically stationary objects on the retina. Two features of this perceptual stability related to saccades are that postsaccadic locations of objects appear invariant relative to their appearance in the presaccadic view, and perception of postsaccadic stimulation is free from interference by remnants of presaccadic stimulation. To generate stability, quantitatively accurate cancellation between retinal input (RI) and extraretinal eye position information (EEPI) must occur, and persisting influences from the presaccadic view must be eliminated. We describe experiments with briefly flashed visual stimuli that have measured (1) the time course of perisaccadic spatial localization, (2) the interfering effects of persisting stimulation prior to the postsaccadic period, (3) the achievement of perceptual stability by removing visual persistence early, and (4) the influence of metacontrast utilizing the normal perisaccadic spatiotemporal distribution of retinal input to prevent interference from visual persistence.

For the steady eye, a generalized cancellation mechanism is analyzed through studying mislocalizations in perceptual orientation and visually guided manual behavior produced by (1) modifying EEPI in observers with experimental partial paralysis (curare) of the extraocular muscles and/or (2) modifying RI by varying visual field orientation (i.e., its pitch and/or roll). The influences of visual pitch and roll derive from the retinal orientations of individual straight lines and their combinations, with the identical lines influencing perceived verticality and elevation. […]

Perceived duration of interstimulus intervals is influenced by the spatial configuration of stimuli. When participants judge the two intervals between a sequence of three stimuli presented with different spatial distances, a greater distance between two stimuli makes the corresponding time interval appear longer (kappa effect, Experiment 1). By employing a choice-reaction time task, we demonstrate that this effect is at least partly due to a facilitating influence of the preceding stimulus on the timing of the subsequent one while the timing of the first stimulus presented is not influenced by the subsequent one. Moreover, reaction times to the subsequent stimulus increased with spatial distance between successive stimuli, and this was valid for a three-stimulus condition (Experiment 2) as well as for a two-stimulus condition (Experiment 3). Thus, our results provide evidence for spatial priming in the temporal kappa effect.

In the “chopstick illusion” (Anstis 1990, 2003) a vertical and horizontal line overlapped to form a cross and followed clockwise circular orbits in counterphase, with one line being at 6 o'clock when the other was at 12 o'clock. The intersection of the lines moved counterclockwise, but it was wrongly perceived as rotating clockwise. This chopstick illusion reveals how moving objects are parsed, based upon the intrinsic and extrinsic terminators of lines viewed through apertures. We conclude that intersections were not parsed as objects, but instead the motion of the terminators (tips) propagated along the lines and was blindly assigned to the intersection. In the similar “sliding rings illusion,” we found that observers could use their eyes to track intersections only when these appeared rigid and not when they appeared to slide. Conclusion – smooth pursuit eye movements are under top-down control and are compelled to rely upon perceptual interpretation of objects.

In the “flash-lag” effect, a static object that is briefly flashed up next to a moving object appears to lag behind the moving object (Nijhawan 2002). We superimposed a flashed spot on a chopsticks intersection that appeared to be moving clockwise along a circular path but was actually moving counterclockwise. We found that the flash appeared displaced clockwise. This was appropriate to the physical, not the subjective direction of rotation, indicating that the flash-lag and the chopstick illusions coexist without interacting. Similarly, the flash-lag effect was unaffected by reversed phi. […]

Anticipation is a hallmark of skilled movements. For example, when removing plates from a loaded tray, the upward force generated by the supporting hand is reduced in anticipation of the reduced load. An adjustment of the postural force occurs as a result of the predicted consequences of the self-initiated action. Although the effect of anticipatory processes is easily discerned in the actions themselves, it is unclear whether these processes also affect our perceptual experience. In this chapter we focus on the relationship between action and the perceptual experience. We begin by reviewing how actions provide reliable predictions of forthcoming sensory information. Following this, we discuss how the anticipation of the time of external events is an important component of action-linked expectations. Finally, we report two experiments that examine how temporal predictions are integrated with the incoming sensory information, evaluating whether this integration occurs in a statistically optimal manner. This predictive process provides the important advantage of compensating for lags in conduction time between peripheral input and the central integration of this information, thus overcoming the physical limitations of sensory channels.

“Real-time sensorimotor control requires the sampling and manipulation not only of parameters representing space but also of those representing time. In particular, when the system itself has inherent processing delays, it invites a situation in which sampled parameters from a peripheral sensor may no longer be valid at the time they are to be used, due to the change in state that took place during the processing delay” (Dominey et al. 1997). In this chapter, we focus on the situation in which a visual stimulus is flashed near the time of a saccade, and the subject's task is to orient the eyes toward the site where the stimulus has been. To perform this task in complete darkness, the subject's brain has to rely on only two signals: retinal error signal and internal eye position signal (iEPS). This is one of the most interesting situations in which the brain has to compute something in the face of specific physical odds (e.g., very long latencies), and we have some hints on how it proceeds. We analyze the time course of the iEPS – which appears quite distorted – using electrical stimulation of brain structures, instead of natural visual stimuli, to provide the goal to be localized. Different hypotheses are then discussed regarding the possible source and possible neural correlate of the iEPS.