Clearly everybody knows the word “multimedia,” yet when people think of it, they usually think of different things. For some people, multimedia equals entertainment. For other people, multimedia equals Web design. For many computer scientists, multimedia often means video in a computing environment. All these are narrow perspectives on multimedia. For example, visual information definitely dominates human activities because of the powerful visual machinery that we are equipped with. In the end, however, humans use all five senses effectively, opportunistically, and judiciously. Therefore, multimedia computing should utilize signals from multifarious sensors and present to users only the relevant information in the appropriate sensory modality.

This book takes an integrative systems approach to multimedia. Integrated multimedia systems receive input from different sensory and symbolic sources in different forms and representations. Users ideally access this information in experiential environments. Early techniques dealt with individual media more effectively than with integrated media and focused on developing efficient techniques for separate individual media, for example, MPEG video compression. During the past few years, issues that span multimedia have received more central attention. Many researchers now recognize that most of the difficult semantic issues become easier to solve when considering integrated multimedia rather than separate individual media.