Skip to main content Accessibility help
Internet Explorer 11 is being discontinued by Microsoft in August 2021. If you have difficulties viewing the site on Internet Explorer 11 we recommend using a different browser such as Microsoft Edge, Google Chrome, Apple Safari or Mozilla Firefox.

Chapter 8: Multimodal Integration and Synchronization

Chapter 8: Multimodal Integration and Synchronization

pp. 82-94

Authors

, International Computer Science Institute, Berkeley, California, , University of California, Irvine
Resources available Unlock the full potential of this textbook with additional resources. There are free resources available for this textbook. Explore resources
  • Add bookmark
  • Cite
  • Share

Summary

A multimedia system, including a human, understands the domain of discussion by correlating and combining partial information from multiple media to get complete information. In most situations, without correlating the information from multiple data streams, one cannot extract information about the real world. Even in those systems where multimedia is for humans’ direct consumption, all correlated information must be presented, or rendered, for humans to extract information that they need from the multimedia data. It is well known that humans combine information from multiple sensors and apply extensive knowledge from various sources to form the models of the real world in which they live. Humans also use different media, assuming the familiarity and knowledge of a recipient, in communicating their experiences and knowledge of the world. Most human experiences are multimedia and are easily captured, stored, and communicated using multimedia.

We already discussed the nature of sensory data and the emerging nature of documents that are increasingly inherently multimedia. Here we discuss the nature of individual sensor streams with a goal to convert them to a multimedia stream. We also start the discussion on how a computing system should view multimedia to develop systems that are not just a simple combination of multimedia, but are inherently multimedia such that they are much more than the sum of their component media elements.

About the book

Access options

Review the options below to login to check your access.

Purchase options

eTextbook
US$98.00
Hardback
US$98.00

Have an access code?

To redeem an access code, please log in with your personal login.

If you believe you should have access to this content, please contact your institutional librarian or consult our FAQ page for further information about accessing our content.

Also available to purchase from these educational ebook suppliers