A multimedia system, including a human, understands the domain of discussion by correlating and combining partial information from multiple media to get complete information. In most situations, without correlating the information from multiple data streams, one cannot extract information about the real world. Even in those systems where multimedia is for humans’ direct consumption, all correlated information must be presented, or rendered, for humans to extract information that they need from the multimedia data. It is well known that humans combine information from multiple sensors and apply extensive knowledge from various sources to form the models of the real world in which they live. Humans also use different media, assuming the familiarity and knowledge of a recipient, in communicating their experiences and knowledge of the world. Most human experiences are multimedia and are easily captured, stored, and communicated using multimedia.

We already discussed the nature of sensory data and the emerging nature of documents that are increasingly inherently multimedia. Here we discuss the nature of individual sensor streams with a goal to convert them to a multimedia stream. We also start the discussion on how a computing system should view multimedia to develop systems that are not just a simple combination of multimedia, but are inherently multimedia such that they are much more than the sum of their component media elements.