[Personal note: Media Streams was conceived by Marc Davis and developed between 1991 and 1997 at the Machine Understanding Group of the MIT Media Laboratory and at Interval Research Corporation by Marc Davis, Brian Williams, and Golan Levin [myself]. I was involved from the project's inception as the principal designer of its iconic visual language and as a contributing designer of its interface and interaction.]
Media Streams is a system for annotating, retrieving, repurposing, and automatically assembling digital video. It uses a stream-based, semantic representation of video content with an iconic visual language interface of hierarchically structured, composable, and searchable primitives. Media Streams addresses problems of annotation convergence and human-computer communication by creating a standardized, computationally readable and writable visual language for representing consensual interpretations of video content.
The Media Streams video content annotation system uses a vocabulary of more than 6000 icons to represent the characters, objects, behaviors and settings of the broadcast universe. Because they are combined in a grammar with a syntax and semantics—permitting meaningful combinations numbering in the hundreds of millions—the Media Streams icons are not merely an iconography but a true visual language. This generative and searchable language supports gestalt information visualization, quick recognition and browsing of annotations, the potential for global use, and the representation of semantic, relational and temporal video content. Creating the Media Streams lexicon involved knowlege-engineering a sensible relational hierarchy of thousands of concepts, and inventing a consistent and readable set of recombinable sub-iconic graphic elements.
Streams iconic annotation language has numerous advantages over traditional
keyword annotation systems, including its ability to describe relations
between descriptions, its ability to clearly render overlapping and contained
actions, its ability to refer more directly to the intrinsic visual qualities
of the video medium, and its ability to serve as a "consensus" language
for multimedia professionals.
|Detailed images and descriptions of the Media Streams user interface and visual language can be accessed from the links below:|
|[Link] The Media Streams Media Timeline, on which iconic annotations of video are temporally indexed. Each stream in the Media Timeline contains annotations about a unique aspect of video content, such as settings, characters, objects, actions, camera motions, etcetera.|
|[Link] The Media Streams Icon Space, an atemporal, hierarchically-indexed "dictionary" of iconic descriptors. The Icon Space incorporates utilities for icon construction and search.|
|[Link] A small but representative selection of icons from the Media Streams visual annotation lexicon. In the interest of space, the icons displayed here have not been arranged to reflect the grammars or knowledge hierarchies into which they are ordinarily structured.|
A Media Streams bibliography:
Davis, Marc. "Media Streams: An Iconic Visual Language for Video Annotation." In: Proceedings of 1993 IEEE Symposium on Visual Languages in Bergen, Norway , IEEE Computer Society Press, 196-202, 1993b.
Davis, Marc. "Media Streams: An Iconic Visual Language for Video Representation." In: Readings in Human-Computer Interaction: Toward the Year 2000, ed. Ronald M. Baecker, Jonathan Grudin, William A. S. Buxton, and Saul Greenberg. 854-866. 2nd ed., San Francisco: Morgan Kaufmann Publishers, Inc., 1995.
Davis, Marc. "Media Streams: Representing Video for Retrieval and Repurposing." Ph.D. Thesis, Massachusetts Institute of Technology, 1995.
Levin, Golan. "Media Streams 3.0: User's Guide and Manual." Internal document. Palo Alto, California: Interval Research Corporation, 1994. Also included as an appendix in Marc Davis' Ph.D. Thesis (see above).