Recent projects of the Intelligent Information Analysis Group:

• MARVEL: Multimedia Analysis and Retrieval System (demo):

The MARVEL system consists of two components: the MARVEL multimedia analysis engine and the MARVEL multimedia search engine. 

·        The MARVEL multimedia analysis engine – applies machine learning techniques to model semantic concepts in video from automatically extracted audio, speech, visual content.  It automatically assigns labels (with associated confidence scores) to new video data to reduce manual annotation load and improve searching and organizes semantic concepts using ontologies that exploit semantic relationships for improving detection performance.

·        The MARVEL multimedia retrieval engine – integrates multimedia semantics-based searching with other search techniques (speech, text, metadata, audio-visual features, etc.).  It also combines content-based, model-based, and text-based searching for video searching.

Figure: The MARVEL multimedia search engine allows searching over large video repository using automatically generated semantic labels.

• SLAM: Semantic Learning and Analysis of Multimedia

  Bridging the gap between features and semantics....  Multimedia content is an essential part of information technology. However, the difficulty in filtering, searching, and summarizing video has so far hindered the effective utilization of video databases. Users want to filter and query video by high-level (semantic) concepts, while automatic algorithms can extract only low-level features (e.g., color, texture, shape, amount of motion). Bridging this gap is thus the most challenging problem in video (multimedia) indexing, summarization, retrieval, and filtering.