Our Mission

The Department of Intelligent Multimedia Interaction (IMI) at IBM T. J. Watson is a research group concentrating on building next-generation intelligent multimodal, multimedia interaction systems. In particular, we use the word multimodal to refer to various input modalities that can be used by users to express their information seeking needs to a computer, including natural language, gesture, GUI, and gaze. In addition, we use the word multimedia to refer to all output channels that a computer can use to act/react upon users' input, including animated graphics, text, speech, and video.

Our work is centered around the development of new methodologies and metaphors for enabling a full-fledged multimodal, multimedia human-computer conversation. Using the multimodal, multimedia conversation metaphor, users can navigate through large and complex information spaces naturally and effectively by exploiting multiple natural input channels. In addition, presented with a customized multimedia tour of information, users can comprehend rich information easily. Ultimately, our work enables a new computing paradigm, proactive computing, in which computers are no longer passive tools that are driven by human users. Instead, computers will become proactive human companions, which can help us seek and acquire information and knowledge in a much more efficent and effective manner.

Our Strength

Due to the interdisciplinary nature of our mission, IMI is made up of researchers from multiple research disciplines. In particular, our strength lies in the following areas: multimodal conversation systems, automated multimedia authoring, natural speech generation, hetergeneous information representation and inferencing, pervasive UIs, and 2D/3D graphics user interfaces. In addition to the close intra-departmental collaboration, we have also established collaboration ties with researchers within and outside of IBM research.