Multimedia research in IBM covers a wide area. Traditionally, the central issues have been the management of digital images and video, providing efficient encoding, storage, transmission, and display. IBM has been instrumental in providing digital images of archival fine arts quality in partnership with such institutions as the Vatican Library and the Wyeth Brandywine Museum. Recently, a team from the Haifa Research Laboratory developed an interactive guide to the Hermitage Museum in St. Petersburg. IBM researchers played leading roles in the definitions of the international standards for compression and storage of facsimile, image, DVD, and MPEG-1 and MPEG-2 digital video. We participate actively in the follow-ons, MPEG-4 and MPEG-7. Our work on signal processing techniques for digital TV broadcast has led to algorithms for MPEG encoding and decoding, and their implementations as chips now widely used in broadcast TV encoders and set-top box decoders.

Current focuses

Our emphasis is now turning to defining and prototyping interactive services in enhanced TV that take advantage of the set-top box as a new, ubiquitous computing platform. We also continue to explore new algorithms for studio applications of MPEG video encoding in both standard and high-definition TV.

Researchers in Watson, Beijing, and Haifa are building tools for creating both broadcast and lower bandwidth streaming video with interactive content and special effects. We participate in the NIST-sponsored development of next-generation tools and infrastructure for broadcast TV, and also have contributed much of the content of the Hot Video HotMedia line of video-rich media creation tools that IBM now markets.

Our audio/video technologies group at Haifa has been developing technologies for various platforms and architectures. We have worked on audio and video compression and modeling, speech synthesis, as well as processing and streaming of both audio and video over IP networks. We have also developed the Java® Media Framework (JMF) in collaboration with Sun Microsystems.

The Almaden Research Center, with traditional strengths in database technologies, has pioneered the development of tools for analysis, indexing, and organizing image and video libraries. Our latest project in this area includes the Cue Video set of tools for breaking large amounts of video into scenes, linking metadata from various sources to the actual shots, and allowing retrieval of scenes like a given scene.

New Challenges

Video is no longer only a broadcast medium, but has become an interactive environment in which avatars and virtual reality play a role. We see an expanding role for these technologies, and are exploring them in collaboration with several university research groups. Serving video to networks has become a big business. We are researching isochronous streaming technologies for media and data, as well as their applications. Current projects include isochronous stream processing; multicast store, filter, and distribution system.

A major source of new interest and new problems in multimedia is the rapid increase in our capability to sense the environment around us, using inexpensive cameras, intelligent pens and pen-sensing tablets, and even exotic effects like electric field sensing of a user's presence. Recognition of the messages input to a system through these new sensors is an active area of research. (See Human Computer Interaction on p. 26.)

Cameras provide even richer input, but require definition of gestural vocabularies to convey information. The availability of very inexpensive means of tracking facial orientation and eye gaze have stimulated several research efforts in these areas. Tools for assisting those with severe handicaps to interact with a computer using visual input have already been transferred from these efforts, but the ultimate goal is to have techniques usable by everyone and intuitive enough not to require special training.

Please contact Paridhi Verma to obtain copies of the Computer Science Brochure