Understanding and exploiting the principles of Human-Computer Interaction (HCI) is a major factor influencing the continued growth of the information technology industry. IBM Research has a long history of innovative work in this area. Successful systems, such as the Audio Distribution System, the Olympic Message System, and Query by Example, have been refined by a deeper understanding of human capabilities and limitations. HCI research at IBM continues today across our research locations around the globe. Our work spans the creation and refinement of enabling technologies, such as speech recognition and gaze tracking, to fundamental research in the way people gain knowledge.

Research into enabling technologies can open new vistas in the way we interact with our environment. For example, if our environment could see, hear and understand us, it could help us in numerous ways. The goal of the BlueEyes project at the Almaden Research Center is to give computers perceiving ability. Using video cameras, microphones, and other sensors, computers can extract information such as where users are looking, their facial expressions, what they are saying, and how they are gesturing. The computer can then respond to users, performing expected actions, and providing helpful information.

Not only do we want our environment to understand us, we would like better control of our environment. Controlling the appliances in our lives should be easy, efficient, and enjoyable. TrackPoint and ScrollPoint Mouse are successful examples of our research in this area. As computers pervade more devices, other forms of interfacing with them must be developed. Ongoing research includes using eye tracking, speech, and alternative text entry methods to efficiently and naturally interact with future pervasive computing devices and environments.

Combining systems capable of multimedia input and output brings us to a very exciting possibility --- being able to obtain information, not through an assortment of independently predesigned Web sites, but rather through an intelligent multimedia conversation that is tailored to the task you are performing, customized to your personal preferences, and adapted to your context and interaction devices. To realize this vision, we are building an exploratory system called Responsive Information Architect that engages users in automatically generated interactive multimedia, multimodal conversations. We use a combination of knowledge-based planning and machine learning approaches to dynamically determine the contents of the response, to select the media for encoding the contents, and to automatically compose coordinated multimedia streams with embedded interaction controls.

Conversational technologies have been pioneered at IBM since the early '70s, with our research group tackling speech and language problems using unique statistical modeling approaches. IBM's highly successful dictation product, ViaVoice, is the result of a collaboration across multiple research and development sites, and is currently available in 12 different languages. Other recognition projects include transcription of radio broadcast news and people's voice-mail. Related activities include speech biometrics (speaker verification/identification), speech synthesis, and audiovisual speech and speaker recognition. User interface work, speech recognition, and natural language processing have come together in the Donald project, a personal assistant to allow natural interaction with e-mail and calendar applications, and the Personal Speech Assistant, an attachment we have built to allow for natural speech interaction with a Palm Pilot.

As we explore conversational technologies, we must begin studying how people naturally interact with each other. The goal of our work in Social Computing is to design systems that support long-running, coherent, productive communication among people. This can be achieved by designing digital systems that exhibit social translucence, i. e., that make visible socially relevant information about the presence and activity of online participants. People can then draw on their social knowledge (e. g., if there's a crowd, it must be interesting). To explore these ideas, we are developing a system called Loops, which features social proxies that provide minimalist graphical visualizations of the presence and activities of people participating in online interactions.

Finally, how do we form concepts about what we see and hear? In one project, we are studying how stories provide an effective way for people to create and share knowledge. We are developing technologies to help people create, capture, organize, utilize, and understand stories. We are also addressing questions of concept formation and classification learning. (Can you tell a Monet from a Bonnard? How do you learn to make such a distinction?) Our goal is to understand how people learn outside traditional educational contexts, and to explore how technology can facilitate such learning. We are now working with the Museum of Modern Art in New York City to examine how people discover complex abstract concepts and acquire general analytic skills.

Please contact Paridhi Verma to obtain copies of the Computer Science Brochure