Skip to main content

IBM R&D Labs in Israel News

IBM Haifa Research Develops Technology to Aid Human Memory

New Software Helps People Recall Names, Faces and Information.

Haifa, Israel, July 29, 2008 - As people today are overwhelmed with information through the Internet, TV and other sources, and they are faced with the challenge of remembering and recalling key data points on a day to day basis, it would be helpful to have a technology to help remember the names of people, an important statistic, etc. Today, IBM (NYSE: IBM) unveiled a software technology created in its Research Labs that uses the images, sounds, and text recorded on everyday mobile devices to help people recall names, faces, conversations and other important information.

The technology, nicknamed "PENSIEVE" by the IBM team, uses associative recall to make connections between pieces of related information acquired by a person. The advantage of the new technology is its ability to understand the context in which information is captured, connect various pieces of information, and then use this knowledge to help bring the correct data to a person when it is needed.

"This is like having a personal assistant for your memory," said Dr. Yaakov Navon, the lead researcher and image processing expert from IBM's Haifa Research Lab. "Our daily routines are overflowing with situations where we gain new information through meetings, advertisements, conferences, events, surfing the web, or even window shopping. Instead of going home and using a general web search to find that information, this technology, we've nicknamed 'PENSIEVE' helps the brain recall those everyday things you might normally forget."

Today's mobile devices have an endless number of functions that can record information in real time. IBM's new software blends techniques from image processing, GPS information, smart clustering, optical character recognition, speech recognition, and information retrieval to index and tag the information.

Researchers at IBM's Haifa Research Lab in Israel are pairing advanced mobile technologies with memory cues to develop a system that can analyze acquired data, create hooks to related experiences, and use them to populate a person's information management applications. Once the address books and calendars are updated, the technology enables memory recall triggered by time, location or the introduction of related information.

For example, if you meet someone at a conference and use your phone to take their picture and a picture of their business card, the new technology will associate the two pieces of information because they were taken at the same time and location. It then creates a virtual briefcase of information that includes the person's image, the name of the conference where you met, the date and time, and any other relevant information.

The knowledge base is unique in its ability to integrate contextual information with image, time, and location data. Prior to a future meeting with the same person, you can be prompted by your calendar to review your notes about that person, along with other facts you may have recorded or items the system associated for you.

This year, for the first time ever, more people in the world will have a mobile phone than a land-line. Mobile phones already double as a music player, video player, game device, and calendar. IBM recently launched a new initiative to bring even more features and functions to the mobile phone.

Although using mobile phones to take photographs and record sound or video bites is common practice, much of this information tends to get stored away on the device with little information attached to it, rendering it of little value for future use.

Many people encounter situations where you have been introduced to someone but you can't quite recall how you know them. By simply typing the person's name into PENSIEVE, you can recall when and where you met them, and any related information garnered at that time. You could even browse forwards or backwards in time to find out what events transpired before or after the initial meeting.

Another use of this technology is in reconstructing and sharing an experience or memory. If enough media-rich information was collected about a particular event, it can be used to build a more complex visual associative representation of the experience.

"This is where the real power of collaboration kicks in," says Eran Belinsky, research team leader and a specialist in collaboration. "You can recall the name of the person you met right before you entered a meeting by traversing a timeline of your experiences, or share a business trip with colleagues by creating a mashup that shows a map with an animation of your trail and the pictures you took in every location."

This technology builds on IBM Research's expertise in image processing, speech recognition, information discovery and retrieval, collaboration, and much more.