Leading a consortium of European partners from academia and industry, IBM Research is developing a large-scale, distributed peer-to-peer architecture that will make it possible to search audio-visual content using the query-by-example paradigm
IBM recently announced the launch of a new EU 6th Framework project for Search on Audio-visual content using Peer-to-peer Information Retrieval (SAPIR) to build a large scale information community that will make multimedia files more accessible.
SAPIR is geared towards finding new ways to analyze, index, and retrieve the tremendous amounts of speech, image, video, and music that are filling our digital universe. The ultimate result will be a peer-to-peer distributed space that can be searched by content or example rather than using the current methods, which are limited to keywords and text-based tags.
Backed by the European Commission, researchers from the IBM Haifa Lab are leading a consortium of nine partners from industry and academia on a € 4.5 million project to drive the development of search technology. The SAPIR consortium includes experts from industry and academia that bring expertise in the area of multi-modal devices, information retrieval, indexing, peer-to-peer distribution, and audio-visual analysis and social networking.
The consortium includes partners from academia and industry including IBM Research (Israel) Max-Planck-Institut (Germany), University of Padova (Italy), CNR (Italy), Eurix (Italy), Xerox (France), Masaryk University (Brno, Czech Republic), Telef·nica I+D (Spain), and Telenor (Norway).
"Today's popular search engines work within defined boundaries," explained Yosi Mass, project leader for SAPIR at the IBM Research Lab, Haifa. "Sapir's goal is to establish a giant peer-to-peer network, where users are peers that produce audio-visual content using multiple devices and service providers are super-peers that maintain indexes and provide search capabilities.
In a sense, this would be similar to a network of computers that collect information, index it, and then present a uniform searchable space for users. Because the information originates in so many different sourcesincluding web servers, content providers, individual mobile devices, and PCsSAPIR will also include intelligent routing capabilities that point the search in the right direction.
Although today's web content is becoming more and more multimedia based, the most popular search engines still specialize in searching for text tags, using centralized indices. Even when users query for images or videos, these engines are searching using text tags that have been associated with the multimedia files. If content providers don't clearly or accurately describe their multimedia files, or use inaccurate tags that are designed to drive more traffic their way, the current method falls short.
Defining new standards in search methods, SAPIR will incorporate technologies such as voice recognition, image processing, indexing algorithms, sophisticated ranking mechanisms, and real search in audio-visual content. Searching by example rather than text-based queries will allow users to say a word out loud and have the engine look for a similar speech pattern. Another scenario would mean any one could input a picture of a saxophone and have the engine search for similar shapes.
SAPIR will address some of today's most exciting search challenges, starting with the scale of information being addressed. The searchable space created by the massive amounts of existing video and multimedia files greatly exceeds the area searched by today's major engines. The popularity of new video sites is a clear indication of today's vital need to search for multimedia. The massive scale of this data can no longer be handled by the current model of centralized crawling and indexing of all searchable content.
"This is a big challenge that requires analyzing and indexing massive amounts of digital information currently not accessible because of today's limited search techniques," noted Mass. "By working towards establishing a giant network that supports modalities ranging from computers to cell phones and PDAs, we can give users the ability to access and publish content from mobile devices anywhere and anytime."
SAPIR researchers will also study the social networks in our digital universe. Based on the information gathered from indexing, query patterns, and users with shared profiles, much important information can be garnered and new communities can be formed.
"Today, the highly scalable search for media-rich information remains an open arena. One of the keys to giving users a comprehensive search experience is breaking down today's limitations of text-based searches and bounded search spaces," concludes Mass. "We're talking about a giant search engine that works in a scalable, cooperative fashion. This will help bring us closer to the point where users can perform audio-visual searches based on examples or patterns, wherever they are, from their preferred device."