Skip to main content

Is web search enough to win Jeopardy!?

A web search engine has access to an immense source of information and can quickly find relevant web pages given a small number of query terms. To play Jeopardy!, however, you must generate precise answers to the questions. A web search engine doesn't return precise answers; rather, it is designed to return a ranked list of web pages the user may be trying to find. To answer a Jeopardy! clue you must go beyond the search result page, dig into documents that are likely to contain the answer, fetch them, read them, and locate the precise and correct answer within them.

Even if you think you've found an answer to the question, there's still the issue of confidence. Remember, you are penalized for giving a wrong answer, so you must have enough confidence in your answer to decide you even want to attempt answering the question. To obtain this confidence you need to read enough supporting text around the answer to convince yourself that you have found the correct answer. You may even want to see the answer justified by multiple sources, especially if the penalty for getting it wrong is significant.

And then there are the clues with answers that must be synthesized — the answer is a list of items or a logical combination of two or more items. These answers do not appear in any one place. Rather, you must synthesize them from independent sources to form your final answer.

On top of all this you must be ready with a confident answer in a matter of seconds after receiving the clue.

The bottom line is that the Jeopardy! Challenge poses a different kind of problem than what is solved by web search. It demands that the computer deeply analyze the question to figure out exactly what is being asked, deeply analyze the available content to extract precise answers, and quickly compute a reliable confidence in light of whatever supporting or refuting information it finds. IBM believes that an effective and general solution to this challenge can help drive the broader impact of automatic question answering in science and the enterprise.