Designing a Human vs. Machine contest should square off the competitors in a way that allows the audience to transparently size them up pound-for-pound. Since everything a human brings to the table is right there in front of the audience no Ethernet, wireless, 3G, or other lifelines allowed the same should be true for the computer. This is why Jeopardy! and IBM intend to put Watson on the stage in plain sight, with no connections to other computers or data resources of any kind what you see is what you get.
The computer will receive the clue electronically, precisely when the human players see it, and after that they're off. Many comparisons will be made. These range from how much information Watson can store in its memory banks versus what a human can store in its wetware, how much parallelization the human brain is capable of versus what Watson can do, the number of petaflops, etc. All of these will be fascinating to explore if the competition proves a worthy one.
One of the interesting comparisons will be the time it takes for the computer to understand the natural language clues well enough to accurately assess its if it knows the answer. Can the computer do this in time to competitively buzz-in? Humans on Jeopardy! excel in this ability and appear to be able to accurately know if they know the right answer very quickly after seeing the clue.
While computers have demonstrated that they can quickly recall documents based on pre-indexed keywords, knowing that a term from the potentially 1000's of returned results correctly answers the question is a whole other ball game. It requires on-the-fly, deep analysis of large volumes of language and the production of accurate probabilities that a term or combination of terms is the right answer all in time to buzz. And not all clues are created equal, of course. Watson will not be able to answer all the clues with its stored content, but it still must be able to determine whether or not it knows the right answer.
It promises to be an interesting contest indeed.