|
Everywhere Displays
Combining
an overhead projector, a rotating mirror, a video camera and special
software, the Everywhere Displays projector can turn walls, floors,
tabletops or any surface into a virtual, interactive touch screen
display.
The Everywhere
Displays project aims to develop systems that can transform
any surface into a touch screen. The basic idea is to use a projector
to create the display and a camera to recognize user interaction
with the projected image. However, given the current limitations
of image resolution for projectors and cameras, it is extremely
difficult to cover a significant portion of a room with a single
projector/camera pair. To overcome this difficulty, IBM Research
invented the concept of "steerable interfaces", where a pan/tilt
motorized system moves the projector and the camera to where information
is needed by the user.
In our prototypes, an Everywhere Displays (ED)
projector combines an LCD projector, a pan/tilt mirror and a pan/tilt/zoom
camera. The mirror is used to deflect the image of the projector
to surfaces, walls or the floor of a room. To avoid oblique projection
distortion, the projected image has to be warped to compensate for
the perspective distortion. The image of the video camera is processed
using computer vision techniques that detect hand/body activity
on the projected area, so people can interact with the projected
image by simply touching the surface.
A major emphasis in this project is to avoid "wiring"
surfaces with monitors or other sensing devices. In IBM’s vision,
information should be added to space without requiring users to
wear cumbersome special technology such as graphics goggles or mobile
devices. Everything happens simply by projection and vision processing:
light in, light out.
Everywhere Displays projector in a store
The ED projector is a generic input/output
device that can substitute for monitors and kiosks in many scenarios.
The main advantage is flexibility: once installed, the system
can be easily reconfigured to adapt to changes in the environment,
new tasks or even completely new applications, without requiring
re-installation or rewiring of equipment. In addition, the ED
projector can be installed in the ceiling, preventing vandalism
and theft. Projected images easily blend with the real world,
facilitating the creation of augmented reality applications.
IBM has built demonstrations of
the ED projector in different scenarios. In the BlueSpace
project, the ED projector is used in a smart office to create
interactive displays on walls, tables and the floor. In the Next
Generation Store project, the ED projector is used to create interactive
signage on floors, walls and store fronts. It is also used to
augment a retail shelf by detecting user manipulation of a product
and automatically displaying interactive information about the
product. The ED projector is also being used to create two next-generation
augmented reality video games: (1) a TicTacToe game where the
board moves to different surfaces, where it must be found by the
players and (2) a toddler's game where an animated frog jumps
to different surfaces as the child approaches it.
We are developing integrated solutions
for IBM clients that combine the ED projector with digital content
distribution/management systems. The ED projector can be used
to deliver traditional advertisement throughout a retail environment
with advantages over other media devices, such as plasma displays,
and with much greater flexibility and adaptability. Moreover,
we see an opportunity in the retail area for developing a new
set of marketing processes that explore the closeness between
product and information. With the ED projector, usage information
can be overlaid directly on the product. For example, an appliance
dealer could provide interactive instructions on how to operate
a washing machine. Animated characters and imagery can be easily
integrated into the retail environment, creating completely new
shopping experiences.
Work on the Everywhere Display started
in 1999. The first prototype was built in early 2000. In 2001,
IBM applied for the patent on the idea, which was granted in August
of 2002. For two years the project was funded by a special program
inside IBM Research called the "Adventurous Research Program,"
which allowed a team of computer scientists, Human Computer Interaction
(HCI) specialists and visual designers to improve the technology
and study its applications and human interaction capabilities.
An
Architecture and Framework for Steerable Interface Systems,
Anthony Levas, Claudio Pinhanez, Gopal Pingali, Rick Kjeldsen, Mark
Podlaseck, Noi Sukaviriya Proc. of the Fifth International Conference
on Ubiquitous Computing (UbiComp'03). Seattle, Washington. October
12-15, 2003.
Dynamically
Reconfigurable Vision-Based User Interfaces, Rick Kjeldsen,
Anthony Levas, Claudio Pinhanez BEST PAPER AWARD In: 3rd International
Conference on Vision Systems (ICVS'03). Graz, Austria. April 2003.
Embedding
Interactions in a Retail Store Environment: The Design and Lessons
Learned, Noi Sukaviriya, Mark Podlaseck, Rick Kjeldsen, Anthony
Levas, Gopal Pingali, Claudio Pinhanez In Proc. of the Ninth IFIP
International Conference on Human-Computer Interaction (INTERACT'03).
Zurich, Switzerland. September 2003.
Steerable
Interfaces for Pervasive Computing Spaces, Gopal Pingali, Claudio
Pinhanez, Anthony Levas, Rick Kjeldsen, Mark Podlaseck, Han Chen,
Noi Sukaviriya Mark Weiser BEST PAPER AWARD In: IEEE International
Conference on Pervasive Computing and Communications - PerCom'03.
Dallas-Fort Worth, Texas. March 2003.
The
Everywhere Displays Projector: A Device to Create Ubiquitous Graphical
Interfaces, Claudio Pinhanez Proc. of Ubiquitous Computing 2001
(Ubicomp'01), Atlanta, Georgia, September 2001.
Best Paper Award:
R. Kjeldsen, A. Levas, C. Pinhanez. Dynamically
Reconfigurable Vision-Based User Interfaces. Best conference
paper of the 3rd International Conference on Vision Systems (ICVS'03).
Graz, Austria. April 2003.
G. Pingali, C. Pinhanez, A. Levas et al. Steerable
Interfaces for Pervasive Computing Spaces. Mark Weiser award
at the IEEE International Conference on Pervasive Computing and
Communications (PerCom'03). Dallas-Fort Worth, Texas. March 2003.
IBM Recognition:
IBM Outstanding Achievement Award, 2003.
IBM Access Technologies for Persons
with Disabilities Special Patent Incentive Award: C. Pinhanez. Multi-Surface
Interactive Display Projections (filed as US Patent Application
Serial No. 09/779212, February 2001). June 2001.
Personal Awards connected to this work:
HENAAC
Most Promising Scientist with a Graduate Degree award: C. Pinhanez.
Awarded by the Hispanic Engineers National Achievement Awards Conference
(HENAAC). Austin, Texas. October 2003.
“Smart
carts, Veggie Vision in Grocery Stores to Come” by Karen
Vibert-Kennedy, USA Today, October 2003.
Cover article “New
Way to Work”.by Aaron Ricadela, Information Week,
January 28, 2002.
“A
Vision of the Office of the Future” by Claudia Deustch,
The New York Times, Business (USA). January 14, 2002.
“The
World is Your Computer Screen” by Fenella Saunders, Discovery
Magazine (USA), December 2001.
“On
Any Window or Wall, a Portable Touch Screen”, by Yudhijit
Bhattacharjee, The New York Times, Circuits (USA), November 22,
2001. |
 |
 |
|
|
What is the most exciting potential
future use for the work you're doing?
I believe that the ED projector
technology can dramatically change home entertainment and,
in particular, the video game industry. Creating characters
that inhabit the world in which we live and making them
interact with people and objects is going to create a new
paradigm for the entertainment industry that overcomes the
passivity of today's games. At the same time, educational
content can leave the confines of books and monitors and
be overlaid directly in contact with objects, allowing a
student to explore their use and meaning.
What is the most interesting part
of your research?
The most interesting area in the project, in my
view, has been the exploration and understanding of the
human factors involved in integrating imagery and objects
in the world. On many occasions, our beliefs about how people
would react to projected, interactive imagery were completely
wrong, which forced us to closely examine issues related
to visual and cognitive loads involved in the application.
For instance, we have found that users have more difficulty
interacting with projected interfaces overlaid in very functional
objects (such as a bucket) than when the same interface
is projected on a table or wall.
What inspired you to go into this
field?
When I was doing my PhD. at the MIT
Media Lab, I was involved with the creation of interactive
spaces for children and theater, using large-scale projected
screens. After these projects, I was frustrated with the
spatial constraints imposed by screens that tend to force
interaction to happen in front of them. What if I could
easily move the projection around? A night in a disco then
presented a solution.
What is your favorite invention
of all time?
The computer.
|
|
Discipline: