[ IBM Research ]

   Claudio Pinhanez
     Home | Biography | Research | Publications | Links | Personal Web Site | IBM Internal Web Site

 

Projects Developed at the
MIT Media Laboratory

(1993-1999)

Computer Theater

HyperMask

SmartCams

The KidsRoom


Computer Theater

scene of It/I

My objective is to incorporate computers into the dramatic structure of theater by building interactive stages where actors can play with computer-controlled actors or augment/expand the expressiveness of their bodies through the use of computerized devices. These issues are largely discussed in the paper "Computer Theater". A first experiment (called "SingSong" ) involved a human clown interacting with computer-controlled creatures. In the fall of 1997, I designed, produced, and directed a full scale theater play called "It/I" where one of the characters is played autonomously by a computer system. The system monitors the human character actions on stage through cameras. More information is available in the "It/I" and in the paper "It/I: a Theater Play Featuring an Autonomous Computer Graphics Character".

Short movies: scene I, scene II, audience participation.


HyperMask (with Sony CSL)

HyperMask is a system which projects an animated face onto a physical mask, worn by an actor. As the mask moves within a prescribed area, its position and orientation are detected by a camera, and the projected image is adjusted to match the mask. The lips of the projected face are automatically synchronized in real time with the voice of the actor, who also controls the face's expressions (using facial technology developed at the Morishima Lab of Seikei University). As a theatrical tool, HyperMask enables a new style of storytelling, as demonstrated in the Emerging Technologies exhibition during SIGGRAPH'99. More information is available in the paper "Hyper
Mask - Talking Head Projected onto Real Object"
.

Hypermask movie.

hypermask of Lincoln

SmartCams and Intelligent TV Studios

close-up of chef

I am building cameras for TV shows which operate without a cameraman -- SmartCams. The vision system which controls the camera receives high-level commands asking, for instance, for a close-up of a talent. The system employs both a previous knowledge of the script of the show, and on-line supervised binding of words to image regions. Details can be found in the publication "Intelligent Studios: Modeling Space and Action to Control TV Cameras".

The KidsRoom

I was a member of the team who designed and built "The KidsRoom", an immersive, story-based interactive space for children. Using images, lighting, sound, and computer vision action recognition technology, a child's bedroom was transformed into an unusual world for fantasy play. Objects in the room became characters in an adventure, and the room itself actively participated in the story, guiding and reacting to the children's choices and actions. More details in "The KidsRoom" and in the paper "The KidsRoom: A Perceptually-Based Interactive and Immersive Story Environment".

Short movies: furniture, florest and river, monsters.

kidsroom scene

Last update: 05/27/03

Contact IBM Legal Privacy Orders IBM home IBM research