| Projects Developed at the
MIT Media Laboratory
My objective is to incorporate computers into the dramatic structure of theater by building interactive stages where actors can play with computer-controlled actors or augment/expand the expressiveness of their bodies through the use of computerized devices. These issues are largely discussed in the paper "Computer Theater". A first experiment (called "SingSong" ) involved a human clown interacting with computer-controlled creatures. In the fall of 1997, I designed, produced, and directed a full scale theater play called "It/I" where one of the characters is played autonomously by a computer system. The system monitors the human character actions on stage through cameras. More information is available in the "It/I" and in the paper "It/I: a Theater Play Featuring an Autonomous Computer Graphics Character".
|HyperMask (with Sony CSL)|
HyperMask is a system which
projects an animated face onto a physical mask, worn by an actor. As the
mask moves within a prescribed area, its position and orientation are
detected by a camera, and the projected image is adjusted to match the
mask. The lips of the projected face are automatically synchronized in
real time with the voice of the actor, who also controls the face's expressions
(using facial technology developed at the Morishima Lab of
As a theatrical tool, HyperMask enables a new style of storytelling,
as demonstrated in the Emerging
Technologies exhibition during SIGGRAPH'99.
More information is available in the paper "Hyper
|SmartCams and Intelligent TV Studios|
|I am building cameras for TV shows which operate without a cameraman -- SmartCams. The vision system which controls the camera receives high-level commands asking, for instance, for a close-up of a talent. The system employs both a previous knowledge of the script of the show, and on-line supervised binding of words to image regions. Details can be found in the publication "Intelligent Studios: Modeling Space and Action to Control TV Cameras".|
I was a member of the team who designed and built "The KidsRoom", an immersive, story-based interactive space for children. Using images, lighting, sound, and computer vision action recognition technology, a child's bedroom was transformed into an unusual world for fantasy play. Objects in the room became characters in an adventure, and the room itself actively participated in the story, guiding and reacting to the children's choices and actions. More details in "The KidsRoom" and in the paper "The KidsRoom: A Perceptually-Based Interactive and Immersive Story Environment".
Last update: 05/27/03