What if we could put information in places? More precisely, what if we could associate relevant information with a place and perceive the information as if it were really there? WorldBoard is a vision of doing just that on a planetary scale and as a natural part of everyday life. For example, imagine being able to enter an airport and see a virtual red carpet leading you right to your gate, look at the ground and see property lines or underground buried cables, walk along a nature trail and see virtual signs near plants and rocks, or simply look at the night sky and see the outlines of the constellations. (See Figure 1.)

Figure 1

Since the pioneering work of Ivan Sutherland in 1968¹, the vision of putting information in places has been a key goal of researchers developing augmented reality systems.² Unlike virtual reality systems³ that allow users to experience a completely virtual or simulated world, augmented reality systems allow users to experience a mixed reality that combines virtual objects with real-world objects. Video special effects, as seen in commercials, television programs, and movies, offer a glimpse at some of the possibilities when artificial images can be seamlessly combined with real images--for example, cars that seem to dissolve before one's eyes offering cut-away views, or animated characters in the kitchen encouraging kids to eat their breakfast. Unlike video special effects, augmented reality systems support the perception of real special effects--or special effects happening right where a person is in real time and in real space. For example, imagine a person walking into a parking lot and looking at a car while wearing special eyeglasses, or looking through the viewfinder of a special video camera, who is then able to see a cut-away view of the car exposing the complete exhaust system perfectly aligned with the real car. That person is perceiving a real special effect or experiencing augmented reality.

In a 1992 paper, Warren Robinett describes some of the ways that augmented reality can be used to extend human perception, beyond the more familiar ways that devices like the telescope and microscope extend human perception.⁴ The primary difference, of course, is that devices like the telescope and microscope show us only what is actually there, whereas augmented reality systems can superimpose useful information drawn from any aspect of human culture or imagination. For example, the constellations are both physical reality (position of stars) and human invention (mythology projected onto the heavens). For another example, a telescope can help us see pinpoints of light circling Jupiter, but an augmented reality system can allow us to perceive the moons with their projected orbits, names of the moons and other useful information, whether or not Jupiter is above or below the horizon, and whether or not it is daytime or nighttime. How might electronic expansion of human perception change our relationship to the world and to information about the world?

The science of human perception, the technology of augmented reality, the art of special effects, and the culture of the information age come together to enable WorldBoard and potentially change our notion of place. A stable notion of place has been fundamental to the way we live our lives;^5,6 we build mental models of objects in spatial array around us in places; we go to places to do routine things; we put things in places for reasons; different individuals or organizations control what can and cannot be done in a place or put in a place; we all control some limited number of places and expect that things we put in those places will be there for us. WorldBoard, if it can be made broadly, uniformly, and cost-effectively available, changes our notion of place in some fundamental ways. First, a new conceptual category of thing (nonphysical information) can now seemingly be in a place. Second, since information takes up no real physical space, the same place can appear differently depending on who is perceiving it and for what purpose (i.e., by tuning to a different WorldBoard channel). Third, many of the most useful properties of a place, such as its history, can be stored with the place.

To explore the implications of WorldBoard as it relates to putting information in its place, this paper is structured around six questions:

1. Initial concept: What is the basic idea behind WorldBoard?
2. Historical background: What have the pioneers achieved?
3. Technical feasibility: Can it really be done?
4. Potential applications: Will it be truly useful?
5. Social implications: Will it catch on?
6. Reflections: Are we even close in our thinking?

Initial concept

Even though the utility of associating information with places goes beyond education, educational users are often very tolerant of experimental infrastructures (witness the evolution of the Internet). Hence, the education community was identified as a good initial audience and codevelopment partner for WorldBoard. With the education community in mind, three key design goals and a four-step development plan were created as part of the original WorldBoard effort. The purpose was to get a simple operational infrastructure in place that would evolve over time. The first stage of WorldBoard development would not require technologies that support a complete augmented reality system.

WorldBoard was originally conceived as a planetary chalkboard for twenty-first-century learners, allowing them to post and read messages associated with any place on the planet. In the mid-1990s, WorldBoard was seen as the logical culmination of an effort to improve educational tools--cognitive tools, social tools, and perception tools. As part of a National Science Foundation (NSF)-funded project, a consortium of industry, university, and government organizations began an investigation aimed at improving the quality and availability of educational software.^7-13 In the course of that effort, three kinds of technology were developed: authoring tools to more easily create educational software content, on-line learning communities to exchange and improve the content, and new paradigms in mobile computing to support learning in context.

More generally, the view at that time was that the quality and availability of educational materials could be improved if people were empowered with cognitive tools to create educational materials, social tools to collaboratively improve the educational materials, and perception tools to access the educational materials in context.

For example, a student might write a report about the life cycle of frogs, create a simulation of the life cycle using an authoring tool, and display the report and simulation on a Web site inviting others to comment and link to the report. On a series of field trips to a real frog pond, a student might use a digital camera to take pictures to add to the report, as well as use a hand-held computer to gather sensor data about water and air temperature. In this example, content creation, collaborations, and conversations, as well as authentic contexts, were seen as part of a more complete learning experience than traditional classroom lectures. Pointers to much of this earlier work can be found at the Educational Object Exchange Web site (http://www.eoe.org), maintained by an educational research, nonprofit organization that is carrying on one thread of this earlier investigation.

Design goals. The early WorldBoard effort adopted three key design goals: (1) to be operational (at least partially) on a planetary scale from the start, (2) to be able to improve rapidly as technology advances, and (3) to be so simple and useful that people use it in everyday life.

First, for a variety of reasons, the decision was made that some initially realistic accuracy level (to within 1 meter) for planetary positioning should be selected. An imaginary 1-meter cube provides six faces (up, down, north, south, east, and west) to post and access information. A 10-meter cube would be larger than most rooms and much larger than many objects of interest in daily life. However, a 10-meter scale is easy to navigate on high-resolution satellite images or aerial photographs, and is an achievable accuracy level for certain positioning devices. Just for reference, the Global Positioning System (GPS) can be used to provide 20- to 100-meter accuracy quite reliably in most outdoor settings.¹⁴ A differential GPS can improve this by an order of magnitude to provide 2- to 10-meter accuracy quite reliably in areas where differential services are available.¹⁵ Therefore, either additional positioning technologies or manual interfaces are required to allow a person to navigate to any particular 1-meter cube.

The information on a square-meter face might be organized to look like information on a poster or bulletin board (i.e., pictures, documents, URL references). Because many people might want to post messages to the same place or restrict access to posted information, the notion of password-protected WorldBoard channels was introduced. Positioning was to be accomplished either manually (using a map or other interface to select a particular cube for writing or reading information), or automatically (using a location-aware device). Manual positioning would allow people to post and retrieve messages from another location, while automatic positioning with location-aware devices requires their physical presence. For example, in anticipation of an upcoming field trip to a national park, a student might use a browser to navigate to the park, and then leave programmatic commands at various locations to gather data on temperature, light, and humidity. Later, when the student is exploring the park, the student's location-aware hand-held computer would automatically collect the desired data as the student moved to those locations, time- and space-stamping the data as well. Furthermore, a teacher could post audio reminders to be triggered when students approach certain locations. For example, when students cross a bridge they could receive a short voice note, or a page call, to look upstream and see the waterfall that exposes particular geological formations being studied by the class, or as a warning to be very careful as they are entering a fragile ecology zone of the park.

A cubic meter may sound like an ad hoc way to divide up space, and it certainly is (though a similar coordinate system will be described in a later section). This was intended to be a somewhat challenging, but not altogether unrealistic, starting point. If learners equipped with WorldBoard browser-based tools and location-aware devices could write and read multimedia messages posted anywhere on the planet, they could use the cubic-meter space like posters for a science fair project. One way to move away from this overly simple model is to recognize that where natural surfaces (buildings or rocks) or other objects (appliances or trees) exist, additional methods of associating information with the places are possible.

Nevertheless, the simple cubic-meter model did allow for a kind of zooming in and out. For example, a user might project all of the information from adjacent cubes onto a larger cube (10-meter view or 100-meter view). This could be valuable for looking for a particular kind of information within a certain radius. One could even imagine using the size of a message as an indication of its importance. For example, children might leave a ten-mile-square message floating above the city skyline for their parents, or a much smaller note that appears to be on the surface of a desk or a window in an office.

A second key design goal has been to allow WorldBoard to improve rapidly as underlying technologies advance. For example, as positioning technology improves, users can post information to smaller spaces more accurately. Ultimately, as decimeter, centimeter, and better accuracy is reliably achieved for both indoor and outdoor locations, users will be able to set the color of a particular cubic centimeter in space. Voxels, or volumetric pixels, allow users to create LEGO**/Logo-like objects in places. Furthermore, as display technology improves, rather than seeing two-dimensional information posted to the faces of cubes on a hand-held computer, a user might use stereoscopic display glasses to see complete three-dimensional (3D) views of the information created out of voxels or rendered in other ways. However, before these techniques become commonplace, considerable content could be made available to be viewed via hand-held computer displays with meter-level positioning accuracy.

A third key design goal has been to keep the initial WorldBoard design simple, so that people might get started even before mobile location-aware devices become widely available. The most basic view of WorldBoard is simply as a spatially addressable bulletin board, containing geocoded (longitude, latitude, and elevation) messages or Web pages. It is important to note that this view of WorldBoard is much simpler than an operational augmented reality system. For example, people might access information on a WorldBoard through a traditional Web browser on a desktop machine, perhaps using a map to spatially navigate to the place. Once a user has navigated to a location, the user might view close-up photographs of that location (up, down, north, south, east, west), and use standard graphical tools to mark up the image, outlining objects of interest, adding arrows or other graphics, and posting messages or even pieces of software that run when location-aware devices get to that location and trigger the code. A public WorldBoard channel might allow people to ask questions and post answers to a particular location ("What is the name of the plant located here?"); a private WorldBoard channel might provide high-quality geological or biological information, or perhaps even advertising with the potential of providing services that go beyond existing billboards. Getting started with projects like these is straightforward. For example, the Confluence Project¹⁶ is one of many efforts to collect a sampling of geocoded photographs of specific longitude and latitude points from around the world.

Development plan. The original WorldBoard proposal described a four-stage development plan:

1. WorldBoard servers: First, identify a reasonable way to associate information with places on a planetary scale. Given coordinates for one of the six faces of a meter cube, a channel number, and a password, the server could serve up a Web page to the client. This information would be authored and accessed from existing Web browsers.
2. WorldBoard clients: Second, identify a mobile capability to author and access the information associated with places on a planetary scale. A location-aware device with navigation, authoring, and global wireless communication capabilities would be needed--probably, a device with camera and pen input, and either a manual map interface or an automatic differential GPS for positioning.
3. WorldBoard glasses: Third, make use of positioning and display advances to create the illusion of seeing (and more generally perceiving) information in places. In addition to the basic client capability, by using kinematic GPS for subcentimeter orientation platform capabilities, glasses (or palm-sized monocles) could be used to display information objects coregistered with reality, rather than simply appearing on a cube face.
4. WorldBoard services: Fourth, make use of new organizations to provide education, safety, entertainment, and industry-specific services on spatial information channels. Organizations might eventually provide archiving, design, and other associated information services, employing information architects and designers.

As part of the NSF grant, prototypes of various aspects of WorldBoard were implemented, but the complete vision has yet to be realized. Since the original proposal was published in 1996, the notion of a WorldBoard has been evolving and key technologies have been advancing. For example, today there are even companies, such as GeoPerception,¹⁷ as well as projects, such as Neighborhood Web,¹⁸ that have the explicit goal of allowing the Web and other useful information to be everywhere perceptible. In the next section, some of the pioneering work on which WorldBoard and related efforts draw is presented.

Historical background: Three threads

The primacy of the physical world is the starting point for much of the research discussed in this section. Nevertheless, there is also the belief that appropriate technology tools can augment human capabilities¹⁹ and thereby enhance daily life, combining virtual with physical reality. A key part of such augmentation tools is the display technology--though communication and positioning technologies are also important and will be highlighted in the next section. The quest to see electronic information objects in real physical spaces has been approached using three types of displays:

1. Head-mounted see-through display glasses (monocular and binocular)
2. Hand-held palm-sized displays that are portholes into information spaces
3. Projectors that superimpose images on the environment or in free space

Using see-through, hand-held, or projector displays, everyday places and objects can gain new electronic properties without losing their familiar physical properties.

Head-mounted displays. One of the earliest uses of head-mounted displays was in 1968. Ivan Sutherland published the first paper describing an operational augmented reality system using a head-mounted display.¹ The fundamental idea was to present the user with a perspective image that changed as the user moved. The system used half-silvered mirrors to allow the user to simultaneously see the displayed materials (wire-frame images that appeared on miniature cathode ray tubes) and real objects in the room. Displayed materials could be made either to hang disembodied in space or to coincide with maps, desktops, walls, or the keys of a typewriter. Sutherland and his colleagues created both a mechanical head-position sensor and an ultrasound sensor that were used to track the location and orientation of the head-mounted display and hence the user's head. The mechanical head-position sensor was rather large and uncomfortable to use, but resulted in a sure method to measure head position. The ultrasonic head-position sensor also measured head position, but after a few minutes its cumulative errors were objectionable.

While the user's movements were restricted to a region about six feet square, a far cry from a planetary-scale infrastructure, nevertheless the key concept of creating the illusion of seeing three-dimensional objects in real space was achieved. Furthermore, it is interesting to note that one of the images that Sutherland was working with was a wire-frame "room" that was a simple six-faced cube with the letters "C," "F," "N," "S," "E," and "W" on the six faces. As will be described in the next section, head-mounted display glasses have been decreasing in size and weight. Already, they have attained the form of normal prescription reading glasses or sunglasses.

In the 1980s there was a resurgence of interest in head-mounted displays for virtual reality applications, military heads-up displays for pilots (e.g., Apache helicopter pilots), and medical applications for surgeons. In the 1990s, heads-up display glasses have become a standard component of wearable computers, such as IBM's VisionPad concept computer.²⁰ In the next few paragraphs, examples of these evolving applications of head-mounted displays for augmented reality are briefly discussed.

In 1991 Ron Azuma²¹ and his colleagues at the University of North Carolina at Chapel Hill first demonstrated to the public, in the Tomorrow's Realities Gallery of the ACM SIGGRAPH conference in Las Vegas, a scalable tracking system for head-mounted displays. The bulk and weight of the displays was still a major problem, but by using optical sensors mounted on the head unit to detect infrared beacons in the ceiling, this group was able to demonstrate substantial improvement over magnetic trackers²² widely in use at that time. Azuma points out that even when the position and orientation of display glasses is reasonably well-known and useful content has been authored for a physical place, there remains the problem of preserving the illusion that a virtual object is actually part of the real world. This requires proper alignment and registration of the virtual objects to the real world, in spite of user movements, tracking errors, and a variety of system delays and device latencies. This is a well-known problem in movie special effects, but because special effects can be done off line, the movie special effects task is significantly easier than the demands of providing real-time special effects for a mobile augmented-reality experience. Even tiny errors in alignment and registration are quickly noticeable. This problem will be examined in more detail in the next section.

Also in the early 1990s, Feiner and his colleagues at Columbia University²³ described an office of the future in which a person wears a see-through head-mounted display to superimpose graphical information on objects in the office environment. For example, superimposed pictures might let a person see inside a printer, copier, or filing cabinet to show how to service it or to locate a document. This prototype augmented-reality system used a Reflection Technology Private Eye (720 x 280 resolution, red lines, and letters) and a LogiTech 3D position and orientation tracking system (ultrasonic transmitters and receivers). The focus of this work was on the complexity of authoring information presentations for augmented-reality spaces. Like other high-end multimedia and special-effects productions, this form of information presentation requires significant skill and time to produce. Feiner demonstrated that knowledge-based systems could be built to automate the design of presentations that explain how to perform 3D tasks. KARMA (knowledge-augmented reality for maintenance assistance) was a test-bed system for automating the design of augmented realities that explain maintenance and repair tasks. KARMA was based on Feiner's IBIS (intent-based illustration system). Feiner also developed a system termed "architectural anatomy" that allowed a user to "see through" walls and view wiring, plumbing, and other infrastructure.

More recently, Feiner and his colleagues²⁴ describe a prototype wearable augmented-reality system. Their tour-guide application provides information about a university campus (i.e., names of buildings and Web information about academic departments). This prototype augmented-reality system used an i-O Display Systems i-glasses** head-worn display (quarter-VGA-resolution color display) and a Trimble DSM (direct sequence modulation) GPS receiver with differential correction services provided by Differential Corrections, Inc. to achieve about 1-meter accuracy. Wireless communication was accomplished with a campus-wide network of radio base stations and an NCR WaveLAN** radio modem (2 megabits per second). Hand-held computers, including Apple Newtons**, were also used experimentally as display devices in this work.

Starner and his colleagues at the Massachusetts Institute of Technology (MIT) Media Lab²⁵ describe a wearable augmented-reality system that "tracks the user's location through computer vision techniques without any off-body infrastructure." Earlier systems required a beacon architecture, which meant placing active or passive identifier tags on objects in the physical environment. In 1995, projects at the MIT Media Lab began binding virtual data to physical locations to support minitours of the laboratory. The purely computer-vision approach to position determination described in this most recent work by Starner uses hidden Markov model (HMM) techniques. HMM techniques can be used to model the environment as a set of states with transitions, and then match the camera input to the model to produce a probability of being in a particular state (location) of the model (world). This approach to positioning may be especially useful when combined with inertial sensors, GPS, and other information sources that can improve the robustness and reliability of tracking information. Also, noteworthy in this work is the fact that the prototype actually used two cameras: one for observing the environment, and a second one for observing the user. In the case of the Patrol Game (battlefield simulation) application, utilizing the second camera enabled the computer to be aware of the user's current task and resource level, thus allowing timely information to be displayed to help the user. This work illustrates that increases in contextual and user information taken together can lead to more intelligent and natural user interfaces.

Rekimoto and his colleagues at Sony Corporation²⁶ describe "a system that allows users to dynamically attach newly created digital information such as voice notes or photographs to the physical environment, through wearable computers as well as normal computers. Similar to the role that Post-it** notes play in community messaging, we expect our proposed method to be a fundamental communication platform when wearable computers become commonplace." The display is based on the Sony Glasstron** (monocular see-through heads-up display), a CCD (charge-coupled device) camera, and an infrared sensor. Nortel Network's NetWave AirSurfer** is used for wireless communication. This prototype supports infrared beacons and visual identification markers as contextual information. The authors describe a "time-machine mode" for authoring information to previously visited remote locations, as well as the ability to send e-mail messages to particular locations. Additionally, they describe the possibility of using a standard Web browser to access messages that have been attached to particular locations. In this way, users can work with normal computers to interact with the information spaces, as well as the mobile wearable computers to interact with the information in context. The potential utility of virtual notes on restaurants, office equipment, and other objects in the physical environment is also discussed. Rekimoto et al. also discuss the use of a hand-held display system called NaviCam, or a magnifying-glass approach to augmented reality as discussed next.²⁷

Hand-held displays. Two problems with the use of head-mounted displays for augmented reality applications are: (1) head-mounted displays are still conspicuous when worn, even when they approximate the form of normal reading glasses, and (2) users of head-mounted displays sometimes complain of nausea as a side effect of prolonged use.^28,29 Hand-held displays with cameras that superimpose information on the real scene have advantages in certain situations. Hand-held displays can be stored and brought out only when needed; more than one person can look at the display at the same time; the physical appearance may be more acceptable than glasses; and the nausea effects may be eliminated or less pronounced. In this section, a few key examples of research efforts that have used hand-held displays for augmented reality applications are presented.

In the early 1990s as part of the Chameleon project,^30,31 Fitzmaurice and colleagues developed the notion of spatially aware computers. These are tools to perceive electronic information in "a world where electronic information will ultimately be everywhere." The goal of this effort was to look for ways of associating electronic information with physical objects in the environment. The information would then be viewed, not on a large fixed display on a desk, but on a small, mobile display that would act as a window (porthole) into the information space. The hand-held computer and display needed both spatial awareness and physical environment sensing capabilities in order to create the illusion of merging the electronic and physical worlds. Different information could be presented to the user depending on the orientation of the hand-held unit. For example, weather information, travel itineraries, and geographical points of interest might be easily accessed. In addition, a user could attach a voice annotation to a selected object. To remind the user of the presence of the voice annotation, a graphical note was superimposed on the video data.

In order to avoid being flooded and overwhelmed with the sheer quantity of electronic information that might eventually be everywhere, they proposed the need to adopt the notion of situated information spaces. The electronic information associated with physical objects could be associated and collocated with those objects. The physical objects anchor the information, providing hot spots and retrieval cues for the user. Fitzmaurice's team also introduced the notion of mediator objects that act as interfaces between the physical and computational environments. For example, when a whiteboard acts as an electronic mediator, notes made on the hand-held unit can automatically be transferred and appear on the electronic whiteboard.

Instead of viewing and manipulating a computerized world through a large stationary computer and display, Fitzmaurice and colleagues proposed to shift to a new model in which people carry around a very small hand-held computer that acts as a personal display of information spaces. The displays are aware of their surroundings and change depending on the situation in which they are immersed. However, it is interesting to note that their prototype actually used a camera pointed at a large workstation monitor; the video was then fed into a small hand-held unit. An Ascension Bird** six-degree-of-freedom input device was attached to a small display (from Casio Electronics) to provide reasonably responsive (50 millisecond delay) position, translation, and rotation information, but only within a three-foot cube range (about the same usable area as Sutherland's system).

In the mid-1990s, Abowd^32,33 and his colleagues at Georgia Institute of Technology worked with context-aware hand-held computers in a project known as CyberGuide. Apple Newtons and other hand-held computers detected infrared signals from beacons in the ceiling to provide information about particular locations in a room. In one scenario, visitors were each given a CyberGuide unit and could walk around a demo room to get information about all of the projects on display. The Smithsonian Institution has used a similar system for traveling shows, but instead of automatically detecting their location, users must enter a number associated with a particular display of interest.

Projector displays. Besides head-mounted displays and hand-held devices with cameras, a third technique for combining and aligning real and virtual objects is to project information onto real-world surfaces. While this approach has clear limitations (projectors are not very portable, require lots of power, and most real-world surfaces do not make particularly good projection screens), nevertheless it has been explored for applications where a wall or desktop surface is the primary focus of attention for mixed reality interactions.

Early experiments projecting information onto surfaces (using projection displays and reading information from the environment using cameras) were performed by Myron Krueger in 1969. Krueger³⁴ reasoned that interfaces should know about people and the environment (both user-aware and context-aware, as in the Starner work previously cited). He created environmental technology systems known as VideoDesk, VideoTouch, and VideoPlace. Much of Krueger's work was the exploration of a new medium for human-computer interaction. Other researchers focused on using the technique to accomplish office automation tasks or achieve levels of fidelity with corresponding real-world alternatives, for ex-ample Knowlton,³⁵ Schmandt,³⁶ Wellner,^37-39 and MacKay.⁴⁰ Wellner and his research team created a system known as the DigitalDesk that used electronic ink and superimposed images on paper documents on a desktop. However, Wellner and others note that paper is easier to read than most computer screens today; it is cheap, universally accepted, tactile, and portable, and use of paper is growing about 20 percent annually.⁴¹ Most recently, Raskar⁴² described spatially augmented reality (SAR), where virtual objects are rendered directly within or on the user's physical space. Raskar's work explores the benefits that derive when several individuals can interact with the information at the same time. Other related uses of projection displays in creating 3D information environments are the RWAV (Room With A View) system⁴³ and the CAVE** (Cave Automatic Virtual Environment) system.⁴⁴

While the projection techniques seem inherently limited today as a means of creating a global infrastructure for information in places, they are worth at least a passing mention for two reasons. First, glasses and hand-held devices are primarily for personal use, whereas projection tends to be used to promote social interactions and communications. Second, projection techniques can be used to simulate environments in which any and all surfaces can potentially act like a display. Many trade-offs exist in designing augmented-reality experiences, and an exploration of alternative display technologies helps crystallize some of the issues. For example, while socially sharing an augmented-reality experience may be important in some situations (projection display), there may be other times when individuals in a group wish to have distinct augmented reality experiences in the same place at the same time (head-mounted or hand-held displays).

Technical feasibility

The technical feasibility of WorldBoard can be evaluated with respect to the four-step development plan originally proposed (see the previous section). In this section, each step in the development plan is introduced, along with issues (including some nontechnical issues), followed by a more detailed discussion of the technical feasibility issues. Five key aspects of WorldBoard feasibility are: positioning technology, communication technology, display technology, simple user experience, and critical mass of geocoded content. In the case of WorldBoard services and geocoded content, both largely economic issues, there is the technical issue of how to rapidly bootstrap content and make it easily accessible.

WorldBoard servers: Geospatial portals. The first stage of WorldBoard requires the creation of Web sites or portals that allow people to easily associate information with places on a planetary scale. Given a global coordinate for a specific cubic meter--one of the six faces of the meter cube, a channel identification, and a password--the Web site should serve up a Web page to the client. This information should be authorable and accessible from existing Web browsers.

Creating this stage of WorldBoard will require a substantial effort, although it is technically feasible. The issues include: What is the global coordinate system that is used to address cubic meters? What is the user experience for authoring and accessing information? What are the basic applications available to users through their browsers? How much storage is required? How are multiple servers to be networked together to provide a seamless user experience?

The UTM (Universal Traverse Mercator) coordinate system is used around the world for topographical maps, with northing and easting offsets expressed in 1-meter units.⁴⁵ In addition, UTM is a worldwide grid of 1200 zones: 60 6-degree zones extending eastward from the International Date Line, and 10 8-degree zones above and 10 below the equator. Although UTM is not currently usable in polar regions (the edge of a zone nearer the pole is shorter than the edge further from the pole, as the pole is a point of convergence), extensions have been proposed.

The first questions users may have are: Where am I in UTM coordinate space? How can I see what has been posted to a particular place? What if I am off by a few meters? How can I post something to a particular place? Unfortunately, without location-aware devices accurate to the cubic meter, answering the "where am I" question is quite a bit of work. First, we consider three less-than-friendly user experiences, and then a more ideal proposal.

Three possible user experiences are: (1) "drill down" on satellite maps or other maps that have been annotated with UTM coordinates (and then make careful measurements off observable points); this is possible with Microsoft's TerraServer**, ⁴⁶ (2) enter a mailing address or telephone number and then look up UTM coordinates given to some local landmark that have previously been entered into a database (and then make careful measurements off the observable landmark), or (3) settle for less than meter-level accuracy, or make up ad hoc relative coordinates from some local landmark.

None of these seems particularly appealing or likely to catch on. However, consider two points: (1) for certain applications, this level of accuracy is already routine and therefore available--archaeological digs, property surveys, building construction, geographic information systems for roads and public utilities, and (2) for many applications, this level of accuracy really does not matter or only matters with respect to a local coordinate space whose global coordinates can safely float or be tied to an arbitrary point until further refinement is needed. The user experience becomes much easier in many ways if only 100-meter accuracy is required. However, while this may be suitable for certain applications, it too easily sidesteps important issues that must be addressed if WorldBoard is ever to become a truly user-friendly technology that is simple to use and useful in everyday life.

A more ideal user experience would be to either fly down smoothly to a location (as opposed to progressively drilling and waiting, as on current systems) or, after entering an address or location, be placed at a standard entry point to that location and be able to move through the space in a mode compatible with that space. For example, this is how many 3D virtual environment games work today, but rather than fictional worlds a very accurate model of the real world could be used. In today's 3D games, the user appears as an avatar in a virtual world, and the mode of transportation is on foot, in a vehicle, or in an airplane. Moving a pointing device and a throttle (or rate controller), the user is comfortably able to move through the space to any number of locations. Many of these game engines are freely available on the Web; some even include the source code and tools to build custom worlds.⁴⁷ Again, freely available tools bring the possibility of bootstrapping WorldBoard by working with the education community--a project for children around the world could be to build 3D models of their communities with meter-level accuracy. Governments could give an excellent starting point by providing a topographically accurate foundation based on satellite images that would have rough terrain features included. By mapping textures from aerial images on regions, or directly using pictures of the outside of buildings and roads for textures, quite recognizable models of the world could be created in a straightforward manner. Some efforts with this goal are already underway, such as the Virtual L.A. project, which is creating a meter-level accuracy 3D model of the entire Los Angeles basin.⁴⁸

Of course, the Virtual L.A. model and others like it will be incomplete, in part, because some spaces are private, although this changes over time.⁴⁹ For example, if a house is for sale, and there is an "open house" sales event, people are allowed to roam freely, but once the house is sold, unless one is invited in, the space is private. This does raise a social acceptance issue: who wants to have the floor plan of their home as part of a public interface to a spatial information store? David Gelernter's notion of Mirror Worlds⁵⁰ includes dynamically updated models of the world, and of Mirror Worlds he says: "Its goal is merely to convert the theoretically public into the actually public. What was always available in principle merely becomes available in fact."

With a 3D world-view-interface, users could quickly navigate to where they are or where they want to be to post or read information. By pressing a key and making a selection, the 1-meter-cube grids could be quickly overlayed on the world view and users could post to the faces of the cubes. Alternatively, users could post information directly to the surfaces of walls in buildings or the ground outside.

The basic tools that users (as producers and consumers) would need are: 3D world-touring tools, 3D world-construction tools, tools to post messages (Web pages) to any of the six faces of cubic-meter grid overlayed on the world, tools to see information that is available in a region, at different scales (zoom in and zoom out) and sorted by creation date, who posted the information, and keyword searches, etc.

The storage requirements for WorldBoard user experience are a function of the fidelity desired. If simple, coarse-wire frames are used, the storage is quite modest. Modeling a typical city block or suburban neighborhood might require on the order of 1 megabyte to allow someone familiar with the area to navigate; adding some simple textures and details might require about 10 megabytes. However, if numerous unique textures are used and submeter detail is provided, the storage requirements can quickly soar to a gigabyte or more. A sense of the current response rate for these types of models when accessed through a Web browser can be experienced by visiting vendor Web sites.^51,52

The problem of networking multiple servers together to provide a seamless user experience is also challenging. Ideally, anyone should be able to set up a WorldBoard server. However, everyone's WorldBoard servers should be able to interoperate in a manner that provides a consistent user experience. If some organization or individual sets up a WorldBoard server it is either to provide a place for others to post and access geocoded information, or to deliver geocoded information to a wide audience. A service provider must first make available to users a channel identification, which could be a URL, or literally a channel number or name, assuming that registration organizations are set up. Next the service provider must make available to users a model (3D world-view interface) of the space to be navigated, though a user may prefer a different model, raising an interoperability issue. And finally, the service provider must provide useful content that is stored, or more likely, dynamically updated on the WorldBoard cubes, directly attached to a 3D model, or in some other way available for users. To improve the response time, users may choose to cache much of this on their local systems and only receive small packets that update geospatial regions of interest.

WorldBoard clients: Mobile and location-aware. The second stage of WorldBoard requires the development of mobile ways to author and access information associated with places on a planetary scale. The simplest version of the client would be a mobile wireless Web browser with a manual interface for positioning. A more complete client would be location-aware, using appropriate automatic positioning technology, such as GPS.

Again, the issues are numerous, but overall an interesting initial subset of WorldBoard client functionality is technically feasible. Issues include: Can the necessary functionality be packaged in a small, lightweight, mobile device that has adequate battery life, processing power, and storage capacity to be viable? What is the coverage area of the wireless communication, bandwidth, cost, and communication standard employed? What is the coverage area of the positioning technology, its accuracy and ability to produce orientation as well as location data, and the rate of position updates?

Packaging the necessary functionality for a mobile WorldBoard client is very near at hand.⁵³ Already, a number of interesting devices exist with subsets of WorldBoard client functionality:^54-57

1. Magellan Corporation's GSC 100** product, the first hand-held global satellite communicator, can send and receive e-mail anywhere in the world, and built-in GPS allows the e-mail messages to be geocoded. The device is 8 by 3.5 by 1.75 inches and can store about 100 messages.
2. Garmin International's NavTalk** product combines a mobile cell phone with a GPS receiver and includes map displays. Calling another NavTalk phone shows the position of the person being called on the map.
3. Nokia's 7110 product combines a mobile cell phone and hand-held PC with wireless Web browsing capabilities.
4. 3Com Corporation's PalmPilot** PC has add-ons that can provide GPS and mapping software. For example, DeLorme's Earthmate** GPS receiver using the Solus** Promapping application can download multiple maps and routing directions from an on-line server, Street Atlas USA** 6.0, Topo USA**, or AAA Map'n'Go**.⁵⁸

Currently, these devices have limited display resolution (text and simple graphics), limited battery life (less than a day or two at most for very heavy usage), and limited and expensive bandwidth (kilobytes per second that cost up to several cents per second). Nevertheless, portable color game machines and videophones are appearing that may soon support interfaces similar to the proposed 3D world-view interface for WorldBoard.^59-61 For WorldBoard park and museum tours, local storage similar to game machines or portable CD or DVD (digital video disk) players are also a possibility.

Wireless communication capability for the WorldBoard clients could make use of cellular and satellite telephone systems. The primary issues are coverage area, bandwidth, and cost. The demand for cellular telephones in the United States, Europe, and Japan is driving the creation of extensive infrastructure, and competition is beginning to lower costs, though lack of global communication standards limits progress. Overcoming these limits, satellite communication systems for consumers, such as Iridium LLC's 66-satellite network, provide excellent global coverage, but at a cost (about $3000 for a telephone and upwards of $1.79 per minute at voice bandwidths) and size disadvantage.⁶² Also, Orbital Sciences Corporation is shipping the Magellan GSC 100 product mentioned earlier. For short-range wireless communication at higher, less expensive bandwidths, many solutions exist,⁶³ and Bluetooth⁶⁴ is emerging as a very short-range wireless communication standard for information appliances. Overall, industry analysts are predicting a "Moore's Law" of bandwidth, in which prices are halved every 18 months due to the efforts of carriers such as Frontier Corporation, IXC Communications, Level 3 Communications, Qwest Communications International, and The Williams Companies.⁶⁵

GPS is likely to be the core positioning technology for WorldBoard clients, because of both its global scope and plummeting costs. For this reason, it is worthwhile to understand GPS strengths and limitations in depth. Leick⁶⁶ has written a technical introduction to the Global Positioning System and satellite surveying. A summary is included in the Appendix.

Commercial GPS systems are produced by several established players,^54,55,67-69 as well as an increasing number of innovative start-up companies.^70-72 To improve the accuracy of GPS products, DGPS (differential GPS) services are also broadly available in the United States.¹⁵ For even greater accuracy, ground-based stations (pseudolites) can be used as in the Trimble 7400MSi GPS receiver, which provides real-time kinematic, centimeter-accurate position updates computed five times a second with latency of 2/10ths of a second.⁶⁷

For some applications, alternative positioning techniques are preferred or can be used in conjunction with GPS. Especially indoors, GPS alternatives have been used, including: local beacons and vision recognition,²⁵ textured light sources,⁷³ ultrasonics,⁷⁴ and accelerometers.⁷⁵ Ultrasonic positioning technologies, based on triangulation using timing, phase shift, and signal strength data along with other techniques, can provide accuracies of about 5 centimeters in areas of about 10 square meters. Accelerometers provide information about acceleration, and by integrating twice, position can be estimated (acceleration |P8 time = velocity, velocity |P8 time = distance). Each integration adds errors, and without resetting, the errors eventually become so large that the position estimate is no longer accurate.⁷⁶ Inertial navigation systems (INSs) used in cars are based on accelerometers, and solve the error reset problem by relying on turns in the road and accurate maps. A very accurate positioning system that relies on no off-body infrastructure should be possible by combining suitably accurate accelerometers with binocular vision systems to reset errors. As will be discussed in the next section, the binocular vision systems can also provide the information needed to accurately align virtual objects with images of the world, creating real-time/space special effects.

WorldBoard glasses: Overlays. The third stage of WorldBoard is to use advances in positioning, display, and special effects (computer graphics) to create the illusion of seeing (and more generally perceiving) information in places. The hand-held PC with a camera is one of the simplest devices with the capability to display information objects coregistered with reality (that is to overlay and align virtual objects to create the illusion of persistence when an observer moves around). However, eyeglasses that could be worn almost all the time would have the advantage of providing a long-term sustained illusion of seeing virtual objects in the world, if negative physiological side-effects of today's heads-up displays (HUDs) can be overcome. Cameras could be supplemented with additional sensors to provide increased awareness of the environment.

For hand-held devices and glasses, what resolution, brightness, and environment illumination matching are required to make overlayed images convincing? Can the nausea effects often associated with heads-up displays be overcome with improved speeds, resolutions, and understanding of the human perception system? Can worn displays be made suitably stylish and socially acceptable? Other than a camera, what additional environment-sensing capabilities might be needed or useful? What will the user experience be like to have motion artifacts, obstructions, and other unexpected changes in the physical environment? How will object identification and relative spatial coordinates be handled for overlaying information on mobile objects?

The original WorldBoard prototype used Virtual i-O's i-glasses, and provided only a quarter VGA color image in a somewhat bulky headset.⁷⁷ MicroOptical Corporation has introduced a display with the same resolution but in a form that is much closer to normal reading glasses.⁷⁸ Microvision is working on the Virtual Retinal Display** technology, which projects images directly onto the human retina and has potential advantages for achieving high-resolution image requirements for realistic augmented reality.⁷⁹ Other companies, including Displaytech,⁸⁰ Sony Electronics,⁸¹ and IBM,²⁰ have introduced miniature displays. There has even been progress on creating a bionic retina that can be used to restore sight to the blind.⁸² Of course, hand-held displays with built-in cameras are also advancing rapidly, and are quite suitable, and even preferred, for many applications.^60,61

While the form factors and resolution of mobile display technology are improving rapidly, one of the key challenges is the software that can combine real and virtual images into a realistic composite. This is an especially challenging task as the user moves around, since a tracker must be accurate to a small fraction of a degree in orientation and a few millimeters in position; otherwise the illusion of a virtual object in the real world will be destroyed. Azuma²¹ suggests the following demonstration to understand the problem: "Take out a dime and hold it at arm's length. The diameter of the dime covers approximately 1.5 degrees of arc. In comparison a full moon covers .5 degrees of arc. Now imagine a virtual coffee cup sitting on a real table 2 meters away from you. An angular error of 1.5 degrees in head orientation moves the cup 52 millimeters. Clearly, a small orientation error could result in a cup suspended in midair." Or even more simply, close your left eye, and hold a finger very near your right eye to block out some object in the room. Moving your head even the slightest amount (or even a slight vibration of your finger) causes objects at the edge of your finger to be eclipsed. Even very small positioning or orientation errors will cause a noticeable jittering of virtual images overlayed on real objects. Additional sources of error include latency in the tracker and graphics software, which show up when the head is moving rapidly (up to 300 degrees per second). New algorithms to improve the speed and quality of coregistration of an augmented reality image are being developed, and this is an active area of research.^83,84 The two main approaches are feature-based and global image techniques.⁸⁵ Feature-based approaches use recognizable beacons in a scene for registration, whereas global image techniques process all pixels in an image using optical flow or other techniques.

While a discussion of the physiology of the eye and the psychological sensation of sight are beyond the scope of this paper, the absolute limit of resolution, assuming an individual rod or cone is the limit, is between 0.3 and 0.5 minutes of arc⁸⁶ for human perception. For glasses with a 1.5-inch lens that covers about 120 degrees of arc, this translates to an upper bound on resolution of about 10 000 dots per inch (dpi). A more realistic lower bound is based on the fact that most readers can perceive improvements in printed font quality on paper only up to about 600 dpi. Even if displays achieve these resolutions with adequate brightness, the psychology of binocular sight creates a host of additional challenges. Stereopsis, lighting conditions, and motion effects all contribute to the psychological sensation of sight, and deviations from our expectations about the relationship of perceived images to reality can result in difficulty with focusing and a sensation of nausea.²⁹ Fortunately, there are many useful applications for WorldBoard prior to complete solutions for the visual overlay and coregistration problems.

WorldBoard services. The fourth step in the WorldBoard development plan, though it will be going on in parallel with all the others, is to involve existing and new organizations to provide education, safety, entertainment, and industry-specific services on geospatial information channels. New organizations might eventually provide archiving, design, and other associated WorldBoard information services, employing information architects and designers.

Issues include: What are the standards and protocols for combining GPS and the Internet? Who will pay for the creation of geocoded content? What will the role of consumers, businesses, governments, and new organizations be? What will users' experience be in dealing with many WorldBoard information channels?

WorldBoard cannot succeed without geocoded content or, more simply, content that has been spatially tagged with descriptors as to where it will be useful. It took years for a critical mass of HTML (HyperText Markup Language) content on HTTP (HyperText Transfer Protocol) servers to be created before Web browsers emerged as the "killer app" for the Internet.^27,87,88 Proprietary e-mail systems and on-line content services did not move to open Internet standards until a critical mass of Web-browsable content was available and being actively explored and created.

Hence, the first step in WorldBoard economic feasibility has to be the availability of large quantities of geocoded content. Proprietary geocoded content services will likely emerge (they already exist in numerous GIS [geographic information system] databases). However, broad adoption of WorldBoard-like capabilities cannot occur until large quantities of geocoded content are publicly available. Researchers and others must have a reason to use WorldBoard, as well as a way to easily read and write spatially addressable messages. Spatially addressable messages are sometimes directed at a particular person in a place, but are often directed at anyone in a place.

There are many potential sources of on-line geocoded content. Government agencies are one obvious producer. In fact, many states are developing geospatial information infrastructure strategies.⁸⁹ A government initiative, to make geospatial data easily accessible to citizens and businesses to stimulate economic development, could be an important first step to economic feasibility for WorldBoard services. The federal government could take the lead to improve general safety, emergency response, and disaster planning. For example, many costly accidents happen each day. Backhoes and bulldozers accidentally cause millions of dollars' worth of damage by cutting underground cables and pipes.⁹⁰ The easy access to small rental backhoes and the increase in costly accidents may cause an insurance and legislative backlash to occur at some point.

In the United States, legislative action surrounding geocoded information has already begun. Federal Communications Commission (FCC) regulation E911 requires all cellular phone manufacturers to provide a mechanism for locating telephones on which the emergency number 911 is dialed to within about 100 meters, no later than October 1, 2001. In some regions over 30 percent of all 911 calls are made on mobile phones, and this percentage is on the rise. The federal government paid for the GPS system, and funding the development of WorldBoard servers populated with geospatial data possessed by government agencies would be an important step toward further economic development.

Several protocols for GPS-based addressing and routing for the Internet have been proposed. For example, Julio C. Navas of Rutgers University⁹¹ has been working to integrate the concept of physical location into the current design of the Internet, which relies on logical addressing. He proposes "georouting" and "geocasting" to send geocoded messages (such as "bridge out ahead") to mobile computing devices. Geographic routing (georouting) uses polygonal geographic destination information in the geographic message header for routing. His approach uses about eight bytes of information to address any .1 square-mile region in the world. In addition, Navas proposes a GeoARP protocol to populate areas with objects of interest. Like the ARP (Address Resolution Protocol), where individual hosts respond with their IP (Internet Protocol) addresses to an ARP broadcast, in the GeoARP protocol, hosts respond with their GPS coordinates.

Once government agencies "prime the pump," the travel and tourism businesses might move geocoded content onto WorldBoard servers. The "killer app" for this industry might be virtual tourism, helping consumers plan vacations, perhaps selecting the hotel room or table at a restaurant with the best view. Tourists might browse information left by others about hotels, restaurants, and local services.

Other sources of geocoded content include: educational activities (part of the original WorldBoard vision), utility companies, satellite data, and scientific fieldwork. Several companies are producing database and authoring tools for geocoded information.^92-95 In addition, applications with a geospatial component that are in the works for the Palm VII** are suggestive of sources of geocoded content: movie times and locations, ATM (automatic teller machine) locator services, traffic and road conditions, driving directions, weather conditions, sports and local news, Internet "yellow pages" businesses locators, and parcel tracking services.⁹⁶

Ultimately, like the Web, the economic feasibility of WorldBoard businesses may derive from advertising. As visitors are going on virtual tours, they could see virtual billboards. Later as consumer electronics companies produce viable WorldBoard mobile clients and glasses, along with information "in its place," consumers may see advertisements from the sponsors who put them there. In the next section, a range of WorldBoard applications is discussed.

Potential applications

WorldBoard-like technologies provide an opportunity to contextualize some of the vast quantities of spatial data in the world. From this perspective, WorldBoard is a method for organizing information that provides a simple place for individuals and businesses to put information and a simple way to find and share information. While WorldBoard was originally motivated by educational considerations, as outlined in an early section of this paper, many business opportunities exist for WorldBoard. For example, as previously mentioned with respect to improving public safety, one can easily imagine the benefits of utilizing GIS (geographic information system) data about underground buried pipes and cables. Visualizations of where cables and pipes are buried could help construction equipment operators avoid costly accidents. (See Figure 2.) Unfortunately, almost every day, backhoe operators accidentally damage underground pipes and cables. In the case of pipes, environmental cleanup costs alone often run into millions of dollars per incident. The frequency and cost of these accidents are tracked at the Web site http://www.underspace.com/. Insurance companies, construction equipment manufacturers, and government organizations are just some of the many organizations with a vested interest in promoting improved safety by making use of GIS information and location-aware devices.

Figure 2

Over the next decade, many devices will quite likely become location-aware. Our cars, wristwatches, phones, computers, and just about anything else that is mobile and has a chip in it will be location-aware and communication-enabled. FCC regulation E911 will be one of the driving forces affecting communication-enabled devices. As previously discussed, E911 mandates that all cellular phones sold in the United States must be location-aware by October 1, 2001, to provide better emergency response to 911 calls. All of these mobile, location-aware devices will very likely have unique digital identification codes as well. Even ignoring WorldBoard applications that derive value from an ability to associate information with a place, the implications of ubiquitous location-aware devices are quite significant. First, theft of devices will become more difficult, when devices can "phone home" or simply stop working when their authorized user or owner is not operating them. Second, inventory control, transportation, and shopping efficiencies will improve to unprecedented levels. Imagine the cost savings when a company can at the push of a button get a complete inventory as well as the location of assets. Third, individuals will waste much less time looking for destinations, lost things, and each other in crowded places. Paul Saffo⁹⁷ of the Institute for the Future foresees the day when it will be cheaper to know the location of most packages being shipped than to pay the postage to send them to their destination. In sum, many forces (in addition to WorldBoard-like efforts) will be driving the trend toward greater numbers of location-aware devices.

In this section, the benefits of improved ways to organize and utilize spatial information are explored, as well as the benefits of devices that provide users with location-aware and context-aware applications and services. Location-aware applications benefit from access to the knowledge of where a user is. Context-aware applications benefit from knowing not only where a user is, but also information about the activity the user is engaged in and local environmental conditions. Location and time of day can be very powerful predictors of likely activities and environmental conditions, as well as indices to on-line information that may be of great relevance to a user--for example, to access a local weather report.

Recent augmented-reality applications. In the process of performing some complex task, a user's performance (speed, accuracy, reliability) may be enhanced through the use of timely and appropriate additional information. However, interrupting the process to refer to a printed manual or even to listen to and act on vague bits of advice ("OK, line it up with the valve to your right") can be distracting and confusing. In November 1999, at the First IEEE Workshop on Augmented Reality, a number of augmented-reality applications were described in which a user was provided with additional information during the performance of a task. This was done in ways designed not to distract or confuse the user, but to enhance performance on a task. Navab⁹⁸ described an industrial augmented-reality system to assist service and maintenance personnel working on complex pipelines in power or chemical factories--overlaying blueprint information, color-coding pipes and wires, and labeling specific valves and assemblies. Starner and colleagues⁹⁹ at the MIT Media Lab described an augmented reality system, called Stochasticks, to enhance the game of billiards--overlaying alignment, angle, and banking information. Molineros¹⁰⁰ described an augmented-reality system for evaluating assembly sequences in robot assembly planning. Curtis¹⁰¹ described an augmented reality system for use in airplane factories for constructing aircraft wire-bundle assemblies--overlaying the path of current wire through the bundle. Reiners¹⁰² described an augmented-reality system for assisting the assembly of a door lock into a car door--overlaying hidden or obscured areas and sequencing steps. Berger¹⁰³ described a medical application to guide treatment for eye disease--utilizing overlays from preoperation planning. Satoh¹⁰⁴ described an augmented-reality air hockey game with a virtual puck.

Billinghurst¹⁰⁵ described an augmented-reality communication space to see and interact with others. Social augmented-reality systems such as this are still rare. Nevertheless, group authoring expeditions are beginning to occur. The Digital Explorers Society (DEX, http://www.dex.com/) promotes the notion of digital expeditions that record data in context. DEX members include ecotourists, adventurers, and technologists.

Pascoe and his colleagues at the University of Kent developed a prototype system that combined a 3Com PalmPilot with a Garmin GPS 45 for use by an ecologist who spent two months observing giraffes in Kenya.¹⁰⁶ Not only could ecologists more easily collect geocoded data, but they could also post messages that might be of value to other ecologists doing similar observations at the same location at a later time. As a result of this experience, a number of integration problems became apparent due to the variety of hardware and software components that needed to be put together to build this kind of mobile context-aware solution. To address these integration problems Pascoe has proposed a contextual information service (CIS) architecture on which to build future systems. Pascoe notes that just as standards were needed in the early days of the Internet, common protocols for integrating diverse hardware and software are needed now to create "plug-and-play" contextual information services that can combine components from various vendors and researchers.

In addition, Pascoe¹⁰⁶ and others¹⁰⁷ have begun to define categories of context-aware applications. For example, Pascoe describes four basic capabilities that can be used in defining context-aware applications.

1. Sensing: Sensor data presented to user (e.g., "you are here" on a map)
2. Adaptation: Sense and adapt application behavior (e.g., clock sets itself to local time on entering a new time zone)
3. Resource discovery: Sensing and adapting to use local physical resources (e.g., a mobile computer identifies a local printer on which to print)
4. Augmentation: Sensing and adapting to use local physical and virtual resources (e.g., virtual signs appear over buildings as part of a tour)

Within this framework, the augmented-reality applications described earlier can be seen to use sensing (positioning, cameras) capabilities, to adapt generic information templates, and to augment the information from the physical world to support users performing various tasks. In the next section, an alternative capability framework is proposed for WorldBoard applications.

WorldBoard application capabilities. The opportunity for WorldBoard applications exists when there is a good answer to the question: Where is the best place to put a particular piece of information? Sometimes the answer to this question is obvious because the information has spatial attributes, and other times the answer must be arrived at indirectly through a series of inferences about its spatial utility:

1. Spatial information. A question that can be asked about any information resource is: Does this information resource contain spatial data? Some information has spatial attributes or dimensions associated with it, which give it either a natural place to be stored or a natural way to be visualized (or more generally perceived) in a spatial context. For example, all of the following have spatial attributes--geographic information system (GIS) data, telephone books and "yellow pages," address books, maps, architectural plans, and CAD (computer-aided design) diagrams. By some estimates, over 50 percent of business data have spatial attributes. Your location and the location of each of your possessions are important spatial attributes. Any place you are trying to get to, or plan to go to, as well as all the places that you have been, have spatial attributes. When you use a camera to take a picture, your location and orientation are spatial attributes that can be used to add geocoding metadata to the picture. Thus all photographs have associated spatial data, and in general all human artifacts have a creation location, current location, location history, etc.
2. Spatial utility. A second question that can be asked about any information resource is: Where should this piece of information be put to maximize its value to a person or organization? Some information is more useful or makes more sense at one place or set of places than another. For example, advertisements are most effective when placed in heavy traffic areas where they are likely to get more attention. Demographic data can have spatial attributes (zip codes), and specific advertisements have utility attributes that correspond to particular demographic values. Sports statistics about a baseball player may or may not have spatial attributes, but when a player is at a particular ball park, the player's statistics have higher utility to fans watching the game than the statistics for a player who is not at the park.

WorldBoard applications arise when useful information can be put exactly where it is needed most. Often businesses and individuals know exactly where they want information to be or how they would like to perceive the information, but for reasons of cost, insufficient physical space, insufficient structural integrity of materials, esthetics, or convenience, information ends up in a suboptimal place. WorldBoard will have economic and quality-of-life benefits in direct proportion to how well it supports (1) improved placement of information resources (associating information with places), and (2) improved perception of information.

WorldBoard benefits arise from new capabilities that developers can incorporate into applications. These new capabilities are:

1. The ability to easily associate information resources with a place
   • Messages and signs at a location--virtual signs that are less expensive or contain more details than physical signs, advertisements, warnings, labels, names of things in multiple languages, names of plants, names of buildings, discussion lists with questions and answers, reminders, personal postcards, navigation aids, real-estate buyer and seller information, archaeological and ecological records, safety and emergency information signs and warnings required by fire, police, and emergency response organizations
   • Virtual objects at a location--virtual works of art, entertaining objects, educational objects, virtual instruction manuals, and educational simulations of objects at a location
   • Programs and interactive characters at a location--data collection programs that are triggered when location-aware devices arrive at a location, virtual tour guides in museums, parks, etc., sales persons, theme-park characters, and historical characters
2. The ability to perceive information about or at places in new ways
   • Ability to see hidden parts of things--virtual X-ray vision through walls, clouds, and underground; cutaways into man-made and natural things; ability to see through buildings, obstructions, and the surface of the earth; ability to see buried infrastructure such as pipes and cables and building infrastructure such as wiring, plumbing, and physical structure; ability to see construction and excavation blueprints
   • Ability to see invisible things, both natural and cultural--normally invisible sensor data, radiation levels, lines of force, microscopic structure, infrared, night vision, other parts of the electromagnetic spectrum; property lines, rights of way; satellite trajectories; dynamic processes, special effects; and constellations and names of planets
   • Ability to change the appearance of things--real-estate landscaping, full potential of properties, and color of walls
   • Ability to add highlights and overlays--color- coded parts of a complex scene, such as factory pipes in a heating plant; highlighted architectural characteristics, crumbling infrastructure, other perspectives that highlight or downplay various aspects of a visual scene; storage location of hazardous, flammable, or toxic materials in warehouses and in transportation yards as well as on highways
   • Ability to see what was or what could be--historical records, personal records; architectural plans, city plans, Olympics host competition model sites
3. The ability to receive location-based consumer information services--"Yellow pages" and spatial queries on GIS databases; traffic, weather reports; queries on related map-based information; and tracking information for personal possessions

Each of these abilities will allow the creation of many vertical applications, but the question of what are the broad-based horizontal applications for WorldBoard remains unanswered.

Communication medium and perception tool. WorldBoard can be viewed as both a new type of communication medium and a new type of perception tool. Communication media, such as television, the Web, telephone, radio, and publications, allow individuals and organizations to send and receive messages that entertain, educate, inform, and market as well as amplify our social awareness. Perception tools, such as vision-correction glasses, microscopes, telescopes, and night vision goggles, amplify our senses. In this section, both perspectives are examined to generate a few basic questions and then suggest some possible answers.

Viewed as a public communication medium we can ask: How might businesses use information in places to support commerce, advertising, and information presentation? Just as businesses now put information on the Web or on billboards, they may put information in places using WorldBoard-like systems--but only if (1) enough customers are "tuned in" to WorldBoard channels, and (2) it is easy to create content for WorldBoard in a cost-effective manner. Alternatively, viewed as a personal communication medium we can ask: How might individuals creatively use the infrastructure for mundane personal tasks? For example, how might family members use WorldBoard to place virtual greeting cards or reminders in places where other members of the family are most likely to see them?

One possible answer to these questions is that WorldBoard may evolve by first becoming a Web portal, where WorldBoard authoring tools are simply standard tools for creating Web pages, and WorldBoard perception tools are simply Web browsers. As mentioned earlier, in the proposed first step of the WorldBoard development plan, using the Web to bootstrap WorldBoard is a realistic and likely scenario. For example, MapQuest⁹⁵ and Microsoft's TerraServer⁴⁶ allow anyone with a Web browser to "zoom in" on any part of the planet, either with a map (MapQuest) or with satellite and aerial photographs (TerraServer). Yahoo!** and other portals provide map and direction services as well.

A possible next step is to allow users to post information to these servers in password-protected channel areas. Archaeologists might post precise photographs and locations of artifacts to allow colleagues in remote locations to inspect their finds, and families might purchase cameras that automatically geocode and upload photographs to be browsed by far-away family members. Colleagues or family members could leave a spatial bookmark on their WebTV** browser, and quickly check on any updates in posted information--perhaps returning comments of their own on the new additions.

Viewed as a perception tool, we can ask how workers might be able to do their jobs more efficiently, effectively, enjoyably, or safely, or what might happen if workers could visualize the inert knowledge, locked in remote databases, in appropriate contexts. But how will the workers see the information, since it is unlikely that any time soon they will be wearing heads-up displays? Again, viewed as a perception tool, we can ask how individuals in museums or national parks might "virtually zoom in" on a tree or a rock and get annotated information about the microscopic structure of an organism. When learning about GPS, for instance, a learner might look up and get a virtual planetarium with the actual positions of GPS satellites shown as they trace their paths in the sky. But how will individuals get access to WorldBoard-capable display systems? Will ordinary people start making extraordinary observations when equipped with these devices? How might enhanced perception lead to new discoveries? How could enhanced perception tools amplify the collective efforts of many people?

One possible answer to these questions is that WorldBoard may evolve from location-based information services on location-aware cellular phones and hand-held computers. Hotel and restaurant chains, gasoline stations, taxi companies, and other service providers could benefit by providing information about the location of nearby services. As these devices connect to the Internet, consumers may be able to access WorldBoard-like portals described earlier to view and post geocoded information. In particular, if these devices contain cameras (either for teleconferencing, taking pictures, scanning business cards, or sending faxes), then the door is opened to providing superimposed information on the displayed camera image. Probably, the first application of information overlay will be to provide travel information about the names of streets, buildings, directions, and other information easily packaged in an overlayed sign or graphic. In addition, if large numbers of users are routinely carrying around a device that allows them to take a picture, annotate it, and post it to a WorldBoard channel, the threshold will be lowered for rapidly collecting massive amounts of time-and-place-stamped information from many perspectives. Amateur botanists or entomologists can easily upload information with accurate time-and-place stamps that can then be inspected automatically or manually to see if new species have been discovered.

The information explosion is well documented,¹⁰⁸ as are the frustrations of users who often complain that information is too hard to find and to use conveniently.¹⁰⁹ WorldBoard provides one way to improve the convenience of both finding and using certain kinds of information.

If WorldBoard could be realized, it might make finding (accessing) information more convenient by improving the precision and context sensitivity of search and retrieval technology through the use of spatial cues. Information in places is context-sensitive information. Just as we keep wrenches in the garage and forks in the kitchen, WorldBoard promotes the view of a place for information and information in its place. The business goal of making high-quality information materials more available might be achieved if low-cost information appliances could allow customers to access information in a context where it is most valuable. WorldBoard capabilities would encourage creators of information to think about where the information belongs and how best to put it there. Information architects might then be asking: "Where, when, and how will a person be most able to make use of the information I have to offer?" We need communication media and perception tools that help optimize finding and using information if we are to make the most effective use of the vast wealth of resources we are creating, both as individuals and as a whole society.

If WorldBoard could be realized, it might improve the convenience of using certain kinds of information as well. When information is more optimally conveyed to the human perception system, it can be more efficiently utilized with less mental and physical effort. For example, imagine the difference between looking up at the sky and seeing the constellations and names of stars in place, vs looking at a book containing the information and having to move one's head back and forth to verify that one is actually mapping the information onto the appropriate part of the sky. Typically, a person is juggling a flashlight, the wind is blowing the paper, and because of the time it takes for one's eyes to go between daylight vision and night vision, the experience leaves much to be desired. Planetariums have been constructed to give people the illusion of seeing information about the sky in its proper place. WorldBoard capabilities allow for the construction of personal planetariums and shared personal planetariums--combining our perception of the real sky with the information our culture has created about it.

In sum, the need for applications can be supported by either the communication-medium or perception-tool aspects of the technology. Information associated with places can be easier to find and to process than out-of-context information. Of course, not all information can be contextualized in this way.

Social implications

Will this idea catch on? Or will putting information in places merely be an oddity, a technological "side show," that never quite worked right or had enough utility to become a truly viable global information service? Perhaps negative social implications will be discovered that limit adoption. In this section, the technical feasibility of WorldBoard will be assumed, but social issues will be examined. While not exhaustive, all of these issues have been raised by others as objections to WorldBoard.

Personal privacy. While there are arguments for tracking prisoners and certain medical patients, tracking an individual's location has the potential to be a significant invasion of privacy. The potential for misuse of tracking information is significant.^110,111 If an organization is tracking its equipment for inventory control purposes, and an employee is in possession of a communicating location-aware device, then the employer can know the detailed whereabouts of the employee.

Nevertheless, organizations already possess personal and highly sensitive information about employees, and new policy and procedures will need to be developed. Furthermore, tracking of children will also be possible. This may be especially useful at theme parks or in other crowded public places. The trade-offs will include privacy concerns, enhanced security and safety, and perhaps enhanced services that can only be provided if some personal information is shared. Sharing of personal information is increasingly the basis of Web-based business models.¹¹²

Technological "haves" and "have-nots." Access to technology is an issue of increasing concern to a number of organizations.¹¹³ What if WorldBoard becomes a reality, but because of its cost, sections of the population become increasingly marginalized or relegated to less desirable careers and living conditions? This might become a genuine concern, because the incremental cost of WorldBoard-enabled devices over simple Web browsers or cellular phones could be significant due to higher communication bandwidth, processor power, storage capability, and battery power requirements. However, the business model of sharing personal information to obtain free or low-cost devices seems to be on the rise. Alternatively, society has many mechanisms for dealing with inequities other than business model innovations. If Web access or any other type of technology access became the major rallying cry of a large portion of the population, these other methods of addressing the inequity (including taxes and entitlement legislation) would most likely be considered. A more serious cost concern may be for schools. If WorldBoard does prove to be of significant value in education, will schools have the funds to provide WorldBoard-capable devices to their students? Again, as the business experiments unfold (like the ZapMe!** netspace,¹¹² which gives schools free or low-cost computers in exchange for allowing students to see targeted advertisements a certain percentage of the day), it will be easier to determine whether there is a socially responsible way to use such methods to address the issue.

Propaganda, altering perceptions of reality, and graffiti. The ability to transform perception has important ramifications. For example, a politician might urge citizens to become aware of the crumbling infrastructure of a city by tuning in a WorldBoard information channel. The channels could be loaded with the politician's particular world view, amplifying what the politician sees that needs changing. A shared world view can help mobilize and coordinate the actions of many citizens--the power of the press to shape views and opinions is nothing new. Nevertheless, the notion of a socially constructed reality could take on a new, more literal meaning. But will coherence emerge, or simply a cacophony of views? The benefits of contextualized information could be lessened if there are too many WorldBoard channels to search and choose from, and if the material on any channel is authored by too many individuals with too many styles and viewpoints. Will there be a channel for Democrats and a channel for Republicans, each highlighting issues with a political agenda attached?

WorldBoard also supports virtual graffiti. If WorldBoard becomes real, it is likely that one WorldBoard channel will be totally open to whomever wants to post information into a space. Recall that a WorldBoard channel is a mechanism for allowing security, privacy, and protection of what is posted to WorldBoard. There will be many channels, and companies will have their own channels with their version of reality available. Why then would anyone ever look at the "anything goes" channel? Curiosity is probably the main motivation for readers, and publishing to a potentially large audience for writers. Some viewers might be curious about what others have posted to the coordinates of the walls or ceiling in the Oval Office or other rooms in the White House. What channels will exist, who can post to them, and what the rights are of perceivers and posters are just some of the open questions. The Internet will undoubtedly be the basis for WorldBoard, not only as technological underpinning, but in legal precedent on thorny issues as well.

Media addiction and disconnecting from reality. How do citizens of the information age in the United States spend their time? We are tremendous producers and consumers of information. A report published by the United States Department of Commerce¹¹⁴ estimates the media usage of an average citizen in 1995 to be a total of 3401 hours, broken down by media category and hours-per-year usage (see Table 1).

Background music or televisions in waiting rooms, restaurants, and other places provide an information ambiance to surround people. In addition to media usage that can be quite passive, the average U.S. citizen makes about six telephone calls a day, lasting on average three minutes each. The average person spends more than ten hours a day using media or making telephone calls, more time than sleeping!

When a new media technology appears, we hear stories of individuals becoming addicted to using that technology. In a fully functional WorldBoard, changing the colors of the walls, the appearance of furniture, or even personal appearance when one looks in the mirror is possible. Some viewers may prefer the "rose-colored glasses" of WorldBoard to the harsh realities of the real world. As we invent computers that are aware of our emotional state or real physiological needs, rose-colored glasses may alter our perceptions of reality to cheer us up or make things seem brighter or illuminated by brighter sunlight.

More realistically, what happens if individuals start depending on WorldBoard for their safety, income, or other essential parts of life, and the technology fails, locally or on a global scale? Again, this problem is not unique to WorldBoard, but is inherent in a society dependent on a technological infrastructure for its smooth operation. Nevertheless, if the complete WorldBoard vision is realized, greater and greater levels of technological dependence will be encouraged as more and more aspects of life benefit from WorldBoard capabilities.

Terrorism and malicious use of information. Technologies can be misused or used for undesirable ends. For example, even something as seemingly harmless as the wristwatch was viewed warily when it was invented over three hundred years ago.¹¹⁵ The newspapers at the time published a story of how an enemy equipped with accurate chronometers could launch a devastating synchronized attack across all of England. More recently, newspapers warned that terrorists could use commercially available GPS systems to obtain information that would allow them to direct missile attacks. Also, in the same way that backhoe operators could use visualizations of buried cables and pipes to avoid accidents, a malicious operator could use the information to inflict maximum damage in a remote area.

Reflections: Trends and alternatives

The realization of a global infrastructure for associating information with places and supporting enhanced perception services will very likely unfold in surprising ways. More often than not, the way we imagine emerging technologies, and the way they actually turn out is quite different. Thus in concluding this paper it is reasonable to ask: What are the confirming trends indicating that we might be on the right path in our thinking? What are the alternatives? Are we even thinking about the capability of putting information in its place in a reasonable way?

Confirming trends. To date, the potential applications of WorldBoard seem to be largely "vertical" markets. A broad-based horizontal "killer app" for WorldBoard, geospatial browsers, and augmented reality has not been identified.^27,87 Nevertheless, WorldBoard can be seen as part of three trends:

1. Atoms to bits: Why deal with a physical object, when the information object is easier to make and manipulate (e.g., physical to CAD models, typewritten page to word processor document)?
2. Stationary to mobile device: Why go to the communication device or information appliance, when the device can go with you (cellular phone, wearable radio, television, or watch)?
3. Senses to instruments: Why settle for normal senses, when instruments can bring you more information (e.g., corrective lenses, hearing aids, telescope, microscope, radar, sonar, night-vision binoculars, sensors)?

As we learn to create and interact with information objects in real space, the relationship between people and information will be changed. One could argue that our success as individuals and as a species depends on our ability to record, manipulate, and access information. All too often we find ourselves struggling to remember, scrambling to find the right person to call, flying off or dashing off to a meeting, fumbling with books or devices to look things up, or hopelessly awash in too much information. Imagine instead a world where information is right where we need it most, readily at hand.

Throughout human history, the relationship between people and information has been of fundamental importance. The cognitive age of humans started when we used representations of the real (sound, gesture, symbols) to refer to the real.¹¹⁶ Each new representation has advantages and disadvantages. For example, Socrates argued against books, since they could merely remind us of the thoughts of others and did not support true inquiry and questioning unless the author accompanied the book. Our relationship with information is determined in part by our methods of producing, communicating, and consuming information, as well as our methods for establishing the ownership, privacy, and quality of information. Consider a sampling of milestones in this story, summarized in Table 2. (A generation is about 15 to 20 years.)

No one would deny that each of the milestones in Table 2 has had a significant and profound impact on our human relationship to information or our ability to use information more effectively. If this accelerating trend continues, the next decade will result in multiple new milestones of historic significance. WorldBoard is but one of many technology forecasts that describe a new relationship between people and information. Gelernter's Mirror Worlds and TimeStreams,⁵⁰ as well as Weiser's ubiquitous computing¹¹⁷ are other examples of predictions of fundamental changes in our relationships to information and technology. While technical barriers still exist, these might well be overcome quite soon, and then the social and economic barriers would be all that remain.

Unexpected alternatives. Before describing some of the ways in which WorldBoard might turn out differently than has been described here, it will be useful to look at an earlier technology forecast. In particular, we examine a forecast that, like WorldBoard, was motivated by a desire to create a new relationship between people and information.

In July 1945, The Atlantic Monthly published an article entitled "As We May Think," by Dr. Vannevar Bush, then Director of the U.S. Office of Scientific Research and Development. In the article118 Bush called for a new relationship between people and the sum of their knowledge. To motivate the need for the new relationship, Bush cited problems caused by an overabundance of information becoming increasingly too large to conveniently search, as well as information not getting to those best able to utilize it (note that finding and utilizing information are core aspects of WorldBoard):

	The summation of human experience is being expanded at a prodigious rate, and the means we use for threading through the consequent maze to the momentarily important item is the same as was used in the days of square-rigged ships Mendel's concept of the laws of genetics was lost to the world for a generation because his publication did not reach the few who were capable of grasping and extending it. This sort of catastrophe is undoubtedly being repeated all about us as truly significant attainments become lost in the mass of the inconsequential Publication has been extended far beyond our present ability to make use of the record.
Bush proposed technologies ("instrumentalities") that we now recognize as cameras, microfilm, speech recognition, artificial intelligence, and the Internet as the basis for redefining the relationship between people and information. The new relationship would use technologies to help people produce, store, manipulate, and consult the "record of the race." However, more than 50 years later, while devices like those that Bush predicted do in fact exist, their physical form and internal operations are substantially different. For example, consider Bush's description of the "memex," a device for storing all books, records, and communications, to be consulted with exceeding speed and flexibility:119
	It consists of a desk, and while it can presumably be operated from a distance, it is primarily the piece of furniture at which he works. On the top are slanting translucent screens on which material can be projected for convenient reading. There is a keyboard and sets of buttons and levers. Otherwise it looks like an ordinary desk if the user inserted 5,000 pages of material a day it would take him hundreds of years to fill up the repository Most of the memex contents are purchased on microfilm ready for insertion On the top of the memex is a transparent platen. On this are placed longhand notes, photographs the depression of a lever causes it to be photographed onto the next blank page On deflecting one of the levers to the right he runs through the book before him, each page in turn being projected at a speed which just allows a recognizing glance of each one He can add marginal notes and comments, taking advantage of one possible type of dry photography, and it could even be arranged so that he can do this by a stylus scheme, such as is now employed in the teleautograph seen in railroad waiting rooms. . .

The point is simply that in trying to make technology forecasts, existing technologies (microfilm for storage) and ways of thinking (mechanical levers) serve as the point of departure for our thoughts as we project forward. In order to seem plausible, the forecasts are based on directly relevant existing or emerging technologies. So one of the ways the proposed WorldBoard, based on global positioning, global wireless communications, and mobile display technologies, may change is that even more radical capabilities may emerge that provide more efficient means for implementing information in places. For example, breakthroughs in any of the following areas (or more likely areas not considered here at all) could create alternative realizations of WorldBoard on very different technological foundations:

1. Human perception and memory model extrapolated--vision recognition capabilities that allow a camera to know where it is (exact location) and what it is looking at. (This is the human perception and memory model extrapolated.)
2. Molecular marker model extrapolated--ubiquitous computing aerosols or paints that can be sprayed on any material to provide unique and customizable digital identifiers that are easily sensed^120,121
3. Projection displays extrapolated to eliminate the projector--a variation of the ubiquitous computing aerosol that can be sprayed on any material as a clear coating that can change its optical qualities, turning any object coated with the material into an optical chameleon
4. Human-computer interface model extrapolated--bionic sensor and effector advances that better leverage or directly amplify existing human-computer interface capabilities

The bottom line is that while there is clear utility to being able to better associate information with places, it is less clear how that capability will ultimately be realized. As the "memex" example helps to illustrate, WorldBoard will probably be reali