Pervasive

3D Graphics

Visual and Geometric Computing

Recently a new class of PDAs with greater color depth and resolution, faster processors, and more memory has become available. These higher-powered devices provide a platform that can support compute-intensive applications such as 3D graphics.

Applications such as Mechanical Computer Aided Design (MCAD), which rely on interactive 3D viewing to communicate information to end-users, can be developed for handheld devices without sacrificing image quality or performance. IBM Research has prototyped a 3D Viewer for PDAs. This project presented interesting technological challenges due to the limitations of handheld computers: The small screen size, restricted input capabilities, and limited processing power mandated efficient use of resources and novel approaches to user interactivity.

The viewer prototype can interactively render 3D lit and textured models. Additionally, a user may annotate and save specific views of the model. IBM is exploring an application that involves loading CAD models from a parts database into the PDA viewer, interacting to create the desired view, annotating it with design revision notes, and transferring the annotated model back to the server. Other applications such as global navigation or catalog shopping are all feasible with this technology.

Full-size screenshots

Our 3D viewer prototype can currently render models with 15-20K triangles, smoothly shaded or textured, at 12 frames-per-second. The user interacts with the model by moving the stylus on the screen. Moving the stylus in the center of the screen rotates the model. The bottom and left sides are for translations. Sliding the stylus on the right side of the screen zooms the model in and out.

Technical Details

The 3D Viewer is designed to process a subset of VRML 2.0. Lack of file system support in Palm OS requires that 3D data be encapsulated in the Palm PDB format. A VRML 2.0 to Palm dB conversion utility is used to accomplish this. Although the data format is identical for the Palm and Pocket PC, internal storage is different. Geometry and attribute data may span multiple records on the Palm larger memory allocations. A scheme has been deployed
to split larger data across multiple records in the conversion utility and reconstruct it in the viewing application.

The 3D Viewer consists of (1) database (2) geometry processing (3) rasterization and (4) gui/display. The C code is organized into state specific functions that are dynamically selected based on rendering state. A depth buffer and color buffer are allocated from available memory. Although Pocket PC devices typically have 16-bit frame buffers, an 8-bit color buffer is chosen to accommodate the Palm restrictions and to achieve faster blit speeds. The renderer is designed to batch process vertex buffers to minimize instruction cache misses. Additionally, buffer sizes are chosen specific to the target processors cache size to maximize cache utilization. Floating point emulation on the Palm predicates the use of integer operations wherever possible. Backfacing polygons are culled early in the geometry processing stage. The raster engine consists of 10 different triangle processors that are switched-in based on state.
Rasterization is optimized for triangle processing and z, <s,t>, and/or <r,g,b> coordinate values are interpolated over the triangle. Textures are perspective corrected. A scheme for properly mapping interpolated <r,g,b> values to valid indices in the color table has been devised. User interface is similar on both platforms although differences in Palm and Pocket PC GUI tool kits prevent an identical look and feel. Stylus, buttons, or thumb wheels are used for input. Screen regions are used to specify different user interactivity.

The prototype 3D viewer provides an annotation function that can be used to add text comments to user-selected views of a 3D model. Several views can be saved and retrieved on demand from a menu. Annotations in new or existing views can be easily added, deleted, or edited.

In our prototype, 3D models are transferred to the PDA from a data repository via a docking station or wireless network. A user typically selects models to be transferred to the PDA using a web-based application running on a PC connected to the server. The selected models are processed by a transcoder, which automatically prepares the models for viewing on each specific handheld device. For example, depending on the charactristics of the target device, the model could be simplified and/or compressed, textures reduced in size, or lighting pre-computed.

The synchronization mechanism also takes care of transferring the text-annotated views back to the server.