DRIVE: A tool for developing, deploying, and managing distributed sensor and actuator applications

Sensors and actuators play an important role in many industrial applications. For example, a manufacturing process-control application might use temperature, pressure, and acidity sensors to determine the current state of a chemical or metallurgical process and actuators such as valves, heaters, and coolers to control that process. An automated assembly line might use photo sensors and tactile sensors to guide robotic actuators during manufacturing operations. With the advent of RFID (radio-frequency identification) technologies, people and physical assets in a production environment can be identified and tracked in real time for such purposes as enforcing safety and security policies or improving equipment utilization. An RFID-enabled supply chain can provide timely inventory and product-demand data captured at dock doors, back rooms, shelves, and points of sale to upstream suppliers in order to improve the supply-chain efficiency by reducing inventory and avoiding “stock-outs.”¹ Whereas the broad adoption of RFID is still in progress, its potential benefits to business performance have already propelled early adopters to incorporate sensor/actuator applications as an integral part of their business processes. The value of sensor/actuator applications can be viewed as providing a bridge between events and actions in the physical world and business processes.

Developing, deploying, and managing sensor/actuator applications take place in a challenging environment in which: 1) technology adoption has to be gradual; 2) deployment choices are many and varied; and 3) the diversity of development methods and tools requires highly skilled personnel. We discuss each of these factors in turn.

As with the introduction of any emerging technology, the adoption of a sensor/actuator application is typically gradual in terms of scale and degree of integration. Deployment might start with a proof-of-concept prototype and gradually evolve into a fully integrated component of the enterprise. The adoption cycle may take from several months to several years, in which time the deployment of the application is affected by the introduction of new technologies and by changes in the supporting network and computing capabilities, and in the nature and scale of business operations.

Sensors and actuators are the interfaces between the physical world and cyberspace. They are placed at the locations where physical events are detected and actions are carried out in response. The geographically distributed nature of sensors and actuators leads to a wide variety of deployment choices. To achieve better responsiveness and to reduce dependency on the enterprise infrastructure, computing capability may be placed close to the edge of the network for information processing at the cost of higher deployment and management needs. On the other hand, to minimize the cost involved in deploying and managing remote computing nodes, a centralized data center may be used to host all the required information-processing needs with increased latency and extra reliance on network availability. As a result, many deployment scenarios may call for a compromise between these two options. For example, within a single enterprise, computing capabilities may be placed within larger facilities, where workload is high and responsiveness is important, and a central data center may serve smaller facilities, where reducing the operation cost is paramount.

The application logic of a typical sensor/actuator application deals with multiple concerns, such as interactions with the physical environment, interactions with the business processes that the application supports, mapping sensor data to business-level events, and mapping business-level decisions to commands executable by actuators. Developers are faced with a large number of programming models and implementation choices, each of which addresses a particular aspect of the application. The lack of a unified programming model and tooling support is compounded by the various computing platforms on which the application may be deployed. This diversity results in a steep learning curve and the need for highly skilled personnel.

The tool presented here, Distributed Responsive Infrastructure Virtualization Environment (DRIVE), is an attempt to address these challenges. DRIVE supports the entire life cycle of sensor/actuator applications by providing both an integrated development environment (IDE) and an execution environment. The development methodology in DRIVE, which is component based² and model driven,³ enables developers to better create, reuse, and customize solution assets over a multiphase technology adoption cycle. Developers create reusable software components for tasks such as data collection, event processing, and device control. These components can be either implemented by using textual programming languages or by connecting other components to form a data-flow diagram. In either case, developers are only responsible for implementing the core event-handling logic, while DRIVE generates other necessary code for message passing and invocation, thus reducing the development skills required. DRIVE further separates the application model, which captures the business logic, from the deployment model, which reflects the physical computing infrastructure. This allows the administrators to configure and deploy applications on various infrastructure topologies.

Many aspects of DRIVE are rooted in the software-engineering literature. The reusable DRIVE components act as event-processing agents and form an event-processing network, as defined by Luckham,⁴ when they are wired together to form a data-flow graph. The notion of using a data-flow graph to model a system, with nodes representing activities that produce and consume data and edges representing the flow of data from a producing activity to a consuming activity, is well known. Martin and Estrin⁵ present an early formalization of data-flow graphs, which were popularized as software-engineering tools by Yourdon and Constantine.⁶ The Structured Analysis and Design Technique⁷ introduces the notion of recursively defining a data-flow graph by using lower-level data-flow graphs to define activities in higher-level data-flow graphs.

Edwards⁸ uses a model-driven approach to achieve high configurability and deployment scalability of component middleware publish-and-subscribe (or pub/sub) services. Configurability is achieved by separating quality of service attribute specifications from application logic and delaying their specification until deployment time. Deployment scalability is achieved by federating event channels to minimize network traffic. Model-driven middleware tools are used to generate configuration models and deploy federated event services more efficiently.

Rouvoy⁹ advocates a mixed approach for building adaptable middleware that combines the complementary strengths of component-based and model-driven approaches.

Component-based visual modeling languages are used for embedded software development in tools such as IBM Rational* System Developer,¹⁰ Microsoft Robotics Studio,¹¹ and even the LEGO** Mindstorms** Robotics Invention System 2.0**.¹² These tools generate executable code that runs on a single processor node. A similar approach is used in engineering and simulation tools such as Ptolemy II,¹³ NI LabVIEW**,¹⁴ and Simulink**.¹⁵ These systems target the development/simulation aspects through rich programming features. In contrast, DRIVE balances language features with runtime efficiency and places additional focus on deployment.

The rest of the paper is organized as follows. There are two principal sections and a concluding section. The first of the two major sections contains an overview of DRIVE. It includes the development model, tools for development and testing, and the runtime environment. The other major section contains a case study: dock-door receiving. It first discusses the associated best-practice design pattern and then it describes the development and the multiphase deployment of the application in a production environment. The paper concludes with a discussion of the results of the case study and plans for future work.

DRIVE includes an IDE and a runtime environment to support the life cycle of sensor/actuator applications, as shown in Figure 1. The IDE is built on Eclipse¹⁶ and includes plug-ins that enable developers to model, implement, assemble, test, and deploy sensor/actuator applications following a component-based approach. The runtime environment supports multiple component execution containers running on multiple computing nodes. A set of containers may form an administrative domain, which is managed by a domain controller. The containers in a domain appear as a single, virtualized container to the application components. A library server (shown as the circle in the center of Figure 1) provides a code repository where components are stored, browsed, and retrieved by the IDE or the runtime environment.

Figure 1

For developing applications, a graphical editor is used to model the application as a collection of interacting components. From the component model, an automated code generation function included in the IDE generates Java** “skeleton” source code that allows the developer to “fill in” the application logic. The developer is thus shielded from the complexity and the specifics of the underlying system; this allows the developer to focus on the application logic. In addition, the application tends to be more reusable, because its design is independent of the system architecture.

To deploy an application, the components are assigned to the available execution containers in the domain using a domain editor in the IDE or an administrative console for the domain. Note that the application is developed independently of the domain model that captures assignments of the component instances to the containers. The communication mechanism between components, regardless of whether they are deployed in the same or different containers, is managed by DRIVE and transparent to the application. This separation of the application development from the deployment infrastructure supports variations in topology and allows infrastructure to evolve throughout a technology adoption cycle from proof-of-concept, to pilot, to full-scale deployment, without incurring significant additional development costs.

The DRIVE development model is component based. The application logic is encapsulated in components, and these are the smallest units for code development, publishing, reuse, and deployment.

Component interaction and the type system
The component interaction model of DRIVE defines the following base concepts: event, port, parameter, and interface. An event is defined as a message sent from a producer to a consumer. Some authors make the distinction between an event as something that happens and an event object as the representation of the event in a computer system.⁴ Because we are dealing with a programming system, we use the term event to refer to the event objects (messages). Each message contains a typed data value. Components interact with each other by producing and consuming events through ports. A DRIVE port is named and strongly typed: every event passing through a port has the same data type as the type of the port. This may be a primitive type (boolean, double, long, object, string, or time), an array type with elements of a specified type, or a structure type with named fields of specified types. The primitive type object represents an opaque object reference that a component may receive in input events or may send in output events; it is not examined or manipulated by the DRIVE runtime. The primitive type time represents a date plus the time of day.

Depending on whether events are flowing into a component or out of it, a port is further classified as an input port or an output port. Viewed from outside the component, an output port is an event producer whereas an input port is an event consumer. To allow customization of component instances, a set of parameters, which are named and strongly typed, can be associated with a component. A component may have one or more interfaces; an interface exposes some or all ports and parameters of a component. For example, sensors (actuators) may be modeled using interfaces that produce (consume) events. Furthermore, several components may have the same interface, which allows various devices, from simple ones, such as sensors and actuators, to complex ones, such as devices with both sensing and actuating capabilities, to be defined in a logical and reusable way.

For readers familiar with the Service Component Architecture (SCA) developed by The Open SOA Collaboration, it helps to draw analogies between the SCA assembly model and the DRIVE component model.¹⁷ In both models, the fundamental notion is the component, which makes business functions available for use by other components. The interface concept in DRIVE corresponds to the notion of service in SCA.

Visual language for modeling
The DRIVE IDE includes a graphical editor to visually create components and model their interaction. The diagrams constructed by the editor can be viewed as a domain-specific modeling language. We use two Eclipse frameworks to create the modeling language. The Eclipse Modeling Framework (EMF) is used to create the meta-model for the language. The Graphical Editor Framework (GEF) is used to create the visual editor for the language. An alternative approach would be to define the DRIVE visual modeling language as a profile on Unified Modeling Language** 2 (UML** 2) and to extend existing UML-based modeling tools. We favor the EMF/GEF approach because EMF generates highly compact and efficient model code and GEF allows us to customize the editor and provide a superior user experience. The EMF-generated model package is further customized to provide additional functionality. It is shared between the tools and the runtime environment.

Component reuse
To facilitate component reuse, DRIVE uses the library server as a repository of components. A developer can access library servers through the component palette view in the IDE. The view provides various ways to organize the components available through the servers. When using the graphical editor to author a new component that includes references to components in the repository, a developer can drag a component from the component palette view and drop it on the canvas of the graphical editor.

DRIVE does not define the implementation of the library server. Instead, it provides an extensible way for a particular library server to be plugged into the component palette view. By default, DRIVE uses a built-in library-server implementation based on the OSGi** Bundle Repository (OBR).¹⁸ The library server, which can also be accessed from the IDE, allows developers to publish their components and perform tests locally. The library server also serves the containers at run time, as discussed below.

The DRIVE IDE is implemented in Java as a set of Eclipse 3.2 plug-ins, as seen in Figure 1. The development tasks are supported by the plug-ins labeled Component Modeling and Assembly and Code Generation. The plug-in Testing Support handles testing of individual components (unit test).

DRIVE allows multiple implementations of the same component, that is, implementations that are functionally equivalent and are interchangeable as long as they adhere to the contracts implied by their (shared) interfaces. The DRIVE IDE allows a developer to visually model components and their interfaces. Figure 2 shows such a visual model for component DuplicateFilter. The shaded rectangle at the top is the interface area, which contains interfaces of the component, whereas the larger bottom rectangle is the implementation area, which contains the component implementation. The interface area of Figure 2 contains the single interface DuplicateFilter with two input ports (input ports are shown on the left) and one output port (shown on the right).

Figure 2

In general, a component can be either atomic (i.e., indivisible, cannot be split into smaller components) or composite. The notions of atomic and composite components correspond to the two tasks in software development referred to as programming-in-the-small and programming-in-the-large.¹⁹ Programming-in-the-small refers to writing code in order to create atomic components, whereas programming-in-the-large refers to creating composite components by connecting available (already implemented) components. Whereas conventional programming languages are the appropriate tools for programming-in-the-small, programming-in-the-large requires “module interconnection languages,” which operate at a higher level of abstraction, where internal implementation details of components are not visible. Currently, a DRIVE atomic component can be implemented either in Java or in EventScript,²⁰ a regular-expression-based language specifically designed for event processing. A composite component is an assembly of components, each of which is either atomic or composite, connected by unidirectional event pipes. A DRIVE composite component is created using the event-flow diagram (EFD) language as the implementation language. In addition, DRIVE supports the use of finite state machines (FSMs), which are specified using a graphical editor, for implementing atomic components or for adding stateful behavior to the EFD implementation of a composite component.

Developers use the graphical component editor and the syntax-highlighting EventScript editor to model and implement the solution components. Then, an incremental builder creates executable code from these models. The DRIVE code generator translates visual models of components stored in Extensible Markup Language Metadata Interchange (XMI) format into specialized subclasses of abstract base classes representing runtime components. Runtime instances of these subclasses represent ports and parameters, and handle event movement between ports.

Atomic components in Java
To implement an atomic component in Java, the programmer first adds invocation rules to the component model using the component editor, as depicted in Figure 2. An invocation rule specifies a Java method name and a set of input ports and output ports associated with the method. The code generator then creates skeletal source code for the methods specified in the invocation rules of an atomic component. Each of these methods has a parameter corresponding to the values of the triggering events, and a parameter corresponding to each output port of the component. When there are events available at all of the input ports specified in a rule, the corresponding Java method is invoked. The output-port parameters support operations to create an outgoing event and cause it to be emitted, upon returning from the method, and delivered to all target consumers of the event. The application developer is responsible only for filling in the bodies of the methods in the DRIVE-generated skeleton. As shown in Figure 3 for atomic component DuplicateFilter, the declaration of the instance variable bag and the statements in the onCmd and onInput method bodies are written by the application developer, whereas the rest of the code is generated automatically by DRIVE.

Figure 3

Atomic components in EventScript
EventScript²⁰ is an event-processing language that provides a higher level of abstraction and expressivity than Java for a large class of event-processing tasks, such as recognizing patterns of events that constitute a compound event, managing event sequences that are structured in single or nested loops, filtering events, reformatting events, handling interleaved sequences of input events and temporal deadlines, and “demultiplexing” a single interleaved event stream into logically independent sub-streams. EventScript uses regular expressions,²¹ a concept familiar to many programmers, to declaratively specify the stream of input events. Actions are specified as snippets of procedural code embedded in these regular expressions. Control flow, the management of time-based actions, and the tracking of the state of the computation are handled implicitly by EventScript rather than by the programmer.

The DRIVE IDE provides an integrated EventScript editor and the DRIVE runtime manages event delivery to and from EventScript components. Because the runtime implementation of an EventScript atomic component is simply a finite-state machine repeatedly performing table-driven state transitions in response to input events and time-based events, its implementation is simple and fast.

Finite state machines
A finite state machine is a well-known mechanism for characterizing the behavior of control elements, such as a controller in a coin-operated vending machine. DRIVE provides a graphical FSM editor. The behavior of a component can be implemented by adding states and transitions to the drawing canvas of the component model, as shown in Figure 4 (the FSM on the right is used to implement component Controller). Each transition in the FSM has a trigger expression and, optionally, an action block. The trigger expression can refer to the value of the triggering event and any parameters defined in the component. The trigger expressions of all outgoing transitions from the current state are evaluated whenever there is an incoming event at any of the input ports of the FSM. The action blocks can contain a sequence of statements to produce output events or to set parameter values. An integrated FSM code generator creates executable Java code based on the visual representation of the FSM and the specifications of the expressions and action blocks associated with the states and the transitions of the FSM.

Figure 4

Composite components
A programmer can create a composite component by drawing an EFD on the canvas of the graphical editor. An event-flow diagram is an assembly graph of subcomponents connected by event pipes, as shown in Figure 4. The subcomponents in an EFD can be specified in two ways—either as a nested subcomponent or as a reference to a component or an interface. A nested subcomponent is declared within the scope of the parent composite component and therefore is not visible from the outside. A nested subcomponent can be either atomic or composite and can be implemented using any of the supported methods. Complex composite components can be created by the hierarchical composition of EFDs. A reference is used to obtain services, exposed as interfaces, provided by existing or yet-to-be-implemented components. Developers can browse existing components in the Component Palette view and create references by drag-and-drop operations. A reference is not executable until it is bound to an implementation of the referenced interface or component.

The binding of a reference to an implementation can be specified using either of two protocols: drive and bind. The drive protocol specifies the name of the component implementation using its Uniform Resource Identifier (URI) path. This protocol instructs the runtime container to create a new instance of the specified implementation. The bind protocol specifies the name of a deployed and running instance of a component. Regardless of the protocol, DRIVE supports both early and late binding. In early binding, the target URI is specified at the development time. In late binding, the administrator can change the target URI during or after the deployment of a component. For readers familiar with SCA, the binding concept in DRIVE is analogous to the wiring concept using in SCA assembly diagrams.

To facilitate the mixing and matching of independently written subcomponents that may involve minor differences in the format of the event data, DRIVE allows the developer to activate a simple transformation, specified as a Java expression of the event value, to be performed on each event passing through an event pipe.

When the operation of a component involves several modes (or states), each of which can be modeled as an EFD, we refer to such a component as a stateful composite component. A stateful composite component is implemented as an event-flow diagram coupled with an FSM, both drawn on the same canvas. Both the FSM and EFD operate exactly as described earlier, with the exception that subcomponents and event pipes can be selectively enabled or disabled based on the state of the composite component.

Support for unit testing
The DRIVE IDE allows a developer to subject a component to a unit test before publishing the component to the code repository. To test a component in the current workspace, the developer first specifies a sequence of input events (trace) and the expected output events for the test. Then, the developer creates a local testing container and instantiates the component. The tool generates input events according to the specified trace and feeds them to the component. The output events generated by the component are collected and compared against the expected values. The trace may specify a simulated time at which an input event occurs.

This section describes the features of DRIVE in application deployment and management and the support of the administrators.

Implementation of the runtime environment
The DRIVE runtime is based on Equinox, an Eclipse open-source project that implements the Open Service Gateway initiative (OSGi) R4 specification. The DRIVE runtime is packaged in OSGi bundles, as are the DRIVE application components. The most important runtime bundles are: container, transport, agent, and EventScript engine.

The container bundle contains abstract base classes representing components, the runtime container itself, and the factory. The component base classes, which are used by all DRIVE components, provide functions such as hierarchy management, name resolution, subcomponent creation, and component wiring. The runtime container manages the component over its life cycle, by performing tasks such as loading, instantiation, temporary disabling and enabling, and destruction of component instances. It also handles cross-module reference bindings.

Another important function of the container is the transport of events in a distributed environment. All events are placed into a global priority queue. A scheduler monitors this queue and is responsible for invoking a component when the events associated with its input ports are available. Output events generated by the component are received by the container. If the target consumer is local to the container, a new invocation task is scheduled using the Java-object representation of the event occurrence. If the consumer is in a remote location, the event is serialized and sent to the target runtime using a transport link. The container uses a factory object to dynamically load and instantiate components. It interacts with the library server in order to resolve and install, on demand, bundles containing the requested components.

The transport bundle provides a mechanism to move events from one container to another. It is used by the container and is not visible to the application-level code. The transport interface is declared as an OSGi service, thus allowing the actual implementation to be plugged in.

The agent is the control point of a runtime instance. The domain controller issues commands to the agent, which in turn controls the container and the transport in order to carry out the request.

The EventScript engine is the runtime execution engine of the EventScript language. The IDE compiles EventScript regular expressions into FSM transition tables. These tables are loaded at runtime and passed to the EventScript engine, which performs the state transitions triggered by input events.

Domain modeling
To simplify the task of deployment and management, especially rolling out similar deployment structures to multiple target environments, DRIVE uses a domain model to capture the computing infrastructure and component deployment. The core concepts in the domain model are domain, runtime, component, binding, and assignment. A domain consists of a collection of component instances and a collection of container instances. The component instances form the sensor/actuator applications. The containers represent the physical computing infrastructure, and are typically managed by a single administrative entity. The term runtime is used to represent a container instance. It is characterized by the Internet Protocol (IP) address of the host computer and the port number at which the controller agent listens. (Note that multiple container instances can be deployed on a single computing node, provided that their agents listen at distinct ports.)

The domain model also includes the bindings from references to interface implementations, which allow independently developed application components to collaborate at run time without any code changes. As described earlier, a composite component models the logical interconnection among subcomponents, but not the physical placement of the component with respect to computers in the domain. The placement information is captured at deployment time as an assignment in the domain model. An assignment maps a component to a runtime in the domain. A subcomponent can either inherit the assignment from its parent or override it. This gives the administrator a convenient way to manage application deployments over a distributed computing infrastructure. The DRIVE IDE includes a form-based editor for creating and editing domain models. Each domain is captured in a single model and stored as an Extensible Markup Language (XML) file.

The domain controller is a central Java 2 Platform Enterprise Edition (J2EE**) application that manages all container instances within a domain. After a domain model is created, the administrator can simply import the model into the domain controller using a Web-based administrative console. From an administrator's point of view, the domain controller administrative console presents a single, virtualized image of the entire distributed computing infrastructure.

The domain controller provides daily operational management functions of a domain. When a domain is operational, administrators can use the administrative console to perform the following management tasks: adding or removing a runtime, installing or uninstalling a module, modifying the bindings of a module, changing the parameters of a module, starting or stopping a module, and monitoring events flowing in a module.

Domain testing and deployment to production environments
Besides the domain editor, DRIVE provides tooling support to test an entire domain locally in the integrated testing environment. The Domain Test Launcher allows a developer to create a test domain launch configuration by choosing a domain model from the workspace and optionally specifying the number of local test containers. The domain testing tool then creates a local version of the domain controller and runtime containers. DRIVE gives the tester the option to use components residing in the workspace, instead of a library server, thus shortening the testing cycle. While a test domain is active, a developer can use the Domain Explorer view to connect to the domain controller and perform management operations such as starting and stopping modules, setting parameters, and monitoring events. A developer can easily verify the binding among components before deploying the domain model to a production environment.

After modeling and testing, a domain can be deployed to a production runtime environment, which consists of a production domain controller instance and a number of containers running on the runtime nodes in the domain.

In this section a proposal is first presented for a design pattern for sensor/actuator applications, and then an example sensor/actuator application is used to illustrate the use of DRIVE for developing and deploying the application. The example, an RFID-instrumented receiving-dock door, is based on a use case common to distribution centers in the supply chains of retail enterprises. It illustrates the development and phased deployment of the dock-door receiving application in a large distribution center with 200 receiving-dock doors. The project starts with a single-door proof-of-concept implementation and progresses to a five-door limited pilot and on to the full-scale rollout.

Component-based development, model-driven development, or service-oriented architecture by themselves do not guarantee the application is reusable. For better reusability, we propose to use the three-layer application design pattern shown in Figure 5.

Figure 5

The bottom layer, the capability layer, exposes the capabilities of physical devices to the higher levels of the hierarchy. Components in the capability layer are usually adapters wrapped around operating-system device drivers, but could also be application programming interfaces for accessing Web services or message queues.

The top layer is the application logic layer. DRIVE components in this layer implement the core business logic using concepts that directly correspond to requirements. For example, the dock-door receiving application requires the service of a motion sensor. There are many different ways in which a computing device can interface with, and control, a motion sensor: via a serial port, a General Purpose Input/Output (GPIO) pin, or TCP/IP (Transmission Control Protocol/Internet Protocol). A good interface design should capture the most salient feature of the motion sensor—that is, it sends out an event when motion is detected—and defer the binding to the actual implementation, thus isolating the common, reusable portion of application logic from the binding.

The design concepts of the application logic layer are not usually directly related to the capabilities of physical devices exposed by the capability layer. For example, a light-stack component in the application logic layer may use names of colors in its interface, whereas the underlying physical device is connected to GPIO pins of an RFID reader and can only be controlled by applying high or low voltages to certain output pins. It is the role of the middle layer, the capability abstraction layer, to bridge this gap. In other words, components in the capability abstraction layer match the requirements from the top layer with the capabilities in the bottom layer.

There is a clear analogy to operating system design: the capability layer corresponds to the device-driver layer in an operating system, the capability abstraction layer corresponds to the hardware abstraction layer, and the application logic layer corresponds to user-level code.

In this scenario, a receiving process involves accepting incoming pallets of goods and stowing these in a put-away area. The business objective is to manage this process by verifying that the pallets are indeed intended for the receiving location and that no cases are missing from the pallets. In addition, the successful delivery of a shipment is to be recorded and a notification is to be sent to both shipper and receiver so that higher-level business processes, such as payment or in-store restocking, can be initiated. If carried out manually, the receiving process is error-prone and expensive and for these reasons was rarely used prior to the advent of RFID technology.

Figure 6 shows an RFID-instrumented receiving-dock door. The motion sensor and light barrier detect the entry and exit of a pallet and trigger events that demarcate the beginning and end of a tag data collection cycle. The RFID reader collects the tag IDs and the red, amber, and green indicators in the light stack provide visual feedback for the operator.

Figure 6

When there are no pallets in the receiving dock, the RFID reader is turned off and all three indicator lights are off. When the motion sensor detects the entry of a pallet, the RFID reader is turned on and the amber light is turned on to indicate that the portal is now operational. The tags read by the reader are filtered to remove duplicates. Unique tags are stored temporarily in local memory. When the pallets leave the dock, as detected by the light barrier installed behind the dock, the RFID reader is turned off. Meanwhile, a validation request with all the pallet tags just scanned is formed and sent to the external validation process. The receiving process then waits for the validation result to arrive. If the validation succeeds, the green light is turned on to reflect successful receipt of a shipment, indicating that the operator can move the pallets to the put-away area. If the validation fails, the red light is turned on to alert the operator to turn back the shipment. In either case, all indicator lights are turned off after 5 seconds so that the next receiving cycle can begin.

Figure 4 shows DDR, the DRIVE component that implements the application logic for dock-door receiving. DDR is implemented as a composite component. The event-flow diagram shows the event flows in the application. The Controller subcomponent is implemented as a finite state machine. It models six distinct states within a tag data collection cycle. DDR makes references to four abstract capabilities—TagCollection, Motion, Barrier, and LightStack—which are indicated as green boxes. Also shown in this figure is component FaultDetector, which is described below.

Component FaultDetector
The role of component FaultDetector, whose implementation uses EventScript, is to detect antenna performance problems. If the total tag read count of any antenna attached to the reader (i.e., the number of RFID tags read by that antenna during a read cycle) is consistently below a threshold specified as a percentage of the total tag read count for all antennas, a problem such as radio interference or bad cabling may have been detected. When this condition is detected, an e-mail notification is sent to the person responsible for handling it.

As shown in Figure 7, FaultDetector is a composite component consisting of three atomic components, Stage1, Stage2, and Stage3. Figure 8 shows the EventScript code for these atomic components. As we demonstrate in this section, many event-processing programming tasks are simplified by constructing a pipeline of EventScript computations such as the one depicted in Figure 7.

Figure 7 Figure 8

Stage1 receives ReaderReport events emitted by component ArraySplitter shown in Figure 4, each event reporting the number of times a given tag ID was read by a given antenna during one read cycle. For each ReaderReport it receives, Stage1 emits through its first output port a CountForAntenna event indicating the number of tags read by the specified antenna. In addition, at one-minute-long intervals Stage1 emits through its second output port a CountForMinute event indicating the total number of tag readings by all antennas during that interval. The EventScript program begins with an input-event declaration indicating that a ReaderReport input event carries a value of the structure type {string antennaID; string tagID; long count}, followed by an output-event declaration indicating that a CountForAntenna output event carries a value of the structure type {string antennaID; long count} and a CountForMinute output event carries a value of type long. These declarations are followed by a regular expression in which sequences of actions are embedded. (The sequences of actions are enclosed in curly braces.)

We are accustomed to regular expressions that are matched against strings, and contain placeholders matching individual characters, but in EventScript, regular expressions are matched against sequences of events, and contain placeholders matching individual events. In Stage1.es, shown in Figure 8, the placeholder ReaderReport(rr) matches an occurrence of a ReaderReport event, and assigns the structure value carried by that event to the variable rr; and the placeholder arrive[.:.:00]() matches the arrival of any time of day consisting of any value for the hour, any value for the minute, and zero (00) seconds (i.e., the start of a minute). If R is a regular expression, the regular expression R* matches zero or more occurrences of event sequences matching R. The regular expression in Stage1.es is of this form, where R matches the sequence of events that occur in any one-minute interval—zero or more ReaderReport events (indicated by another * expression) followed by the arrival of a time of the form .:.:00. Turning to the actions embedded in the regular expression, the variable totalForMinute is set to zero at the start of the pattern for a given minute. After each arrival of a ReaderReport event, totalForMinute is incremented and the expression

!>CountForAntenna({antennaID : rr.antennaID, count : rr.count});

is executed; the action taken is to emit a CountForAntenna event carrying a structure whose antennaID and count fields get their values from the corresponding fields of rr. Moreover, a CountForMinute event carrying the value totalForMinute is emitted at the end of each one-minute interval.

The input-event declarations of Stage2.es correspond to the output-event declarations of Stage1.es. However, the declaration of the input event CountForAntenna contains the clause “group(CountForAntenna.antennaID)”, indicating that incoming CountForAntenna events are grouped by the values of their antennaID fields; in effect, a different instance of Stage2.es executes for each such value, and each CountForAntenna event is directed to the instance corresponding to its antennaID field. Because the input event CountForMinute is declared without a group clause, CountForMinute events are directed to all instances of the component. Thus the regular expression for Stage2.es specifies the pattern of events observed for any one antenna: zero or more repetitions of the pattern for a one-minute interval, each consisting of zero or more CountForAntenna events for that antenna, followed by a CountForMinute event. At the end of a minute, each instance computes the share of all tag readings for the one-minute interval attributable to its antenna, emitting a GoodMinute event if this share is greater than or equal to THRESHOLD (e.g., 2 percent) and a BadMinute event otherwise. A GoodMinute event carries the antenna ID corresponding to the instance (denoted by the construct group[0]); a BadMinute event carries the antenna ID and the antenna's (deficient) share of the reader's tag readings.

Stage3.es receives GoodMinute and BadMinute input events, both grouped by antenna ID so that a separate instance of the component executes for each antenna. The role of this component is to emit a LowShare output event for each disjoint block of three consecutive one-minute intervals in which the share of the antenna is below the threshold; that is, for each disjoint sequence of three consecutive BadMinute events. (It is the resulting stream of LowShare events that the FaultDetection component feeds to component Notification in Figure 4.) The LowShare output carries the average reading share for the three BadMinute events that triggered it. The regular expression matches repeated occurrences of sequences that culminate in a LowShare event. Each such sequence consists of a subsequence that does not trigger a LowShare event (consisting of zero or more GoodMinute events, each preceded by zero, one, or two BadMinute events), followed by three consecutive BadMinute events.

Proof-of-concept deployment
During the initial proof-of-concept phase, a single receiving dock door is instrumented with an RFID reader (of fictitious brand XYZ) and other sensors as depicted in Figure 6. The motion sensor and light barrier are connected to the GPIO input pins of the reader and the light stack to the GPIO output pins.

Component XYZReader is created by adapting the device driver provided by the vendor. It exposes two interfaces: TagCollection for the reader functionality and GPIO for the I/O capability.

To provide the higher-level capabilities required by the application logic, component Portal is created. It references the TagCollection and GPIO interfaces (services), and exposes four interfaces: TagCollection [shown in Figure 9(A)], which is a direct pass-through to the low-level interface, and Motion, Barrier, and LightStack [LightStack is shown in Figure 9(B)], which are implemented as simple EFDs. Notice that the connections to the GPIO reference reflect the actual physical wiring, facilitating simple solution verification and customization.

Figure 9

For this proof-of-concept phase, a single server runtime is used in the domain. Three DRIVE modules—one instance each of XYZReader, Portal, and DDR—are deployed to the domain. Since there is only a single runtime (named server), all components (and their subcomponents) are assigned to this runtime. The domain model is created in the domain editor and then deployed to the domain controller, which manages the DRIVE container on runtime server to carry out the specified application behavior.

Pilot deployment
In the second (pilot) phase, five doors are instrumented with the same physical setup used in the previous phase. In addition, embedded controllers on the RFID readers are activated and used as part of the DRIVE domain. The deployment requirements are easily supported by DRIVE:

• Five additional runtimes are added to the domain model and configured using their IP addresses.

• Four additional instances each of XYZReader, Portal, and DDR are deployed to the domain.

• To take advantage of the processing power of the embedded controllers and offload computation from the server, the Reader instance, the Portal instance, and the Controller subcomponent of the DDR instance are each assigned to the runtime executing on a particular runtime controller.

As a result, the computation is easily distributed over the existing infrastructure without requiring any reengineering of the original solution.

Using reader virtualization to correct an antenna performance problem—During the trial runs of pilot deployment, component FaultDetector generated a number of warnings indicating possible reader/antenna performance degradation. The problem was determined to be caused by the long antenna cable that goes over the portal to the antenna on the opposite side of the reader. A solution is devised to shorten the cable length. Instead of serving a single portal, the two antennas of a reader will be shared by adjacent portals (One additional set of devices is needed at the end of the row of portals.)

Because component DDR does not specify the implementation of TagCollection, this change in hardware configuration can be easily accommodated in the abstraction layer. In addition to the original four interfaces, it exposes a fifth one named ContributionAnt, which corresponds to the antenna that is not used by the local portal. It also references to an additional NeighborAnt in its implementation of the TagCollection interface. The tag output stream from the referenced TagCollection is split by its antenna ID. Tags from ant0 and tags from the NeighborAnt are combined to form the output of the exposed TagCollection interface, thus creating a virtual reader for the application logic. Tags from ant1 are fed to the exposed ContributionAnt interface. The application logic layer component DDR remains unchanged. It still binds to the four interfaces exposed by Portal, namely TagCollection, Motion, Barrier, and LightStack. Additionally, the NeighborAnt reference of each Portal is bound to the ContributionAnt interface of its neighbor Portal. The implied peer-to-peer communication among the Portal instances does not need to be dealt with explicitly. DRIVE runtime handles the inter-runtime message transport between runtimes transparently.

Full-scale deployment
Following the reader virtualization procedure above, the portals worked as expected, and the decision was taken to perform the full-scale deployment to all 200 portals. The results of the experiment are discussed in the concluding section.

During some peak operation hours, excessive CPU load was observed on the server—the large fan-out number caused the server to become a bottleneck. To further offload the computation to the edge of the network, the administrator modified the domain and reassigned the subcomponents ArraySplitter, DupFilter, CaseFilter, and Collector of each DDR instance to the corresponding embedded controller. This simple procedure eliminated the problem.

As sensors and actuators are increasingly being incorporated into enterprise computing systems in order to improve business efficiency, a new breed of technologies is needed to address the challenges of the development and deployment of sensor/actuator applications over an evolving, distributed computing infrastructure. The approach and the tools provided by DRIVE and described in this paper are an example of such a technology.

The previous section describes how the DRIVE prototype was developed and deployed in a production environment. This project demonstrates important benefits are to be gained by the use of DRIVE. Because DRIVE presents to developers a unified programming model, development effort is reduced and the quality of the resulting application is increased. Due to the hierarchical nature of the design pattern, the developers are allowed to focus on the application logic while code-generation tasks further cut down on the development effort. By providing significant support for the application throughout its life cycle, DRIVE enables flexible deployment of the application and reduces the cost of deploying and maintaining the application.

DRIVE provides support for sensor/actuator applications throughout the technology adoption cycle and the application lifetime, accommodating variations in architecture, in infrastructure capabilities, in communication mechanisms, and in specifics of sensors and actuators. The central concept is infrastructure virtualization, which separates the application logic from the infrastructure capabilities. It is possible to deploy an application to run on a single processor in a data center, or as a collaborative set of application components running on distributed processors, without having to modify the application logic. The ability to assign application components to processing nodes at deployment time paves the way to system-level performance optimization.

Reuse of the essential design of an application is also made easier by the concept and practice of device virtualization. DRIVE allows the use of interface references as placeholders for logical devices, which may be bound to different physical devices or by other artifacts playing the role of physical devices at various points in the application life cycle. Device virtualization enables stepwise refinement: The application logic can be designed using a logical model of the sensors/actuators, and the realization of the logical devices can be tailored to the specific deployment needs.

A complete sensor/actuator application deals with a variety of devices, platforms, legacy applications, and communication links. DRIVE provides a framework for application modeling and implementation in a manner that is independent of these variations, thus reducing the complexity of application development and the need for specialized skills. The framework builds on a unified programming model supported by an integrated set of tools, including graphical modeling and automatic code-generation capabilities.

DRIVE enables application developers to focus on business logic rather than the mechanics of its implementation. Developers express the business logic in terms of an abstract model. Detailed code sequences for interfacing with hardware and middleware, and for choreographing the reactions to asynchronous sensor inputs, are generated automatically. This saves labor, eliminates any opportunities for clerical error, and makes the high-level model more generally applicable.

Separation of concerns is not only a good software engineering practice, but it also enables independent participants in a sensor/actuator application ecosystem to contribute with clearly defined roles and boundaries. Device vendors provide device drivers that can be plugged in into a DRIVE application. Application developers focus on the development of reusable, business-specific logic. Solution integrators address customer- and deployment-specific requirements, integrating application components with hardware devices, legacy applications and business processes, and testing and deploying them on the target infrastructure. System administrators manage the infrastructure capabilities and deployed applications.

A lesson we have learned from experience is the importance and difficulty of incorporating graphical user interfaces (GUIs) for end users—for example, the control console of an assembly line, the touch screen of a vending machine, or a point-of-sale terminal—into sensor/actuator applications. These GUIs are traditionally programmed and controlled with languages or protocols, such as Java Server Pages (JSP**) for Web GUIs or the Standard Widget Toolkit (SWT) for native GUIs, which require additional skills and thus increase the overall cost of ownership. GUI programming is fundamentally event-driven: The widgets generate and receive asynchronous events such as button clicks and label-text changes. If we treat these widgets as virtual sensors and actuators, we can apply the visual language used in DRIVE to GUIs as well, thus further unifying the programming models involved in sensor/actuator applications. We are currently extending the DRIVE tool and runtime to model GUIs in this way.

Today, DRIVE supports the separation of deployment concerns from business-logic concerns, and the deferral of deployment decisions until just before production use of the sensor/actuator application. We plan to extend DRIVE support for application deployment in two ways. First, we would like to help the developer choose an optimal deployment. Second, we would like to support run-time reconfiguration of an application in response to varying network conditions.

While DRIVE makes it easy for the developer to impose a mapping of an application to physical resources, today it is up to the developer to choose that mapping. This is a daunting task, because deployment decisions interact with each other. For example, moving two components to the same processor may clear up a backlog by eliminating the communication latency between them, but it may create a new backlog by overloading that processor. Optimal deployment is a multidimensional optimization problem whose analysis depends on intricate queuing models. In one scenario, DRIVE could assist the developer by recommending a deployment based on predicted workloads, including the distribution of event arrival times, and the nominal capacity of processors and communication links. In another scenario, the advice of DRIVE would be based on actual execution profiles.

However, any static assignment is only as good as the assumptions it is based on, and even the data collected in actual execution profiles is subject to variation. Sources of variation include different business workloads (for example, a higher volume of shipments during the holiday season), processor loads, network capacity, and device failures. With DRIVE, binding of resources late in the development process is a promising first step toward even later binding of resources, at execution time every time conditions change. This results in a better-performing system and, in the case of reconfiguration in response to device or processor failure, a more-robust system. DRIVE support for automated reconfiguration in response to resource failure would be an application of the principles of recovery-oriented computing.²²

This work was partially supported by the IT839 project from the Institute of Information Technology Assessment and Ministry of Information and Communication of the Republic of Korea. We thank Yoon-Deock Lee, project manager, and the team members who contributed to the DRIVE prototype system: Woo Chul Jung, Soo Yeon Kim, Sung Wook Kim, Kang Yoon Lee, Myoung Min Sim, and Yong Hun Eom of the Ubiquitous Computing Lab, IBM Korea, and Johnathan Reason and Danny Wong, Thomas J. Watson Research Center, Hawthorne, NY. We also thank our colleague Francis Parr for many valuable discussions.

*Trademark, service mark, or registered trademark of International Business Machine Corporation in the United States, other countries, or both.
**Trademark, service mark, or registered trademark of The LEGO Group, National Instruments Corporation, The MathWorks, Inc., Sun Microsystems, Inc., Object Management Group, Inc., or the OSGi Alliance in the United States, other countries, or both.

Accepted for publication November 28, 2007; Published online May 6, 2008.