iQueue: aggregation of pervasive data

The vision of billions of users connected to millions of services using trillions of devices is fast becoming a reality. The result will be a vast network of mobile communication devices and data sources, including sensors, newsfeeds, web services, and databases. Potential uses of this data span a wide range of application domains. Applications require flexible mechanisms for constructing condensed and refined views of the raw data, possibly in ways unanticipated by the data providers. Aggregation comprises the collection of high volumes of abstractly specified data, the composition of the raw data into less voluminous refined data, and the timely delivery of the refined data to applications. We have written a paper describing the difficult challenges inherent in creating an aggregation system that is sufficiently flexible, scalable, and reliable to address the needs of applications.

The iQueue system will facilitate the writing of applications aggregating pervasive sources of data and feeding pervasive consumers of data. The system will provide services to resolve abstract source and destination data specifications, compose data, and enforce access controls.

An aggregation system includes data sources such as sensors, network services, and files. It includes data consumers such as applications. Finally, it includes data composers, which act as both data consumers and data sources, combining input data to produce output data. Under different conditions, applications may use different data sources for the same computation, so data sources are best specified not by the name of a particular source, but by an abstract data specification listing attributes of the data to be retrieved. A process called resolution determines a data source from which data described by a given data specification can be obtained. The data may then be composed with other data. A data-aggregation application is programmed by specifying combining rules in terms of abstract data specifications.

[Back to Pervasive Synchronization and Messaging home page.]

Last Modified on February 5, 2001