Conditional Messaging & Dependency-Spheres (D-Spheres)

Conditional Messaging and Dependency-Spheres are novel extensions to standard middleware for messaging and object transaction processing.

Conditional Messaging allows an application to define, monitor, and evaluate various conditions on messages, such as time constraints on the receipt or the processing of a message by a set of final recipients who read messages from intermediary destinations like message queues. Messages can be evaluated to an outcome of success or failure based on such application-defined conditions.

Dependency-Spheres extend the conventional model of distributed object transactions to include conditional messages. A D-Sphere represents a new type of gobal transaction context that includes both conventional object transactions and (conditional) messaging. Within a D-Sphere both synchronous object requests and asynchronous messaging operations may occur arbitrarily, and the final outcome of the transaction and messaging become interdependent. The D-Sphere system is a new middleware transaction service that introduces an increased level of reliability for the use of object middleware and message middleware in combination.

Conditional Messaging

Messaging and messaging middleware (as exemplified by IBM's MQSeries and implementations to Sun's Java Message Service JMS) are often used for purposes of enterprise application integration. Applications create, manipulate, store, and (typically asynchronously) communicate messages using messaging middleware. The middleware guarantees reliable and eventual message delivery to intermediary destinations like message queues, but does not readily support the management of application-defined conditions on messages, for example, to determine the timely receipt or the timely processing of a message by a set of final recipients.

Conditional messaging is an extension to standard MQ/JMS middleware

• to define diverse conditions on message delivery and message processing in a structured and flexible way, and to represent these conditions independently of messages as distinct condition objects
• to send messages associated with conditions using a simple indirection API to standard messaging middleware
• to monitor the delivery to and the processing by final recipients using automatically generated internal acknowledgment messages of receipt and of (transactional) processing
• to evaluate conditions to determine a message outcome of success or failure
• to perform actions based on the outcome of a message, including the sending of success notifications to all destinations in case of a message success, or, the sending of compensation messages in case of a message failure

Conditional messaging uniquely shifts the responsibilities for implementing the management of conditions on messages from an application to the middleware. Details can be found in our paper Conditional Messaging: Extending Reliable Messaging with Application Conditions.

Dependency-Spheres

Many enterprise systems employ both object middleware (as exemplified by the OMG's CORBA or component technologies like Sun's Enterprise JavaBeans EJB) and message middleware in combination. However, support for the integration of object and messaging services, in particular for transaction processing across object and messaging components, is very limited. Standard object transactions such as CORBA OTS transactions consist of synchronous requests on distributed objects only, but do not integrate remote asynchronous message deliveries that can affect the outcome of the transaction. This introduces a limitation to enterprise systems.

As discussed in our paper Strategies for Integrating Messaging and Distributed Object Transactions, different levels of integrating distributed object transactions and messages can be pursued. Advanced strategies include, for example, the integration of outgoing deliveries of messages from within an ongoing object transaction to final message recipients beyond intermediary destinations like message queues.

The paper Dependency-Spheres: A Global Transaction Context for Distributed Objects and Messages introduces the concept and middleware service of Dependency-Spheres, a new kind of a global transaction context for distributed objects and messages. D-Spheres integrate the flexibility of messaging and middleware-mediated interactions into distributed object transactions, uniquely making an object transaction and a set of conditional messages dependent on each other.

The Dependency-Spheres middleware service enables the systematic integration of distributed object transaction processing with reliable conditional messaging. It implements a set of transaction-messaging coupling modes as defined in the paper Middleware-Mediated Transaction

A D-Sphere allows

• to demarcate object requests of standard object transactions and a set of conditional messages in a single atomic unit-of-work
• to send messages to remote destinations at any point in time during the course of an ongoing transaction
• to make the outcome of object requests and object transactions dependent on the outcome of conditional messages, and vice versa

The recovery model of Dependency-Spheres combines rollback techniques of conventional object transactions with compensation techniques for messages that were sent with immediate visibility. The Dependency-Spheres middleware therefore ensures guaranteed compensation even in the presence of system failures. The paper Message Queuing Patterns for Middleware-Mediated Transactions describes the three message queuing patterns we used to implement guaranteed compensation: the Fire-and-Remember pattern, the Recoverable State Machine pattern, and the Detached Compensation for Orphaned Trandactions pattern.

Research Prototype

A research prototype of a middleware service that implements and supports the concepts of conditional messaging and D-Spheres has been developed. The prototype is a proof-of-concept implementation.

The service prototype is writen in Java using IBM's WebSphere application server and IBM's MQSeries messaging middleware. The service is designed for applications running in standard Java2 Enterprise Edition (J2EE) and message queueing environments; it supports OTS/JTS transactions and conditional messaging on top of JMS.

D-Spheres and Transactional Web Services

The D-Spheres model is promising also for Web Services environments. We have started to look at evolving the D-Spheres work to support transactions across Web Services as well. For further details, see the Transactional Web Services project page.

The relationship of D-Spheres and Web Services is also discussed in an article by James Snell of IBM's Emerging Technology Group: "The Web services insider, Part 10: Digging into the issues. Reliability and transactions." (at http://www-106.ibm.com/developerworks/webservices/library/ws-ref10/)

Publications

The following papers are available for download or directly from Stefan Tai, stai@us.ibm.com.

1. Stefan Tai, Thomas A. Mikalsen, Isabelle Rouvellou, Stanley M. Sutton Jr. Conditional Messaging: Extending Reliable Messaging with Application Conditions. 22nd IEEE International Conference on Distributed Computing Systems (ICDCS 2002), Vienna, Austria, July 2002.
2. Stefan Tai, Alexander Totok, Thomas Mikalsen, Isabelle Rouvellou. Message Queuing Patterns for Middleware-Mediated Transactions". In Proceedings of the 3rd International Workshop on Software Engineering and Middleware (SEM 2002), Orlando, FL, USA, May 2002.
3. Stefan Tai, Thomas A. Mikalsen, Isabelle Rouvellou, Stanley M. Sutton Jr. Transactions meet MOM: System Support for Integrating Distributed Object Transactions and Messaging in Java and MQ Environments. Extended Abstract/Poster, ACM OOPSLA 2001, October 2001.
4. Christoph Liebig, Stefan Tai. Middleware-Mediated Transactions. In Proceedings 3rd International Symposium on Distributed Objects and Applications (DOA 2001, Rome, Italy), IEEE Computer Society, September 2001.
5. Stefan Tai, Thomas A. Mikalsen, Isabelle Rouvellou, Stanley M. Sutton Jr. Dependency-Spheres: A Global Transaction Context for Distributed Objects and Messages. In Proceedings 5th International Enterprise Distributed Object Computing Conference (EDOC 2001, Seattle, USA), IEEE Computer Society, September 2001.
6. Stefan Tai, Isabelle Rouvellou. Strategies for Integrating Messaging and Distributed Object Transactions. In Proceedings IFIP/ACM International Conference on Distributed Systems Platforms (Middleware 2000, New York, USA), Springer-Verlag LNCS 1795, April 2000.

Project Members

D-Spheres is an effort of the Advanced Enterprise Middleware group at the IBM T.J. Watson Research Center, Hawthorne, New York, USA. Project members are

• Thomas Mikalsen
• Isabelle Rouvellou
• Stefan Tai

For further information on D-Spheres, please contact Stefan Tai (stai@us.ibm.com).