Concurrent publishing with ordering in a queued JMS publish/subscribe environment.
Original Publication Date: 2004-Sep-21
Included in the Prior Art Database: 2004-Sep-21
A locking strategy allowing a multi-threaded JMS publish subscribe broker to maintain correct message sequence when processing a queue of publish requests.
Concurrent publishing with ordering in a queued JMS publish /subscribe environment.
The JMS specification requires that the sequence of publication produced by each publisher are delivered to each subscriber in the same order. If the JMS provider is able to queue the requests (for example to cope with a peak in publisher workload in a large fan out topology) then the provider must process the queued publications in an order which preserves the required publisher order. A common method of achieving this goal is to globally serialize the processing of all publish request. This invention describes a locking design which allows multiple consumers to process the queued publications whilst preserving publisher order.
This invention has been implemented in the MQSeries* publish subscribe broker. For simplicity the invention is disclosed using MQSeries terminology, although the core idea could be used in other JMS providers.
Publications are written as MQ messages which stored on a broker stream queue. The broker schedules a set of threads to process the publication requests.
The broker stream queue is serialized by a simple mutex (only one thread is allowed to issue MQGET concurrently). When the MQGET completes then the publisher thread examines the publication request to discover the publisher identity. The publisher thread then determines if any other publisher thread is currently publishing against that publisher id, if so then the publisher thread is added to a FIFO queue of publisher threads waiting to publish on behalf of this particular publisher id, and then the mutex serializing access to the broker stream queue is released (it is vitally important that the mutex protecting the input queue is not locked while the publisher thread is waiting for control of the publisher id). If no other publisher thread is currently operating on behalf of this publisher then the publisher thread records that it is publishing on behalf of this publisher and releases the mutex serializing access to the input queue. As each publisher thread successfully completes processing a publication request then it checks to see i...