Dismiss
InnovationQ will be updated on Sunday, Oct. 22, from 10am ET - noon. You may experience brief service interruptions during that time.
Browse Prior Art Database

Concurrent Message Processing Algorithm that guarantees no messages will be lost

IP.com Disclosure Number: IPCOM000018747D
Original Publication Date: 2003-Aug-05
Included in the Prior Art Database: 2003-Aug-05
Document File: 3 page(s) / 61K

Publishing Venue

IBM

Abstract

Disclosed is a new concurrent message processing algorithm that guarantees no messages will be lost. It changes the way that the messages are retrieved and acknowledged so that the messages being processed would not be lost at the time of catastrophic failures, such as message consumer crashes or connections is lost.

This text was extracted from a PDF file.
At least one non-text object (such as an image or picture) has been suppressed.
This is the abbreviated version, containing approximately 54% of the total text.

Page 1 of 3

  Concurrent Message Processing Algorithm that guarantees no messages will be lost

   For performance reasons, people often want to concurrently process messages from a JMS queue. The common known approach to do this is illustrated in Figure 1.

Worker Thread

n gsm ssec2. Pro

nru3. ret

3. return

JMS Server Message Consumer

Session

1. retreive msg

4. ack / remove msg

2. Process msg n+1 3. return

The problem with this approach is that the messages being processed could be lost when the message consumer crashes.

Let's begin with the definitions in our context:

JMS Server manages queues and messages in the queues A message consumer is a JMS client that consumes messages from the server. It can be a listener getting messages pushed by the server; or it can be a receiver polling messages from the server. A Session object is a single-threaded context for producing and consuming messages. That is, it creates message listeners and receivers.

In order to not lose messages, the JMS message consumption mechanism is designed in such a way that the consumed messages will not be explicitly deleted from queue until an acknowledgement is received from the message consumer. In the event that the message consumer crashes before acknowledging the JMS server, the messages being consumed will be restored, which will be redelivered when the consumer is back.

However, there is a catch - acknowledgment takes place on the session level: Acknowledging a consumed message automatically acknowledges the receipt of all messages that have been consumed by its session. For example, if a message consumer consumes ten messages and then acknowledges the fifth message delivered, all ten messages are acknowledged.

Worker Thread

2. Process msg n+m

Thread Pool with threads

Figure 1. The common known approach - all messages are retreived in a single session

Worker Thread

[This page contains 1 picture or other non-text object]

Page 2 of 3


1.

Before we explain the reason, let's examine this approach:

In a JMS session, the message consumer retrieves a message from a queue;


2.

The message consumer sends the retrieved message to a worker running in


3.

separate thread; The worker is done with the message processing and returns;


4.

Since the messages are processed by workers concurrently in separate threads, steps from 1 to 3 are repeated. As a result, there could be multiple messages being processes at the same time, which could lead to the message losing problem - Say, multiple messages are being processed concurrently. The time for processing them usually varies. When one of them is done, the acknowledgement is triggered while others are stil...