Adaptive load-based expiration of pending messages in a publish/subscribe messaging system
Original Publication Date: 2008-Dec-23
Included in the Prior Art Database: 2008-Dec-23
The publication describes a message expiry system for managing the storage of messages pending delivery within a publish/subscribe messaging environment.
Adaptive load-based expiration of pending messages in a publish /subscribe messaging system
The problem arises in the situation where a producer of information and a consumer of that information are interconnected via a messaging system which de-couples the producer from the consumer by storing pending "messages" for later delivery. In the case where the consumer can request that the messaging system (or "server") keep information destined for it, regardless of whether the consumer is currently able to receive it or not, it is possible for large amounts of information to build up on the server. If the consumer is unable to receive the information, perhaps due to a network or hardware failure, then information can continue to build up on the server. This information can build up at different rates, depending on the load the server is under, and the amount of information to which the consumer has subscribed.
The system works by providing a means for the server to "expire" certain content automatically, based on how damaging it could be to keep the information. The "damage", in this context, would be caused by the server running out of storage for pending messages, and thus being unable to handle any more work. When a consumer "subscribes" to some information, the server can set a time limit on how long that subscription will last. The choice of time limit is not fixed, and can be calculated and adapted, based on a number of factors. Inputs can include (but are not limited to) the number of messages passing through the server (historically, and currently), the number of other consumers who have subscribed to the same information, the size of the messages, and the amount of information to which the consumer has subscribed.
By allowing the server to adapt in this way, it is easier to prevent damage being caused by poorly behaved clients, malicious clients, or ones which may no longer be able to receive the messages originally intended for them. An alternative solution would have to keep all the messages (which could cause damage), expiring content after a fixed time (whi...