METHOD OF DYNAMICALLY ADJUSTING NUMBER OF ASYNCHRONOUS NOTIFICATION HANDLERS BASED ON LOAD
Original Publication Date: 1999-Dec-01
Included in the Prior Art Database: 2003-Jun-18
Disclosed is a method to dynamically adjust, based on workload, the number of threads that are handling asynchronous notifications. This is done through the use of a controller thread (the "notification handler supervisor" thread) that starts new threads as needed, when the already-existing ("permanent") notification-handler threads cannot handle the load. The newly-started threads ("temporary" notification handlers) only run until the workload goes down enough that they are no longer needed, in which case, they kill themselves. In this way, a number of threads that are appropriate to the current workload are running at any one time. To start the process, only the notification handler supervisor is started. This thread starts a number of permanent notification handlers. Each notification handler looks at the current backlog of unhandled notifications whenever it obtains a notification, and if the backlog is above a certain threshold (most likely based on current number of handlers), the handler signals the supervisor that more threads are needed. The supervisor then starts a number of temporary handler threads based on the current number of handlers and the current backlog of unhandled notifications. Each handler thread simply loops waiting for notifications to handle. Permanent handlers wait forever. Temporary handlers, however, wait only a certain time, and if no notification comes in, they increment a counter of the number of consecutive timeouts they have experienced. If that count gets bigger than a value passed in by the supervisor when the temporary threads were started, the temporary thread kills itself. Otherwise, the temporary handler simply loops back and waits for another notification. When the temporary handlers are first started, they must be started with a higher priority than the supervisor, so that they can begin to work immediately. Otherwise, the supervisor could end up spinning, adding more and more and more handlers before any of the handlers could do any work.