Avoiding interlock and minimising distortion when gathering operational data from multiple threads in a network of processing units

Publishing Venue

IBM

Abstract

In a network of cooperating processing units where there is a need to gather operational data, such as statistics. Gathering points are synchronized with the connectors between the processing units in a network so it is necessary to avoid distortion of the processing flow while gathering and accumulating the operational data. Therefore gathering and collecting points are not interlocked in this design. The solution is to provide an intermediate mechanism referred to here as the Conveyor which provides an asynchronous delivery of gathered operational data from the gathering point to the collecting and reporting point. Thus the relatively lightweight gathering mechanism is decoupled from the more processing intensive accumulation and output of the operational data.

Avoiding interlock and minimising distortion when gathering operational data from multiple threads in a network of processing units

This publication addresses the problem of avoiding interlock between separate instances of a mechanism that gathers operational data from a network of cooperating processing units which transfer computational flow. A collecting mechanism is described which accumulates operational data that is passed to it in gathering containers generated by one or more instances of the gathering mechanism.

    An example of this type of network of cooperating processing units is a message flow transaction in IBM's WebSphere* Message Brokers product. In such a network there is a need to gather operational data ( for example statistics of the progress of a computational flow through the network .) whilst avoiding instrumentation of the processing units. Please refer to Fig 1. below Figure 1:

Gathering Data From Message Flow

message flow transaction

Gatherer

Update

Calls to Gatherer to update Container

nodes

1 2

3

4

5

Container

Container

flow data

thread data

node 1 data

node 2 data

node 3 data

node 4 data

node 5 data

    Gathering points are synchronized with the connectors between the processing units in a network so it is necessary to avoid distortion of the processing flow while gathering and accumulating the operational data. Therefore gathering and collecting points are not interlocked in this design .

    The solution is to provide an intermediate mechanism referred to here as the Conveyor. This is a logical FIFO queue of containers that represent complete sets of operational data gathered during the flow of processing along the path of a network. Each gathering instance hands off a completed gathering container to the Conveyor, which is chained to the end of the Conveyor queue ( please refer to Fig 2.

1

)

    When the Collector is ready to accumulate more data from the gatherin...