Self adaptive buffered read
Original Publication Date: 2005-Sep-21
Included in the Prior Art Database: 2005-Sep-21
Self Adaptive Buffered Read A program application, in order to perform some elaborations, reads records from a sequential input log (on DASD or on Tape), and it expects them within a certain sequence (all before a closure record). A problem arises if some records are written in the used input log in a-synchronous way, and the arrival within the the single sequence closure record is not honored. Disclosed is a mechanism to address this kind of issue.
Self adaptive buffered read
If the needed record did not arrive on time, two decisions can be taken: to consider the input sequence closed or on the contrary to wait in hold till the reading of the needed record. In the first case some info can be missing to the program application that is elaborating the input sequence, while in the other case there will be an overhead of memory/CPU usage to keep all the pending record sequences in memory (not considering the worst case, when the needed record will never arrive, causing a never ending wait).
An alternative solution could be to read the input log finding the needed record if present, but due to the sequentiality of the access, this solution has as side effect the need to repositioning the read to the next record, after a search is issued, meaning that the input log should be closed and reopened and several read without action issued till the repositioning is complete.
The basic idea is to use an ahead read into a circular buffer, having an external parameter to reduce the amount of search performed, and an internal adaptive tuning mechanism that helps to tune the ahead read and the related circular buffer performances.
The solution that is described hereafter uses an ahead-buffered record read mechanism changing the normal sequential read mechanism used to read a sequential input log (DASD or tape stored).
The basic concept is: a specific record is needed within a record sequence. It is assumed that if it does not arrive within an end-sequence record time frame, it will arrive in a reasonable N-records window after the end-record. This since there are two or more asynchronous processes generating the record sequence and since one can be faster than the other, it can occur that sometimes one record is arriving out of sequence, after the end-sequence record. This "out of sequence" situation may be causing problems to the application programs that are using that record sequence.
Tho avoid this at the beginning N records are read in advance by the application program and stored in a circular buffer. The buffer has N positions/slots (where N has a maximum user selectable limit): N-1 positions to hold the records read in advance and 1 to hold the most current record.
There are initially, at program start, N physical (sequential) reads from the input log without the related "normal" processing of these records.
The program engine will get the next record to process from the buffer entries (buffered-read) and not directly from the physical read from the input log, since the physical record read is N records in advance, compared with the program record processing.
Every time a record...