Verification Of The Temporal Synchronization Between Like Database Objects Across Two Or More Concurrent, But Discrete, Systems
Publication Date: 2014-Jan-09
The IP.com Prior Art Database
Disclosed is a general methodology of data object verification that introduces verification as a transaction into database change streams. As a transaction, verification is logged to the database log and processed by replication products for the sole purpose of detecting out-of-synchronization conditions at a target database.
Page 01 of 5
Verification Of The Temporal Synchronization Between Like Database Objects Across Two Or More Concurrent ,
But Discrete , ,
For data objects being synchronized by asynchronous replication techniques, embodiments of the disclosed invention verify that the objects are identical. The objects may include Information Management System (IMS) databases, relational database tables, Virtual Storage Access Method (VSAM) files, etc. Ideally, synchronization can be checked using a point in time reference and data can be compared at the byte level to insure databases are 100% equal. However, point in time comparisons are problematic as databases usually have to be online all the time and changes are almost always in flight. Adding to the complexity, replication products are all asynchronous and therefore databases are never totally synchronized due to latency created by asynchronous processing.
A mechanism is herein disclosed to verify, upon user request, that source and target data objects are in-sync both at the initial start of replication and at regular intervals for the purposes of auditing.
Comparing data objects on a record-by-record basis may not be practical. Static comparisons may require that data objects are off-line or in read-only mode. Database management systems must generally be on-line 7x24x365 and cannot be taken off-line for verification purposes, particularly where databases are terabytes in size.
Replication may be asynchronous. During replication, data objects tend to be slightly out-of-sync due to latency. Embodiments of the disclosed invention perform validation
while replication is occurring and minimize the impact of temporary out-of-sync
conditions resulting from latency.
According to embodiments of the disclosed invention, verification is included as a transaction in database change streams. As a transaction, verification is logged to the database log and processed by replication products for the sole purpose of detecting out-of-synchronization conditions at a target database. The present invention incorporates the following techniques:
• Transactional processing. By adding verification as a transaction, verification information can be written to database change logs and database locking can be utilized to insure a singular point-in-time consistency of one or more database records.
• Data hashing. By hashing source data for verification purposes, the impact of verification logging can be minimized and detection of mismatches at the target due to a lost delete can be detected. As a case in point, if all source IMS segments are sent to a target system to verify they exist, extra IMS segments will not be discovered at the target because a delete was missed. (This will be covered in more detail herein below.)
• Database Trolling. The process to verify data trolls databases in order to guarantee that the database is completely in sync over time. The rate and size
Page 02 of 5