InnovationQ will be updated on Sunday, Oct. 22, from 10am ET - noon. You may experience brief service interruptions during that time.
Browse Prior Art Database

Single Log reader for processing database logs in different formats

IP.com Disclosure Number: IPCOM000238012D
Publication Date: 2014-Jul-25
Document File: 6 page(s) / 141K

Publishing Venue

The IP.com Prior Art Database


A data replication product cannot read and parse database archive logs generated in different formats (OS, Hardware). As one installation of the replication software can only process database logs of that platform, this paper describes a solution where single installation of replication product can read and parse database logs from different operating systems and hardwares.

This text was extracted from a PDF file.
This is the abbreviated version, containing approximately 51% of the total text.

Page 01 of 6

Single Log reader for processing database logs in different formats
A traditional replication software is either co-located with database or processes database logs available remotely from the same database. But in case of business organization that has offices distributed across different regions globally, each office has different standard with respect to hardware and software for their OLTP operations. For example the US office uses AIX, the UK office uses Linux and the India office uses HP Unix for their transactional systems. At the end of the day each office makes the database logs available to a central location where replication software is installed.

Now the role of replication product is to scrape the logs coming from different offices and replicate to the end system like data warehouse for reporting and analytics.

The database logs generated in different platforms will have different formats. As the replication software has only one log reader supporting a specific log format, it cannot read all these logs available in central location making it handicapped in broader way.

The following is an architecture where different source engines (log readers) are required to process database logs in different formats. Even though different logs are centrally available, we still require different source engines due to the inability of log reader processing logs in different formats. To provide the capabilities to log reader to understand and read the logs in different formats is the main problem. The solution is explained in the below sections.

The solution is to have the log reader identify the logs generated in different formats by parsing the first two log blocks and then determine the values of header fields Block Size, Endianess, Database ID. These header fields are different for logs generated from different platforms. The Database ID is primarily used for table structures that could be same across different database versions. This overall methodology is an


Page 02 of 6

improvement to the replication software when processing database logs in different formats is required. The below described example is specific to Oracle but can be extended to other databases like DB2.

The following is the architecture of single log reader processing database logs in different formats.

The log reader will be enhanced to first...