Browse Prior Art Database

System Monitoring In A Parallel Database Replication Apply Processing

IP.com Disclosure Number: IPCOM000234802D
Publication Date: 2014-Feb-07
Document File: 5 page(s) / 136K

Publishing Venue

The IP.com Prior Art Database

Abstract

Disclosed is an efficient monitoring method for health and performance monitoring as well as troubleshooting in a parallel replication apply processing. This method allows near-real-time collection of a large amount of data with a negligible performance impact.

This text was extracted from a PDF file.
This is the abbreviated version, containing approximately 25% of the total text.

Page 01 of 5

System Monitoring In A Parallel Database Replication Apply Processing

In a replication system, the apply program processes the transactions from the source database and applies the transactions to the target database. Parallelism is used to increase performance. The apply agents are the working threads for applying transaction changes on the target. Each agent thread can work for either a particular task or multiple different tasks. In the parallel and complex apply processing, efficient monitoring is highly desirable to show the health and performance. Dedicated threads are used for monitoring. The collected statistics include, but are not restricted to the cross-interval cumulative values (e.g., the number of transactions that have been applied after a specific time point.), the interval-bounded cumulative values (e.g., average and maximum end-to-end latency in a particular sampling interval), and the snapshot values (e.g., the current depth of an internal queue). Each statistic indicates an update on one or multiple statistics, which could be correlated with each other. The following three sections explain the details of this highly efficient monitoring solution in a parallel database replication apply processing.

Shared-memory-based Communication from Agents to a Monitor

Figure 1: A shared-memory model from Agents to a Monitor

A shared memory space is used for cross-thread communication in two directions (from agent threads to a monitor thread and from a monitor thread to agent threads). Shared memory supports concurrent access from both monitoring threads and agent threads. In the shared memory space, each agent thread has dedicated memory that is not accessible to other agent threads. This agent thread and the monitor thread have the read privileges. The shared memory space allows numerous agent threads to simultaneously maintain and update statistics and allows the monitor thread to simultaneously read the statistics. The queue-based solution is almost impractical for collecting the statistics whose update volumes are very high. For example, the number of queue messages could be overwhelming when the statistics are updated after each row-level apply.

The disclosed communication solution provides a hybrid mode that combines agent threads pushing and monitor threads polling. The apply agents maintain the data in their own memory spaces. When appropriate, the working agents will update the data

1


Page 02 of 5

in the shared memory space. An example of this is statistics related to a transaction.

Those statistics should be updated in one atomic unit in the shared memory space. The monitor thread polls the data in the shared memory space.

Clock-based Interval Model


The monitor thread periodically reports the apply-related statistics based on a monitoring interval (monitor_interval). The monitor thread also can generate the statistics-based notification events. Possible reporting methods include:


• The statistics/events inserte...