Browse Prior Art Database

Meta data driven staging table delta calculation

IP.com Disclosure Number: IPCOM000212634D
Publication Date: 2011-Nov-21
Document File: 5 page(s) / 26K

Publishing Venue

The IP.com Prior Art Database

Abstract

Delta calculation component which is driven by meta data and performs operations on database

This text was extracted from a PDF file.
This is the abbreviated version, containing approximately 27% of the total text.

Page 01 of 5

Meta data driven staging table delta calculation

When data is available from snapshots and one is interested in data changes within certain timeslot, calculating deltas is needed to provide such a data view. The process of delta calculation may be related to different types of data or various volumes of data. Flexible solution for that purpose is needed but in the same time, performance of delta calculations should be considered in order to keep the input data processing as quick as possible and being capable of handling large volumes of data. Disclosed is an idea of component meant for delta calculations. As an input, the component receives data of cumulative nature. As an output, the component fills destination database table with current deltas, calculated from two subsequent baselines. Inside the component, staging table (implemented as temporary tables) is used to store current deltas together with recent baselines. The staging table is generated from the definition of destination table.


1.Problem description


There is a snapshot-like source of data (baselines). The baseline provides three types of metrics:
- identification
- current
- cumulative

All identification metrics together identifies a subject described by a baseline. Subsequent baselines for the subject have the same values of the identification metrics. Current metrics may have different values in baselines for the subject. For instance they may report the current state of the metric or their data type does not

justify delta calculation. Cumulative metrics' values can grow. Subsequent baselines for the subject have equal or greater value of the cumulative metric. There can be multiple types of sources of data. Each type of source provides different kind of baselines. The source of data provides meta data about served metrics.

Baselines from all sources are read periodically (sampling). Baselines are put into Staging Delta Calculator (SDC) component. The SDC is responsible for providing deltas between two subsequent baselines of the same subject and type of source. Output data from the SDC is stored into database tables (history tables). There is a history table for each type of source. The history table contains values calculated between two subsequent baselines. The history table's structure is based on its source, i.e. the table has corresponding columns for identification, current and cumulative metrics. The identification and current columns keep the same value as read from the source. The cumulative columns keep delta values. In addition, timestamp column is present and it keeps timestamp value when record was inserted.

Example (1):


Snapshot-like source gives readings from gas meters (type of source: Gas). A gas meter provides set of metrics:
- identification: SERIAL_NUMBER smallint
- current: LOCATION varchar(256), STATUS varchar(32)
- cumulative: GAS_USED bigint

1


Page 02 of 5

One wants to find out when gas is used most often. In order to provide fine-grained anal...