Browse Prior Art Database

DWH Version Change without interrupting production

IP.com Disclosure Number: IPCOM000029870D
Original Publication Date: 2004-Jul-15
Included in the Prior Art Database: 2004-Jul-15
Document File: 6 page(s) / 42K

Publishing Venue

IBM

Abstract

A Data Warehouse stores informational data, that is extracted from the operational data and then transformed for end user decision making, controlled by meta data. Operational efficiency and the precision of adjustments to changing conditions are essential for companies in today's increasing competition. Many parameters in the meta data repository, that control the data processing in a data warehouse, are subject to modifications. The invention describes the technique how a meta data repository and the exploitation of this meta data by the warehouse processes must be designed and implemented to enable a seamless migration from Version N to N+1 of the potential version change areas without interrupting processing.

This text was extracted from a PDF file.
At least one non-text object (such as an image or picture) has been suppressed.
This is the abbreviated version, containing approximately 22% of the total text.

Page 1 of 6

DWH Version Change without interrupting production

Main Idea

Introduction

A Data Warehouse stores informational data, that is extracted from the operational data and then transformed for end user decision making . For example, a data warehousing solution might copy all the sales data from the operational database, perform calculations to summarize the data, and write the aggregated data to summary tables. End-users can query the warehouse databases without impacting the operational databases.

    The typical architecture of data warehouse implementations has to following structure:

E x te rn al c o n su m ers (d a ta m a rts)

S taging T ables

Input Layer:

    Monitors the interfaces to the operational systems, collects (Pull/Push) data from operational systems, checks for completeness and provides the input to the "Transform and Load Layer".

Transform and Load Layer:

    This function is responsible for checking, cleansing, transformation and loading of the detailed information into temporary load tables (Staging Tables) or directly into the final target tables of the data warehouse . In the case of staging tables, the data is transferred via SQL commands, providing the possibilities for further transformations, into the target tables .

Database Layer:

This layer represents the staging tables, the target tables, including all

    A ggregation In te rn al E x te rn al

Meta Data Repository

T arget T ables

S taging T ables

O perational System s

T ransform and L oad

T ransform and L oad

Agenda

Input M onitoring

A ggregation L ayer

T ransform an d L oad L ayer

D atab ase L ayer

In pu t layer:

[This page contains 23 pictures or other non-text objects]

Page 2 of 6

reference tables of the warehouse and the summary tables (See aggregation layer).

Aggregation Layer:

    Data from target tables will be analyzed and aggregated. Result sets will be stored in further tables or files, delivered to external consumers . Meta Data Repository:

    Meta Data triggers the whole processing steps in a data warehouse. The major part of information, that is provided in the repository is about the specification of the incoming data, the business rules for transformation and aggregation and the schema of the tables, that are involved for processing .

The Problem to be solved:

    Operational efficiency and the precision of adjustments to changing conditions are essential for companies in today's increasing competition . Many parameters in the meta data repository, that control the data processing in a data warehouse, are subject to modifications.

    During project lifetime the DWH will be typically confronted with - changes in the interface specification in the Input layer Operational systems will change the structure of their data, that is delivered to the data warehouse.

- adaptation of transformation rules in the Transform and Load Layer Transformations might change caused by new information or by extracting more or modified aggregated information.

- adaptation of table schema in the...