Browse Prior Art Database

A method to manage materialized tables based on the column based store

IP.com Disclosure Number: IPCOM000239428D
Publication Date: 2014-Nov-06
Document File: 4 page(s) / 83K

Publishing Venue

The IP.com Prior Art Database

Abstract

When there are a large of records in a row-based table, it costs time to query to it. Materialized table makes a better query, but not good enough. A column-based materialized table is suggested to improve performance.

This text was extracted from a PDF file.
This is the abbreviated version, containing approximately 57% of the total text.

Page 01 of 4

A method to manage materialized tables based on the column based store

Materialized table is generated to optimize query performance in OLAP system, however, as data grows, it becomes a huge table, which may heavily impact the query optimization effect, besides, it also costs extra storage space.

Typical scenario, Telecom or bank usually has big data volume of transaction records every day, assumed 200,000,000 records per day. Based on these daily records, kinds of application can summarize and analyze customer's behavior, such as monthly records, although considering materialized table for query optimization, the table size is too large.

• Fact table: 200,000,000 records

M

aterialized table:10,000,000 records


The problems of the current materialized table are as below.

Currently, there's only one way to store materialized table, based on row table. Query performance of big materialized table is not good enough.

Consume storage space.

There are below solutions currently to solve the problem, but both with some pain points.

Solution 1: create another table which is column-based. to replace the materialized table.
drawback:
1) application has to explicitly manipulate them,the coupling of app and data increases the complexity of application development
2) additional effort is needed for data moving
Solution 2: Change the table to column-based.
drawback:
1) It will impact the application performance when doing the high stress on CIUD operations.

1



Page 02 of 4

Our...