Dismiss
InnovationQ will be updated on Sunday, Oct. 22, from 10am ET - noon. You may experience brief service interruptions during that time.
Browse Prior Art Database

Methodology and Process for Identifying Data Quality Issues in an Application from the Database Management System

IP.com Disclosure Number: IPCOM000235060D
Publication Date: 2014-Feb-26
Document File: 2 page(s) / 31K

Publishing Venue

The IP.com Prior Art Database

Abstract

A process and methodology to improve a database manager computer program product, by integrating application data quality checks based on temporal and volume comparisons internally into the database manager, providing increased efficiency and throughput over current alternatives

This text was extracted from a PDF file.
This is the abbreviated version, containing approximately 51% of the total text.

Page 01 of 2

Methodology and Process for Identifying Data Quality Issues in an Application from the Database Management System

In a database environment, application code is frequently written to check the quality of the data that is being processed. This can include seeing the amount of data volumes that have changed from one run to another, as significant change, can often forewarn of a data quality error. Another possible sign of data quality error is a table that is not getting updated, where in this instance, the data error is the lack of updates. In this case, the users of the system are not usually aware that a feeder system is not working and their data has not been changed.

This disclosure would notify the user when a particular table is not updated during a pre-subscribed elapsed time. For example, a user wants a notification when a table is not updated for the past 10 days. Another embodiment would be for these checks to be run in an on-demand mode as opposed to a pre-scheduled check. This could be useful for validating the update to the source prior to the running of an application batch schedule.

Systems depend on data being updated by back end processes that sometimes break. When the tables do not get updated, incorrect reporting or processing can occur. Today, to find out when this happens, additional code must be written; this is usually skipped and the problem does not become visible until someone using the data realizes there is a problem.

GLOSSARY

System catalog - A set of tables in a database management system, that contain metadata (data about data) that is used to build and maintain tables in the database

Referential integrity - A function of the relational database management system that validates that relationships that need to be defined between tables exists. For example, If Endicott is a city in NY, as defined in 1 table, another table with NY as a state would have to exist.

The article proposes a technique of adding capabilities at the table level, where the database management system includes functions that help maintain the application level data quality. This would be consistent with functionality in the database where referential integrity is maintained by the database system. This prposed solution would check if needed data updates are happening at the prescribed times, and if so, are the volume of updates that are happening in normal range for the application. These checks will have the possibility of happening on a pre-scheduled or on-demand basis. If problems are identified, the database management system will produce an error that is consistent with other errors t...