Browse Prior Art Database

Inheritance-Based Bufferless Split Stream Extraction using Oblivious Aggregators

IP.com Disclosure Number: IPCOM000242209D
Publication Date: 2015-Jun-26
Document File: 4 page(s) / 73K

Publishing Venue

The IP.com Prior Art Database

Abstract

Disclosed are a unique methodology and an associated apparatus to achieve large-scale migration while maintaining efficiency, accuracy, speed, and data safety. This is implemented by an innovative concurrent framework, applicable to the domain of large-scale data retrieval.

This text was extracted from a PDF file.
This is the abbreviated version, containing approximately 35% of the total text.

Page 01 of 4

Inheritance-Based Bufferless Split Stream Extraction using Oblivious Aggregators

Ecosystems in which data storage and portability is greatly important have many challenges around large data migrations. For example, migrating to another vendor entails exporting data out and importing into another format. In some cases, especially with the advancement in technology pertaining to transactional memory, it might be a migration with the intent of creating an in-memory model for faster queries and data retrieval. Any complex case has diversity in the number of third party as well as vendor specific tools that assist in the migration effort. A seamless solution is needed. Such a solution must entail the use of an embedded capability in the consuming software that is able to manage a large amount of extractions of pertinent data for its own consumption, while minimizing repeated cycles to the source system for additional information.

Multiple solutions exist in support of data migration. These include multiple format conversions, running large queries for extraction, buffer-based migrations, and coupled producer consumer migration. Each of these approaches has drawbacks including, but not limited to, inefficiency, the need for a large amount of memory overhead, poor performance, low rate of extraction, risk of data loss (with an excessive rate of extraction), inaccuracy, circular dependency, and loss of flexibility.

A system is needed to achieve large-scale migration while maintaining efficiency, accuracy, speed, and data safety.

The novel contribution is a unique methodology and an associated apparatus implemented by an innovative concurrent framework , applicable to the domain of large-scale data retrieval.

The novel machine receives as input a simple request at a high level to extract data. The request comprises:

The identity of the back end data server

The identity of the consuming client

The specific data artifacts of interest for retrieval

This high-level query commandeers multiple builders of the imperative programming language paradigm for initialization and execution, which in turn travel down to a backend external system that serves to store the data requiring extraction. This high-level directive for extraction, at the onset, is analyzed by concurrent agents that orthogonally split the query. These agents can further linearly partition the query. These concurrently produced splinter queries can then perform in parallel in an effort to extract data . The extraction is bufferless, whereby the flexible concurrent framework is provided in which the system leverages modularized functions discoverable at runtime to accommodate variable intake of streamed back data . Clients of the novel machine merely

1


Page 02 of 4

inherit from the modularized components and benefit from the core layer of the machine that uses this oblivious aggregator to serve up the distributed data.

The novelty of the solution is entrenched not only i...