Browse Prior Art Database

Method and System of Supporting Asynchronous and Reliable Bulk Processing

IP.com Disclosure Number: IPCOM000185963D
Original Publication Date: 2009-Aug-03
Included in the Prior Art Database: 2009-Aug-03
Document File: 3 page(s) / 65K

Publishing Venue

IBM

Abstract

In this article we describe a general-purpose reliable and scalable bulk uploading service. The service is implemented as a J2EE scheduled task and leverages the features of the J2EE framework.

This text was extracted from a PDF file.
At least one non-text object (such as an image or picture) has been suppressed.
This is the abbreviated version, containing approximately 52% of the total text.

Page 1 of 3

Method and System of Supporting Asynchronous and Reliable Bulk Processing

Disclosed is a general-purpose bulk uploading service. The service is developed under the Java 2 Enterprise EditionTM (J2EETM ) framework. From the functional perspective, the nature of system is similar to the function of well-known UNIX® crontab . The main difference is that this service is developed under J2EE technology. This makes the service becomes more flexible and scalable using service-oriented architecture (SOA) concept. Exposed as web service, this service becomes a platform-independent service. This disclosure provides a common solution for supporting bulk processing service for many bulk processing applications. Another feature of the invention is its reliability, which is achieved by adding failure handling mechanism.

The service supports following two modes. The first mode is called File-based mode. This mode applies to the case where a huge chunk of data is bundled together as a huge file for processing. Each updated bundled file is stored under a pre-defined processing directory. The work manager of J2EE container creates a working thread to process the bundled file. Usually, the bundled file comes from a feed of another business application. The second mode is called directory-based mode. In this mode, we upload received files into the service without bundling. For files belonging to the same category, we upload those files under a same directory, the work manager of WAS J2EE container will create a working thread to process the file. The functionality of the invoked working thread is determined by the category of the files. Therefore it can be uniquely determined by the directory of the files located. In this mode, each file usually consumes very limited computing resources.

There is a limitation of using existing technologies, such as UNIX crontab or JavaTM timer feature
(http://oreilly.com/

processing of large number of files. Solution created with those technologies usually lack extensibility and reusability. The disclosed solution has the extensibility, reusability, and platform-independent features. It is also more scalable as any other J2EE solutions being able to deploy on multiple machines.

The solution (or system) has a configuration file. The configuration file defines the mapping of the application name to the properties of the application in run-time. The properties are: 1) the name of the plug-in used to process the files for this application,
2) the updated directory name in the server side, 3) the working directory to store the file(s) being processed;...