Browse Prior Art Database

Extensible and scalable test environment using event based validation Disclosure Number: IPCOM000188314D
Original Publication Date: 2009-Sep-30
Included in the Prior Art Database: 2009-Sep-30
Document File: 3 page(s) / 50K

Publishing Venue



Extensible and scalable test environment using event based validation. Using asynchronous event based test applications to increase system test efficiency.

This text was extracted from a PDF file.
This is the abbreviated version, containing approximately 32% of the total text.

Page 1 of 3

Extensible and scalable test environment using event based validation

System Testing requires that testing new functionality within a product is tested in a customer like manner. To achieve this, the following requirements need to be considered:-

    Throughput - One of the primary concerns of any customer is how many transactions per second can be run through a system. System Test needs to replicate both the extreme high, low and fluctuating throughputs.

    Configuration - Each customer structures their environment differently, from how many tiers there are, to how a workload flows within the environment. Also, a configuration can change as a workload is running. System Test will need to choose and adapt a varied selection of different configurations during testing.

    Workload - There are various ways of initiating a transaction, from traditional 3270 to the latest SOA protocols. Once initiated there are also different ways of managing the distribution and balancing of that work within their enterprise. Workloads will need to exercise the differing sets of functionality that can be found within the product, being both new and old, in these different ways. System Testing requires workloads that can be sourced from various points and exercise multiple functions concurrently.

    Scale - as with Configuration, customer environments differ in size. System Test needs to be able to model these environments; from the small (10s of servers) to the very large (1000s of servers).

    Integration Testing - System Test has traditionally tested new functionality in isolation of other new functionality. Customers, however, will use the majority of functionality within a product at the same time.

    Unfortunately, with the way System Test traditionally works, to solve one of the above requirements causes other requirement(s) to be unsolvable, for the following reasons:-

    Throughput - To be able to run a high throughput workload the test metrics within the test code has to be limited to lightweight questions, for example, did I crash, did I move from server A to server B. This will allow the test transaction to complete in a very quick time. But to enable lightweight test metrics, the Configuration, Workload and Scale needs to be static. Any change to the Configuration, for example, will require the simple test question to be modified.

    Configuration - To enable System Test to run on various configurations, the test metrics within the test code needs to be able to adapt to the new configuration without having to be rewritten each time there is a change. Therefore, the test metrics will need to become aware of their surroundings so that they can prove that the transaction worked correctly. If the test metrics become aware of their environment then the environment can be altered as the workload is running. The increasing complexity of the test metrics means that Throughput will decrease.

    Workload - As a test transaction can be i...