System and Technique for Automating Diverse Tests Using Consistent Input and Output
Original Publication Date: 2004-Feb-12
Included in the Prior Art Database: 2004-Feb-12
When attempting to automate tests across a diverse set of applications, operating systems, and platforms, the input parameters and resultant outputs can be very widely varied in structure, as well as the tests themselves being vastly different. Each team can use their own methods and tools to automate, but this leads to inconsistency in techniques and results across the test organization. Disclosed in this article is a system and technique to derive automated tests having a consistent look and feel across an organization.
System and Technique for Automating Diverse Tests Using Consistent Input and
An outer shell is designed to accept input parameters and publish test results . Within this shell are hooks to allow diverse methods as the underlying test driver. Most tests can be broken down into a shared atomic set of subprocesses : test system setup and initialization; input load creation; input load generation; parameterized test control; data collection; data reduction and analysis; etc. For many of these subprocesses, the basic structure is the same, regardless of the actual product examined, so standard parameterized coded procedures can be defined, with modifiable interfaces for the different products. Other subprocesses may be unique to the product and require more specialized code - but all can be imbedded into a larger control program consisting of shared code modules.
The figure below summarizes and existing (standardized) automated test paradigm that has been developed. All performance test parameters were catalogued, in order to define a standardized test control file structure (for the .ini file below). This control file is used by a tester to define the parameter variations (for i=1 to n, for j=1 to m, etc.) that in turn define a series of automated performance test runs that the tester desires . (The
.ini file, as well as the control program, PERL modules, etc. below, reside on a controller machine, that ideally is separate from the test client and server machines executing the actual performance tests. However, all control programs could reside on one of the test subject machines.)
The parameter values in the .ini file are fed to a control program (TestDrive.pl below, written in PERL) that calls up the appropriate PERL modules (pm files), e.g., TestUtils.pm, Tec.pm, etc. (see below), some of which may be product specific.
The pm files issue the appropriate commands to either a STAF (Software Test Automation Framework) agent or to Rational Test Manager. STAF is a standardized automation framework (see www.staf.org) that can be installed on a wide variety of operating systems to remotely start and stop processes, programs, etc .,...