A system and method of calculating the result sets of application variables using auto generated input data in Gray-box testing
Original Publication Date: 2010-Jan-12
Included in the Prior Art Database: 2010-Jan-12
In the grey-box test of some applications, we can see there are various variables with type like Boolean, Enumeration, List etc, which are designed to use different values to indicate corresponding status during the transaction flow. E.g. In a ticketing system, the status of a ticket could be: Submit, Assigned, Open, Resolved, Closed. And the variable represent this ticket status in the source code of the ticketing system is A. Relative DB column which represent ticket status is column M in table N. There will be some possible ways which can make the ticket status changed within the above value set, certainly some more variables will be needed, and most of them sources from the value from DB tables and configuration like session and configuration files. Some of them come from a random generation method without any data source. It is very difficult from reviewing the source code to detect and cover all the flows. Manually testing will also be very difficult to cover all the flows because we need to plan and prepare enough value sets which will be used in the functions changing ticket status. So ideally, testers want to verify all the status transition, but through manual code review or studying the function requirements to create test cases, some significant status transition flow may be ignored and consequently some defects may be overlooked. For manual code reviewing process: since the transition flow will be made even cross many source code files so it is very hard to detect all the flows from reviewing the code. For manually testing of the program: since the variables worked as input parameters will have too many value sets and there will be some relationship for the variables built in the code, so it is hard to list all the possible value set of all the variables and test the program using the compact parameter set.
A system and method of calculating the result sets of application variables using auto generated input data in Gray-
Solution introduced in this disclosure will providea way to directly generate a variable transition flow chart or a data table. The chart of the table is come up with the following items:
1. Target variable value sets which could be generated from the program.
2. Combination of each source variable values and relative target variable value as a result.
3. If values for the target variable have path to transit within the set. All possible flows including the source variable value sets which make the transition will be listed.
This result chart of data table may be easy to identify whether there is some confliction against the requirement with all paths of code covered and all input data prepared for the testing.
Here is a flow chart indicate how to generate this result diagram:
The following is the main working flow for the invention on how to get the result described above.
1. We need to find out the target variable which wewant to test. It maybe one or more variables because different developers may use different variable names in their code.
2. Identify whether this variable will be used to update DB data or configuration file.
Above 2 items need developer or tester's manual effort while the following is executed automatically by script or programs.
3. Source code is analyzed to find out the source variables which may have effect on the generation ofthe target variable. Store all the medium variableand relative class functions or java scripts detected.
Values of each of the variables should be grant by an expression, while this expression should be
1. A function (or some calculation of the function)whose definition could be found in the source codes
2. A function (or some calculation of the function)whose definition could not be found in the source codes (A lib function)
3. An expression to generate a value through some calculations which we called source variable
4. Variable is granted in a sql expression which wecalled source variable
5. Variable is directly got from user input in JSP pages or directly from database tables which we called source variable
System will remember the relative variable which wecalled medium variables and the functions. If it is got from database, then which column in which table will be co-related.
For the functions not defined in the source code, whether there are some parameters in the function will be judged and then stored the parameters as medium variable. And the lib function as the function.
For the functions defined in the source code, the parameters will be stored as medium variable and thevariables defined in the function will also be stored as the medium v...