Browse Prior Art Database

Output-Independent CLI Functional Test Harness

IP.com Disclosure Number: IPCOM000028720D
Original Publication Date: 2004-May-27
Included in the Prior Art Database: 2004-May-27
Document File: 2 page(s) / 34K

Publishing Venue

IBM

Abstract

Disclosed is a mechanism to validate Command-line interfaces using knowledge of the CLI's message catalogue to test the semantics of the CLI output instead of doing a direct string comparison. CLIs are often tested by executing a CLI command and validating the output matches an expected output file (called the expect file) through a tool to identify differencs in outputs (e.g., expect, diff). In order to obtain a reasonable test coverage, there are often hundreds or even thousands of test cases written with hard-wired expect files for each one.

This text was extracted from a PDF file.
This is the abbreviated version, containing approximately 58% of the total text.

Page 1 of 2

Output-Independent CLI Functional Test Harness

It is common for software products to use a string catalogue. The CLI itself does not store the message string, it uses a message key to look up the string from the catalogue for display. The expect files should be written to use message keys and arguments instead of hardcoded strings. The comparison tool will lookup the correct message (using the key and arguments) as necessary to construct the expected output string for comparison purposes. This resulting string is compared against the CLI output to see if it matches.

I'll describe the mechanism by way of an example. Let's say we have a CLI command "math" that will evaluate mathematical expressions (e.g., 3+4). There are only two outputs that this CLI can return: a divide by zero error message, and a string indicating the answer.

Here's an example of the message catalogue file:

ANSWER = The answer is {0}
DIVIDE_BY_ZERO = Illegal operation. You attempted to divide by
zero.

Let's say we have suite of serveral test cases with their expected outputs:

CLI command to test Expected Output math "1+0" The answer is 1
math "1-1" The answer is 0
math "-1*4" The answer is -4
math "999/3" The answer is 333
math "3.14/0" Illegal operation. You
attempted to divide by zero.

If the message catalogue file changes, then clearly all the expected outputs need to change as well. For example, if you wanted to test that the math CLI worked with a French catalogue, you would need a new set of e...