Browse Prior Art Database

System for easily updating "known good" artifacts in automated software testing

IP.com Disclosure Number: IPCOM000244484D
Publication Date: 2015-Dec-16
Document File: 1 page(s) / 40K

Publishing Venue

The IP.com Prior Art Database

Abstract

During software development, running automated tests is a common task. When running automated tests a common validation mechanism in the tests is checking against a 'known good' artifact (text file, picture etc). These known good artifacts can become out of step with 'correct' behaviour due to relatively minor changes within the software system under test and this then cause a lot of work updating these known good artifacts. This article describes a mechanism to update these known goods in a more automated way.

This text was extracted from a PDF file.
This is the abbreviated version, containing approximately 62% of the total text.

Page 01 of 1

System for easily updating "known good" artifacts in automated software testing

The system herein operates on information from running automated tests that use 'known good' artifacts. After running a set of tests, the system takes in the list of differences between the 'known good' and actual output for each failing test. With human input it then categorises each of those differences into either:

Acceptable changes in the software under test (which needs part of the 'known good'


1.

artifact to be updated)

Undesirable change in the software under test


2.

    Once these lists have been created, for acceptable differences (ie. category 1 above) the known-good test artifacts can be automatically updated to match the accepted new behaviour.

    The advantage of the system herein is that updating known-good test artifacts manually can be error prone and time consuming. eg. one minor change to the wording of an error message could cause hundreds of tests to fail with the same difference.

    A system which can assist in automatically updating known good artifacts for acceptable differences given relatively limited manual interaction saves time and is less error prone.

    Automatically updating known-good artifacts can be non-trivial depending how the artifacts are stored. For example they could be compressed in archives, or encrypted, and need extra processing steps to be correctly updated. The system herein can include the necessary logic to process updates correctly, further reduci...