Browse Prior Art Database

A Method & System of Image Structure Based Automatic Testing

IP.com Disclosure Number: IPCOM000237775D
Publication Date: 2014-Jul-10
Document File: 6 page(s) / 159K

Publishing Venue

The IP.com Prior Art Database

Abstract

This article introduces a new kind of image based object recognition method for automation testing. This method can solve the issues of cross platform, various resolutions/colors/styles revealed by traditional automation testing methods.

This text was extracted from a PDF file.
This is the abbreviated version, containing approximately 55% of the total text.

Page 01 of 6

&&& System of Image Structure Based Automatic Testing System of Image Structure Based Automatic Testing

Problem statement and background
Choosing the right toolis still a big problem for test automation, especially for mobile GUI test!

Cross platform, multiple app types, rapid script creation, low cost maintenanceshould be the major factors for consideration.

The application objects are the main building blocks of test automation. They consist of edit boxes, lists, links, text objects, etc. In the mobile space there are mainly two ways to recognize objects, each method has several solutions in the market now:

Meth od Nam e

A Method

A Method

Rec og niz ed Obj ect s

How they handle object recognition.

Limitation

OS nati ve obj ect s

Works well on devices with same OS, but different resolution
s.

Two

Virtu al Obje ct Reco gniti on

(Non -Ima ge base
d)


1.

differe nt solutio ns for native object

UI Automation APIs for different mobile platforms:

a. Using javascript interface -- Access its UI element hierarchy by "UIATarget.localTarget().logElementTree();"


b. Using uiautomator API -- Access its UI element hierarchy by uiautomatorviewer tool:

1


Page 02 of 6

<img height="44" id="apthumb3"

src="./image.jpg" width="44" border="0">

and DOM object

High

2.

develo pment skill neede d

Low

3.

stabilit y of test execut ion

High

4.

cost and time intensi ve of script creati on cross platfor m Differe

5.

nt DOM tree in variou s

DO M obj ect s

2


Page 03 of 6

platfor

ms results in low reusab ility of scripts

Vis ual Obj ect Rec ogn itio n

(Im age bas ed)

Im age Obj ect

 Object identificati on is totally based on image recognitio
n.
However, the images/ic ons are easily affected by different OS/resolu tion/langu age.

Look into the traditional Image based automation test, the problems exposed are quite obvious:

3


Page 04 of 6

If there are several similarimages on the same screen, manual assistance should be needed for identifying.

If the color or styleof image has changed, object recognition should fail.

Different screen resolutions would make image sizechange, object recognition may fail.

High effortsare needed for script maintenance.

Our solution is aimed to overcome the obstacles above and get more closed...