A Method & System of Image Structure Based Automatic Testing
Publication Date: 2014-Jul-10
The IP.com Prior Art Database
This article introduces a new kind of image based object recognition method for automation testing. This method can solve the issues of cross platform, various resolutions/colors/styles revealed by traditional automation testing methods.
Page 01 of 6
&&& System of Image Structure Based Automatic Testing System of Image Structure Based Automatic Testing
Problem statement and background
Choosing the right toolis still a big problem for test automation, especially for mobile GUI test!
Cross platform, multiple app types, rapid script creation, low cost maintenanceshould be the major factors for consideration.
The application objects are the main building blocks of test automation. They consist of edit boxes, lists, links, text objects, etc. In the mobile space there are mainly two ways to recognize objects, each method has several solutions in the market now:
Meth od Nam e
Rec og niz ed Obj ect s
How they handle object recognition.
OS nati ve obj ect s
Works well on devices with same OS, but different resolution
Virtu al Obje ct Reco gniti on
(Non -Ima ge base
differe nt solutio ns for native object
UI Automation APIs for different mobile platforms:
b. Using uiautomator API -- Access its UI element hierarchy by uiautomatorviewer tool:
Page 02 of 6
<img height="44" id="apthumb3"
src="./image.jpg" width="44" border="0">
and DOM object
develo pment skill neede d
stabilit y of test execut ion
cost and time intensi ve of script creati on cross platfor m Differe
nt DOM tree in variou s
DO M obj ect s
Page 03 of 6
ms results in low reusab ility of scripts
Vis ual Obj ect Rec ogn itio n
(Im age bas ed)
Im age Obj ect
Object identificati on is totally based on image recognitio
However, the images/ic ons are easily affected by different OS/resolu tion/langu age.
Look into the traditional Image based automation test, the problems exposed are quite obvious:
Page 04 of 6
If there are several similarimages on the same screen, manual assistance should be needed for identifying.
If the color or styleof image has changed, object recognition should fail.
Different screen resolutions would make image sizechange, object recognition may fail.
High effortsare needed for script maintenance.
Our solution is aimed to overcome the obstacles above and get more closed...