Browse Prior Art Database

A system and method to validate UI Layout with image analysis

IP.com Disclosure Number: IPCOM000237271D
Publication Date: 2014-Jun-11

Publishing Venue

The IP.com Prior Art Database

Abstract

This invention proposes an easy way to validate html-based web page layout automatically. The idea utilizes some existing knowledge to process UI to generate model and validate a target UI page. The process to generate model includes – CSS coloring, image analysis, component recognition and generating XML-based model. A target UI page is also processed to a model, and XML comparison method is used to recognize the invalid UI part of a target page.

This text was extracted from a PDF file.
This is the abbreviated version, containing approximately 26% of the total text.

Page 01 of 13

A system and method to validate UI Layout with image analysis

User interface consistency is important to all software vendors. According to Forrester, 86% of enterprise purchase decision-makers consider it a determining factor in their purchase decision, and 93% of customers consider user interface consistency important. UI layout consistency is a key element in UI consistency. It ensures the UI structure hierarchy looks consistency to customer.

Currently, very few tools can validate the UI layout. There are two patent applications on webpage layout validation area, US20100220923 Automatic Page Layout Validation and US20120166933 Verifying Page Layout Automatically.

US20100220923 Automatic Page Layout Validation focuses on how to validate web pages generated from the same template having the same layout. Web page may have multiple blocks and each block will be filled with a color. Then the colored page or a snapshot of the page can be compared by a computer program. Differences or lack of a color indicate layout errors.

US20120166933 Verifying Page Layout Automatically uses layout rule to validate web page. User needs to provide rules of page layout such as size and position of each widget, relation of widget, etc. After layout rules are provided, the system will extract page layout information of validation target page. The information comes from properties of widget or DOM.

The disclosed invention has great advantage to them. User does need to provide layout rules for each widget manually. Layout information doesn't need to be extracted from page properties. Browser interpretation variation doesn't affect page layout validation.

This invention proposes an easy way to validate html based web page layout automatically. A new HTML-to-XML transferring mechanism is proposed. The HTML is the web pages and the XML is the layout information of UI components.

Compared with DOM, the XML is the simplified one that just persist the layout information --- position and inclusion position & relationship. XML generating process is very specific as below,

HTML -> Colored HTML -> Snapshot Picture -> XML

The scripting language JavaScript is introduced to modify the component CSS to highlight the layout and to hide no-layout information such as characters. Then the screen snapshot is taken to be delivered into image analysis to recognize the component color. Each component would have a pre-defined unique color. Key pixel points would record the size, the absolute/relative position of a component, as well as the relationship apart from others. All those layout information, which is computed by image analysis, will be persisted into the XML.

When the layout information is ready, two steps are proposed to validate web page layout: training and scoring. The training would get the metrics for further layout validation, and the scoring is the validation process.

As far as we could see, the advantages are obviously.

1



Page 02 of 13

1. One of the core cl...