Surety is performing system maintenance this weekend. Electronic date stamps on new Prior Art Database disclosures may be delayed.
Browse Prior Art Database

Test coverage management in crowdsourcing process

IP.com Disclosure Number: IPCOM000243187D
Publication Date: 2015-Sep-17
Document File: 3 page(s) / 89K

Publishing Venue

The IP.com Prior Art Database


Quote from Wikipedia: "Crowdsourcing is the process of obtaining needed services, ideas, or content by soliciting contributions from a large group of people, and especially from an online community, rather than from traditional employees or suppliers." The crowdsourcing technique became more popular for online software testing (when our software is available online to the world and we do not have resources to do all test activities on our own). The crowd (for example students from technical universities) are involved in new product tests (awards for top bug hunters). The biggest problem in that technique is crowd management. To be more precise correct test coverage - we do not want several hundred of students to test the same features. The goal is to have the optimal coverage of product features (test paths).

This text was extracted from a PDF file.
This is the abbreviated version, containing approximately 55% of the total text.

Page 01 of 3

Test coverage management in crowdsourcing process

    The idea is to remove/hide GUI elements that has been covered by crowd testing. If all children of a given GUI element has been already covered, that element is removed as well (e.g. if all submenu items has been tested, parent menu item is removed). Thanks to that each member of crowd will see only those parts of product that have not been covered by tests yet.

In addition, the special counters can be added to each gui object so each tester knows how many times this object was executed/visited by others. Thanks to that each member of crowd will see which part of product were not covered by tests and where is the highest probability of finding defects.

1. Presentation of gui objects counters as labels on UI - that can be easily noticed by crowd testers (Figure 1)

    Example: [2] - current gui object was visited 2 times 2. Presentation on UI information about remaining tree nodes - deeper level gui objects (Figure 2)

a) Example:
[12/1] - there is only 1 node in that tree. This node has been visited 12 times [1/5] - there are 5 nodes in that tree. Only 1 has been visited.The current node

was visited once. Remaining nodes were not visited.

or we can use opposite definition - instead of displaying number of visited items display number of not visited (unvisited) items - both approaches may be useful.


[0/1] - there is only 1 node in that tree. There is 0 unvisited nodes in that tree. [4/5] - there are 5 nodes in that t...