InnovationQ and the IP.com Prior Art Database will be updated on Sunday, December 15, from 11am-2pm ET. You may experience brief service interruptions during that time.
Browse Prior Art Database

Method for Creating a Ranking List using Crowd Sourcing Service

IP.com Disclosure Number: IPCOM000237449D
Publication Date: 2014-Jun-18

Publishing Venue

The IP.com Prior Art Database

Related People

Roger Jie Luo Yi Chang ChengXiang Zhai Xiaolong Wang


A method is disclosed for creating a ranking list using crowd sourcing service. The method includes combining pairwise preference of items provided by crowdsourcing participants into a full ranking list. Additionally, a Thurstonian Pairwise Preference (TPP) model is implemented to infer distribution of gold standard ranking lists as parameterized by relevance scores. The TPP model learns domain expertise and truthfulness of crowd sourcing workers as well as query difficulty.