Method for Creating a Ranking List using Crowd Sourcing Service
Publication Date: 2014-Jun-18
The IP.com Prior Art Database
A method is disclosed for creating a ranking list using crowd sourcing service. The method includes combining pairwise preference of items provided by crowdsourcing participants into a full ranking list. Additionally, a Thurstonian Pairwise Preference (TPP) model is implemented to infer distribution of gold standard ranking lists as parameterized by relevance scores. The TPP model learns domain expertise and truthfulness of crowd sourcing workers as well as query difficulty.