Browse Prior Art Database

Validation System for Determining when Ratings Do Not Match Supporting Text. Disclosure Number: IPCOM000245375D
Publication Date: 2016-Mar-04
Document File: 3 page(s) / 86K

Publishing Venue

The Prior Art Database


Disclosed is a system to analyze the textual content entered by the user of an online rating system, and then compare the identified sentiment to the selected symbolic or numerical rating to determine whether the two are congruous. If the text and selected rating are not congruous, then the system prompts the user to make another rating selection that better matches the meaning of the text entry.

This text was extracted from a PDF file.
This is the abbreviated version, containing approximately 51% of the total text.

Page 01 of 3

Validation System for Determining when Ratings Do Not Match Supporting Text.

Often users of a website are asked to give a rating for either a used service or purchased product. Many times, the rating system requests both textual feedback and a symbolic (i.e. "star") or numeric rating. Because these rating systems differ among hosting sites, users might misunderstand the rating system and choose a numerical or symbolic rating value that is the opposite of the intended selection. For example, a rating of "1" might be a high rating on one site and a low rating on another. The user can easily enter the unintended rating. Reviewers can discover the mistake if the user also provides accompanying text and the review reads it; however, averaging systems typically only consider the scores and not the text provided.

A method is needed to solve the problem of the user misunderstanding the rating

system and choosing a rating that has the opposite value from the intended meaning.

The novel solution is a system to determine the sentiment of the textual description, compare it with the associated rating, and thus identify whether the two are contradictory and require amendment.

The system applies content analytics and natural language processing (NLP) to obtain either a positive or a negative score based on a customer's supplied description. The system is trained by comparing text descriptions with previously validated rating scores. The system runs the validated scores against new content and compares a predicted rating based on the text description with the actual associated rating. If the actual rating is significantly different from the predicted rating, based on the content analyzed, then the system alerts the user and provides a prompt (e.g., a scale of high and low ratings) for the user to modify the numeric or symbolic rating to align with the meaning of the entered text.

This system easily incorporates into customer review/rating subsystems and requires no additional effort on the part of the customer using the review system.

For implementation, the system uses content analytics and NLP techniques within an Unstructured Information Management Architecture (UIMA) framework as the base for creating item and service profiles as well as sentiment annotation for product/service review ratings. The system is first trained by taking validated data consisting of ratings and text descriptions. The analytics engine then finds the most common key phrases,

word choices, length, and sentiment that appear for each rating value rating based on text descriptions alone. Because ratings are scale based, the presence of words of a range could be used to fill in gaps when tuning the analytic rating engine.

The system creates a rating prediction for each new text description entered into the system. The system then uses this predicted rating to creat...