Browse Prior Art Database

Automatic Evaluation of Machine Translation

IP.com Disclosure Number: IPCOM000101460D
Original Publication Date: 1990-Aug-01
Included in the Prior Art Database: 2005-Mar-16
Document File: 1 page(s) / 36K

Publishing Venue

IBM

Related People

Nasukawa, T: AUTHOR

Abstract

Disclosed is a device for automatically evaluating the quality of a machine translation system by checking how much of the system's sentence output is changed in post-editing.

This text was extracted from an ASCII text file.
This is the abbreviated version, containing approximately 80% of the total text.

Automatic Evaluation of Machine Translation

       Disclosed is a device for automatically evaluating the
quality of a machine translation system by checking how much of the
system's sentence output is changed in post-editing.

      At present there is no truly automatic machine translation
system whose output can be used as it stands; to use a machine
translation system, a human editor has to fine-tune the output of the
system.  The quantity of this post-editing may be said to indicate
the quality of the sentence output by the machine translation. On
this assumption, our device observes post-editing operations and
assigns a number of points to the translation by evaluating the
importance and quantity of the operations.

      This evaluation is performed by referring to a weight table,
which assigns weights to each post-editing operation. The usual post-
editing operations are as follows:
      *    Substituting words
      *    Moving words
      *    Inserting words
      *    Deleting words

      For each of these operations, weights should be assigned
appropriately as follows:
      *    When a word is replaced by another word that is a
          different part of speech, the points are lower
          than when it is replaced by a word that is the
          same part of speech.
      *    The substitution of words gives lower points than
          the movement of wo...