Browse Prior Art Database

Automatic Construction of Conditional Exponential Models from Elementary Features

IP.com Disclosure Number: IPCOM000112542D
Original Publication Date: 1994-May-01
Included in the Prior Art Database: 2005-Mar-27
Document File: 2 page(s) / 36K

Publishing Venue

IBM

Related People

Brown, PF: AUTHOR [+4]

Abstract

Disclosed is a method for constructing an exponential model of the conditional distribution p(y | x ) given 1) an empirical distribution p tilde (x, y) , which characterizes some set of training data, and 2) a set B of binary features b(x,y)

This text was extracted from an ASCII text file.
This is the abbreviated version, containing approximately 75% of the total text.

Automatic Construction of Conditional Exponential Models from Elementary
Features

      Disclosed is a method for constructing an exponential model of
the conditional distribution p(y | x )  given 1) an empirical
distribution  p tilde (x, y) , which characterizes some set of
training data, and 2) a set  B  of binary features  b(x,y)

  p(y | x ) = Z(x) sup <-1> p sub 0 ( y | x ) exp left lb < sum from
         <f memberof F> of lambda sub f  f ( x, y) > right rb,
where  p sub 0 ( y | x )  is an initial model (for example, the
uniform distribution), F is a set of non-negative feature functions
f(x,y), and lambda sub f is the vote of the feature f, and  Z(x) is
chosen to ensure that  sum from y of p ( y | x ) = 1.  The likelihood
of the training data is defined to be

     L(p tilde, p) = sum from <(x,y)> p tilde (x,y) log p (y | x).

The votes are chosen so as to make the likelihood as large as
possible.

      In the following algorithm for developing the set F
incrementally, A is an auxiliary set of features.

1.  Set A to B and place in F the single constant feature f(x,y)
    identical 1.

2.  For each pair (a,b) memberof A * B , estimate the gain in
    likelihood resulting from the augmentation of F by the features
     f(x,y)=a(x,y)b(x,y) and f prime (x,y)=a(x,y)(1-b(x,y)).

3.  Select those pairs (a,b) for which the gain in likelihood is
    significant.

4.  If no pairs are selected, stop.

5.  For each selected pair (a,b),...