Browse Prior Art Database

Interpolating Images to Higher Resolution using a Feedforward 'Nneural' Network

IP.com Disclosure Number: IPCOM000108583D
Original Publication Date: 1992-Jun-01
Included in the Prior Art Database: 2005-Mar-22
Document File: 3 page(s) / 164K

Publishing Venue

IBM

Related People

Linsker, R: AUTHOR [+2]

Abstract

Disclosed is a method for using an artificial "neural" network and learning algorithm to improve the apparent resolution of images. Given a set of pixel values on a grid, a nonlinear interpolation method is used to compute pixel values on a higher-resolution grid, in such a manner that discontinuities (in luminance, motion, color, or other domains) in the original image are detected and incorporated as discontinuities in the higher-resolution image. The apparent resolution of the final image is thereby enhanced even though information at the higher resolution is not actually available in the original image.

This text was extracted from an ASCII text file.
This is the abbreviated version, containing approximately 40% of the total text.

Interpolating Images to Higher Resolution using a Feedforward 'Nneural' Network

       Disclosed is a method for using an artificial "neural"
network and learning algorithm to improve the apparent resolution of
images.  Given a set of pixel values on a grid, a nonlinear
interpolation method is used to compute pixel values on a
higher-resolution grid, in such a manner that discontinuities (in
luminance, motion, color, or other domains) in the original image are
detected and incorporated as discontinuities in the higher-resolution
image.  The apparent resolution of the final image is thereby
enhanced even though information at the higher resolution is not
actually available in the original image.

      One can trivially generate pixel values at intermediate
locations by linear interpolation of the surrounding pixel elements.
However, additional information, not captured by linear
interpolation, is available at a number of scales in the image.  For
example, orientation and continuity of lines tend to be locally
preserved; a collection of features that are moving coherently tend
to continue to do so.  This invention uses such information.  The
source of information, however, is not explicitly declared.  A
feedforward "neural" network is used to extract the salient
characteristics.

      The network is trained on a set of examples.  For the case of
spatial interpolation, each example input consists of a windowed
region of an image at one spatial resolution (say, grid spacing g),
and the corresponding desired output is the pixel value at a central
position of the window (lying on a higher-resolution grid, e.g., of
spacing g/2). The network is trained to generate output values that
are as close as possible to the desired output values.  During this
process, the elements of the network develop to perform a set of
nonlinear filtering operations.  These operations are then applied to
novel input (not part of the training set) at grid spacing g', to
yield a set of pixel values on a desired higher-resolution grid
(e.g., at spacing g'/2).  The training examples may be obtained in
either of two ways.  If highresolution images (at grid spacing g'/2)
are available for training, then a preferred embodiment uses g'=g.
That is, a degraded version of the original image is used to
reconstruct the original image.  If the available resolution for
training (spacing=g/2) is lower than the resolution of the desired
output images during operation (spacing=g'/2), then a preferred
embodiment uses a degraded (spacing=g) version of the original
(spacing=g/2) to reconstruct the original, and the resulting network
is then applied to images at spacing=g' to generate output images at
spacing=g'/2.  This latter case applies when the only available
training images are those that arrive at a receiver at a resolution
corresponding to grid spacing (g/2)=g'.

      The network consists of N (typically, at least 3) layers of
nodes (artificial "neurons...