Browse Prior Art Database

Language Processing Language Lexical Rule Definition

IP.com Disclosure Number: IPCOM000110066D
Original Publication Date: 1992-Oct-01
Included in the Prior Art Database: 2005-Mar-25
Document File: 4 page(s) / 110K

Publishing Venue

IBM

Related People

Hidalgo, DS: AUTHOR

Abstract

Disclosed is a feature of the Language Processing Language (LPL) that supports the abstracted definition of the lexical rules that describe the syntactic units or tokens that are used in the definition of a formal language. This feature allows the LPL user to specify multiple aspects of the processing of each individual syntactic unit built from the input stream, as well as the mechanical implementation of lexical scanners based on those abstract rule definitions.

This text was extracted from an ASCII text file.
This is the abbreviated version, containing approximately 54% of the total text.

Language Processing Language Lexical Rule Definition

       Disclosed is a feature of the Language Processing
Language (LPL) that supports the abstracted definition of the lexical
rules that describe the syntactic units or tokens that are used in
the definition of a formal language.  This feature allows the LPL
user to specify multiple aspects of the processing of each individual
syntactic unit built from the input stream, as well as the mechanical
implementation of lexical scanners based on those abstract rule
definitions.

      Part of the process of designing a language implementation is
the design of the lexical scanner, which is responsible for building
syntactic units or tokens from the input data stream that represents
the source written in the language.  When the lexical scanner reads a
character code from the input stream it must determine how to use it
in order to build a token that can be used by the language parser to
verify grammatical correctness.  In order to make such determination
the lexical scanner must contain a set of lexical rules that describe
how input character codes are used to build tokens.  A lexical rule
is necessary for each type of token accepted by the input language.
Each rule specifies at least an initial state that is required to be
unique among all the rules applied by the scanner at any given time.
An LPL lexical rule may also contain a main body state and a
termination state.  A state may also reference another lexical ru...