Parsing or Tokenizing Tables Using Finite State Machines to Direct Program Actions As Well As Program Control Flow
Original Publication Date: 1989-Jan-01
Included in the Prior Art Database: 2005-Jan-27
This algorithm is used to implement a tokenizing routine and a parsing routine. A tokenizing routine is one that scans a line of input (usually text or commands) and breaks it up into generic words. For example, it may know what numbers are, what special characters are, etc. A parsing routine is one that interprets the words a tokenizing routine returns in a given context to extract meaning and derive a complete command. It will normally return this information to the caller who will execute the command requested. This design is based upon the finite state machine concept, but extends it slightly by using two tables together to describe the state machine flow in one table as well as the specific parsing/tokenizing actions to be taken with the data in the other table.