Method for automated log processing

Method for automated log processing

Disclosed is a method for automated log processing. Benefits include improved productivity.

Background

              One of the principal

le

methods of troubleshooting software is printing log entries at specific execution points. The user sees this captured information about the state of the system. A log can be an invaluable tool in the investigating the causes of software problems.

              In the best case, the log indicates a single point of failure flagged with an appropriate key word such as error. However, in a great many cases the results are less obvious. The information needed for diagnosing the problem may not be flagged

and

and may be interspersed with other log messages. The conventional best practice is that an experienced engineer reads the log and analyzes the messages. This process is problematic. As the logs become large with perhaps several million messages, the human processing of the

at muchth

information in a meaningful and reliable way becomes almost impossible.

General Description

              The disclosed method includes a database into which log messages are stored. Log messages

are going

typically fit certain patterns in terms of frequency of appearance, relative order, and the range of data that appears in any type of message. For problem diagnosis

,

the significant information is often that which does not fit the normal patterns. Isolating this information for each message and creating a database provides a new tool for problem resolution.

              For example, a line is defined in code as opening %s with file descriptor %d appears in the log file as opening xyz.txt with file descriptor 7. This message appears at line 2457 in the log. The message type is opening … with file descriptor. The data associated with this instance is xyz.txt, 7, and 2457. Another message is defined as closing file descriptor %d and writing %d bytes to file descriptor %d. A user reports that data is missing in the output file xyz.txt. The log database is searched for the file descriptor associated with the filename, 7. A query finds any occurrence of file descriptor 7 and orders the results by line number (see Figure 1). This tool enables an inexperienced person to easily determine that there was an attempt to write 24 bytes to the file after it was closed. Even selecting and sorting records by file descriptor and line number exposes the pattern without writing complicated scripts.

              The key to creating an effective database is the breakdown of log messages into applicable fields. Because a predeter...