Browse Prior Art Database

Method for Maximizing Log Content Value While Minimizing Log Size through Dynamic Log-Level Boosting

IP.com Disclosure Number: IPCOM000131920D
Publication Date: 2005-Nov-21
Document File: 2 page(s) / 29K

Publishing Venue

The IP.com Prior Art Database

Abstract

The idea of logging activity in a system is crucial to the identification and diagnosis of problems (bugs). In most large systems (e.g., device software, e-mail servers servers, the network redirection servers), log items have an associated level (e.g., Critical, Serious, Moderate, Low and Informational, or simply 1, 2, 3, 4, etc.). In the ideal case, the logs would contain all the possible log items. However, that would consume an unacceptable amount of storage space. Consider that it doesn't take much to overflow the capacity of a portable device, and that even though server systems have lots of storage space, log events accumulate much more rapidly than on a portable device. The most frequently encountered solution to the storage problem is to limit the "log level" to only record the most important events. This reduction usually doesn't compromise the ability to identify that a problem occurred. However, without the less important events, diagnosis of the problem (bug) is much more difficult, and often impossible. The main idea is to "boost" the effective log level whenever a higher event happens. To do this, the logging system has to record all the lower events in a circular queue without writing them out to the log itself. Consider that in some environments, log events are not necessarily the same size, so the circular queue can be limited based on size (in bytes) or on the number of log events. It can also be limited by elapsed time or other system specific factors. For advanced applications, it can even be a combination of multiple limiting factors. Whenever a higher-level event happens, the circular queue of the lower-level events would be dumped out first, followed by the higher-level event, optionally followed by subsequent low-level events, for an unlimited duration, or limited by factors similar to (but not necessarily the same as) the ones limiting the aforementioned circular queue. This causes the effective log level to be dynamically "boosted" around the higher-level event. The mechanism described in the previous paragraph maintains chronological order of the log events, which is desirable but not required. In cases where the logging system may have limited time to record an event, it may be preferable to output the higher-level event first to minimize the chance of losing it, followed by the lower-level events which precede it chronologically. In this case, it may be useful to delineate where they end and the chronological order is resumed. A post-processing tool can also fix up the chronological order in such a log by sorting the event timestamps or scanning for the aforementioned delineators. The system may also employ a mixture of the two approaches, for example, using the chronological order on "medium" importance events, and the non-chronological approach on the highest-level events. Note, it's possible for another higher-level event to occur before the log level returns to its original value. In that case, the "boost" may be extended for the full duration past the new higher-level event.

This text was extracted from a Microsoft Word document.
This is the abbreviated version, containing approximately 53% of the total text.

Dynamic Log-Level Boost

Method for Maximizing Log Content Value While Minimizing Log Size through Dynamic Log-Level Boosting

Disclosed Anonymously

The idea of logging activity in a system is crucial to the identification and diagnosis of problems (bugs).

In most large systems (e.g., device software, e-mail servers servers, the network redirection servers), log items have an associated level (e.g., Critical, Serious, Moderate, Low and Informational, or simply 1, 2, 3, 4, etc.).

In the ideal case, the logs would contain all the possible log items. However, that would consume an unacceptable amount of storage space. Consider that it doesn't take much to overflow the capacity of a portable device, and that even though server systems have lots of storage space, log events accumulate much more rapidly than on a portable device.

The most frequently encountered solution to the storage problem is to limit the "log level" to only record the most important events.

This reduction usually doesn't compromise the ability to identify that a problem occurred. However, without the less important events, diagnosis of the problem (bug) is much more difficult, and often impossible.

The main idea is to "boost" the effective log level whenever a higher event happens.

To do this, the logging system has to record all the lower events in a circular queue without writing them out to the log itself.

Consider that in some environments, log events are not necessarily the same size, so the circular queue can be limited based on size (in bytes) or on the number of log events. It can also be limited by elapsed time or other system specific factors. For advanced applications, it can even b...