Browse Prior Art Database

Method and system for application logging to a remote log server through HTTP/HTTPS

IP.com Disclosure Number: IPCOM000202020D
Publication Date: 2010-Dec-01
Document File: 8 page(s) / 86K

Publishing Venue

The IP.com Prior Art Database

Abstract

This invention consists of a method and system for application to use HTTP or HTTPS to log to a remote log server. It is lightweight, it has support in every programing language on every OS, from the simplest cell phone to the largest mainframe. It is easy routable with proxy server in case IP connectivity is limited. Most importantly, HTTP is well understood and accepted in every network infrastructure, no one needs to explain about it, unlike other protocols. It can be encrypted with HTTPS for ultra sensitive environment. Compression is supported natively by HTTP protocol, so does Unicode. All these elements make HTTP a much better protocol for logging than syslog. At the server side, a regular web server (e.g. Apache HTTP Server) could easily be configured to act as a remote log server, accepting HTTP/HTTPS connection from any remote client from any platform. Most web server software have a rich and mature set of rules to parse and filter incoming request in order to direct each request to the appropriate handler module. Finally, most web server can write its log to file or database.

This text was extracted from a PDF file.
This is the abbreviated version, containing approximately 45% of the total text.

Page 01 of 8

Method and system for application logging to a remote log server through HTTP/HTTPS

Logging is critical in a production environment. It keeps trail of application/OS/device successful and failure messages. Logging is a main tool for troubleshooting, for auditing what went wrong after the fact, for keeping track of what changed, and much more. Today, logging is done in every means,

with large variance between them

                                                                                         . Each OS has its own way of logging, Unix and Linux are relatively similar, but other OSs, they all log messages differently and incompatible. Applications are even worse, some applications log to file, or to many files, or to database, or in memory only, or not at all. Some people use their own tool for logging, other simply write to a file

without any ordering or buffering

. It is a mess!

For a long time, syslog has been the de facto tool and protocol used in the Unix world for local logging as well as remote logging through UDP. It has worked

. Outside of the Unix world, there is no standard way to log locally, or to

write to a remote log server. There are many logging frameworks supported by various development environment, but programers are free to choose any

framework, or none at all. If you have firewall between datacenter zones that block UDP, then remote syslog is a pain to use. It's not safe to log outside of secure zone because UDP is not encrypted.

Logging to local files has been a common practice because it's so simple and fast. On a mainframe it's not bad since the hardware is not likely going to fail, and the storage is undoubtly kept highly available. However, on a cloud environment where virtual nodes are brought up and down arbitrarily, logging to local file is equivalent to no logging because the virtual disk goes away with the virtual node. In such environment, logging must always be made remotely to a central log server to be stored and analyzed and audited. In a public or private cloud, the virtual nodes might remain in a partially isolated network, an easily routable protocol must be used for remote logging so that we won't have to setup a log server in every network island.

Logging is generally not considered a feature by development, few people spend effort designing the log module, they just use the simplest tool to do the job, and continuously change it over time to meet more and more serviceability requirements. A better approach is to keep the log module at the source side relatively simple, and allow for more sophisticate rules and filtering at the log server side as need arises.

Log rotation/archive/purge on every node for every application is another tedious subject that system administrator faces everyday...

So, as illustrated previously, there are plenty of reasons proving that existing logging methods are inappropriate for a cloud environment where virtual machines are brought up and wiped away frequently by automated workload scheduler. A new logging method is needed that would be si...