System method and apparatus to use end customer experience to maintain (scale up + block incoming on max system capacity) best fit capacity of logging and monitoring services
Publication Date: 2016-Oct-07
The IP.com Prior Art Database
Disclosed is a system method and apparatus to use end customer experience to maintain (scale up + block incoming on max system capacity) best fit capacity of logging and monitoring services
Page 01 of 5
System method and apparatus to use end customer experience to maintain (scale up + block incoming on max system capacity ) best fit capacity of logging and monitoring services
Usage of logging and monitoring services are essential part of current software applications and cloud deployed workloads.
When an application sends log or metrics that needs to be finally available in a dashboard to logging and monitoring service- they expect
a) Less delay in getting it in a customer viewable dashboard
b) Almost no loss of messages and logs that has been send to the service
From a provider perspective - they have to balance above expectations of all customers .
Following are the problems we have identified in this context
Current logging and monitoring services designs do not consider the actual end user performance degradation while capacity management and auto scaling of logging and monitoring services
There is also no clear solution on how and when the incoming data should be blocked or limited (Automatically) to save monitoring and logging data loss when the logging and monitoring service is not able to scale up and respond to unexpected increase in the incoming data rate, if such a situation arises.
There is proposed
a) a mechanism by which Lag experienced by the end customer is used for auto scaling of logging and monitoring services
b) a mechanism by which Lag and resource limitations is used as a means of deriving the need for capping of incoming data to avoid loss of data.
Components of a typical logging and monitoring service
A typical logging and monitoring service will have
Page 02 of 5
1) Incoming channels - through which applications can send their logs and monitoring messages
2) Queues or brokers - these are the asynchronous queues that detaches incoming channels and actual consumers
3) Consumers or outgoing channels- Based on the capacity of the logging and monitoring services there can be N number of consumers that can consume messages and logs from the queue and put in Dashboard
4) Dashboard -> this is the web(usually http) dashboard or UI where customers of the applications can come and search for the logs and messages
Test client to measure customer experienced LAG
The test client proposed in this disclosure does following
1) it sends test logs and test metrics with timestamps to one of the incoming channels
2) It then monitors the Dashboard and searches and until it sees the test logs and messages in the dashboard with same timestamp.
By doing this - Test client is able to measure the LAG or time delay between the data is send to the system by an application the customer who configured logging and monitoring is able to watch the data in the dashboard.
For example if a message is send at 8:am and Test client able to detect the message in Dashboard at 8:15 am then the LAG in minutes is 15 and LAG in seconds is 15 * 60
Page 03 of 5
Method to use LAG as input to logging and monitoring capac...