Browse Prior Art Database

Method and System for Facilitating Self-Tuning of Serviceability Canaries within Streaming Data Environments

IP.com Disclosure Number: IPCOM000252456D
Publication Date: 2018-Jan-12
Document File: 2 page(s) / 72K

Publishing Venue

The IP.com Prior Art Database

This text was extracted from a PDF file.
This is the abbreviated version, containing approximately 51% of the total text.

1

Method and System for Facilitating Self-Tuning of Serviceability Canaries within Streaming Data Environments

Abstract

A method and system is disclosed for facilitating self-tuning of serviceability canaries within streaming data environments.

Description

Streams is an application platform that deploys and manages user written (distributed system) applications. As part of being a platform, without owning the underlying hardware or software stack, it is often concocting a variety of templates for all the various configuration options. In an environment where bespoke deployments and applications are the norm, even template configurations are an issue. Problem determination is a constant struggle with the shifting environment. Canaries still suffer the problem of not adapting well to custom environments, and self-tuning the parameters improves an attempt to better gauge problems in an irregular environment.

Disclosed is a method and system for facilitating self-tuning of serviceability canaries within streaming data environments. The method and system creates a self-tuning application framework which also includes canaries that are aware of historical events and use a feedback loop to limit false positives and infer better defaults for values moving forward. Given the dynamic and complex nature of stream computing application, a new mechanism is implemented that is based on three categories of metrics learned over time, which auto-tunes the stream computing environment. The metrics that are learned and categorized over time can be such as operating system metrics comprising ulimit, load average, memory consumption and socket exhaustion rate. Application metrics comprising ingest tuple rate, interim tuple rate, checkpoint interval & load, memory profile, base CPU load, per-tuple CPU load, and tuple latency. Streams management metrics comprising resource change rate, user operation rate, memory profile, CPU profile, co-located program impact, failover rates, request latency, and JVM metrics.

In...