Browse Prior Art Database

Avoiding backpressure by allowing code changes

IP.com Disclosure Number: IPCOM000248507D
Publication Date: 2016-Dec-09
Document File: 1 page(s) / 21K

Publishing Venue

The IP.com Prior Art Database

Abstract

Described is a methodology to predict, profile, edit, modify, and redeploy a processing element in a distributed environment to help alleviate or avoid congestion.

This text was extracted from a PDF file.
This is the abbreviated version, containing approximately 63% of the total text.

1

Avoiding backpressure by allowing code changes

Streams programming is a distributed computing environment whereby a series of operators and

processing elements constitute an operator graph. An operator graph is typically deployed to

represent a job.

The slowest part of an operator graph is called the congestion point. More typically this is

referred to as congestion. Congestion occurs when a processing element cannot process tuples as

fast as the rest of the graph. The slowest part of the operator graph dictates the overall throughput

and speed of the application. Therefore, a need exists to assist in preventing the operator from

becoming a bottleneck.

Described here is a methodology to predict, profile, edit, modify, and redeploy a

processing element in a distributed environment to help alleviate or avoid congestion.

Furthermore, hints as to how the code should be changed to eliminate the congestion reasons are

provided. This works by having the streams runtime monitor the operator graph looking for

potential points of future congestion. The aim is to avoid congestion altogether by having the

streams monitor predict congestion points. This is accomplished by looking at the current

backlog of tuples, coupled with incoming tuples rates and overall processing time of a tuple

within the operator.

During some threshold, which could be a fixed amount of time or a percentage full of the

backlog, a signal is sent by the streams runtime. Simultaneously, as the streams runtime

prod...