Dismiss
InnovationQ will be updated on Sunday, Oct. 22, from 10am ET - noon. You may experience brief service interruptions during that time.
Browse Prior Art Database

Over Time timeline correlation of multiple stream operator?s performance across cluster nodes

IP.com Disclosure Number: IPCOM000249493D
Publication Date: 2017-Mar-01
Document File: 6 page(s) / 303K

Publishing Venue

The IP.com Prior Art Database

Abstract

Using IBM streams for over time operator monitoring an co-rellation

This text was extracted from a PDF file.
This is the abbreviated version, containing approximately 33% of the total text.

1

Over Time timeline correlation of multiple stream operator’s performance across cluster nodes

IBMstreamsisIBM’spremiumreal-timeanalyticsplatform.IBMstreamsinstallationsare typicallyrunningonmultiplenodeclusters,thebasicbuildingblockofanIBMStreams applicationisasoftwareconstructcalledan“operator”,sinceanyIBMStreamsinstallationis basicallyaclosedsystemeachoperatorduringitsrunningtimemightormightnotaffectother  operatorsrunningonthesamesystem. Becauseofthisreasonitisveryhardtoco-relatetheeffectofoneoperatoronotheroperators especiallyinenvironmentwheretherearelotsofoperatorsdeployedononeclusterusingsystem toolswhichreflectsthecurrentstateoftheclustersmachinesinrespecttoCPU/Memory/Disk/IO etc..Itishardbutpossibletoreflectthecurrentstate. Thecurrentmethodologyandtoolingislackingawaytocreatehistoricalcorrelationandcross  effectsbetweenoperators.thereforeitisimpossibletoanswer"simple"questionslike“what happenedtwodaysago?"or"didImesssomethingupinthelatestrelease?" OurMethodenablesthecorrelationofmultipleoperatorsbasedonsinglehistoricaltimeline ,for examplewecanseetheeffectofoneoperatoronotheroperatorsduringthelast 6months.Or eventracktheperformanceofasingleoperatorbetweendifferentapplicationreleasesthroughthe  years,basicallyansweringthequestion“DidIdosomethingwhichhasbrokensomething ?”ata specificversion. Streamprocessingisacomputerprogrammingparadigm,equivalentto"dataflow"programming, eventstreamprocessing,andreactiveprogrammingthatallowssomeapplicationstomoreeasily exploitalimitedformofparallelprocessing.Suchapplicationscanusemultiplecomputational  units,suchastheFPUsonaGPUorfieldprogrammablegatearrayswithoutexplicitly  managingallocation,synchronization,orcommunicationamongthoseunits. Thestreamprocessingparadigmsimplifiesparallelsoftwareandhardwarebyrestrictingthe  parallelcomputationthatcanbeperformed.Givenasequenceofdata(astream),aseriesof operations(kernelfunctions)isappliedtoeachelementinthestream.Uniformstreaming,where onekernelfunctionisappliedtoallelementsinthestream,istypical.Kernelfunctionsareusually pipelined,andlocalon-chipmemoryisreusedtominimizeexternalmemorybandwidth.Since thekernelandstreamabstractionsexposedatadependencies ,compilertoolscanfullyautomate andoptimizeon-chipmanagementtasks.Streamprocessinghardwarecanusescoreboarding,for example,tolaunchDMAsatruntime,whendependenciesbecomeknown.Theeliminationof manualDMAmanagementreducessoftwarecomplexity,andtheeliminationofhardwarecaches reducestheamountoftheareanotdedicatedtocomputationalunitssuchasALUs . ThebasicbuildingblockofIBMStreamsapplicationisasof...