Browse Prior Art Database

Meaningful measurements of runbook execution effort Disclosure Number: IPCOM000246833D
Publication Date: 2016-Jul-05
Document File: 3 page(s) / 41K

Publishing Venue

The Prior Art Database


IT operations support centers do have a need to optimize their operational workflows. On metric for measuring efficiency of an operational workflow is through execution time of such a workflow, i.e. how much time does operational staff have to spend on such a workflow. Any time saved for operational staff usually directly related to cost savings for a support center.  However, measuring that time effectively can be challenging for various reasons. This article describes how to address such challenges, and proposes how a meaningful time measurement method for runbooks could be implemented.

This text was extracted from a PDF file.
This is the abbreviated version, containing approximately 52% of the total text.

Page 01 of 3

Meaningful measurements of runbook execution effort

Disclosed is a method for calculating the effective execution time of a runbook. In an IT operations support center it is crucial that incoming events and tickets are responded quickly and accurately. The same resolution procedure may need to be executed multiple times if it resolves a repeating event. Traditionally procedures

were stored as unstructured tribal knowledge (in a form of wiki, pdf, word files, or even paper documents). There are also specialized software solutions for managing runbooks and runbook automation.

In general it is possible to distinguish between three types of runbooks. A manual runbook is just a described procedure, which user has to follow step by step. In semi-automated runbooks some of the steps are replaced by automation (typically: a script) that operator invokes with just one click of a button inside the runbook. There are also fully automated runbooks, which do not require any attention from the operator. The transition from manual runbooks to semi-automated and then into fully automated runbooks requires effort from a Subject Matter Expert (SME) who prepares automations, but in the long term it can reduce cost of IT operations, because automated runbooks do not involve operators time and are executed precisely.

Therefore it is important to identify which runbooks have high cost of effort for operators and which are therefore worth the effort of SMEs to have them automated or at least simplified. The calculation of pure execution time (open web browser) is too naive. User can pause a runbook to dismiss it from her/his work area without cancelling or finishing the execution. In a typical use case, the operator would pause a runbook when a long running command on external system was executed as a result of runbook execution. Operator can pause the runbook execution to work on other runbook at that time. User could also open multiple browser tabs and have multiple runbooks running at the same time.

The idea is to improve estimation of operators' effort for running a runbook by excluding time when operator explicitly paused runbook and to account only fraction of wall clock time, depending of how many other runbooks the operator is executing at the moment. During runbook execution the time when runbook execution is started, completed, paused or resumed are recorded. Start and finish of automation execution is also recorded.

On finish of every runbook execution, the following actions have to be done:

Gather a list of timestamps of significant events during runbook execution, such



beginning and finish of the runbook execution


pause and resume time


start and finish of r...