Browse Prior Art Database

Performance regression measurement using best round trip approach

IP.com Disclosure Number: IPCOM000201844D
Publication Date: 2010-Nov-26
Document File: 2 page(s) / 60K

Publishing Venue

The IP.com Prior Art Database

Abstract

This article proposes the use of a "best round trip" approach to the performance measurement of a computer system compared to the traditional averaging, especially for detecting changes in performance during product development. Whilst system interactions and network delays can cause an unpredictable prolonging of the response time for a given work item, it is impossible for the response time to be less than a specific minimum value. This best round trip is thus more reliable as a performance statistic than the average of a large number of messages or transactions, and can be established more accurately and using less system time/resource.

This text was extracted from a PDF file.
This is the abbreviated version, containing approximately 58% of the total text.

Page 01 of 2

Performance regression measurement using best round trip approach

Performance characteristics are a critical element of software systems, and all software development organisations habitually measure the performance of their products. Software engineering tools are sold to facilitate such measurements.

    One commonly measured performance characteristic of a system is the message rate. In a messaging product, the number of messages processed per second is measured when performing different functions. Typically the system is run flat out for some time interval and number of messages processed is recorded. The average message rate achievable on a particular hardware configuration is useful information for customers in capacity planning.

    Another important reason for measuring performance regularly during product development, is to detect regressions in performance, inadvertently introduced by code changes. For business critical commercial products it is important that each release of a product is no worse than the previous in this regard, so an effective way to detect and measure performance regressions is a vital part of the development process, independently of providing performance information to customers.

    Such measurements are difficult to achieve reliably and repeatedly in a complex system, due to system interactions, network delays, etc. Repeatability is typically achieved by averaging over significant periods of time, which, when measuring many different functional aspects, means heavy resource usage of powerful computers.

    This article proposes the use of a "best round trip" approach to the performance measurement compared to the traditional averaging. Whilst system interactions and network delays can cause an unpredictable prolonging of the response time for a given work item, it is impossible for the response time to be less than a specific minimum value. If you will, the "golden-path" execution, where minimal disturbances delay the message, cannot be bettered. This "best round trip" is thus more reliable as a performance statistic than the average of a large number of messages, and can be established more accurately and using less system time/reso...