Dismiss
InnovationQ will be updated on Sunday, Oct. 22, from 10am ET - noon. You may experience brief service interruptions during that time.
Browse Prior Art Database

Factorial Design for Hardware and Software Performance Evaluation Across an Internet Connection

IP.com Disclosure Number: IPCOM000004553D
Publication Date: 2001-Jan-26
Document File: 10 page(s) / 39K

Publishing Venue

The IP.com Prior Art Database

Related Documents

0471093157: ISBN [+2]

Abstract

Hardware and software reviewers often dismiss the idea of testing across a live Internet connection because of the inherent lack of reproducibility. This is an ideal opportunity to use statistical experimental design, as is done routinely in the physical sciences. For example, one might test two different browsers at two different times of day, applying well-established methods to extract significant trends from the data. Many variables could be included, ranging from Web page content to the client operating system and hardware configuration. As an example, a 2(5-1) fractional factorial design is presented, in which effects of ISP, time of day, type of connection, browser, and client operating system are evaluated. In addition, all two variable interactions are assessed.

This text was extracted from a PPT97 document.
This is the abbreviated version, containing approximately 92% of the total text.

Factorial Design for Hardware and Software Performance Evaluation Across an Internet Connection

Abstract: Hardware and software reviewers often dismiss the idea of testing across a live Internet connection because of the inherent lack of reproducibility. This is an ideal opportunity to use statistical experimental design, as is done routinely in the physical sciences. For example, one might test two different browsers at two different times of day, applying well-established methods to extract significant trends from the data. Many variables could be included, ranging from Web page content to the client operating system and hardware configuration. As an example, a 25-1 fractional factorial design is presented, in which effects of ISP, time of day, type of connection, browser, and client operating system are evaluated. In addition, all two variable interactions are assessed.

Note: To be published anonymously at http://www.ip.com

Problem Definition

Readers of computer industry trade journals value hardware and software reviews and comparative evaluations.

Such reviews often have a large impact on sales for a new product or service.

Problem Definition (continued)

Evaluations that require testing across an Internet connection are problematic, due to inherent variability and unpredictability in several key variables, including

Internet service provider, network traffic (and time of day), type of Internet connection, client and server hardware configuration, client and server operating system, browser, etc.

Problem Definition (continued)

As a result of these variables, quantitative evaluations have generally been avoided across an Internet connection

What is needed is a way to obtain reliable quantitative data in such a situation

Proposal –Factorial Designed

Although the computer and electronics industry are accustomed to low statistical variability, the physical and life sciences routinely must live with variability, and have learned to apply advanced statistical methods to glean quantitative trends.

It is proposed to utilize experimental design (factorial design) to evaluate software and hardware across an Internet connection.

Example: A 25-1 Fractional Factorial Design

The next slide shows an example of the type of study that is possible.

It is a 25-1 fractional factorial design.

Analysis of the data in this and other types of factorial designs is explained in the attached references

Note that any indicator of performance can be evaluated, such as download speed, connection speed, etc.

References: 1) Statistics for Experimenters: An Introduction to Design, Data Analysis, and Model Building

by George E. P. Box, J. Stuart Hunter, William G. Hunter 1st edition (June 1978) John Wiley Sons; ISBN: 0471093157; 2) Improving Quality through Planned Experimentation by Ronald D. Moen, Thomas W. Nol...