Computer Benchmark Evaluation and Design of Experiments: A Case Study

The author defines installation tuning and promotes it as an important area of concern for computer configurations. Examples include deciding on the paging configuration for a particular workload and partitioning available memory into system and user areas. Available tuning options are often difficult to select due to poor understanding of their effects, so analytic models rarely exist for these areas. The author illustrates the use of various models to assess the significance of installation tuning factors. A case study is presented in which a statistical model for response time is developed incorporating the key factors of user load, paging, and main storage configuration. A case study is presented in which a statistical design of experiments to computer performance analysis is presented. General statistical issues pertaining to the design and analysis of computer performance experiments are identified and illustrated.