Book contents
- Frontmatter
- Contents
- Preface
- Acknowledgements
- 1 Introduction
- 2 Metrics of performance
- 3 Average performance and variability
- 4 Errors in experimental measurements
- 5 Comparing alternatives
- 6 Measurement tools and techniques
- 7 Benchmark programs
- 8 Linear-regression models
- 9 The design of experiments
- 10 Simulation and random-number generation
- 11 Queueing analysis
- Appendix A Glossary
- Appendix B Some useful probability distributions
- Appendix C Selected statistical tables
- Index
9 - The design of experiments
Published online by Cambridge University Press: 15 December 2009
- Frontmatter
- Contents
- Preface
- Acknowledgements
- 1 Introduction
- 2 Metrics of performance
- 3 Average performance and variability
- 4 Errors in experimental measurements
- 5 Comparing alternatives
- 6 Measurement tools and techniques
- 7 Benchmark programs
- 8 Linear-regression models
- 9 The design of experiments
- 10 Simulation and random-number generation
- 11 Queueing analysis
- Appendix A Glossary
- Appendix B Some useful probability distributions
- Appendix C Selected statistical tables
- Index
Summary
‘The fundamental principle of science, the definition almost, is this: the sole test of the validity of any idea is experiment.’
Richard P. FeynmanThe primary goal of the design of experiments is to determine the maximum amount of information about a system with the minimum amount of effort. A well-designed experiment guides the experimenter in choosing what experiments actually need to be performed. From the resulting measurements, the experimenter can determine the effects on performance of each individual input factor, and the effects of their interactions. The form of the experimental design also allows a quantitative evaluation of the error inherent in the experimental measurements relative to the overall system response.
A key assumption behind the design of experiments is that there is a nonzero cost associated with performing an experiment. This cost includes the time and effort required to gather the necessary data, plus the time and effort needed to analyze these data to draw some appropriate conclusions. Consequently, it is important to minimize the number of experiments that must be performed while maximizing the information obtained.
Good experiment design allows the experimenter to
isolate the effects of each individual input variable,
determine the effects due to interactions of the input variables,
determine the magnitude of the change in the system's output due to the experimental error, and
obtain the maximum amount of information with the minimum amount of effort by limiting and controlling the number of experiments that must be performed.
Types of experiments
The simplest design for an experiment varies one input (factor) while holding all of the other inputs constant.
- Type
- Chapter
- Information
- Measuring Computer PerformanceA Practitioner's Guide, pp. 157 - 180Publisher: Cambridge University PressPrint publication year: 2000