Overview

As discussed in the earlier chapters, controlled experiments and questionnaires offer HCI researchers the opportunity to deal with numbers and the hope is that in doing so we can reach solid, secure results like other sciences. Of course, as discussed in Chapter 1, there are a lot of pitfalls in ensuring that these numbers are meaningful. Even if they are avoided there is still the problem that people vary – we all take different amounts of time to do routine tasks such as finding a link on a web page or copying a paragraph from one page in a document to another. Thus, we need to be sure that what we see in our numerical data is not just natural variation between people but variation due to the real differences between interfaces and their effects on people. Statistical methods allow us to do this.

The purpose of this chapter is to look at the two sorts of data that occur frequently in HCI: data from controlled experiments and data from questionnaire studies. We will discuss how statistics can be used to distinguish natural variation from systematic difference, or in other words, how to see the wood for the trees. It is also worth saying that the purpose of this chapter is not to help you choose and execute different statistical tests. There are a lot of excellent textbooks out there, some of which will be recommended later, where you can find out the nuts and bolts of statistics.