Overview

As discussed in the earlier chapters, controlled experiments and questionnaires offer HCI researchers the opportunity to deal with numbers and the hope is that in doing so we can reach solid, secure results like other sciences. Of course, as discussed in Chapter 1, there are a lot of pitfalls in ensuring that these numbers are meaningful. Even if they are avoided there is still the problem that people vary – we all take different amounts of time to do routine tasks such as finding a link on a web page or copying a paragraph from one page in a document to another. Thus, we need to be sure that what we see in our numerical data is not just natural variation between people but variation due to the real differences between interfaces and their effects on people. Statistical methods allow us to do this.

The purpose of this chapter is to look at the two sorts of data that occur frequently in HCI: data from controlled experiments and data from questionnaire studies. We will discuss how statistics can be used to distinguish natural variation from systematic difference, or in other words, how to see the wood for the trees. It is also worth saying that the purpose of this chapter is not to help you choose and execute different statistical tests. There are a lot of excellent textbooks out there, some of which will be recommended later, where you can find out the nuts and bolts of statistics.

Overview

Eyetracking records eye movement and provides information on what people look at. Understanding how people look gives researchers an insight into how people think, especially in areas of cognition such as attention. The advantage of eyetracking is that it also gathers this information in real time and to a high level of detail.

The eye movement recorded by eyetrackers is a combination of two main behaviours: first, fixations, where the eye is relatively still; second, saccades, where the eye moves rapidly between fixations (Rayner, 1998; Salvucci and Goldberg, 2000). Fixations are usually of more interest, since these are the times when the eye receives the most detailed information.

The academic world has investigated tracking eye behaviour for over 100 years. However it really started to flourish in the 1960s and 1970s, being used in the realms of cognitive sciences, language and advertising (Jacob and Karn, 2003; Rosbergen, 1998, cited by Radach et al., 2003). In the past few years however, it has been used more widely in academia and commercially. Early eyetrackers were very expensive and used bespoke equipment and software. They were also cumbersome for participants as sensitivity to head motion meant equipment such as bite bars and chin rests were used to reduce movement (see Figure 3.1). Advances in technology have greatly decreased the cost of eyetracking equipment and improved its accessibility and marketability.

Introduction

Writing is hard, and it is easy to postpone doing it. There seem to be many natural reasons to postpone writing, like you don't know what to write yet so you can't start. Our natural inclinations, however, are counter-productive. This chapter provides many reasons to start writing now. Writing now will improve your self-esteem, it will help you write better and it will help you do the work you are writing about – there are many other benefits this chapter covers. In short, writing is formative; it is the most important activity of your project, and is integral to it, not just a description of what you did. This chapter does not tell you everything you need to know about writing, but it tells you the most important secret: write now.

The advice in this chapter is written concretely as if for helping people write project reports (undergraduate, Masters or PhD theses). But the arguments apply equally to writing research proposals, job applications, novels or research papers.

Writing (noun) and writing (verb)

Writing (noun) was invented around six thousand years ago by the Sumerians. Today, we take writing for granted, yet in truth writing has a magic effect on us. Little marks, on a screen, on paper, on a road sign, anywhere, convey information – thoughts and emotions – from the writer to the readers.

What is information?

In this section we are going to try to quantify the notion of information. Before we do this, we should be aware that ‘information’ has a special meaning in probability theory, which is not the same as its use in ordinary language. For example, consider the following two statements:

1. (i) I will eat some food tomorrow.

2. (ii) The prime minister and leader of the opposition will dance naked in the street tomorrow.

If I ask which of these two statements conveys the most information, you will (I hope!) say that it is (ii). Your argument might be that (i) is practically a statement of the obvious (unless I am prone to fasting), whereas (ii) is extremely unlikely. To summarise:

1. (i) has very high probability and so conveys little information,

2. (ii) has very low probability and so conveys much information. Clearly, then, quantity of information is closely related to the element of surprise.

Consider now the following ‘statement’:

1. (iii) XQWQYK VZXPU VVBGXWQ.

Our immediate reaction to (iii) is that it is meaningless and hence conveys no information. However, from the point of view of English language structure we should be aware that (iii) has low probability (e.g. Q is a rarely occurring letter and is generally followed by U, (iii) contains no vowels) and so has a high surprise element.

The above discussion should indicate that the word ‘information’, as it occurs in everyday life, consists of two aspects, ‘surprise’ and ‘meaning’.

When I wrote the first edition of this book in the early 1990s it was designed as an undergraduate text which gave a unified introduction to the mathematics of ‘chance’ and ‘information’. I am delighted that many courses (mainly in Australasia and the USA) have adopted the book as a core text and have been pleased to receive so much positive feedback from both students and instructors since the book first appeared. For this second edition I have resisted the temptation to expand the existing text and most of the changes to the first nine chapters are corrections of errors and typos. The main new ingredient is the addition of a further chapter (Chapter 10) which brings a third important concept, that of ‘time’ into play via an introduction to Markov chains and their entropy. The mathematical device for combining time and chance together is called a ‘stochastic process’ which is playing an increasingly important role in mathematical modelling in such diverse (and important) areas as mathematical finance and climate science. Markov chains form a highly accessible subclass of stochastic (random) processes and nowadays these often appear in first year courses (at least in British universities). From a pedagogic perspective, the early study of Markov chains also gives students an additional insight into the importance of matrices within an applied context and this theme is stressed heavily in the approach presented here, which is based on courses taught at both Nottingham Trent and Sheffield Universities.

Chance and information

Our experience of the world leads us to conclude that many events are unpredictable and sometimes quite unexpected. These may range from the outcome of seemingly simple games such as tossing a coin and trying to guess whether it will be heads or tails to the sudden collapse of governments or the dramatic fall in prices of shares on the stock market. When we try to interpret such events, it is likely that we will take one of two approaches – we will either shrug our shoulders and say it was due to ‘chance’ or we will argue that we might have have been better able to predict, for example, the government's collapse if only we'd had more ‘information’ about the machinations of certain ministers. One of the main aims of this book is to demonstrate that these two concepts of ‘chance’ and ‘information’ are more closely related than you might think. Indeed, when faced with uncertainty our natural tendency is to search for information that will help us to reduce the uncertainty in our own minds; for example, think of the gambler about to bet on the outcome of a race and combing the sporting papers beforehand for hints about the form of the jockeys and the horses.

Before we proceed further, we should clarify our understanding of the concept of chance. It may be argued that the tossing of fair, unbiased coins is an ‘intrinsically random’ procedure in that everyone in the world is equally ignorant of whether the result will be heads or tails.

Counting

This chapter will be devoted to problems involving counting. Of course, everybody knows how to count, but sometimes this can be quite a tricky business. Consider, for example, the following questions:

1. (i) In how many different ways can seven identical objects be arranged in a row?

2. (ii) In how many different ways can a group of three ball bearings be selected from a bag containing eight?

Problems of this type are called combinatorial. If you try to solve them directly by counting all the possible alternatives, you will find this to be a laborious and time-consuming procedure. Fortunately, a number of clever tricks are available which save you from having to do this. The branch of mathematics which develops these is called combinatorics and the purpose of the present chapter is to give a brief introduction to this topic.

A fundamental concept both in this chapter and the subsequent ones on probability theory proper will be that of an ‘experience’ which can result in several possible ‘outcomes’. Examples of such experiences are:

1. (a) throwing a die where the possible outcomes are the six faces which can appear,

2. (b) queueing at a bus-stop where the outcomes consist of the nine different buses, serving different routes, which stop there.

If A and B are two separate experiences, we write A ∘ B to denote the combined experience of A followed by B.