Introduction

An essential requirement of all measurements in clinical practice and research is that they are reliable. Reliability is defined as ‘the degree to which the measurement is free from measurement error’ (Mokkink et al., 2010a). Its importance often remains unrecognized until repeated measurements are performed. To give a few examples of reliability issues: radiologists want to know whether their colleagues interpret X-rays or specific scans in the same way as they do, or whether they themselves would give the same rating if they had to assess the same X-ray twice. These are called the inter-rater and the intra-rater reliability, respectively. Repeated measurements of fasting blood glucose levels in patients with diabetes may differ due to day-to-day variation or to the instruments used to determine the blood glucose level. These sources of variation play a role in test–retest reliability. In a pilot study, we are interested in the extent of agreement between two physiotherapists who assess the range of movement in a shoulder, so that we can decide whether or not their ratings can be used interchangeably in the main study. The findings of such performance tests may differ for several reasons. For example, patients may perform the second test differently because of their experience with the first test, the physiotherapists may score the same performance differently or the instructions given by one physiotherapist may motivate the patients more than the instructions given by the other physiotherapist.