As noted in earlier chapters, measuring instruments for observational methods consist of coding schemes in the hands (and minds and eyes) of trained observers. Like all measuring instruments, they need to be calibrated; we need to be assured that the instruments are accurate. With human observers, this means demonstrating that two observers independently coding the same stream of behavior produce essentially the same results – or, perhaps better, that an observer agrees with a gold standard, a version that has been prepared by experts and is presumed accurate. It is not an exaggeration to call such demonstrations of observer agreement the sine qua non of observational methods. Without such demonstrations, we are left with individual narratives – perhaps fascinating, perhaps insightful, perhaps useful for generating research questions – but nonetheless narratives of unknown reliability.

Nothing is absolute, and a suitable level of agreement between two independent observers does not by itself guarantee accuracy: After all, two observers could share similar deviant views of the world. But almost always in the behavioral research world, observational methods require attention to and demonstration of observer agreement – either two observers with each other or one observer with a gold standard.