Book contents
- Frontmatter
- Contents
- Figures
- Preface
- 1 Introduction to Observational Methods
- 2 Coding Schemes and Observational Measurement
- 3 Recording Observational Data
- 4 Representing Observational Data
- 5 Observer Agreement and Cohen’s Kappa
- 6 Kappas for Point-by-Point Agreement
- 7 The Intraclass Correlation Coefficient (ICC) for Summary Measures
- 8 Summary Statistics for Individual Codes
- 9 Cell and Summary Statistics for Contingency Tables
- 10 Preparing for Sequential and Other Analyses
- 11 Time-Window and Log-Linear Sequential Analysis
- 12 Recurrence Analysis and Permutation Tests
- Epilogue
- Appendix A Expected Values for Kappa Comparing Two Observers
- Appendix B Expected Values for Kappa Comparing with a Gold Standard
- References
- Index
6 - Kappas for Point-by-Point Agreement
Published online by Cambridge University Press: 05 June 2012
- Frontmatter
- Contents
- Figures
- Preface
- 1 Introduction to Observational Methods
- 2 Coding Schemes and Observational Measurement
- 3 Recording Observational Data
- 4 Representing Observational Data
- 5 Observer Agreement and Cohen’s Kappa
- 6 Kappas for Point-by-Point Agreement
- 7 The Intraclass Correlation Coefficient (ICC) for Summary Measures
- 8 Summary Statistics for Individual Codes
- 9 Cell and Summary Statistics for Contingency Tables
- 10 Preparing for Sequential and Other Analyses
- 11 Time-Window and Log-Linear Sequential Analysis
- 12 Recurrence Analysis and Permutation Tests
- Epilogue
- Appendix A Expected Values for Kappa Comparing Two Observers
- Appendix B Expected Values for Kappa Comparing with a Gold Standard
- References
- Index
Summary
In the previous chapter we introduced Cohen’s (1960) classic kappa and discussed a number of issues associated with it and its use. In this chapter we describe how kappa can be used with each of the data types described in Chapter 4. The basic kappa computation remains basically unchanged, but some data types require matching and tallying procedures that differ from the classic Cohen’s kappa. Where procedures differ, we regard the result as a variant of Cohen’s kappa and, to make the distinction clear, provide a name for that variant. To provide orientation (and to let you select which sections of this chapter may best fit your circumstances), Figure 6.1 links data types with their appropriate kappa variant. The remaining sections of this chapter detail appropriate kappa procedures for each data type.
Event-Based Agreement: the Alignment Problem
When events are demarcated before coding, as turns in a transcript, the classic Cohen’s kappa described in the previous chapter is appropriate. However, when observes first segment the stream of behavior into events – that is, decide where the seams between events occur – and only then code the events, the situation is more complicated.
- Type
- Chapter
- Information
- Publisher: Cambridge University PressPrint publication year: 2011