Hostname: page-component-cd9895bd7-jn8rn Total loading time: 0 Render date: 2025-01-05T14:24:41.989Z Has data issue: false hasContentIssue false
  • English
  • Français

A Generalized Family of Coefficients of Relational Agreement for Numerical Scales

Published online by Cambridge University Press:  01 January 2025

Robert F. Fagot
Robert F. Fagot*
Affiliation:
University of Oregon
*
Request for reprints should be sent to Robert F. Fagot, Department of Psychology, University of Oregon, Eugene, OR 97403-1227.
Get access Rights & Permissions [Opens in a new window]

Abstract

A family of coefficients of relational agreement for numerical scales is proposed. The theory is a generalization to multiple judges of the Zegers and ten Berge theory of association coefficients for two variables and is based on the premise that the choice of a coefficient depends on the scale type of the variables, defined by the class of admissible transformations. Coefficients of relational agreement that denote agreement with respect to empirically meaningful relationships are derived for absolute, ratio, interval, and additive scales. The proposed theory is compared to intraclass correlation, and it is shown that the coefficient of additivity is identical to one measure of intraclass correlation.

Keywords

agreement coefficientmeaningfulness theoryscale typeintraclass correlation
Type
Original Paper
Information
Psychometrika , Volume 58 , Issue 2 , June 1993 , pp. 357 - 370
Copyright
Copyright © 1993 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

The author thanks the Editor and anonymous reviewers for helpful suggestions.

References

Adams, E. W., Fagot, R. F., Robinson, R. E. (1965). A theory of appropriate statistics. Psychometrika, 30, 99–127.CrossRefGoogle ScholarPubMed
Alper, T. M. (1985). A note on real measurement structures of scale type (m, m + 1). Journal of Mathematical Psychology, 29, 73–81.CrossRefGoogle Scholar
Birnbaum, M. H. (1982). Controversies in psychological measurement. In Wegener, B. (Eds.), Social attitudes and psychophysical measurement (pp. 401–485). Hillsdale, NJ: Erlbaum.Google Scholar
Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 37–46.CrossRefGoogle Scholar
Fagot, R. F. (1991). Reliability of ratings for multiple judges: Intraclass correlation and metric scales. Applied Psychological Measurement, 15, 1–11.CrossRefGoogle Scholar
Fagot, R. F., Mazo, R. M. (1989). Association coefficients of identity and proportionality for metric scales. Psychometrika, 54, 93–104.CrossRefGoogle Scholar
Fisher, R. A. (1924). The influence of rainfall on the yield of wheat at Rothamsted. Philosophical Transactions of the Royal Society of London, 213, 89–124.Google Scholar
Green, B. F. (1956). A method of scalogram analysis using summary statistics. Psychometrika, 21, 79–88.CrossRefGoogle Scholar
Hays, W. L. (1988). Statistics 4th ed.,, New York: Holt, Rinehart & Winston.Google Scholar
Kendall, M. G., Stuart, A. (1961). The advanced theory of statistics (Vol. 2), London: Griffin.Google Scholar
Lahey, M. A., Downey, R. G., Saal, F. E. (1983). Intraclass correlation: There's more there than meets the eye. Psychological Bulletin, 93, 586–595.CrossRefGoogle Scholar
Luce, R. D., Krantz, D. H., Suppes, P., Tversky, A. (1990). Foundations of Measurement, Vol. 3: Representation, axiomatization, and invariance, New York: Academic Press.Google Scholar
Narens, L. (1981). A general theory of ratio scalability with remarks about the measurement-theoretic concept of meaningfulness. Theory and Decision, 13, 1–70.CrossRefGoogle Scholar
Narens, L. (1981). On the scales of measurement. Journal of Mathematical Psychology, 24, 249–275.CrossRefGoogle Scholar
Narens, L. (1985). Abstract measurement theory, Cambridge, MA: MIT Press.Google Scholar
Roberts, F. S. (1979). Measurement theory, Reading, MA: MIT Press.Google Scholar
Shrout, P. E., Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86, 420–428.CrossRefGoogle ScholarPubMed
Stevens, S. S. (1946). On the theory of scales of measurement. Science, 103, 677–680.CrossRefGoogle ScholarPubMed
Stevens, S. S. (1951). Mathematics, measurement, and psychophysics. In Stevens, S. S. (Eds.), Handbook of experimental psychology (pp. 1–49). New York: Wiley.Google Scholar
Stevens, S. S. (1968). Measurement, statistics, and the schemapiric view. Science, 161, 849–856.CrossRefGoogle ScholarPubMed
Stine, W. W. (1989). Interobserver relational agreement. Psychological Bulletin, 106, 341–347.CrossRefGoogle Scholar
Stine, W. W. (1989). Meaningful inference: The role of measurement in statistics. Psychological Bulletin, 105, 147–155.CrossRefGoogle Scholar
Suppes, P., Zinnes, J. L. (1963). Basic measurement theory. In Luce, R. D., Bush, R. R., Galanter, E. (Eds.), Handbook of mathematical psychology, Vol. I (pp. 1–76). New York: Wiley.Google Scholar
Winer, B. J. (1971). Statistical principles in experimental design 2nd ed.,, New York: McGraw-Hill.Google Scholar
Zegers, F. E. (1986). A family of chance-corrected association coefficients for metric scales. Psychometrika, 51, 559–562.CrossRefGoogle Scholar
Zegers, F. E., ten Berge, J. M. F. (1985). A family of association coefficients for metric scales. Psychometrika, 50, 17–24.CrossRefGoogle Scholar