Hostname: page-component-745bb68f8f-kw2vx Total loading time: 0 Render date: 2025-01-08T09:56:06.786Z Has data issue: false hasContentIssue false
  • English
  • Français

Improving the Crossing-SIBTEST Statistic for Detecting Non-uniform DIF

Published online by Cambridge University Press:  01 January 2025

R. Philip Chalmers
R. Philip Chalmers*
Affiliation:
Department of Educational Psychology, The University of Georgia
*
Correspondence should be made to R. Philip Chalmers, Department of Educational Psychology, The University of Georgia, 323 Aderhold Hall, Athens, GA 30602 USA. Email: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

This paper demonstrates that, after applying a simple modification to Li and Stout’s (Psychometrika 61(4):647–677, 1996) CSIBTEST statistic, an improved variant of the statistic could be realized. It is shown that this modified version of CSIBTEST has a more direct association with the SIBTEST statistic presented by Shealy and Stout (Psychometrika 58(2):159–194, 1993). In particular, the asymptotic sampling distributions and general interpretation of the effect size estimates are the same for SIBTEST and the new CSIBTEST. Given the more natural connection to SIBTEST, it is shown that Li and Stout’s hypothesis testing approach is insufficient for CSIBTEST; thus, an improved hypothesis testing procedure is required. Based on the presented arguments, a new chi-squared-based hypothesis testing approach is proposed for the modified CSIBTEST statistic. Positive results from a modest Monte Carlo simulation study strongly suggest the original CSIBTEST procedure and randomization hypothesis testing approach should be replaced by the modified statistic and hypothesis testing method.

Keywords

DIFnon-uniform DIFbidirectional biasSIBTESTCrossing-SIBTEST
Type
Original Paper
Information
Psychometrika , Volume 83 , Issue 2 , June 2018 , pp. 376 - 386
Copyright
Copyright © 2017 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

Special thanks to two anonymous reviewers for providing insightful comments that improved the quality of this manuscript. Correspondence concerning this article should be addressed to R. Philip Chalmers.

References

Chalmers, R.P., (2012). mirt: A multidimensional item response theory package for the R environment, Journal of Statistical Software, 48(6), 1–29.CrossRefGoogle Scholar
Chalmers, R. P., (2016). SimDesign: Structure for organizing Monte Carlo simulation designs. R package version 1.0. https://CRAN.R-project.org/package=SimDesign.Google Scholar
Chalmers, R.P., Counsell, A., Flora, D.B., (2016). It might not make a big DIF: Improved differential test functioning statistics that account for sampling variability, Educational and Psychological Measurement, 76(1), 114–140.CrossRefGoogle Scholar
Chang, H-H, Mazzeo, J., Roussos, L., (1996). DIF for polytomously scored items: An adaptation of the SIBTEST procedure, Journal of Educational Measurement, 33(3), 333–353.CrossRefGoogle Scholar
Dorans, N.J., Kulick, E., (1986). Demonstrating the utility of the standardization approach to assessing unexpected differential item performance on the Scholastic Aptitude Test, Journal of Educational Measurement, 23(4), 355–368.CrossRefGoogle Scholar
Edgington, E.S., (1987). Randomization tests. New York, NY:Maecel Dekker.Google Scholar
Guttman, L., (1945). A basis for analyzing test–retest reliability, Psychometrika, 10, 255–282.CrossRefGoogle ScholarPubMed
Kuder, G.F., Richardson, M.W., (1937). The theory of the estimation of test reliability, Psychometrika, 2, 151–160.CrossRefGoogle Scholar
Li, H-H, Stout, W., (1996). A new procedure for detection of crossing DIF, Psychometrika, 61(4), 647–677.CrossRefGoogle Scholar
Lord, F.M., Novick, M.R., (1968). Statistical theory of mental test scores. Reading, MA:Addison-Wesley.Google Scholar
Shealy, R., Stout, W., (1993). A model-based standardization approach that separates true bias/DIF from group ability differences and detect test bias/DTF as well as item bias/DIF, Psychometrika, 58(2), 159–194.CrossRefGoogle Scholar
Sigal, M.J., Chalmers, R.P., (2016). Play it again: Teaching statistics with Monte Carlo simulation, Journal of Statistics Education, 24(3), 136–156.CrossRefGoogle Scholar
Thissen, D., Steinberg, L., Wainer, H., (1993). Detection of differential item functioning using the parameters of item response models. In Holland, P.W., Wainer, H.(Eds.), Differential item functioning. (pp 67–113). Hillsdale, NJ:Lawrence Erlbaum.Google Scholar