Hostname: page-component-cd9895bd7-8ctnn Total loading time: 0 Render date: 2025-01-05T15:36:14.314Z Has data issue: false hasContentIssue false
  • English
  • Français

Item-focussed Trees for the Identification of Items in Differential Item Functioning

Published online by Cambridge University Press:  01 January 2025

Gerhard Tutz  and
Moritz Berger
Gerhard Tutz*
Affiliation:
Ludwig-Maximilians-Universität
Moritz Berger
Affiliation:
Ludwig-Maximilians-Universität
*
Correspondence should be made to Gerhard Tutz and Moritz Berger, Ludwig-Maximilians-Universität, Munich, Germany. Email: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

A novel method for the identification of differential item functioning (DIF) by means of recursive partitioning techniques is proposed. We assume an extension of the Rasch model that allows for DIF being induced by an arbitrary number of covariates for each item. Recursive partitioning on the item level results in one tree for each item and leads to simultaneous selection of items and variables that induce DIF. For each item, it is possible to detect groups of subjects with different item difficulties, defined by combinations of characteristics that are not pre-specified. The way a DIF item is determined by covariates is visualized in a small tree and therefore easily accessible. An algorithm is proposed that is based on permutation tests. Various simulation studies, including the comparison with traditional approaches to identify items with DIF, show the applicability and the competitive performance of the method. Two applications illustrate the usefulness and the advantages of the new method.

Keywords

Rasch modeldifferential item functioningrecursive partitioningitem-focussed trees
Type
Original Paper
Information
Psychometrika , Volume 81 , Issue 3 , September 2016 , pp. 727 - 750
Copyright
Copyright © 2015 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Amthauer, R., Brocke, B., Liepmann, D., & Beauducel, A. (1973). Intelligenz-Struktur-Test (IST 70), Göttingen: HogrefeGoogle Scholar
Amthauer, R., Brocke, B., Liepmann, D., & Beauducel, A. (1999). Intelligenz-Struktur-Test 2000 (IST 2000), Göttingen: HogrefeGoogle Scholar
Amthauer, R., Brocke, B., Liepmann, D., & Beauducel, A. (2001). Intelligenz-Struktur-Test 2000 R (IST 2000 R), Göttingen: HogrefeGoogle Scholar
Beauducel, A., Liepmann, D., Horn, S., & Brocke, B. (2010). Intelligence-Structure-Test. English version of the Intelligenz-Struktur-Test 2000 R (I-S-T 2000 R). Göttingen: Hogrefe.Google Scholar
Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society. Series B (Methodological), 57, 1289–300.CrossRefGoogle Scholar
Berger, M. (2015). DIFtree: Item Focused Trees for the Identification of Items in Differential Item Functioning. R package version 1.1.0.Google Scholar
Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, J. C. (1984). Classification and regression trees, Monterey, CA: WadsworthGoogle Scholar
Bühner, M., Ziegler, M., Krumm, S., & Schmidt-Atzert, L. (2006). Ist der I-S-T 2000 R Rasch-skalierbar?. Diagnostica, 52, 3119–130.CrossRefGoogle Scholar
Ciampi, A., Chang, C. -H., Hogg, S., McKinney, S., McNeil, I., & Umphrey, G. (1987). Recursive partitioning: A versatile method for exploratory data analysis in biostatistics. Biostatistics, New York, USA: D. Reidel PublishingGoogle Scholar
Clark, L., Pregibon, D., Chambers, J., & Hastie, T. (1992). Tree-based models. Statistical models in S, Pacific Grove, CA: Wadsworth & Brooks 377–420.Google Scholar
Holland, P. W., Thayer, D. T., Wainer, H., & Braun, H. I. (1988). Differential item performance and the Mantel–Haenszel procedure. Test validity, Hillsdale, NJ: Lawrence Erlbaum 129–145.Google Scholar
Holland, W., & Wainer, H. (1993). Differential item functioning, Hillsdale, NJ: Lawrence Erlbaum AssociatesGoogle Scholar
Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6, 265–70.Google Scholar
Hothorn, T., Hornik, K., & Zeileis, A. (2006). Unbiased recursive partitioning: A conditional inference framework. Journal of Computational and Graphical Statistics, 15, 651–674.CrossRefGoogle Scholar
Hothorn, T., & Lausen, B. (2003). On the exact distribution of maximally selected rank statistics. Computational Statistics and Data Analysis, 43, 121–137.CrossRefGoogle Scholar
Kim, S-H, Cohen, A. S., & Park, T-H (1995).Detection of differential item functioning in multiple groups. Journal of Educational Measurement, 32, 3261–276.CrossRefGoogle Scholar
Lord, F. M. (1980). Applications of item response theory to practical testing problems, New York, USA: RoutledgeGoogle Scholar
Magis, D., Beland, S., & Raiche, G. (2013). difR: Collection of methods to detect dichotomous differential item functioning (DIF) in psychometrics. R package version 4. 5.Google Scholar
Magis, D., Bèland, S., Tuerlinckx, F., & Boeck, P. (2010). A general framework and an R package for the detection of dichotomous differential item functioning. Behavior Research Methods, 42, 3847–862.CrossRefGoogle Scholar
Magis, D., & De Boeck, P. (2014). Type I error inflation in DIF identification with Mantel–Haenszel an explanation and a solution. Educational and Psychological Measurement, 74, 4713–728.CrossRefGoogle Scholar
Magis, D., Raîche, G., Béland, S., & Gérard, P. (2011). A generalized logistic regression procedure to detect differential item functioning among multiple groups. International Journal of Testing, 11, 4365–386.CrossRefGoogle Scholar
Magis, D., Tuerlinckx, F., & De Boeck, P. (2015). Detection of differential item functioning using the lasso approach. Educational and Behavioral Statistics, 40, 2111–135.CrossRefGoogle Scholar
Millsap, R., & Everson, H. (1993). Methodology review: Statistical approaches for assessing measurement bias. Applied Psychological Measurement, 17, 4297–334.CrossRefGoogle Scholar
Morgan, J. N., & Sonquist, J. A. (1963). Problems in the analysis of survey data, and a proposal. Journal of the American Statistical Association, 58, 415–435.CrossRefGoogle Scholar
Osterlind, S., & Everson, H. (2009). Differential item functioning, Thousand Oaks, CA: Sage Publications, IncCrossRefGoogle Scholar
Quinlan, J. R. (1986). Induction of decision trees. Machine Learning, 1, 81–106.CrossRefGoogle Scholar
Quinlan, J. R. (1993). Programs for machine learning, San Francisco: Morgan Kaufmann PublisherIncGoogle Scholar
Raju, N. S. (1988). The area between two item characteristic curves. Psychometrika, 53, 4495–502.CrossRefGoogle Scholar
Ripley, B. D. (1996). Pattern recognition and neural networks, Cambridge: Cambridge University PressCrossRefGoogle Scholar
Rogers, H. (2005). Differential item functioning. Encyclopedia of statistics in behavioral science, Chichester: WileyGoogle Scholar
Schauberger, G., & Tutz, G. (2015). Detection of differential item functioning in Rasch models by boosting techniques. British Journal of Mathematical and Statistical Psychology. doi:10.1111/bmsp.12060.CrossRefGoogle Scholar
Schmidt-Atzert, L. (2000). Intelligenz-Strukturtest 2000 R. Zeitschrift für Personalpsychologie, 1, 52–55.Google Scholar
Schmidt-Atzert, L., Hommers, W., & Hess, M. (1995). Der I-S-T 70: Eine Analyse und Neubewertung. Diagnostica, 41, 108–130.Google Scholar
Shih, Y-S (2004).A note on split selection bias in classification trees. Computational Statistics and Data Analysis, 45, 457–466.CrossRefGoogle Scholar
Shih, Y-S, & Tsai, H. (2004). Variable selection bias in regression trees with constant fits. Computational Statistics and Data Analysis, 45, 595–607.CrossRefGoogle Scholar
Soares, T., Gonçalves, F., & Gamerman, D. (2009). An integrated bayesian model for DIF analysis. Journal of Educational and Behavioral Statistics, 34, 3348–377.CrossRefGoogle Scholar
Strobl, C., Boulesteix, A-L, & Augustin, T. (2007). Unbiased split selection for classification trees based on the Gini index. Computational Statistics & Data Analysis, 52, 483–501.CrossRefGoogle Scholar
Strobl, C., Kopf, J., & Zeileis, A. (2015). Rasch trees: A new method for detecting differential item functioning in the Rasch model. Psychometrika, 80, 2289–316.CrossRefGoogle Scholar
Strobl, C., Malley, J., & Tutz, G. (2009). An introduction to recursive partitioning: Rationale, application and characteristics of classification and regression trees, bagging and random forests. Psychological Methods, 14, 323–348.CrossRefGoogle ScholarPubMed
Swaminathan, H., & Rogers, H. J. (1990). Detecting differential item functioning using logistic regression procedures. Journal of Educational Measurement, 27, 4361–370.CrossRefGoogle Scholar
Thissen, D., Steinberg, L., Wainer, H., Holland, P. W., & Wainer, H. (1993). Detection of differential item functioning using the parameters of item response models. Differential item functioning, Hillsdale, NJ: Erlbaum 67–113.Google Scholar
Trepte, S., & Verbeet, M. (2010). Allgemeinbildung in Deutschland - Erkenntnisse aus dem SPIEGEL Studentenpisa-Test, Wiesbaden: VS VerlagGoogle Scholar
Tutz, G., & Schauberger, G. (2015). A penalty approach to differential item functioning in Rasch models. Psychometrika, 80, 121–43.CrossRefGoogle ScholarPubMed
Van den Noortgate, W., & De Boeck, P. (2005). Assessing and explaining differential item functioning using logistic mixed models. Journal of Educational and Behavioral Statistics, 30, 4443–464.CrossRefGoogle Scholar
Zumbo, B. (1999). A handbook on the theory and methods of differential item functioning (DIF), Ottawa: National Defense HeadquartersGoogle Scholar