2.1. Item Vector Representation

The drawback that occurs in the representation of M2PL two-dimensional items is that only one surface or contour can be examined at a time. This problem can be solved by representing items in the two-dimensional latent ability plane as a vector. This is accomplished using the following guidelines (Reckase, Reference Reckase2009):

• • All vectors lie on lines that pass through the origin.

• • Vectors can lie only in the first and third quadrants because the a-parameters are constrained to be positive.

• • Vectors representing easy items lie in the third quadrant; those representing difficult items lie in the first quadrant. (Note that if $a_1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$a_{1}$$\end{document} is negative the vector will lie in the second quadrant and if $a_2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$a_{2}$$\end{document} is negative the vector will lie in the fourth quadrant.)

• • The tail of the vector lies on the $p=$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$p =$$\end{document} .5 equiprobability contour and the vector is always orthogonal to this contour.

Using the derived information for multidimensional discrimination and difficulty, these vectors are created where the length of the vector indicates how discriminating the item is equal to MDISC (4). The tail of the vector lies on the $p=.5$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$p =.5 $$\end{document} equiprobability contour. The signed distance from the origin perpendicular to this contour is the unidimensional analog of difficulty, MDIFF (5). The angular direction from the ${\theta }_1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta _{1}$$\end{document} -axis indicates the ( ${\theta }_1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta _{1}$$\end{document} , ${\theta }_2)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta _{2})$$\end{document} -composite of ability that the item i is best measuring:

$\begin{array}{c}{\alpha }_i={cos}^{-1}\left(\frac{a_{1i}}{\text{MDISC}}\right).\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \alpha _{i}=\cos ^{-1}\left(\frac{a_{1i}}{\hbox {MDISC}}\right) . \end{aligned}$$\end{document}

As illustrated in Fig. 3, vectors are projections, indicating the composite direction of maximum discrimination or maximum slope onto the latent ability plane. As is shown, the corresponding contour with the response surface over the third quadrant is removed so that the projected item vector is illustrated in relationship to the underlying contour surface. The greater the discrimination of the item, the steeper the response surface, causing the corresponding contours to become closer together and the greater the length of the vector. Vectors of easy items appear in the third quadrant and vectors representing difficult items are in the first quadrant.

Figure. 3 Illustration of the direction of maximum slope for a compensatory item projected onto the latent ability plane to form its item vector.

Angle item vectors differ by only a few degrees these nuances cannot be attributed solely to phrasing or vocabulary. Item writers and psychometricians need to examine the content sectors that contain different item contents or test specifications. These sectors help determine which ( ${\theta }_1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta _{1}$$\end{document} , ${\theta }_2)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta _{2})$$\end{document} -composites an item measures best (Ackerman, Reference Ackerman1991). Ultimately, these content sectors will help in understanding or defining the ${\theta }_1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta _{1} $$\end{document} and ${\theta }_2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta _{2 }$$\end{document} latent abilities and defining an imposed unidimensional score scale.

Item vectors are often color-coded based on their content classification. Ideally, vectors with the same content should cluster in a narrow content sector, indicating that they are measuring similar ( ${\theta }_1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta _{1}$$\end{document} , ${\theta }_2)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta _{2})$$\end{document} -composites. For example, consider a standardized graduate admissions test with 101 items. In Fig. 4, observe how different content areas occupy unique sectors. The one vector in quadrant two had a negative a ${}_{\text{2}}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$_{\textrm{2}}$$\end{document} value.

Figure. 4 Item vectors for a standardized 101-item test with three content areas.

2.2. The Validity Sector

Ackerman (Reference Ackerman1992) defined the validity sector as the sector containing vectors of items measuring practitioner-determined valid composites. Unlike the 101-item test shown above, most standardized tests yield vectors that can be enclosed by a 30 ${}^{\circ }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ $$\end{document} –45 ${}^{\circ }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ $$\end{document} degree sector as illustrated in Fig. 5. In this figure, the green item vectors (and red dotted RC) are enclosed in a 45 ${}^{\circ }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ $$\end{document} validity sector and are believed to be vectors of items measuring the valid ( ${\theta }_1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta _{1}$$\end{document} , ${\theta }_2)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta _{2})$$\end{document} -composites as described in the test’s specifications. Items whose vectors (red) fall outside the validity sector should be examined for DIF because they are likely measuring invalid or nuisance dimensions. DIF has the potential to occur if the groups of interest differ in their distributions of underlying abilities on these invalid skill composites. Ma et al. (Reference Ma, Ackerman, Ip and Chung2023) demonstrated that differences in multidimensional latent ability distribution on the invalid dimension can result in DIF especially when items measure primarily the invalid dimension. The insight gained from item vectors and the validity sector can guide psychometricians in refining assessments, ensuring validity, and understanding the intricate interplay of latent abilities.

Figure. 5 A validity sector enclosing item vectors (green) for a 60-item standardized test. Vectors outside the sector (red) are measuring composites that could result in DIF. (Color figure online)

2.3. Score Scale Consistency Using Conditional Centroids

Another way to understand how DIF can occur from a two-dimensional perspective is to examine whether an imposed unidimensional score scale consistently represents the same ( ${\theta }_1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta _{1}$$\end{document} , ${\theta }_2)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta _{2})$$\end{document} -composite as one progresses across the observable score scale. When different parts of the unidimensional score scale represent different skill composites, score scale consistency breaks down.

For example, consider the 101-item test represents a dimensional scenario in which there is a lack of score scale consistency. That is, the same ( ${\theta }_1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta _{1}$$\end{document} , ${\theta }_2)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta _{2})$$\end{document} -composite is not being measured equally well throughout the observable score range. To examine scale consistency, “centroid” plots can be created to show ( $\overline{{\theta }_1},\overline{{\theta }_2})$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\bar{\theta _{1}},\bar{\theta _{2}}) $$\end{document} for each number correct observed score, x, (i.e., ( $\overline{{\theta }_1},\overline{{\theta }_2}\left)\right|X=x)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\bar{\theta _{1}},\bar{\theta _{2}})\vert X=x)$$\end{document} . This plot should be linear across the latent ability plane, indicating consistent measurement of ( ${\theta }_1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta _{1}$$\end{document} , ${\theta }_2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta _{2}$$\end{document} ) across the scale. In the top illustration of Fig. 6, centroid plots for each content category are graphed across the score range. The Analytic Reasoning tends to be linear representing primarily differences in ${\theta }_{\text{1}}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta _{\textrm{1}}$$\end{document} . Logical Reasoning and Reading Comprehension represent differences in ${\theta }_2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta _{2}$$\end{document} , but not consistently ${}_{.}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$_{\mathrm {. }}$$\end{document} However, the results of this test are reported as a single score. The lower figure shows the centroid plot for the total test score scale. Scores in the range from 0 to 35 represent differences in the ${\theta }_1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta _{1}$$\end{document} -ability. Scores in the range from 35 to 80 indicate differences in the ${\theta }_2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta _{2}$$\end{document} -ability. Scores from 85 to 100 represent proficiency differences in the upper ${\theta }_1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta _{1}$$\end{document} -ability. Work by Strachan et al. (Reference Strachan, Cho, Ackerman, Chen, de la Torre and Ip2022) found that if there is a confounding of difficulty and dimensionality (e.g., easy items measure one dimension and difficult items measure a second dimension) the composite may not be linear.

Such inconsistency makes score interpretation and certain psychometric procedures such as equating and computer adaptive testing pool development incredibly challenging. This variation also affects DIF procedures that group examinees according to their number correct scores for conditional analyses because different scores reflect different skills.

Figure. 6 Conditional centroid plots for each content category (top) and total test score (bottom).

2.4. Reference Composite: Mapping a 2PL Scale in a Ttwo-Dimensional Latent Space

Wang (Reference Wang1985) demonstrated that when calibrating multidimensional data, the estimated unidimensional 2PL model essentially combines latent abilities into a weighted composite known as the reference composite (RC). The RC is key because it indicates the ( ${\theta }_1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta _{1}$$\end{document} , ${\theta }_2)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta _{2})$$\end{document} -composite being best measured by the unidimensional IRT $\theta$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta $$\end{document} -scale and the number correct score scale. It is important to note that for, say a two-dimensional two-item test, the RC ( ${\theta }_1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta _{1}$$\end{document} , ${\theta }_2)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta _{2})$$\end{document} -composite does not directly correspond with the composite skills being measured by either of the two items. This has important implications for how to substantiate or define the unidimensional scale for a test containing two-dimensional items. The RC is a useful tool for demonstrating the parallelism of multiple two-dimensional test forms. That is, after calibration and rescaling, the RCs of parallel forms should align closely within measurement error and other constraints based on the test specification (e.g., cut score measurement precision, content constraints).

For the two-dimensional case, Wang (Reference Wang1985) determined that the RC is a function of the L’A’AL matrix, where A is the n x 2 matrix of discrimination parameters for a given n-item test and L is the Cholesky decomposition of the underlying ${\theta }_1-{\theta }_2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta _{1}-\theta _{2}$$\end{document} variance–covariance matrix, $\Omega$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Omega $$\end{document} . The angle between the positive ${\theta }_1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta _{1}$$\end{document} -axis and the RC can be calculated as the arccosine of the first element of the eigenvector associated with the larger of the two eigenvalues of the L’A’AL matrix.

A simple example will help to clarify. Assume a two-item case where $a_1=$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$a_{1}=$$\end{document} 1.3, $a_2=.4$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$a_{2} =.4 $$\end{document} and $d=-1.2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$d = -1.2$$\end{document} for Item 1 and $a_1=$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$a_{1}=$$\end{document} .4, $a_2=$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$a_{2} =$$\end{document} 1.3 and $d=$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$d =$$\end{document} $-$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$-$$\end{document} 1.2 are the parameters for Item 2. The item vectors for Item 1 and Item 2 are, respectively 17.10 ${}^{\circ }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ $$\end{document} and 72.90 ${}^{\circ }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ $$\end{document} from the positive ${\theta }_1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta _{1}$$\end{document} -axis, respectively. Assume further a group of examinees, Group A, whose two-dimensional underlying ability is $N\left(\left(\begin{array}{c}0\\ 0\end{array}\right),,,\left(\begin{array}{cc}1.0& .0\\ .0& .5\end{array}\right)\right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$N\left[ \left({\begin{array}{*{20}c} 0\\ 0\\ \end{array} } \right) ,\left({\begin{array}{*{20}c} 1.0 &{} .0\\ .0 &{} .5\\ \end{array} } \right) \right] $$\end{document} and Group B’s underlying distribution is $N\left(\left(\begin{array}{c}0\\ 0\end{array}\right),,,\left(\begin{array}{cc}.5& .0\\ .0& 1.0\end{array}\right)\right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$N\left[ \left({\begin{array}{*{20}c} 0\\ 0\\ \end{array} } \right) ,\left({\begin{array}{*{20}c} .5 &{} .0\\ .0 &{} 1.0\\ \end{array} } \right) \right] $$\end{document} . The Cholesky decomposition of ${\Omega }_{\text{A}}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Omega _{\textrm{A}}$$\end{document} is $\left(\begin{array}{cc}1.0& .0\\ .0& .7071\end{array}\right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\left[ {\begin{array}{*{20}c} 1.0 &{} .0\\ .0 &{} .7071\\ \end{array} } \right] $$\end{document} for both groups and the L’A’AL matrix is equal to $\left(\begin{array}{cc}1.8500& .7353\\ .7353& .9250\end{array}\right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\left[ {\begin{array}{*{20}c} 1.8500 &{} .7353\\ .7353 &{} .9250\\ \end{array} } \right] $$\end{document} . This matrix is negative definite. The eigenvalues of this matrix are 2.2562 and.5187 and the eigenvector that corresponds to the larger eigenvalue is $\left(\begin{array}{c}-.8753\\ -.4835\end{array}\right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\left[ {\begin{array}{*{20}c} -.8753\\ -.4835\\ \end{array} } \right] $$\end{document} . The squared elements sum to 1.0, so this eigenvector can be considered as direction cosines. The angle associated with the RC can be computed by taking the arccosine of the absolute value of the elements since the L’A’AL matrix is negative definite. These calculations indicate that the RC for the Reference group lies 28.91 ${}^{\circ }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ $$\end{document} from the positive ${\theta }_1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta _{1}$$\end{document} -axis. Similarly, the RC for the Focal group lies 61.09 ${}^{\circ }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ $$\end{document} from the positive ${\theta }_1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta _{1}$$\end{document} -axis. Notice the angular difference between the RCs is 32.18 ${}^{\circ }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ $$\end{document} , which is the angular between the red dashed RC and green dashed RC figure in Fig. 7. The underlying joint and marginal distributions, RCs, and item vectors are also displayed in the figure.

Trying to compare scores from different RCs can be problematic because it could result in examinees being ordered differently on the two RCs. Ramsay (Reference Ramsay1990) and Junker and Stout (Reference Junker and Stout1991) examined the effects of differential ordering in a DIF context. Even though DIF procedures that condition upon the number correct score may not be sensitive to the dissimilar substantive interpretation of the two RCs, they are sensitive to differential ordering. This situation is graphically illustrated on the right in Fig. 7. In this diagram, two examinees, X and Y, would have one ordering if they are in the Focal group and orthogonally mapped onto the RC ${}_{\text{FOC}}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$_{\textrm{FOC}}$$\end{document} , but a reverse ordering if they belong to the Reference group and are mapped onto RC ${}_{\text{REF}}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$_{\textrm{REF}}$$\end{document} .

In an attempt to overcome scaling problems, DIF methodologies use different approaches. DIF analyses that use IRT methodology (Raju Reference Raju1988) require that before comparing item characteristic curves for Reference and Focal groups, the item parameters for one group must be rescaled and placed on the other group’s scale. This can be accomplished using either a mean–mean or mean–sigma rescaling (Kolen and Brennan, Reference Kolen and Brennan2014). It should be noted that these linking/rescaling procedures apply optimally when the reference composites have identical composite directions. They are designed to account for mean and standard deviation scale differences that are established in calibrating response data to fit a unidimensional model.

DIF Methodologies are not designed to compensate for scale differences that would occur when scales represent different ability composites. The greater the angular separation between the RCs, the less effective rescaling becomes. For example, consider an extremely unrealistic case. If the RC for the Reference group is 10 ${}^{\circ }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ $$\end{document} , the RC for the Focal group is 70 ${}^{\circ }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ $$\end{document} , placing the Focal group parameters on the Reference group’s scale would adjust primarily ${\theta }_1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta _{1}$$\end{document} differences, but the Focal group’s scale would primarily measure ${\theta }_2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta _{2}$$\end{document} . This problem is discussed further in Study 1 below. Several researchers, including Li and Lissitz (Reference Li and Lissitz2000) and Oshima et al. (Reference Oshima, Davey and Lee2000), examined linking from a multidimensional perspective which would align the RC’s for two distinct groups, before adjusting for scale differences.

It should be further noted that the Mantel–Haenszel accomplishes rescaling by including the studied item in the calculation of the total score. Sibtest calculations utilize a regression correction. It is important to recognize that these rescaling techniques work optimally when the RC’s angular directions are similar. Because the RC direction can vary for different content subsets of items, Shealy and Stout (Reference Shealy, Stout, Holland and Wainer1993a, Reference Shealy and Stout1993b) recommend that practitioners identify valid test items to ensure the conditioning score (i.e., RC) represents a valid composite. As the number of items on a test increases, (say > 40), the influence of any one item decreases. Usually, the RC lies within the validity sector.

Additional research by Ackerman and Xie (Reference Ackerman and Xie2019) compared Camilli’s approach two other two unidimensional approaches to explore how well they capture the representation of two-dimensional latent ability space. Carlson (Reference Carlson2017) and Strachan et al. (Reference Strachan, Ip, Fu, Ackerman, Chen and Willse2020) conducted research in which the RC is nonlinear when there is a confounding of difficulty and dimensionality. Additionally, Ma et al. (Reference Ma, Ackerman, Ip and Chung2023) evaluated the efficiency of DIF detection using both Camilli’s approach and the projective IRT approach (Ip, 2010) when potential DIF items fell outside the validity sector.

Figure. 7 RCs for two groups having different underlying ability distributions based on a two-item test (left) and orthogonal mappings upon the composites for two examinees, X and Y, (right).

Figure. 8 RCs for the 101-item test for the three subsections and the total test.

The contour plot of the standardized 101-item test is displayed in Fig. 8. Within this plot are the RCs for the three individual content areas, as well as the total test. The RCs align with the direction of the sector containing content vectors. Interestingly, the wide angular range of the content RCs results in curved contours for higher score categories, resembling patterns seen in the noncompensatory model (10).

2.5. Centipede Plot: Graphically Mapping the Two-Dimensional IRT Latent Ability Space Onto the Unidimensional IRT Scale

This RC enables one to create an interesting visualization of how the two-dimensional latent ability plane gets mapped onto the unidimensional ability scale. This mapping can be illustrated by a “centipede” plot. In this type of plot, the compensatory model (6) test characteristic curve is first drawn in the RC-direction. Then, vectors are drawn from the generated $\left({\theta }_1,,,{\theta }_2\right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\left(\theta _{1},\theta _{2} \right) $$\end{document} to their expected score, obtained using the estimated $\widehat{\theta .}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\hat{\theta . }$$\end{document} Figure 9 displays two perspectives of a centipede plot for a 40-item test: one from a side view and another from an overhead view. The vertical axis represents the proportion correct true score. Vectors are displayed for a sample of 200 examinees. It is informative to observe which $\left({\theta }_1,,,{\theta }_2\right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\left(\theta _{1},\theta _{2} \right) $$\end{document} -combinations map onto the same proportion correct true scores. This information helps psychometricians and test developers explore how regions of the latent ability space with opposite $\left({\theta }_1,,,{\theta }_2\right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\left(\theta _{1},\theta _{2} \right) $$\end{document} -profiles, (high, low) vs (low, high), get mapped onto the same conditioning number correct score and thus will be in the same 2x2 contingency table often used in DIF approaches such as the Mantel–Haenszel.

Figure. 9 Two perspectives illustrating the mapping of the two-dimensional latent abilities onto the expected number correct score scale.

3.1. Illustration of Changes in a Two-Dimensional Latent Ability Distribution can Affect $\hat{a}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\hat{a}$$\end{document} and $\hat{b}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\hat{b}$$\end{document} Values Using Camilli’s Formulation

Using (6) and (7), one can observe how changes in the underlying two-dimensional distribution of an examinee population impact the direction of the RC and consequently affect estimated a-parameters. As an item’s vector angle approaches the angle of the RC, the estimated unidimensional $\hat{a}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\hat{a}$$\end{document} -parameter increases. Conversely, when an item’s vector angle deviates from the RC’s angular direction, the estimated $\hat{a}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\hat{a}$$\end{document} becomes smaller. For illustrative purposes, consider a generated test of 19 items. The vectors of these items span angles in 5 ${}^{\circ }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ $$\end{document} increments from 0 ${}^{\circ }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ $$\end{document} to 90 ${}^{\circ }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ $$\end{document} . Assume all items have an MDISC value of 1.5 and an MDIFF value of $-$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$-$$\end{document} 1.0. (Note in (6) the d-parameter is added in the logit, thus these would be considered to all be difficult items.) The vectors for this test are shown in Fig. 11. These parameters were selected for illustration purposes only and would never reflect an actual test. In most cases, real tests have more items and item vectors typically lie within a relatively narrow validity sector (e.g., 30 ${}^{\circ }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ $$\end{document} -45 ${}^{\circ })$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ )$$\end{document} . Only in extreme distributional cases would the RC be pulled out of the validity sector.

Figure. 11 Item vectors for a hypothetical 19-item test.

Using (6), $\hat{a}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\hat{a}$$\end{document} values were calculated for all items under four different two-dimensional latent ability distributional conditions: $\text{N}\left(\left(\begin{array}{c}\text{0}\\ \text{0}\end{array}\right),\text{,},\left(\begin{array}{cc}\text{1}& .0\\ .0& \text{1}\end{array}\right)\right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{N}\left[ \left({\begin{array}{*{20}c} \textrm{0}\\ \textrm{0}\\ \end{array} } \right) \textrm{,}\left({\begin{array}{*{20}c} \textrm{1} &{} \mathrm {.0}\\ \mathrm {.0} &{} \textrm{1}\\ \end{array} } \right) \right] $$\end{document} , $\text{N}\left(\left(\begin{array}{c}\text{0}\\ \text{0}\end{array}\right),\text{,},\left(\begin{array}{cc}\text{1}& .5\\ .5& \text{1}\end{array}\right)\right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ \textrm{N}\left[ \left({\begin{array}{*{20}c} \textrm{0}\\ \textrm{0}\\ \end{array} } \right) \textrm{,}\left({\begin{array}{*{20}c} \textrm{1} &{} \mathrm {.5}\\ \mathrm {.5} &{} \textrm{1}\\ \end{array} } \right) \right] $$\end{document} , $\text{N}\left(\left(\begin{array}{c}\text{0}\\ \text{0}\end{array}\right),\text{,},\left(\begin{array}{cc}\text{1}& .0\\ .0& .5\end{array}\right)\right),\text{N}\left(\left(\begin{array}{c}\text{0}\\ \text{0}\end{array}\right),\text{,},\left(\begin{array}{cc}.5& .0\\ .0& \text{1}\end{array}\right)\right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{N}\left[ \left({\begin{array}{*{20}c} \textrm{0}\\ \textrm{0}\\ \end{array} } \right) \textrm{,}\left({\begin{array}{*{20}c} \textrm{1} &{} \mathrm {.0}\\ \mathrm {.0} &{} \mathrm {.5}\\ \end{array} } \right) \right] , \textrm{N}\left[ \left({\begin{array}{*{20}c} \textrm{0}\\ \textrm{0}\\ \end{array} } \right) \textrm{,}\left({\begin{array}{*{20}c} \mathrm {.5} &{} \mathrm {.0}\\ \mathrm {.0} &{} \textrm{1}\\ \end{array} } \right) \right] $$\end{document} . The RC-angles for these cases are 45 ${}^{\circ }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ $$\end{document} , 52.15 ${}^{\circ }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ $$\end{document} , 60.22 ${}^{\circ }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ $$\end{document} ; and 22.78 ${}^{\circ }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ $$\end{document} , respectively. As the correlation increased, the RC began to shift in the 45 ${}^{\circ }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ $$\end{document} direction. When the variances of the two groups are unequal, the RC shifts toward the axis of the ability with the greater variance. The results are displayed in Fig. 12 (top). For each of the four distributional conditions, the $\hat{a}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\hat{a}$$\end{document} ’s tend to approach the MDISC value (4) as an item’s angular direction aligns more closely with that of the RC. These trends are depicted in the plot by the respective colored vertical lines. The range of $\hat{a}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\hat{a}$$\end{document} was (.52 (0 ${}^{\circ }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ $$\end{document} ) to.66(45 ${}^{\circ }\left)\right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ ))$$\end{document} , (.44(0 ${}^{\circ })$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ )$$\end{document} to.67(60 ${}^{\circ }\left)\right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ ))$$\end{document} , (.34(0 ${}^{\circ })$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ )$$\end{document} to.72(75 ${}^{\circ }\left)\right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ ))$$\end{document} , and (.34(90 ${}^{\circ })$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ )$$\end{document} to.72(15 ${}^{\circ }\left)\right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ ))$$\end{document} for the four respective conditions.

Additionally, two more conditions were considered in the calculation of $\hat{b}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\hat{b}$$\end{document} : $N\left(\left(\begin{array}{c}\text{1}\\ \text{0}\end{array}\right),\text{,},\left(\begin{array}{cc}\text{1}& .0\\ .0& \text{1}\end{array}\right)\right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$N\left[ \left({\begin{array}{*{20}c} \textrm{1}\\ \textrm{0}\\ \end{array} } \right) \textrm{,}\left({\begin{array}{*{20}c} \textrm{1} &{} \mathrm {.0}\\ \mathrm {.0} &{} \textrm{1}\\ \end{array} } \right) \right] $$\end{document} and $N\left(\left(\begin{array}{c}\text{0}\\ \text{1}\end{array}\right),\text{,},\left(\begin{array}{cc}\text{1}& .0\\ .0& \text{1}\end{array}\right)\right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$N\left[ \left({\begin{array}{*{20}c} \textrm{0}\\ \textrm{1}\\ \end{array} } \right) \textrm{,}\left({\begin{array}{*{20}c} \textrm{1} &{} \mathrm {.0}\\ \mathrm {.0} &{} \textrm{1}\\ \end{array} } \right) \right] $$\end{document} . Although mean differences do not affect the RC or $\hat{a}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\hat{a}$$\end{document} , they do affect $\hat{b}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\hat{b}$$\end{document} , along with changes in variances and correlations. Figure 12 (bottom) shows how the estimated difficulties change as the items’ measurement angles vary in reference to the RC (denoted by the colored vertical lines). The range of $\hat{b}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\hat{b}$$\end{document} was 1.41(0 ${}^{\circ }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ $$\end{document} ) to.99(45 ${}^{\circ }\left)\right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ ))$$\end{document} , 1.62 (0 ${}^{\circ })$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ )$$\end{document} to 1.00(50 ${}^{\circ })$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ )$$\end{document} , 2.01(0 ${}^{\circ })$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ )$$\end{document} to 1.00 (60 ${}^{\circ })$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ )$$\end{document} , $-$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$-$$\end{document} 2.01(90 ${}^{\circ })$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ )$$\end{document} to $-$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$-$$\end{document} 1.00 (30 ${}^{\circ })$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ )$$\end{document} , $-$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$-$$\end{document} 2.82(0 ${}^{\circ })$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ )$$\end{document} to $-$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$-$$\end{document} 1.41(90 ${}^{\circ })$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ )$$\end{document} and $-$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$-$$\end{document} 2.82(90 ${}^{\circ })$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ )$$\end{document} to $-$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$-$$\end{document} 1.41(0 ${}^{\circ })$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ )$$\end{document} for the six respective conditions. As an item’s angular direction approaches the angle of the RC, the closer the $\hat{b}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\hat{b}$$\end{document} value will be to the item’s MDIFF value (5).

Figure. 12 Estimated a-(top) and b values (bottom) for different underlying ability distributions.

Figure 13 presents a composite graph containing the 19 M2PL response surfaces and the RC plane (depicted in green, top left panel). The RC plane intersects the latent probability space at a 45 ${}^{\text{o}}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{\textrm{o }}$$\end{document} angle from the ${\theta }_1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta _{1}$$\end{document} axis. Additionally, the 19 ICCs are graphed in the RC plane (right panel). The top right panel shows the test characteristic surface and the corresponding contours, marked by the reference composite (indicated by the red arrow). Drawn in blue is the test characteristic curve (sum of the estimated 19 unidimensional ICCs using (4) and (9). This illustrates how closely the analytical estimates of a and b align with the two-dimensional model. Notably, the unidimensional TCC is not as steep as the TCS, indicating that the discrimination parameters may be underestimated. Wang (1986) previously compared 2PL estimates derived from generated and real data with estimates using the two-dimensional compensatory model. However, more research needs to be done in this area.

At the bottom of Fig. 13 are the 19 ICCs. The red ICCs represent vectors furthest from the 45 ${}^{\circ }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ $$\end{document} and have the lowest $\hat{a}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\hat{a}$$\end{document} values or flattest ICCs. The green ICCs correspond to item vectors with angles closest to the RC angle that have the largest $\hat{a}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\hat{a}$$\end{document} values or steepest ICCs.

Figure. 13 Composite graph illustrating the 19 M2PL item response surfaces and green RC plane (top left); the test characteristic surface, contour, reference composite (red arrow), and unidimensional TCC (top right); and, the 19 estimated unidimensional ICCs colored by item vector angle (bottom). (Color figure online)

In the next sections, three studies are examined. Each study provides a different perspective on how DIF can occur when response data are two-dimensional. The goal is to provide further insights for testing practitioners enabling them to conduct more informed DIF analyses and better understand the underlying causes when DIF occurs.

4.1. Case 1: Unequal ${\mu }_{\theta 1}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mu _{\theta 1}$$\end{document} - and Unequal ${\mu }_{\theta 1}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mu _{\theta 1}$$\end{document} Values: Uniform Bias

This case has two parts, one in which the Reference and Focal groups have mean differences in ${\theta }_1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta _{1 }$$\end{document} and mean differences in ${\theta }_2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta _{2}$$\end{document} . ECD differences were 2 and $-$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$-$$\end{document} 2, respectively. There were no $\hat{a}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\hat{a}$$\end{document} differences. However, when the Reference group had the greater ${\theta }_{\text{1}}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta _{\textrm{1}}$$\end{document} -mean, Item 10 was much easier for the Reference group with $\hat{b}=1.78$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\hat{b} = 1.78 $$\end{document} compared to $\hat{b}=-.24$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\hat{b} = -.24 $$\end{document} for the Focal group. Conversely, the Reference group had the greater mean for ${\theta }_2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta _{2}$$\end{document} , the Reference group’s $\hat{b}=-2.30$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\hat{b} = -2.30$$\end{document} versus $\hat{b}=1.73$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\hat{b} = 1.73$$\end{document} for the Focal group. Notably, the RC-angle was 26.68 ${}^{\text{o}}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{\textrm{o }}$$\end{document} for both groups. Since these examples resulted in only $\hat{b}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\hat{b}$$\end{document} differences, they were an indication of uniform DIF.

4.2. Case 2: Unequal $\eta$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\eta $$\end{document} Variance: Nonuniform Bias

In Case 2a, the Reference group has ${\sigma }_{{\theta }_1}^2=$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sigma _{\theta _{1}}^{2}=$$\end{document} 2.5 and ${\sigma }_{{\theta }_2}^2=$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sigma _{\theta _{2}}^{2}=$$\end{document} 1. and $\rho =.4,$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\rho =.4,$$\end{document} whereas the Focal group has ${\sigma }_{{\theta }_1}^2=$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sigma _{\theta _{1}}^{2}=$$\end{document} .5 and ${\sigma }_{{\theta }_2}^2=$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sigma _{\theta _{2}}^{2}=$$\end{document} 1.0 and $\rho =.4.$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\rho =.4.$$\end{document} Note that the ${\sigma }_{{\theta }_1}^2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sigma _{\theta _{1}}^{2}$$\end{document} is five times larger for the Reference group. In Case 2b, the variance differences were on ${\theta }_2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta _{2}$$\end{document} : ${\sigma }_{{\theta }_1}^2=$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sigma _{\theta _{1}}^{2}=$$\end{document} 2.5 for the Reference group and ${\sigma }_{{\theta }_2}^2=$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sigma _{\theta _{2}}^{2}=$$\end{document} .5 for the Focal group. The ECD was -.31 ${\theta }_{\text{1}}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta _{\textrm{1}}$$\end{document} and the RC angles were 16.40 ${}^{\circ }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ $$\end{document} and 42.05 ${}^{\circ }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ $$\end{document} for the Focal group and 37.78 ${}^{\circ }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ $$\end{document} and 18.23 ${}^{\circ }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ $$\end{document} for the Reference group for Case 2a and Case 2b, respectively. In both of these cases, both $\hat{a}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\hat{a}$$\end{document} and $\hat{b}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\hat{b}$$\end{document} differed between the two groups resulting in nonuniform DIF.

4.3. Case 3: Unequal Correlations: Nonuniform Bias

In this case, the ${\theta }_1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta _{1}$$\end{document} - and ${\theta }_2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta _{2}$$\end{document} means and variances are equal but the correlations differ: Reference ${\rho }_{\theta 1,\theta 2}=.8$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\rho _{{\uptheta 1,\uptheta 2}} =.8$$\end{document} and Focal ${\rho }_{\theta 1,\theta 2}=.2.$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\rho _{{\uptheta 1,\uptheta 2}} =.2.$$\end{document} The expected difference is $E\left({\eta }_R,|\theta \right)-E\left({\eta }_F,|\theta \right)=$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$E\left[ \eta _{R}\vert \theta \right] -E\left[ \eta _{F}\vert \theta \right] =$$\end{document} .6 $\theta$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta $$\end{document} . The RC-angles were 16.17 ${}^{\circ }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ $$\end{document} for the Focal group and 40.67 ${}^{\text{o}}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{\textrm{o }}$$\end{document} for the Reference group. Again, both $\hat{a}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\hat{a}$$\end{document} and $\hat{b}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\hat{b}$$\end{document} differed between the groups indicating nonuniform DIF.

Using the software package Mathematica (Wolfram, 2020), Ackerman and Xie (Reference Ackerman and Xie2019) created a DIF Graphical Simulator. This simulator allows researchers to change the underlying two-dimensional latent distributions for the Reference and Focal groups as well as the M2PL item parameters for a given suspect item. A graphical example is given in Appendix C.

5.1. Background: Mantel–Haenszel and Sibtest DIF Detection Methods

When calculating the MH-statistic for item i, we consider two groups of examinees: the Reference group and the Focal group. The Focal (F) group typically represents a minority group (e.g., Hispanic examinees). The Reference (R) group is frequently a nonminority group (e.g., White examinees). Examinees from each group are matched based on their number correct score.

For each score category j, a 2 x 2 contingency table (Table 4) is created. This table notes the frequency of correct and incorrect answers for each group, along with the marginal and total frequencies. It is tacitly assumed that examinees in the same contingency table are matched on their latent abilities, that they used to respond to the item being examined. The centipede plot above (Fig. 9) illustrates how even though examinees may have the same number correct score, they could have quite different two-dimensional latent ability profiles.

Table 4 A 2 x 2 contingency used in the MH computation.

Summing over the contingency tables for item i and using a continuity correction, the MH statistic is calculated as

$\begin{array}{c}{\text{MH}}_i=\frac{{\left(\left({\sum }_j,A_j-{\sum }_{j}E\left(A_j\right)\right),-,\frac{1}{2}\right)}^2}{{\sum }_{j}Var\left(A_j\right)},\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \hbox {MH}_{i}=\frac{\left[ \left| \sum \nolimits _j {A_{j}-\sum \nolimits _j {E\left(A_{j} \right) } } \right| -\frac{1}{2} \right] ^{2}}{\sum \nolimits _j {Var\left(A_{j} \right) } }, \end{aligned}$$\end{document}

where the expected value of Cell A frequency is given as

$\begin{array}{c}E\left(A_j\right)=\frac{N_R{N}_{1.j}}{N_{..j}}\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} E\left(A_{j} \right) =\frac{N_{R}N_{1.j}}{N_{..j}} \end{aligned}$$\end{document}

and the variance of cell A frequencies equals

$\begin{array}{c}Var\left(A_j\right)=\frac{N_{\mathrm{Rj}}{N}_{\mathrm{Fj}}{N}_{1.j}{N}_{0.j}}{{\left(N_{..j}\right)}^2\left(N_{..j},-,1\right)}\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} Var\left(A_{j} \right) =\frac{N_{Rj}N_{Fj}N_{1.j}N_{0.j}}{\left(N_{..j} \right) ^{2}\left(N_{..j}-1 \right) } \end{aligned}$$\end{document}

Typically, the MH statistic is used to test the null hypothesis that for each raw score category j the odds of a Reference group examinee answering the item correctly equals the odds that a Focal group examinee will answer the item correctly (Holland et al., Reference Holland and Thayer1988). That is, if $p_{\mathrm{Rj}}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$p_{Rj}$$\end{document} and $p_{\mathrm{Fj}}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$p_{Fj}$$\end{document} ; are the probabilities of a Reference and Focal group examinee answering the item correctly, respectively, and $q_{\mathrm{Rj}}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$q_{Rj} $$\end{document} and $q_{\mathrm{Fj}}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$q_{Fj}$$\end{document} are the probabilities of a Reference and Focal group examinee answering the item incorrectly, respectively,

$\begin{array}{c}{H}_o:\frac{p_{\mathrm{Rj}}}{q_{\mathrm{Rj}}}=\frac{p_{\mathrm{Fj}}}{q_{\mathrm{Fj}}}\text{j}=1,\dots ,\text{K}\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} H_{o}: \frac{p_{Rj}}{q_{Rj}}=\frac{p_{Fj}}{q_{Fj}}\, \textrm{j}=1, \ldots , \textrm{K} \end{aligned}$$\end{document}

is tested against the alternative of uniform DIF,

$\begin{array}{c}{H}_1:\frac{p_{\mathrm{Rj}}}{q_{\mathrm{Rj}}}=\alpha \frac{p_{\mathrm{Fj}}}{q_{\mathrm{Fj}}}\alpha \ne 1,j=1,\dots ,K\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} H_{1}: \frac{p_{Rj}}{q_{Rj}}=\alpha \frac{p_{Fj}}{q_{Fj}} \alpha \ne 1, j=1, \ldots , K \end{aligned}$$\end{document}

where $H_o$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$H_{o}$$\end{document} is the null hypothesis, $H_1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$H_{1}$$\end{document} is the alternative hypothesis, and $\alpha$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha $$\end{document} is the common odds ratio in the K 2 x 2 tables. Uniform DIF occurs when the rescaled, unidimensional item response functions differ only in difficulty. When $H_o$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$H_{o}$$\end{document} is true, MH is distributed as ${\chi }^{\text{2}}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^{\textrm{2}}$$\end{document} with 1 degree of freedom. It should be noted that for a given examinee, the score (i.e., 0 vs 1) on the item being examined is part of the conditioning score.

DIF according to (Shealy and Stout, Reference Shealy, Stout, Holland and Wainer1993a, Reference Shealy and Stoutb), should be conceptualized by examining the difference in certain marginal item characteristic curves for the two groups of interest:

$\begin{array}{c}P\left(X_i=1,|\Theta =\theta \right)\int P_i\left(\theta ,,,\eta \right)f\left(\eta ,|,\theta \right)\text{d}\eta ,\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} P\left({X_{i}=1}\vert {\varTheta =\theta }\right) \int P_{i} \left[ \theta ,\eta \right] f\left(\eta \vert \theta \right) \textrm{d}\eta , \end{aligned}$$\end{document}

where $P_i\left(\theta ,,,\eta \right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$P_{i}\left[ \theta ,\eta \right] $$\end{document} is the M2PL model (Eq 1) and $f\left(\eta \right|\theta )$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$f(\eta \vert \theta )$$\end{document} is a specified group’s conditional distribution of the nuisance dimension, $\eta$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\eta $$\end{document} , given a fixed value of $\theta$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta $$\end{document} , the target ability. That is, for a fixed value of $\theta$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta $$\end{document} , $P_i(X_i=1|\Theta =\theta )$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$P_{i}(X_{i}=1\vert \varTheta =\theta )$$\end{document} is obtained by averaging $P_i$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$P_{i}$$\end{document} [ $\theta ,\eta$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta , \eta $$\end{document} ] over $\eta$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\eta $$\end{document} . That is, $P_i(X_i=1|\Theta =\theta )$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$P_{i}(X_{i}=1\vert \varTheta =\theta )$$\end{document} is the ICC if the differences in the nuisance direction are integrated out. Y*

An estimate of the SIBTEST test statistic is given as

$\begin{array}{c}{\hat{\beta }}_U=\sum _{h=0}^n{\hat{p}}_h\left({\overline{\text{Y}}}_{\mathrm{Rh}},-,{\overline{Y}}_{\mathrm{Fh}}\right)\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \hat{\beta }_{U}=\sum \limits _{h=0}^n \hat{p}_{h} \left(\bar{{\textrm{Y}}}_{ Rh}-\bar{Y}_{Fh} \right) \end{aligned}$$\end{document}

where

$\begin{array}{c}{\hat{p}}_h=\frac{\left(G_{\mathrm{Rh}},-,G_{\mathrm{Fh}}\right)}{{\sum }_{j=0}^n\left(G_{\mathrm{Rh}},-,G_{\mathrm{Fh}}\right)},\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \hat{p}_{h}=\frac{\left(G_{Rh}-G_{Fh} \right) }{\mathop {\sum }\nolimits _{j=0}^n \left(G_{Rh}-G_{Fh} \right) } , \end{aligned}$$\end{document}

and G ${}_{\text{Rh}}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$_{\textrm{Rh}}$$\end{document} and G ${}_{\text{Fh}}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$_{\textrm{Fh}}$$\end{document} are the number of examinees in the Reference and Focal groups at the valid score X $=$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$=$$\end{document} h. The Sibtest test statistic is computed as

$\begin{array}{c}{\beta }_U=\frac{{\hat{\beta }}_U}{\hat{\sigma }\left({\hat{\beta }}_U\right)}.\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \beta _{U}=\frac{\hat{\beta }_{U}}{\hat{\sigma }(\hat{\beta }_{U})}. \end{aligned}$$\end{document}

where the standard deviation in the denominator is calculated as

$\begin{array}{c}\hat{\sigma }\left({\hat{\beta }}_U\right)={\left(\sum _{h=0}^n,{\hat{p}}_k^2\left(\frac{1}{G_{\mathrm{Rh}}},{\hat{\sigma }}^2,\left(Y,|,h,,,R\right),+,\frac{1}{G_{\mathrm{Fh}}},{\hat{\sigma }}^2,\left(Y,|,h,,\right)\right)\right)}^2,\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \hat{\sigma }(\hat{\beta }_{U})=\left\{ \sum \limits _{h=0}^n {\hat{p}_{k}^{2}\left[ \frac{1}{G_{Rh}}\hat{\sigma }^{2}\left(Y\vert h,R \right) +\frac{1}{G_{Fh}}\hat{\sigma }^{2}\left(Y\vert h, \right) \right] } \right\} ^{2}, \end{aligned}$$\end{document}

The test statistic has an approximate N(0,1) distribution when no DIF is present. Unlike the MH statistic, an examinee’s score on the studied item is not part of the conditioning score. Sibtest resolves rescaling issues by means of a regression correction (Shealy and Stout, Reference Shealy, Stout, Holland and Wainer1993a, Reference Shealy and Stoutb).

5.2. DIF Detection with Different Conditioning Scores

Using the Mantel–Haenszel and Sibtest statistics, Ackerman and Evans (Reference Ackerman and Evans1994) examined the impact of different conditioning scores on DIF results. Specifically, they looked at generated two-dimensional compensatory data where $\theta$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta $$\end{document} was the valid skill and $\eta$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\eta $$\end{document} represented the invalid skill. The testing scenario involved a 30-item test measuring ( $\theta$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta $$\end{document} , $\eta )$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\eta )$$\end{document} -composites spanning measurement angles from 0 ${}^{\circ }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ $$\end{document} to 90 ${}^{\circ }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ $$\end{document} , in 3 ${}^{\circ }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ $$\end{document} increments. A vector of these items is displayed in Fig. 15. All items had a difficulty parameter ( $d_i)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$d_{i})$$\end{document} value of zero and an MDISC value of 1.5. The Reference and Focal groups had different latent ability distributions. The bivariate normal distributions for the Reference and Focal were $N\left(\left(\begin{array}{c}\text{1}\\ -1\end{array}\right),\text{,},\left(\begin{array}{cc}\text{1}& .4\\ .4& \text{1}\end{array}\right)\right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$N\left[ \left({\begin{array}{*{20}c} \textrm{1}\\ \mathrm {-1}\\ \end{array} } \right) \textrm{,}\left({\begin{array}{*{20}c} \textrm{1} &{} \mathrm {.4}\\ \mathrm {.4} &{} \textrm{1}\\ \end{array} } \right) \right] $$\end{document} and $N\left(\left(\begin{array}{c}-1\\ \text{1}\end{array}\right),\text{,},\left(\begin{array}{cc}\text{1}& .4\\ .4& \text{1}\end{array}\right)\right),$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$N\left[ \left({\begin{array}{*{20}c} \mathrm {-1}\\ \textrm{1}\\ \end{array} } \right) \textrm{,}\left({\begin{array}{*{20}c} \textrm{1} &{} \mathrm {.4}\\ \mathrm {.4} &{} \textrm{1}\\ \end{array} } \right) \right] ,$$\end{document} respectively. Three different sample size pairings were used but results were similar for each pairing. The results shown here are for the pairing N ${}_{\text{Ref}}=$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$_{\textrm{Ref }}=$$\end{document} 1000, and N ${}_{\text{Foc}}=500$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$_{\textrm{Foc}} = 500$$\end{document} .

The purpose of comparing the four different conditioning scores is to illustrate that DIF can occur when one has not accounted for the complete latent ability space. Here are the details for each conditioning variable:

1. – To condition on $\theta$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta $$\end{document} , the transformation used is X ${}_{\theta }=10\left(\theta \right)+25$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$_{{\uptheta }} = 10(\theta ) + 25$$\end{document} . In this case, the ability $\eta$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\eta $$\end{document} is not accounted for and DIF should increase the more the ability $\eta$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\eta $$\end{document} is required (i.e., as the angle of the item vector increases toward 90 ${}^{\circ })$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ )$$\end{document} .

2. – To condition on $\eta$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\eta $$\end{document} , the transformation used is X ${}_{\eta }=10\left(\eta \right)+25$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$_{{\upeta }} =10(\eta ) + 25$$\end{document} was used. When conditioning on $\eta$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\eta $$\end{document} the ability $\theta$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta $$\end{document} is not accounted for, and DIF should increase the more the ability $\eta$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\eta $$\end{document} is required (i.e., as the angle of the item vector decreases toward 0 ${}^{\circ })$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ )$$\end{document} .

3. – The number correct score is equivalent to the case where $\theta =\eta$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta = \eta $$\end{document} , (i.e., the RC angle is at 45 ${}^{\circ }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ $$\end{document} ). Items that require an equal weighting of both skills, (i.e., $a_1=a_{2)}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$a_{1} = a_{2)}$$\end{document} , should show no DIF. However, DIF should increase as items require more of $\theta$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta $$\end{document} -skill (item vectors approach 0 ${}^{\circ })$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ )$$\end{document} or more of $\eta$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\eta $$\end{document} (items approach 90 ${}^{\circ })$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ )$$\end{document} .

4. – Finally, to condition on ( $\theta$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta $$\end{document} , $\eta )$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\eta )$$\end{document} the latent ability plane was divided into 64-square regions using an 8 x 8 grid (Fig. 16). All examinees in the same square of the grid were assigned the same conditioning score, (i.e., examinees in the square -3 $\le$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\le $$\end{document} ${\theta }_{\text{1}}<-2.25$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta _{\textrm{1}} < -2.25$$\end{document} and $2.25<\eta \le$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$2.25 < \eta {\le }$$\end{document} 3 would be assigned a conditioning score of 1). Note that the conditioning score does not enter the calculation of either DIF statistic, but rather ensures that examinees with the same abilities are placed into the same 2 x 2 conditioning table. When the conditioning score is a function of ( $\theta$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta $$\end{document} , $\eta )$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\eta )$$\end{document} , the complete latent ability is accounted for and there should be no DIF.

Figure. 15 Item vectors for the 30-item symmetric test and conditioning score composite directions.

Figure. 16 Overlaying an 8 x 8 grid on the ( $\theta$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta $$\end{document} , $\eta )$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\eta )$$\end{document} latent ability plane with Reference (red) and Focal (green) underlying and marginal distributions. (Color figure online)

Results of the DIF analyses using the four different conditioning scores are illustrated in Fig. 17. When the conditioning score was the number correct, both DIF procedures consistently identified items 1–9 and 22–30 as showing DIF 100% of the time. When the conditioning score was a linear transformation of the generating ${\theta }_{\text{,}}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta _{\textrm{, }}$$\end{document} items 9–30 were consistently rejected 100% of the time. Slight differences were observed between MH and Sibtest ${\beta }_U$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\beta _{U}$$\end{document} results. ${\beta }_U$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\beta _{U}$$\end{document} appeared to be more sensitive to DIF. Its rejection rate increased faster than MH as the angular composite of the item deviated from 0°. This is shown by the fact that the rejection rate reached or exceeded .9 by Item 7 for ${\beta }_U$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\beta _{U}$$\end{document} , whereas it did not occur until Item 9 for MH. Note that rather than using the Sibtest regression correction for ${\beta }_U$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\beta _{U}$$\end{document} , the conditioning variable was based on a latent trait parameter.

Figure. 17 MH and SIBTEST DIF results by item for each of the four condition scores.

As hypothesized, the opposite results occurred when the valid test direction was along the $\eta$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\eta $$\end{document} -axis. That is, items measuring $\theta$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta $$\end{document} (i.e., Items 1–23) consistently exhibited DIF in favor of the Reference group. For the final analysis, the 64 score categories matching examinees on both $\theta$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta $$\end{document} and $\eta$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\eta $$\end{document} were used as the conditioning scores. The results, as shown in Fig. 19, showed no DIF for any of the 30 items, regardless of the DIF procedure. However, it is important to recognize that this purely hypothetical testing situation would not occur in practice. Its purpose was to illustrate the significance of underlying group distributional differences interacting with items that measure a spread of two-dimensional composite skills. Identifying the specific composite skills being measured remains a genuine challenge for testing practitioners. Several studies have examined using multiple conditioning scores, such as those by Clauser et al. (Reference Clauser, Nungester and Swaminathan1996) and Mazor et al. (Reference Mazor, Hambleton and Clauser1998), in an attempt to condition on the complete latent ability space.

6.1. The Two-Dimensional Noncompensatory MIRT Model

Up to this point, the discussion has centered around how DIF can occur when items measure invalid skill composites and the two groups of interest have different ability distributions related to the invalid skill. Interestingly, DIF can also occur when the underlying two-dimensional ability distributions are identical and the vectors corresponding to the test items lie in a very narrow validity sector (e.g., a unidimensional test). This phenomenon was examined by Ackerman and Evans (Reference Ackerman and Evans1994), Bolt and Johnson (Reference Bolt2009), and Ackerman et al. (Reference Ackerman, McCallaum and Ngerano2014). They found that DIF can occur when a two-dimensional test contains items for which different groups of students use distinct approaches to solve the same problem. These divergent solution strategies could occur due to pedagogical differences in how students were taught to information, particularly in items such as “story” problems. For instance, one group might have been explicitly taught to combine pieces of information, whereas another group was not instructed to integrate or combine these pieces.

In Ackerman and Evans (Reference Ackerman and Evans1994) and Ackerman et al. (Reference Ackerman, McCallaum and Ngerano2014), the different strategies were are modeled using two distinct MIRT models. The integration strategy was modeled using the compensatory model (6), and the nonintegration strategy was modeled using the MIRT noncompensatory model developed by Sympson (1978). This does not allow for compensation and can be expressed as

(11) $\begin{array}{c}{P}_{\mathrm{NC}}\left(u_{\mathrm{ij}}=1,|,{\theta }_{1j},{\theta }_{2j},a_{1i},a_{2i},b_{1i},b_{1i}\right)=\left(\frac{1.0}{1.0+e^{\left(a_{1i},(,{\theta }_{1j},-,b_{1i}\right)}}\right)\left(\frac{1.0}{1.0+e^{\left(a_2,(,{\theta }_{2j},-,b_{2i}\right)}}\right).\\ \end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} P_{NC}\left({u_{ij}=1}\vert {\theta _{1j},\theta _{2j},a_{1i},a_{2i},b_{1i},b_{1i}}\right) =\left[ \frac{1.0}{1.0+e^{\left(a_{1i}(\theta _{1j}-b_{1i} \right) }}\right] \left[ \frac{1.0}{1.0+e^{\left(a_{2}(\theta _{2j}-b_{2i} \right) }}\right] .\nonumber \\ \end{aligned}$$\end{document}

This model is essentially the product of two 2PL (1) models, with a discrimination and difficulty parameter for each dimension. Unlike the compensatory model (6) which assumes the abilities from different dimensions can compensate for each other, the noncompensatory model treats them independently. P ${}_{\text{NC}}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$_{\textrm{NC}}$$\end{document} ’s multiplicative nature ensures that P ${}_{\text{NC}}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$_{\textrm{NC }}$$\end{document} can never be larger than the maximum value of either dimension’s 2PL model.

Figure. 18 Contour and difference plots for a matched noncompensatory and compensatory item.

The multiplicative nature of this model causes the response surface equiprobability contours to become curved. That is, unlike the compensatory model contours which are always parallel lines, the noncompensatory model contours are always parallel curves, the larger the a values, the more discriminating the item and the closer together the equiprobability curves. A contour plot of a noncompensatory response surface where $a_1=a_2=1.6$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$a_{1} = a_{2} = 1.6$$\end{document} and $b_1=b_2=-.47$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$b_{1} = b_{2} = -.47$$\end{document} is displayed in the left panel in Fig. 18. In the left panel, the letters A, B, and C denote three different ( ${\theta }_1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta _{1}$$\end{document} , ${\theta }_2)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta _{2})$$\end{document} -profiles, (high, low), (low, low), and (low, high), respectively. Notice that all lie on the same equiprobability contour or have the same probability of correct response.

6.2. DIF Study Simulation Using Matched Compensatory and Noncompensatory Items

A 30-item test was created where the primary focus was on measuring ${\theta }_1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta _{1}$$\end{document} and ${\theta }_2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta _{2}$$\end{document} equally (i.e., the item vectors were enclosed in a narrow validity sector from 40 ${}^{\circ }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ $$\end{document} to 50 ${}^{\circ }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ $$\end{document} .) For the Reference group, all 30 items were modeled using the compensatory model (6). For the Focal group, for items 1–10 and 15–30 the probability of correct response followed the compensatory model, but for items 11–14, the probability of correct response was determined using the noncompensatory model (11) that matched their compensatory counterparts. To estimate noncompensatory item parameters that would match the compensatory for items 11–14, the approach proposed by Spray et al. (Reference Spray, Davey, Reckase, Ackerman and Carlson1990) was used. Specifically, PNC parameters were estimated by minimizing the function

$\begin{array}{c}\sum _{i=1}^N{\left\{[P_C\left({\theta }_i,,,a,,,d\right)-P_{\mathrm{NC}}({\theta }_i,\hat{a},\hat{b})]\right\}}^2\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \sum \limits _{i=1}^N {\{[P_{C}\left(\varvec{\theta }_{\varvec{i}},\varvec{a},d \right) -P_{NC}(\theta _{i},\hat{{\varvec{a}}},\hat{b})] \}}^{2} \end{aligned}$$\end{document}

for 2000 randomly generated examinee abilities from the latent underlying bivariate normal distribution, $N\left(\left(\begin{array}{c}\text{0}\\ \text{0}\end{array}\right),\text{,},\left(\begin{array}{cc}\text{1}& .4\\ .4& \text{1}\end{array}\right)\right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$N\left[ \left({\begin{array}{*{20}c} \textrm{0}\\ \textrm{0}\\ \end{array} } \right) \textrm{,}\left({\begin{array}{*{20}c} \textrm{1} &{} \mathrm {.4}\\ \mathrm {.4} &{} \textrm{1}\\ \end{array} } \right) \right] $$\end{document} . The Nminimize function in Mathematica (2020) was used for this optimization. This process was repeated for 10 replications for each of the four items to ensure that the estimates obtained were not unduly influenced by the samples selected or the starting values. The matched set of parameters for items 11–14 are displayed in Table 5. In Fig. 18, the center panel contains the equiprobability contour plot for the matched Item 14 compensatory item. The right panel displays the compensatory—noncompensatory difference contour for item 14. From this plot, it appears that the compensatory item 14 and the noncompensatory item 14 would produce similar probabilities of correct response for examinees in the first and third quadrants, but noticeably different probabilities for examinees in the third and fourth quadrants.