3.1. Setup

In our exposition, y _ijce is the position of political party i on issue j, in country c, as indicated by expert e on an ordered scale. While we focus on experts, e can be generally thought of as any coder of issue positioning. The probability $\Pr ( y_{ijce} = k)$, where the response option k = 1, ⋅ ⋅ ⋅ , K, is determined by an appropriate link function between y _ijce and the latent continuum $y_{ijce}^\ast$. The link function can be adapted to incorporate other kinds of measures such as word counts for manifestos.

Two party-specific factors are systematically associated with $y_{ijce}^\ast$, namely a party's position on D ideological dimensions, θ_ic, and a party's idiosyncratic preferences, γ _ij. The discrimination parameters of an issue in ${\mathbb R}^D$ are given by β_jc and capture the strength and direction of the association between latent ideological traits and issue positions. Importantly, the subscript c allows for cross-national variation in these associations, so that ideological conflict can play out differently at the issue-level in different locations. In keeping with random-effects IRT models, we postulate β _jcd = β _jd0 + β _jcd1, where β _jd0 is the mean and β _jcd1 is a country-issue-specific error term (cf. De Jong et al., Reference De Jong, Steenkamp and Fox2007; Fox and Verhagen, Reference Fox, Verhagen, Davidov, Schmidt, Billiet and Meuleman2018). This way of modeling the discrimination parameters constitutes one of our main contributions.

The idiosyncratic shocks are applied to the item difficulty parameters of the model. Specifically, for each party and each issue, we postulate a difficulty of γ _ij. This is an additive component that makes it more or less likely for political parties to embrace a particular position, regardless of θ_ic. Thus, two parties with identical θ_ic can still have different positions on an issue.

Two additional factors influence the relationship between latent and manifest variables, namely heteroskedasticity in the scale of experts’ errors, σ _e (cf. Harvey, Reference Harvey1976), and country-level variation in scaling, ζ _c. The expert-specific term captures variation across experts, whereas the parameter ζ _c is akin to the scale parameter in A-M models (Hare et al., Reference Hare, Armstrong, Bakker, Carroll and Poole2015) and captures cross-national differences in response behavior.

Combining everything so far, we obtain the following model:

(1)$$y_{ijce}^\ast = {( {\boldsymbol \theta}_{ic}^\top {\boldsymbol \beta}_{\,jc} + \gamma_{ij}) \zeta_c\over \sigma_e}.$$

We use an ordered probit specification to link $y_{ijce}^\ast$ and y _ijce, using a set of ordered cutpoints α _jc1, ⋅ ⋅ ⋅ , α _jcK−1 (Samejima, Reference Samejima1969). Similar to the discrimination parameters, the cutpoints vary across issues and countries. We model them as α _jck = α _jk0 + α _jc1 + α _c, where α _c is akin to the shift parameter in the A-M model, which captures country-specific shifts in responses (Hare et al., Reference Hare, Armstrong, Bakker, Carroll and Poole2015), and α _jc1 is a country-issue-specific error term. The response probabilities are now given by:

(2)$$\matrix{\Pr ( y_{ijce} = 1) & = \Phi \left(y_{ijce}^\ast - {\alpha_{\,jc1}\over \sigma_e} \right),\; \cr \Pr ( y_{ijce} = k) & = \Phi \left(y_{ijce}^\ast - {\alpha_{\,jck}\over \sigma_e} \right)- \Phi \left(y_{ijce}^\ast - {\alpha_{\,jck-1}\over \sigma_e} \right),\; \cr \Pr ( y_{ijke} = K) & = 1 - \Phi \left(y_{ijce}^\ast - {\alpha_{\,jcK}\over \sigma_e} \right),\; }$$

where Φ denotes the cumulative standard normal distribution function.

The model's innovation is to allow for variation in the discrimination parameters and thresholds. In addition, it combines research on idiosyncratic shocks (Lauderdale et al., Reference Lauderdale, Hanretty and Vivyan2018) and A-M scaling (Hare et al., Reference Hare, Armstrong, Bakker, Carroll and Poole2015). Indeed, the model generalizes the A-M approach to settings with multiple observations of indicators for units of analysis, whereas that approach is typically used with single observations of indicators for units. This allows us to estimate country-item-specific scale and shift parameters in addition to the country-specific scale and shift parameters estimable via the A-M approach.

3.2. Priors

We estimate our IRT model using Bayesian inference, as implemented in Stan (Carpenter et al., Reference Carpenter, Gelman, Hoffman, Lee, Goodrich, Betancourt, Brubaker, Guo, Li and Riddell2017), with the following priors. Parties’ positions on latent ideological dimensions are drawn from a standard normal distribution: $\theta _{icd} \sim {{\cal N}}( 0,\; 1)$. Further, $\gamma _{ij} \sim {{\cal N}}( 0,\; \sigma _{\gamma })$. Throughout, the hyperparameters σ for priors of the idiosyncratic shocks (σ _γ), the random effects on the discrimination parameters (σ _β) and cutpoints (σ _α), as well as expert-specific errors (σ _σ) and country-specific scale parameters (σ _ζ) are drawn from a truncated normal distribution, $\sigma \sim {{\cal N}_ + }( 0,\; 1)$, to ensure that σ > 0.

The mean item parameters are also drawn from normal distributions: (1) $\beta _{jd0} \sim {{\cal N}}( 0,\; 1)$ and (2) $\alpha _{jk0} \sim {{\cal N}}( 0,\; 1)$ (plus an ordering constraint). The random effects for both sets of parameters are drawn from multivariate normal distributions, specifically ${\boldsymbol \beta }_{cd1} \sim {{\cal M}VN}( {\bf 0},\; {\bf \Sigma }_{{\boldsymbol \beta }_d})$ and ${\boldsymbol \alpha }_{c1} \sim {{\cal M}VN}( {\bf 0},\; {\bf \Sigma }_{{\boldsymbol \alpha }})$. The covariance matrices here are estimated from the data by first multiplying scale parameters σ _β and σ _α with two vectors of length J drawn from uniform Dirichlet distributions (${{\cal D}}( 1)$). This results in vectors τ _β and τ _α as σ _β and σ _α are distributed across each element of the vectors constrained to sum to 1. Then, diagonal matrices with diagonal elements consisting of these vectors are multiplied with correlation matrices, Ω, drawn from LKJ-priors with shape = 4 so that, e.g., Σ_α = diag(τ _α)Ω_αdiag(τ _α) (Barnard et al., Reference Barnard, McCullogh and Meng2000). The country-specific parameters for the ordered cutpoints are drawn from a standard normal distribution: $\alpha _c \sim {{\cal N}}( 0,\; 1)$.

Finally, the scale of expert-specific errors is drawn from a symmetric Dirichlet distribution, where the hyperparameter 1/σ _σ is equal across experts: $\sigma _e \sim {{\cal D}}( 1/\sigma _{\sigma })$. This constitutes an uninformative prior over the resulting vector and assures that the resulting estimates cannot be negative. This is also the case for the country-specific scale parameters: $\zeta _c \sim {{\cal D}}( 1/\sigma _{\zeta })$.

3.3. Identification

Our IRT model is not identified without further constraints, which pertain to location, scale, and rotation of the latent dimensions (see Bafumi et al., Reference Bafumi, Gelman, Park and Kaplan2005). We fix the scale and location by re-scaling estimated ideological positions θ to a standard normal distribution after each iteration of the sampling procedure. We address the rotational invariance problem by constraining the discrimination parameters to 0 for selected issues on selected dimensions (see below) and by setting starting values for party positions based on prior knowledge.

The introduction of idiosyncratic preferences and random effects on item parameters introduce the potential for novel identification problems regarding location. We solve this by fixing the mean of those parameters to 0 for each issue and parameter. Introducing σ _e and ζ _c raises potential identification issues for the scale of the estimated latent parameters. Thus, we impose a mean of 1 for those parameters.

We assess convergence of our model via the $\hat {R}$ convergence diagnostic, which is below 1.1 for all parameters (Gelman and Rubin, Reference Gelman and Rubin1992).

4.1. Data

In our application, we use expert-level CHES data (2019), which covers 15 Western-European countries, 21 issues, 129 political parties, and 191 experts (Jolly et al., Reference Jolly, Bakker, Hooghe, Marks, Polk, Rovny, Steenbergen and Vachudova2022). For each issue, experts position the party on an ordered response scale. The issues are selected to tap into three distinctive areas of political conflict, namely the economy (five items), social/cultural issues (ten items), and the European Union (six items). For more on the data, see Appendix 1 in the SI.

One major advantage of CHES is that it contains multiple observations per party and issue from different experts. On the one hand, this allows us to disentangle ideology from idiosyncrasy. On the other, this allows us to estimate σ _e. Other data sources that have this feature are voter placements of parties or party manifestos, if there are multiple coders.

We impose a two-dimensional ideological structure to explain variation in party preferences. One dimension captures conflict over the economy, specifically the role of the state, whereas the second dimension captures conflict over culture with poles that have sometimes been described as green-alternative-libertarian (GAL) and traditional-authoritarian-nationalist (TAN) (Bakker et al., Reference Bakker, Edwards, Jolly, Polk, Rovny and Steenbergen2014a). For identification purposes, we set the discrimination parameter of the issue “economic intervention” to 0 for the cultural dimension. We also set the discrimination parameter for “immigration policy” on the economic dimension equal to 0. Substantively, this specifies that these two issues are entirely unrelated to these two ideological dimensions.

4.2. The ideological component

How did parties place on the ideological components? What patterns do the item discrimination parameters reveal? Figure 1 displays the correlation between the standardized mean expert placements on economic left–right and GAL–TAN (horizontal axes) and estimates of θ_ic from our model (vertical axis). Overall, the values strongly correlate: r = 0.93 for the economic dimension and r = 0.90 for the cultural dimension. We interpret this as high face validity of our model.

Figure 1. Ideological placements of parties: experts versus estimates.

Figure 2 is a box plot of the mean item discrimination parameters, β_jc, with the boxes reflecting cross-country variation.Footnote ² The absolute size of the parameters indicates how strongly issues connect with the ideological dimensions (i.e., how well does an issue distinguish parties on either side of a dimension?), whereas the sign indicates the nature of the ideological politicization of an issue (e.g., does the economic right or the economic left favor a policy?). The figure highlights that the core issues associated with the economic dimension are very similar across Western Europe (visible in the first row of the figure). For deregulation, economic intervention, income redistribution, and spending versus tax cuts, there is hardly any variation in the parameters.

Figure 2. Discrimination parameters across countries, dimensions, and issues.

On the social/cultural issues, there is more heterogeneity (visible in the second row). This is particularly true for urban–rural policy emphases, the role of religious principles in politics, and the status of regions. Of the latter issue, it could be said that it is salient in only a few countries (e.g., Belgium, Spain, and the United Kingdom) and can be framed in both economic and cultural terms. Considering crime, the environment, immigration, nationalism, and social lifestyle, there is considerably less cross-national variation in the discrimination parameters. Those issues appear to define the cultural GAL–TAN dimension uniformly in Western Europe.

The greatest variation in item discrimination parameters can be found for the issues related to European integration (visible in the third and final row). Here the inter-quartile distances (the box lengths) and the whiskers are sizable compared to the other issues. In some countries, such as the United Kingdom, the European Union mostly seems to be a cultural issue, showing a much larger discrimination parameter for that than for the economic dimension (see Figures 4 and 5 in the SI). In other countries, such as Greece, Europe appears to be contested mainly on the economic dimension.

The payoff of our modeling strategy is that we unconfound the discrimination parameters from the scaling factor ζ _c. If we do not do this, then at least part of the variation in the association between items and latent variables may be due to differences in cross-national response behavior. That is not the case with our setup.

4.3. The idiosyncratic component

Our model allows for party-specific shocks, γ _ij, relative to the ideological component structuring parties’ issue preferences. Figure 3 shows the size of those shocks for the various issues. The issues are sorted by the standard deviations of the shocks.

Figure 3. Idiosyncratic shock estimates across issues and parties.

The first thing to observe is that the shocks can be sizable, meaning that for certain issues party preferences are driven not by ideology alone, but also by idiosyncratic preferences. If the image so far has been one of relative agreement about the issues that discriminate in the two-dimensional space, the current image is one of distinct heterogeneity.

The largest shocks appear for five issues: (1) the role of religious principles in politics; (2) the urban–rural focus of parties; (3) their regional policy stances; (4) the environment; and (5) social lifestyle. All of those issues discriminate on the cultural dimension. It is clear, however, that some parties take positions that deviate from their cultural ideology. Further analysis shows this to be true of the agrarian party family for urban–rural concerns, the confessional family for religious principles and social lifestyle, the green family for the environment, and the regionalist family for regional issues (see Figure 6 in the SI).

4.4. Other results

In our discussion thus far, we have focused on parameters associated with party-specific factors influencing parties’ issue positions, namely parties’ latent ideological placement (θ_ic), discrimination parameters (β_jc), and idiosyncratic shocks (γ _ij). In doing so, we have highlighted the face validity of this model, variation in the politicization of issues across Western Europe, and the extent to which parties’ positions are determined by factors other than ideology. These are the quantities most clearly associated with our primary interest, namely substantive sources of DIF.

For brevity, we present the resulting estimates of other parameters regarding measurement-specific sources of DIF from this model in the SI: the country-issue-specific shocks to difficulty parameters (α _jc; Figure 7), the country-specific scale and shift parameters (ζ _c and α _c; Figures 8 and 9), as well as expert-level errors (σ _e; Figure 10). In Appendix 3 in the SI, we show via model comparisons that the single sole component that accounts for the largest increase in model performance relative to a conventional IRT model is the inclusion of idiosyncratic preferences. Overall, the results show that our model disentangles different sources of DIF that represent different sources of both substantial and measurement-specific variation in the association between manifest and latent variables across countries.