Experimental Procedure

Respondents (N = 331) were randomly assigned to one of sixteen descriptions of a proposed military plan for a targeted killing, and then were asked to evaluate the proportionality of the plan using their legal expertise. Vignette survey experiments are widely used in social sciences, and were found to have strong external validity in predicting behaviour of both citizens (Reference Hainmueller, Hangartner and YamamotoHainmueller, Hangartner, and Yamamoto 2015) and professionals (Reference Peabody, Luck, Glassman, Jain, Hansen, Spell and LeePeabody et al. 2004). The versions of the military plan differed with regard to the operational goal and to the attendant infringement of rights. Overall, we used four “goal treatments” and four “rights infringement treatments,” which yielded a four by four, between-subjects, fully crossed factorial design. Table 2 presents the two sets of vignettes that were used to construct the 16 experimental conditions. Each condition combines one of the “goal treatments” and one of the “rights infringement treatments.”

Table 2. Experimental Treatments

Note: The 16 experimental conditions are based on all the possible pairings of these variations of the “goal” and “rights infringement” parameters.

The “goal treatments” were as follows: G1 – a punitive goal; G2 – a preventive goal; G3 – prevention of a more severe harm; and G4 – prevention of a more severe harm by targeting more senior terrorist operatives. The “rights infringement treatments” were as follows: R1 – targeting terrorists who are in a car in an urban area of Gaza, without mentioning the option to apprehend them; R2 – the same plan preceded by a remark that apprehending the terrorists is not a feasible option; R3 – identical to R2, except the targeted car is in a nonurban area, and therefore, the plan presents a lesser risk to uninvolved bystanders; R4 – identical to R3, except the car is supplanted by a motorcycle, thus reducing the risk of uninvolved passengers being inside the target.Footnote ¹⁷

Measuring Proportionality Judgments

The dependent variable – the proportionality judgment – was measured using a set of five questions. Four questions covered the four decision stages: (1) Is the goal of the military plan worthy (yes/no)? (2) Is the military plan adequate and effective for achieving the goal (yes/no)? (3) Is the military plan a practical means to achieve the goal with minimal infringements of human rights (yes/no)? (4) Is the proportion between the advantage to be gained by implementing the military plan and the expected infringement of human rights adequate (yes/no)? Since this study constitutes the first attempt to measure proportionality judgments empirically, we conducted a series of validation analyses for the measurement items, which are reported in detail in the online appendix.

The fifth question was prompted by the above-discussed doctrinal ambiguity of proportionality analysis and was intended to elicit an unstructured proportionality judgment (henceforth, “free PA”). Respondents were asked to indicate their judgment regarding the proportionality of the proposed plan on a scale of 1 (very disproportional) to 6 (very proportional). Respondents were informed that the boundary between un-proportionality and proportionality lay between 3 and 4 on the scale.

Responses to these five questions yielded three binary measures of proportionality judgments for each respondent, as displayed in Figure 1. The first measure is proportionality in the “strict sense,” as prescribed by IHL. It assigns the value of 1 to the affirmative answer to the forth item and 0 otherwise. The second measure is a “full PA,” as required by the Targeted Killing decision. This measure assigns the value of 1 to affirmative answers to all four items, and 0 otherwise. The third measure is a binary version of the response to the “free PA” item; it assigns the value of 1 to a score between 4 and 6, and 0 otherwise. Thus, respondents were presented with legal questions they expected a proportionality analysis to address, while we obtained results regarding both PA versions, the comprehensive (“full PA”) and the less comprehensive one (“strict-sense balancing”).

Figure 1. Question wording, and three measures of proportionality judgments.

As the three measures of proportionality judgment yielded an internally consistent index (alpha = .866), they were treated as three repeated proportionality measures for each respondent. The analysis controlled for the type of measure, and thus provided estimates for the effects of the doctrinal version (“strict-sense balancing,” “full PA,” and “free PA”) on the judgment. Further details are provided in the Estimation strategy section.

Measuring Ideological Preference

The effect of respondents’ legally irrelevant preferences on proportionality judgments were estimated based on their self-reported political ideology. Political ideology serves as a simplifying heuristics that helps individuals interpret the world, by “making assertions or assumptions about human nature, historical events, present realities, and future possibilities” (Reference Jost, Federico and NapierJost, Federico, and Napier 2009; Reference Ball and DaggerBall and Dagger 2010; Reference Erikson and TedinErikson and Tedin 2010; Reference Ball and DaggerBall and Dagger 2010). Indeed ideology is the most important spatial dimension guiding vote choice (Reference Duch, May and Ii ArmstrongDuch, May, and Armstrong 2010) and policy preference (Reference RingeRinge 2005; Reference Sulitzeanu-Kenan and HalperinSulitzeanu-Kenan and Halperin 2013). In Israel, the central domain aligning right- and left-wing supporters is the hawk versus dove approach to the Arab-Israeli conflict (Reference Benoit and LaverBenoit and Laver 2006; Reference Shamir and ArianShamir and Arian 1999). If proportionality judgments are indeed susceptible to legally irrelevant preferences (H3), hawks are expected to be more likely than doves to judge military plans for targeted killings in the context of this conflict as proportional (and vice versa). Each respondent's political ideology relative to the hawk-dove continuum was assessed using a single-item measure, in which respondents were asked to indicate their political position on a scale of 1 (extreme right) to 5 (extreme left). For each respondent we also recorded a set of covariates which included age, gender, and religiosity.Footnote ¹⁸

Estimation Strategy

To identify the effects of factual variation in the “worthy goal” and “rights infringements” parameters, respondents were randomly allocated to 16 experimental conditions. Supporting Information Table A4 in the online appendix verifies that the randomization process was successful, by demonstrating that practically all individual-level variables are balanced across the experimental treatments. As the three proportionality measures for each respondent turned out to be internally consistent, the data were structured as a repeated measures dataset of 3 × 331 = 993 observations.

Equation (1) presents our basic method for estimating proportionality judgments; it accounts for the factual variation, the doctrinal versions, and individual-level characteristics:

(1) $Logit\left[E\right(Y_{ij}=1\left|{\left(K,D,X\right)}_{ij}\right]={\alpha K}_{ij}+{\beta D}_{ij}+{\gamma X}_{ij}$

where Y is a binomial variable that represents a judgment that the plan is proportional [Y = 1] or not [Y = 0], by respondent i, based on proportionality type j, conditional on a vector of factual variation K, a vector of doctrine alternatives D and a vector of individual characteristics X. We estimate the effects of factual variation on proportionality judgment adjusting for multiple individual-level covariates, using generalized estimating equations (Reference Liang and ZegerLiang and Zeger 1986). Generalized estimating equation is a method developed for handling clustered data, in which the observations within each cluster are intercorrelated. Given that each set of three proportionality judgments are clustered within respondent, we used generalized estimating equations to account for the correlations among observations. To address within-subject correlations, we cluster standard errors within respondent. To obtain detailed estimates for the “goal” and “rights infringement” treatments, doctrine alternatives, and ideological differences, we estimate proportionality judgments using the following elaboration of (1):

(2) $Logit{\left(Y\right)}_{ij}={\alpha }_0+{\alpha }_1{Preventive\left(g2\right)}_i+{{\alpha }_{2}Severity\left(g3\right)}_i+{\alpha }_{3}Senior{\left(g4\right)}_i+{\alpha }_{4}LRM{\left(r2\right)}_i+{\alpha }_{5}nonunrban{\left(r3\right)}_i+{\alpha }_{6}motorcycle{\left(r4\right)}_i+{\alpha }_7{FullPA}_j+{{\alpha }_{8}FreePA}_j+{\alpha }_9{Ideology}_i+{\alpha }_{10}{female}_i+{\alpha }_{11}{age}_i+{\alpha }_{12}{relig.}_i$

where the first six independent variables comprise two sets of three indicator variables, each set representing the four conditions with regard to the operation goal (with “punitive goal” as reference) and the potential infringement of rights (with the plan with most rights-infringement potential as reference), respectively. $FullPA$ and $FreePA$ are indicator variables for the type of proportionality analysis (“strict-sense” as reference). Next are individual-level characteristics (hawk-dove ideology, gender, age, and religiosity).

To evaluate the impact of factual variation on proportionality judgments as per Hypotheses 1 and 2, we first need to define the minimal set of empirical expectations required to satisfy these hypotheses. As noted above, the proportionality doctrine assigns positive utility to a “worthy goal” and negative utility to infringement of rights. However, it does not specify the relative weight of the “goal” and “rights infringement” factors, nor does it stipulate the impact of changes in these factors on utility (Reference AleinikoffAleinikoff 1987; Reference PetersenPetersen 2013; Reference TsakyrakisTsakyrakis 2009; Reference WebberWebber 2010). This particular vagueness poses a problem in stating clear empirical expectations regarding behaviors that are congruent with this doctrine and those that are not. Despite this vagueness, we can specify certain restrictions on the pattern of judgment which would qualify the doctrinal guidance. Such a pattern must be consistent with the dominance principle (Reference Hadar and RussellHadar and Russell 1969; Reference Kariv and SilvermanSee also: Kariv and Silverman 2013), in the sense that a policy should be preferred to another if it yields unambiguously greater utility (i.e., increases the share obtained from the goal, or decreases the potential for rights infringement). This entails that, when comparing proportionality judgments across such ordered set of policies, the pattern of judgment must be monotonic.Footnote ¹⁹ Only under this condition can we conclude that it conforms to a certain proportionality doctrine. Implementing this restriction in our analysis suggests that Hypotheses 1 and 2 are supported only if coefficients ${\alpha }_1$–${\alpha }_3$, and ${\alpha }_4$–${\alpha }_6$, respectively, have positive, and monotonically increasing values. This means that, ceteris paribus, the probability of a plan being judged as proportional should never decrease when moving to a scenario with a more worthy goal or with a lesser potential for rights infringements.Footnote ²⁰

The effect of ideological (hawk-dove) preferences on proportionality judgments is estimated in two ways. First, in order to test Hypothesis 3, we simply estimate the size and statistical significance of coefficient ${\alpha }_9$ in (2). If such an association is found, we proceed to determine the mechanisms through which these preferences shape proportionality judgments. One possible account is that one's prior propensity for judging targeted killings as proportional depends on one's ideology. The second is that one's ideological preference influences the weight assigned to different factual considerations, thereby affecting information processing, as suggested in the motivated-cognition literature. This indirect effect of ideology on proportionality judgments, as per Hypothesis 4, can be tested by comparing the effect of the different treatments employed on hawks versus doves. For this purpose, we add to (2) interaction terms for ideological groups (hawks/doves) and the treatments within the “goal” and “rights infringements” factors:Footnote ²¹

(3) $\mathrm{Logit}{\left(Y\right)}_{ij}={\alpha }_0+{\alpha }_1{\mathrm{Goal}}_i+{\alpha }_2{\mathrm{Rights}\text{\_}\mathrm{infringe}.}_i+{\alpha }_3{\mathrm{Full}PA}_j+{\alpha }_4{\mathrm{Free}PA}_j+{\alpha }_5{\mathrm{Hawk}\times \mathrm{Goal}}_i+{\alpha }_6{\mathrm{Hawk}\times \mathrm{Rights}\_\mathrm{infringe}.}_i+{\alpha }_7{\mathrm{Hawk}}_i+{\alpha }_8{\mathrm{female}}_i+{\alpha }_9{\mathrm{age}}_i+{\alpha }_{10}{\mathrm{relig}.}_i{}^{21}$

These interaction terms allow us to test Hypothesis 4 by estimating the differences between the receptivity of proportionality judgments to factual variation across ideological groups.Footnote ²²

To assess the implication of the doctrine version on proportionality judgments, we first assess the size and significance of coefficients ${\alpha }_7$ and ${\alpha }_8$ in (2). Next, to test whether the doctrine version affects the level of receptivity to factual variation (Hypothesis 5), we add to (2) interaction terms for the doctrine version and ideological preference:

(4) $Logit{\left(Y\right)}_{ij}={\alpha }_0+{\alpha }_1{Goal}_i+{\alpha }_2{Rights\_infringe.}_i+{\alpha }_3{FullPA}_j+{\alpha }_4{FreePA}_j+{\alpha }_5{FullPA}_j\times {Goal}_i+{\alpha }_6{FullPA}_j\times {Rights\_infringe.}_i+{\alpha }_7{FreePA}_j\times {Goal}_i+{\alpha }_8{FreePA}_j\times {Rights\_infringe.}_i+{\alpha }_9{Ideology}_i+{\alpha }_{10}{female}_i+{\alpha }_{11}{age}_i+{\alpha }_{12}{relig.}_i$

Lastly, to test whether the doctrine version affects the level of susceptibility to legally irrelevant preferences (Hypothesis 6), we add to (2) interaction terms for the doctrine version and ideological preference:

(5) $\mathrm{Logit}{\left(Y\right)}_{ij}={\alpha }_0+{\alpha }_1{Goal}_i+{\alpha }_2{Rights\_infringe.}_i+{\alpha }_3{FullPA}_j+{\alpha }_4{FreePA}_j+{\alpha }_5{FullPA}_j\times {Ideology}_i+{\alpha }_6{FreePA}_j\times {Ideology}_i+{\alpha }_7{Ideology}_i+{\alpha }_8{female}_i+{\alpha }_9{age}_i+{\alpha }_{10}{relig.}_i$

Receptivity of Proportionality Judgments to Normatively Relevant Factual Variation

We estimate the effects of factual considerations on proportionality judgments by fitting to our data. Detailed results are reported in Table 3 and presented graphically in Figure 2B below.

Figure 2. The effects of “goal” and “rights infringement” treatments on the probability of “proportional” judgment (95% CI). [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

Table 3. Generalized Estimating Equations predicting Proportionality Judgments

Note: The generalized estimating equations employ logit link function and unstructured correlation structure. Coefficients are reported in logit values. Statistical significance levels are represented as follows: + p < 0.1, *p < 0.05, **p < 0.01, ***p < 0.001. “Goal treatments” is a four-scale ordinal variable representing the four “goal” treatments; “Preventive attack,” “Severe attack” and “Senior terrorists” are indicator variables for the “goal” treatments, with “punitive attack” serving as a reference category; “Rights infringement treatments” is a four-scale ordinal variable representing the four “rights infringement” treatments (note that a higher value indicates less potential for rights infringements); Least restrictive means”, “Car in nonurban area” and “Motorcycle in nonurban area” are indicator variables for the “rights infringement” treatments, with “no mention of alternative” serving as a reference category; “Hawk-dove” is a five-scale ordinal variable with 1 representing “extreme hawk” and 5 representing “extreme dove”; “Hawk” is an indicator variable for respondents who identified as either “extreme hawk,” “hawk” or “center” (“dove” and “extreme dove” as reference); “Gender” is an indicator variable representing a female respondent (male as reference); “Religiosity” is a four-scale ordinal variable with 1 representing “ultraorthodox” and 4 representing “secular”; “Online sample” is an indicator representing respondents who participated in the online sample (pen and paper sample as reference); “Full PA” and “Free PA” are indicator variables, with “strict sense” as a reference category.

Figure 2A displays the average probability of judging the operation as proportional for each treatment of the “goal” factor,Footnote ²³ controlling for all other variables, which are set at their respective mean values.Footnote ²⁴ The overall effect of “goal” treatments was statistically significant (χ ² =12.51, p = 0.005) based on post estimation Wald test. As shown in Figure 2A, the probability that the operational plan would be judged as proportional increases with the rise in the worthiness of the goal (moving from left to right on the X-axis). The overall effect of the “rights infringements” treatments are likewise statistically significant (χ²= 37.53, p < 0.0001) based on post estimation Wald Test. As per Figure 2B, the probability of an operation being judged as proportional increases with the decrease in the anticipated rights infringement. Note that both of these trends are monotonic, in that none of the treatments is associated with a reverse trend.Footnote ²⁵ These finding align with the empirical expectations regarding Hypotheses 1 and 2, which are presented in the Estimation strategy section, suggesting that proportionality judgments are receptive to doctrinally relevant factual variation.

Although, overall, the susceptibility of proportionality judgments with factual variations in the “goal” and “rights infringement” factors conformed to our expectations, some specific treatments within these two factors do not appear to affect the judgment at a statistically significant level. The difference between a punitive and a preventive operation resulted in a significant increase of 20.0 percentage points (p = 0.012) in the probability of judging the operation as proportional. Conversely, the effect of implying that the expected attack (to be averted by the operation) is likely to be severe was −1.2 percentage points and statistically insignificant (p = 0.986). Likewise, the marginal effect of stipulating that the operation targets a senior and important terrorist operative resulted in an increase of 7.6 percentage points, which is statistically insignificant (p = 0.302).Footnote ²⁶ With regard to the influence of the “rights infringement” factor on the probability that the operation would be judged as proportional, a putative lack of a nonlethal alternative to the operation had a marginal effect of 13.6 percentage points, which is marginally significant (p = 0.10). The reduction in the risk to uninvolved bystanders resulting from locating the targeted car in a nonurban area had a significant effect of 23.7 percentage points (p = 0.002). Finally, supplanting a car by a motorcycle and thereby increasing the precision of the attack, yielded an insignificant marginal effect of 1.9 percentage points (p = 0.476). The specific treatments that did not result in a statistically significant effect may be accounted for by the ineffective phrasing of a particular treatmentFootnote ²⁷ and/or sample size.

Inasmuch as both factors produced a significant overall effect and clearly monotonic patters, the lack of a significant effect for some specific treatments does not detract from the validity of the results, which suggest that proportionality judgments appear to be receptive to factual variation in line with doctrinal expectations. The specific treatments that did not result in a statistically significant effect may be accounted for by ineffective phrasing of a particular treatment and/or sample size. Notwithstanding these potential methodological reasons, comparing the set of treatments that were found to affect proportionality judgements and those that did not, suggests that respondents make a discrete distinction between a punitive and preventive goals but do not appear to be affected by the size of the preventable harm in weighing the worthiness of the goal. Respondents also appear to care whether a nonlethal alternative is available, and the level of risk to bystanders. However, once risk to bystanders was reduced, no indication was found that the selectiveness of the operation (i.e., the level of certainty of targeting only the suspected terrorists and not individuals accompanying them) was consequential to respondents’ decisions. While the latter considerations might be limited to professionals versed in the tactical nuances of such operations, the former null finding regarding the facts of the operational goal – expected severity of attack to be averted and the seniority of the targeted terrorists – is more surprising, as they (especially preventable severity) pertain to a central aspect to be considered in proportionality analysis, i.e. the importance of the operation. We discuss these results further in the discussion section.

The existing literature on proportionality, and particularly those studies that formally specify the theory (Reference AlexyAlexy 2010; Reference PetersenPetersen 2013; Reference VeelVeel 2010) implicitly suggest that proportionality judgment relies on the additive effects of the two conflicting values. However, as our interest in this study is in the characteristics of the actual way in which legal experts conduct the proportionality analysis, we performed a further analysis that assesses whether the weight given to either of the two values is influenced by changes in the other conflicting value.Footnote ²⁸

To estimate these possible interactions between the two factors we fitted a model that adds all the possible interactions between the levels of the two factors (“worthy goal” and “rights infringement”) to (2). Detailed results are reported in the left column of Supporting Information Table A5 in the online appendix. The results do not support an interaction model. None of the specific interaction terms were statistically significantly. Contrasts of the marginal linear predictions based on this model resulted in two main effects – for the “worthy goal” (χ ² =12.73, p = 0.005) and the “rights infringement” (χ ² =37.76, p < 0.001) – and an insignificant interaction effect (χ ² =5.76, p = 0.764). Given that some specific treatments within the two factors were not found to affect the proportionality judgments at a statistically significant level, we fitted a further model, that compares condition G1 to the three other levels of the “worthy goal” factor, collapses conditions R3 and R4 of the “rights infringement” factor, and includes all the possible interactions between these levels of the two factors. Detailed results are reported in the column titled “Reduced Interaction Model” in Supporting Information Table A5 in the online appendix. The substantive findings for this model are similar to the full interaction model. None of the specific interaction terms are statistically significantly, and contrasts of the marginal linear predictions resulted in two main effects – for the “worthy goal” (χ ² =10.84, p = 0.001) and the “rights infringement” (χ ² =31.84, p < 0.001) – and an insignificant interaction effect (χ ² =.33, p = 0.848). These results suggest that the pattern of proportionality judgments in response to the experimental treatments is consistent with an additive conception of proportionality analysis.

The effects of the “worthy goal” and “rights infringement” factors on proportionality judgments are concurrent and integrated, as is demonstrated in Figure 3, which maps the factual variation into proportionality judgments in a three-dimensional graph. The two horizontal axes represent the “goal“ and “rights infringements” treatments, and the vertical axis represents the probability of the plan being judged as proportional. Thus, the figure provides a comprehensive visual representation of the receptivity of proportionality judgments to factual variation.

Figure 3. A three-dimensional mapping of normatively relevant factual variations into proportionality judgments. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

The Effects of Ideology on Proportionality Judgments

Figure 4 is based on (2) with a minor specification change: the ordinal ideology variable is treated as categorical rather than ordinal. It presents the mean probability of the operation being judged as proportional for respondents with different positions on the hawk-dove ideological scale, controlling for all other variables (which are set at their respective mean values). This analysis clearly shows that proportionality judgments are strongly associated with ideology (χ² = 62.26, p < 0.0001). Even if we ignore the two extreme categories on both sides of the ideological spectrum (jointly constituting 8.8% of the sample), the mean difference in the probability of judging a given operation as proportional decreases from 90.9% for “hawks,” to 78.1% for “center” (χ² = 9.90, p = 0.007), and to 52.1% for “doves” (χ² = 17.21, p < 0.001) – both differences being sizable and statistically significant.Footnote ²⁹ These findings provide support for Hypothesis 3.

Figure 4. Mean probability of “proportional” judgment across ideological positions (with 95% CI) Note: only 8 respondents occupy the “extreme hawk” category, all of whom judged their respective scenario as proportional (under the three different proportionality formulae). [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

As our results indicated an association between ideology and proportionality judgment, we proceeded to explore the mechanism for this influence, as per Hypothesis 4. To this end we fitted (3) to our data. This model includes interaction terms which allow us to estimate the difference in treatment effects for hawks versus doves. Accordingly, we split our sample into two ideological groups: “hawks” (extreme hawks, hawks, center; N = 200) and “doves” (doves, extreme doves; N =127).Footnote ³⁰ For simplicity, we replaced the two sets of three dummy variables with two ordinal variables – “goal” and “rights infringements.” The full analysis is presented in Table 3. Both interaction terms turned out to be statistically insignificant (Hawk × Goal: z = −.23, p = 0.821; Hawk × Rights infringements: z = −.93, p = 0.353), providing no support to Hypothesis 4. Put differently, there is no indication that the factual variation had systematically different effects on the judgments of hawks versus doves.Footnote ³¹

Proportionality Judgments vis-à-vis Doctrinal Versions

Figure 5 is based on (2). It presents the expected probability of the operational plan being judged as proportional for the three proportionality types: “strict-sense” (the IHL rule); “full PA,” involving the “worthy goal,” “suitability,” “necessity” (LRM) and “strict-sense balancing” stages (the Targeted Killing decision); and “free PA” – the unstructured judgment. Figure 5 shows distinct differences in the probability of judging the operation as proportional under the three different proportionality formulae (χ² = 67.45, p < 0.0001). The expected probability under the “strict-sense” alternative, with all other variables set at their mean values, is 75.9%. Applying the “full PA” alternative resulted in a reduction of 24.0 percentage points (z = −8.23, p < 0.001), yielding the probability of 45.2%. The probability under the “free PA” alternative was 80.3%, significantly higher than under “full PA” (z = 9.06, p < 0.001), and 4.4 percentage points higher than under the “strict-sense” decision (z = 2.11, p = 0.035).

Figure 5. Mean probability of “proportional” judgment across proportionality formulae (with 95% CI). [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

Thus, although the unstructured judgment (“free PA”) was solicited last, after respondents had replied to the entire set of questions regarding all the considerations involved in the decision, it was on average more similar to the one under the “strict- sense” rather than under “full PA” alternative. The sizable effect of applying “full PA” did not come as a surprise. It is reasonable to expect that the requirement that the proposed plan should clear all the decision stages would reduce the probability of approval.

To test whether a more comprehensive proportionality analysis increases the receptivity of judgements to normatively relevant factual variation (Hypothesis 5), we fitted (4) to our data. This analysis, reported in Table 3, allowed us to estimate the differences in the effect of factual variation in the “goal” and “rights infringement” parameters under each doctrinal version (“strict-sense balancing,” “full PA,” and “free PA”). All four interaction terms are negative, suggesting that the receptivity of judgments to factual variation under “full PA” and “free PA” is lower than under the “strict-sense balancing.” However, of the four interaction terms, only the “Free PA” X Goal is statistically significant (z = −2.02, p = 0.044). These results, thus, do not allow us to reject the null hypothesis that proportionality judgments under “full PA” and under “strict-sense balancing” are equally receptive to factual variation. Figure 6 presents the predicted probability of the proposed plan being judged as proportional across the “goal” (upper panel) and “rights infringements” (lower panel) treatments, and doctrine versions. The pattern of receptivity is very similar for the “goal” and “rights infringements” factors. In both, the incline of the three slopes (for three doctrine versions) are similar, with the slope representing the “strict-sense balancing” (blue) being slightly more pronounced compared to “full PA” (red) and “free PA” (green). Thus, receptivity to factual variation appears to be somewhat higher under the less comprehensive version of PA – “strict-sense balancing”. This difference, however, is statistically insignificant.

Figure 6. The effects of “goal“ (upper panel) and “rights infringement“ (lower panel) treatments on the probability of “proportional“ judgment across proportionality types. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

Finally, to test whether proportionality type affects the susceptibility of judgements to legally irrelevant preferences, we fitted (5) to our data. This analysis, reported in Table 3, allowed us to estimate the differences in the effect of ideological preferences on proportionality judgments for different doctrine versions. The interaction terms suggest that such an effect is stronger under “full PA” (albeit at a marginally significant level), and under “free PA,” than under “strict-sense balancing” (Hawk–Dove X “Full PA”: z = −1.89, p = 0.058; Hawk–Dove X “Free PA”: z = −2.56, p = 0.010). These results provide no support to Hypothesis 5, which predicted a lesser effect of legally irrelevant preferences under “Full PA” than under “strict-sense balancing.” The differences in the effect of ideological preferences under the three types of proportionality judgments are displayed in Figure 7, which plots the predicted probability of the proposed plan being judged as proportional across the set of ideological preferences and for the three doctrine versions. The three curves are similar, but the slopes of the “Full PA” (red) and “Free PA” (green) curves are slightly more pronounced than of the “strict-sense balancing” curve (blue). In other words, the effect of ideological preferences on proportionality judgments appears to be somewhat larger under the two former PA types than under the latter.

Figure 7. The effects of ideological preferences across proportionality types. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]