2. The link between rebellion, ideology, policy congruence and valence

Experimental studies that seek to uncover how voters view MPs’ partisan dissent vary significantly in their research designs, the contextual information provided to respondents, and the countries in which they are conducted. But they are remarkably consistent in their findings that voters express support for rebels. Some studies present survey respondents with vignettes that describe dissent devoid of the policy context in which it occurs, focusing instead on MPs who are more rebellious over the long term, regardless of policy content (e.g., Campbell et al., Reference Campbell, Cowley, Vivyan and Wagner2016), others with respect to a policy lacking ideological content or without specifying the direction of policy change (Bøggild and Pedersen, Reference Bøggild and Pedersen2020; Besch and López-Ortega, Reference Besch and López-Ortega2021; Bøggild, Reference Bøggild2020), and still others by categorizing a hypothetical MP's ideology on the basis of multiple policy positions without directly linking them to the policy position of the respondent (Carson et al., Reference Carson, Koger, Lebo and Young2010). Studies also vary in the nature of comparisons they ask respondents to make. Some show respondents a number of comparisons between two hypothetical MPs and ask them to choose one MP over the other (or to rate both) in the context of a paired conjoint experiment (e.g., Campbell et al., Reference Campbell, Cowley, Vivyan and Wagner2016; Wagner et al., Reference Wagner, Vivyan and Glinitzer2019). Other studies use a factorial vignette design, presenting each respondent with only one single vignette about a single MP (e.g., Carson et al., Reference Carson, Koger, Lebo and Young2010; Bøggild and Pedersen, Reference Bøggild and Pedersen2020; Besch and López-Ortega, Reference Besch and López-Ortega2021). The variation in design choices makes it difficult to disentangle the effects of the design from the effects of particular explanations about why voters support rebels. Moreover, none of the existing designs test the impact of substantive policy congruence between voters and the MPs on support for rebellion. Additionally, studies present evidence that voters value independence as a trait in an MP (e.g., Campbell et al., Reference Campbell, Cowley, Vivyan and Wagner2016). However, they have not shown experimentally that respondents associate independence and other valence characteristics with rebellion.Footnote ² Despite this variation in designs, the studies consistently find that rebellion is an asset when seeking voter support with few exceptions. For example, in the US Congress, Harbridge and Malhotra (Reference Harbridge and Malhotra2011) find support for crossing the aisle in general, but also find that strong partisans punish members of Congress who engage in too much bipartisanship.Footnote ³ Likewise, Campbell et al. (Reference Campbell, Cowley, Vivyan and Wagner2016) find that strong party identifiers are less likely to reward rebellion in the context of British politics. Nevertheless, the findings across these various settings are remarkably consistent in demonstrating that voters express comparatively strong support for MPs engaging in party dissent.

The design of our study differs from the existing studies in important ways. We specifically design our study to examine whether the rebel effect runs through policy and ideological congruence. Of the previous studies only Carson et al. (Reference Carson, Koger, Lebo and Young2010) consider ideological congruence, but they do not explicitly consider congruence on specific, salient policy issues. Instead, they construct a measure of latent ideological distance between the respondent and the member of Congress. In contrast, we chose a series of salient policy proposals that relate to relevant ideological conflicts across countries.

We also provide the respondents with a richer information environment than most previous studies. Respondents are given additional information on whether the public supports the bill, whether the MP's party is in government, whether the MP's vote is pivotal for the outcome of the parliamentary process, or whether the bill is adopted or rejected, among other attributes. The amount of contextual information may make it more difficult to find an effect for being a rebel as respondents may gloss over information about dissent—manipulation checks, however, reveal that they do not—but it also creates a more realistic environment and allows us to potentially rule out alternative explanations. It also prevents respondents from inferring left-out information and perhaps attributing that left-out information to an act of rebellion. To compare our findings with those from existing studies and so that we are not simply adding to the number of designs used, we also incorporate treatments that mirror the conditions in these previous studies.

Before theorizing and testing the reasons behind voter support for rebellion, we would like to be confident that an effect of rebellion exists. Ideally, we would replicate the results from previous experiments that find voter support for rebel MPs. In the context of our experiment, we would hope to find that voters do, indeed, want MPs to be like the rebellious MPs portrayed in our vignettes and also that voters feel represented by these rebels. This leads us to our first hypothesis.

Assuming that we find support for hypothesis 1, we then move on to test reasons for voter support for rebels. The existing literature has suggested that voter support for rebellion can be primarily attributed to voters associating rebellion with valence characteristics (e.g., Campbell et al., Reference Campbell, Cowley, Vivyan and Wagner2016; Wagner et al., Reference Wagner, Vivyan and Glinitzer2019). These characteristics include an MP's perceived independence, strong personal convictions, honesty, and other positive attributes. But because existing designs do not specifically look at the ideological congruence between rebel MPs and respondents with respect to particular policies, we cannot rule out that rebellion, instead, serves as a signal that the MP stands up for the policy wishes of the voter. If a respondent, who has no information about policy congruence, sees an MP engaging in an act of rebellion, they may (possibly subconsciously) assume that the MP is rebelling to support a position that they would favor. Moreover, the literature to date has demonstrated that voters prefer MPs that they perceive to be independent thinkers, a trait one is likely to associate with MPs willing to buck their party's wishes, but it has not been shown experimentally that voters are more likely to view rebels as independent compared to MPs who remain loyal to their party.

We suggest that voters might support rebellion, not because they value the act of rebellion, per se, but rather because it serves as a signal that the MP stands up for the policy wishes of the voter. By examining support for rebels conditional on substantive policy congruence, we are able to tease out the direct effect of rebellion from the effect of rebelling possibly signalling policy congruence. If we find that voters value rebellion, but that the value of rebellion disappears once controlling for policy congruence, we can conclude that voters’ support for rebellion runs through policy congruence. In other words, voters support rebels because they think that those rebels likely represent them better on policy concerns.

If the rebel effect remains after controlling for policy congruence with the respondent, it suggests that other factors may be behind support for rebels. Support for rebellion might arise, not because rebellion is indicative of individual-level congruence, but rather because it signals to voters that MPs are better at representing a larger group of voters. Voters may believe that rebels are better at representing society, even if they do not represent their own policy views on a specific issue. Our third hypothesis moves beyond individual congruence to ask whether rebellion might signal an MP's concern to represent the electorate at large.

Of course, we also wish to test experimentally whether respondents attribute valence characteristics with rebellion as the previous literature has suggested that they do. Thus, we test our fourth and final hypothesis.

If we fail to find support for hypotheses 2 and 3, but we find support for hypothesis 4, it would suggest support for rebels does, indeed, run primarily through rebellion signaling valence characteristics.Footnote ⁴

The main objective of the experiment is to explore whether and why voters appreciate rebellious activity. Nevertheless, we expect that the government participation of the MP's party, the sex of the MP, and the number of terms an MP has served in parliament affect whether voters like rebellious MPs without stating theoretically grounded predictions of the direction of these effects. Such contextual features will enter the decision situation we create in the experiment.

3. Experimental design

In our main survey, we present respondents with a series of hypothetical vignettes as part of a larger survey on political attitudes.Footnote ⁵ The experiment was conducted on nationally representative samples in France, Germany, Italy, and the United Kingdom (N = 14,000). In particular, these four European democracies vary with respect to (1) the level of importance that MPs place on supporting their party (van Vonno et al., Reference van Vonno, Depauw, Hazan, Andeweg and Deschouwer2014), (2) how much party unity MPs believe there should be (Willumsen, Reference Willumsen2017), and (3) the degree of unity that MPs actually display (Sieberer, Reference Sieberer2006; Depauw and Martin, Reference Depauw and Martin2008). France and the United Kingdom have tended to have quite high levels of party unity while Germany and Italy have experienced somewhat lower levels (see Depauw and Martin, Reference Depauw and Martin2008), and German and British MPs are more likely to say that MPs should vote with their party than MPs in France and Italy (van Vonno et al., Reference van Vonno, Depauw, Hazan, Andeweg and Deschouwer2014). Further, these countries vary institutionally with respect to their electoral systems, which may also impact how voters view rebellion (Wagner et al., Reference Wagner, Vivyan and Glinitzer2019). Given that we implement the same vignette design and definitions of issues across these four countries, we are able to ascertain the robustness of our experimental results on rebel MP behavior across the different national contexts and issues described.

After a series of questions about their socio-demographics, political attitudes, and positions on various policy issues, respondents were asked to assess five experimental vignettes. Each vignette describes the action of a particular MP on a parliamentary vote and corresponds to a bill in a different policy area. We asked respondents for their preferred option for each policy issue before the experiment. Respondents always saw the bills in random order. The five bills are about either (1) removing or establishing environmental regulations for businesses; (2) strengthening or weakening ties with the EU; (3) making it easier or harder for foreigners to immigrate; (4) increasing or decreasing the level of social spending financed by taxation; and (5) increasing or decreasing the range of areas in which the EU can make policy. The wording for each bill and its corresponding policy area are shown in Table 1. After reading about the MP, respondents were asked how they felt about the MP on two outcome measures.

Table 1. Wording of the presented bills

The first vignette was introduced with the following frame: “Voting on bills is a key task of Members of Parliament (MPs). You will now see five hypothetical situations. In each of them, MPs vote on a different bill. We would like you to think about each situation separately when responding to the questions.”

Factorial vignette design

On the next screen, all respondents read a short paragraph describing the topic of the bill, the attributes of the MP (gender and tenure), the MP party's government status, and the voting outcome on the bill in parliament (vote and pivotality). These attributes are uniformly randomized at the vignette level. Most importantly, some MPs rebel against their party by voting differently than their party leadership, other MPs vote in the same way as their party leadership. We also randomize the provision of information about the bill's direction and public opinion on the bill at the respondent level. This provides some respondents with a cue about policy congruence between their preferences, the public's preferences, and the MP's behavior, allowing us to test our hypotheses 2 and 3. In both cases, we either provide no information about the bill's direction or its public support, or we tell respondents the direction of the bill and/or whether the public was for or against the bill.

Table 2 shows the full set of randomly assigned attribute levels. Note that the precise wording of the direction of the bill varies by policy issue (i.e., see Table 1) and that we simplify the presentation of the design here by just focusing on whether the respondent's preference on the issue and the action of the MP in the vignette align or not.Footnote ⁶

Table 2. Attributes and attribute levels

Table 3 shows the nine possible versions of the vignettes that result from the randomization of the bill's direction and public opinion at the respondent level.Footnote ⁷ In the Baseline version, we only provide information on the bill type, but not on bill direction or public opinion, which leaves respondents ignorant about both the MP's position on the bill and the public's view. We also implement six other versions. Four of them only show information on one of the two respondent-level attributes: we tell respondents that the public was against (Against public) or for the bill (With public); or reveal that the MP's behavior aligned (With voter) or did not align (Against voter) with their preferences. These versions are needed to test our hypotheses 2 and 3 to observe how the rebel effect changes if we provide information on the hypothesized mechanisms. We further implement two of the four remaining possible versions, in which information on both factors is provided (With voter–Against public and Against voter–With public), as this allows us to address some concerns about people drawing inferences from information on one factor about the other. We do not implement all nine but only seven versions due to resource constraints.Footnote ⁸

Table 3. Summary of vignette versions implemented at the respondent level

Outcome measures

If, as many have argued (e.g., Campbell et al., Reference Campbell, Cowley, Vivyan and Wagner2016; Wolkenstein and Wratil, Reference Wolkenstein and Wratil2021), rebellion is primarily a cue to voters about the type and quality of representation that they receive, our outcome measures should focus on whether respondents feel represented. To capture voter perceptions of representation by MPs, we use two measures that tap into personal feelings about representation and whether an MP represents an ideal MP in society. First, we consider whether respondents would like to be personally represented using the following question: “How well do you feel represented by this MP?” (“1—Not at all” to “7—Very well,” feel represented). Second, we ask respondents to reflect upon whether the displayed MP should serve as an ideal type for other politicians using the following question: “Do you agree or disagree with the following statement: Every [nationality] MP should be like this MP.” (“1—Strongly disagree” to “7—Strongly agree”; every MP like this MP). Thus, we specifically prompt respondents to think about representation when considering the MPs’ actions, as well as the extent to which they would like other MPs to engage in the same behavior as the MP they observe. This allows us to capture the depth of support for such behavior.Footnote ⁹

We furthermore elicit respondents’ associations of being a rebel with particular character traits of the MP after the last of our five vignettes by asking: “Which of the following character traits would you say describes the MP best? Please choose up to 3 of them,” with the list of possible answers including “independent,” “loyal to the party,” “honest,” “strong personal convictions,” “defends interest of his/her voters,” “willing to compromise,” “untrustworthy,” “selfish,” “disrespectful,” “decisive,” “irresponsible,” “cowardly,” “unreliable,” and “stupid” (MP traits).Footnote ¹⁰

Identification and estimation

To test hypothesis 1, we compute the difference in marginal means of the outcome measures feeling represented and every MP like this MP for profiles that contain a rebel and those that do not. In other words, the effect of rebellion on an outcome measure is estimated by computing $\tau \, \text {( Rebel) } = {1\over {\it N} }\sum _{\it i} ^{\it N} \text {Outcome}_i\, \text {( Rebel) } - {1\over {\it M} }\sum _{\it i} ^{\it M} \text {Outcome}_i \, \text {( No rebel) }$, where Outcome_i (Rebel) is the answer of respondent i on the outcome measure when being shown a vignette that features a rebelling MP, Outcome_i (No rebel) when a vignette with a non-rebelling MP is displayed, and N + M = 70,000 is the number of respondent-vignette observations in the study. In a fully factorized vignette experiment such as the one presented here, this difference between marginal means is known as Average Marginal Component Effect (AMCE; Hainmueller and others (Reference Hainmueller, Hopkins and Yamamoto2014)).Footnote ¹¹ We find evidence in support of hypothesis 1 if τ (Rebel) > 0.

To determine whether voters take rebellion as a cue for policy congruence with themselves (hypothesis 2) or with public opinion (hypothesis 3), we compute the share of the total effect of rebellion that is driven by policy congruence. This quantity measures the extent to which policy congruence mediates the effect of rebellion—that is, how much of the effect of rebellion is explained by the mediator policy congruence. In line with much literature, we conceive of this mediation effect as a causal mechanism, the process through which the treatment affects the outcome of interest (Imai et al., Reference Imai, Tingley and Yamamoto2013).

Following Acharya et al. (Reference Acharya, Blackwell and Sen2018) we compute the total effect of rebellion, the effect of rebellion when no information about policy congruence is provided as in the Baseline version of our design (see Table 3), τ (Rebel|Baseline), and subtract from it the component of the rebel effect that we know is not driven by policy congruence, the controlled direct effect. More precisely, the controlled direct effect is given by τ (Rebel|With voter) or τ (Rebel|With public). This leaves us with the rebel effect that runs through policy congruence, a quantity called the “eliminated effect” (Acharya et al., Reference Acharya, Blackwell and Sen2018, p. 357). Whenever τ (Rebel|Baseline) − τ (Rebel|With voter) > 0 we would say that we have found evidence that perceptions of representation are due to rebellion's link with policy congruence between the MP and the respondent (hypothesis 2). Equivalently, whenever τ (Rebel|Baseline) − τ (Rebel|With public) > 0 we find evidence that perceptions of representation are due to rebellion because of congruence between the MP and public opinion (hypothesis 3). The first term is the total effect of rebellion, which we operationalize as the effect of rebellion when information about policy congruence is absent (i.e., in our Baseline version). We subtract from this term the rebel effect that cannot be due to congruence because we hold congruence constant. We are left with the rebel effect on the outcome measures that must emerge from congruence.

Finally, we test hypothesis 4 by comparing the probability that a respondent attributes positive characteristics to rebels and non-rebels using the question following the final vignette in which we ask respondents to report attributes they associate with the MP they just saw. We find evidence for hypothesis 4 if respondents are more likely to select the attributes “independent,” “honest,” “willing to compromise,” and “defends interest of their voters” when they are shown the profile of a rebel MP than when shown a loyal MP. We also expect respondents who saw a rebel MP in their final vignette to be less likely to select “loyal to the party” compared with respondents who saw a non-rebellious MP.

Table 4 summarizes the empirical tests for all hypotheses. Note that some of the hypotheses can also be tested in further, alternative ways (e.g., as we have various additional vignette versions or expect some heterogeneity in treatment effects).Footnote ¹²

Table 4. Summary of empirical tests of the hypotheses

We estimate all quantities of interest in a linear regression framework with the main coefficient of interest being the one on the treatment indicator. Because respondents make several choices, unless otherwise indicated, we report standard errors clustered at the respondent level.

5. Follow-up experiment

As the diminishing rebel effect has not been found by previous studies (e.g., Campbell et al., Reference Campbell, Cowley, Vivyan and Wagner2016), we designed a supplementary, follow-up experiment to explore potential design effects and substantive mechanisms that could explain why support for rebels becomes less liked over the course of the vignette tasks.

We hypothesize that the diminishing rebel effect may either be due to design effects or substantive cognitive mechanisms in respondents’ minds. With regard to design effects, we are particularly concerned that we use different outcome questions in our experiment than Campbell et al. (Reference Campbell, Cowley, Vivyan and Wagner2016) and only present a single vignette instead of pairs, as these authors do. With regard to substantive cognitive mechanisms, we hypothesize two possible explanations for the diminishing rebel effect. (1) Surprise: rebellion in politics is a rare phenomenon—respondents may be surprised by it at first, and that surprise may generate an effect that wears off starting with the second vignette; (2) Reflection: on initial encounter, respondents may view rebellion positively, but repeated exposure to the possibility of rebellion may cause them to rethink and re-evaluate, retrieving more negative considerations about rebellion.

Our implementation of the follow-up experiment slightly deviates from the main experiment. First, it uses a UK-only sample of 2,307 respondents and was conducted after Brexit. We, therefore, do not ask respondents about competences of the EU and focus on the remaining four issues, only (see Table 1). Second, we only use the Baseline version of the vignette, meaning that respondents do not receive information about whether the rebellion goes in a direction that they or the public prefer.Footnote ¹⁸

The core of the follow-up experiment are five different treatment arms we designed to test each of the hypothesized explanations for the diminishing rebel effect. Beginning with design effects, we use the paired vignette treatment, in which respondents are presented with two sets of four choice tasks. One-fifth of respondents assigned to this treatment arm completed four choice tasks with two vignettes first—seeing two situations with MPs side by side—before completing four tasks with a single vignette afterwards. The “untreated” four-fifths of respondents see the single-vignette tasks first before also completing the paired-vignette tasks. In the alternative outcome treatment, we ask a random sub-set of respondents who see the single vignette design first “Based on this information, how much would you like to have this MP as your MP in the House of Commons?” instead of our own outcome measures. Both of these design choices combined are implemented in Campbell et al. (Reference Campbell, Cowley, Vivyan and Wagner2016), who find a stable rebel effect. Results show that using a paired vignette design delivers positive rebel effects for every MP like this MP but still provides evidence for a wearing off of any rebel effect in later vignettes. We find no rebel effect using the alternative outcome measure.Footnote ¹⁹ Thus, we can be confident that our findings of diminishing effects are not merely a function of these design choices.

Hence, we test the two substantive cognitive mechanisms that could explain the diminishing rebel effect by randomly assigning those three-fifths of respondents seeing the single vignette design first as well as our outcome questions to three further treatment arms: (1) in the before reminder treatment arm, we alert the respondents before they see the first vignette that MPs sometimes vote against their party and ask them to comment on rebels in parliament in an open-ended question; (2) in the between reminder treatment, we randomly assign a reminder either before the third or the fourth vignette. Both treatments speak to the first mechanism by making the idea of a rebellious MP available in respondents’ minds when they make a decision, diminishing the surprise of seeing a rebel. (3) In the MP traits treatment, we elicit respondents associations of character traits with rebellious MPs not only after the last vignette as in the main experiment, but randomly assigned either after the first, second or third vignette. Here, we wish to measure whether respondents perceptions of rebels turn sour with being exposed to a rebel either repeatedly or in later vignettes, speaking to the reflection mechanism.

In these treatment arms, we use as baseline observations those choice tasks from the between reminder treatment where no reminder, and from the MP traits treatment where no question on character traits have been shown yet (vignettes 1 and 2 or 1, 2 and 3).Footnote ²⁰ For the baseline observations, we replicate the drop in the positive effect of rebellion on the every MP like this MP outcome measure between 1st and 2nd vignette out of the four vignettes shown to all respondents. While the AMCE of rebellion is 0.19 ( − 0.03,  0.40) in the first vignette, it is −0.01 ( − 0.24,  0.23) in the second vignette as shown at the top of Figure 5 (top two black markers); the difference in AMCE is not significant.Footnote ²¹ The AMCEs of rebellion on the outcome measure feeling represented are negative in both the first and second vignette, −0.17 ( − 0.38,  0.04) and −0.19 ( − 0.42, − 0.05), respectively.

Figure 5. AMCE of rebel attribute on outcome variables feeling represented and every MP like this MP for baseline observations and observations with reminder. Note: We report 95 percent (thin line) and 90 percent confidence bounds (thick line) based on clustering standard errors at the respondent level.

We test the surprise mechanism by comparing the AMCE of rebellion for the baseline observations to those treatment observations that received a screen reminding them about the possibility of rebellion either before any or between vignettes (before reminder and between reminder treatments). We posited that when rebellion does not come as a surprise to respondents, the rebel effect should disappear. Figure 5 demonstrates that when participants are shown a reminder of the possibility of rebellion before one of the four vignettes (depending on treatment assignment, gray markers), the rebel effect is zero or even negative for the every MP like this MP outcome measure from the first vignette and drops further negative for the feeling represented outcome measure from the first and second to the third vignette. In other words, we find evidence for the mechanism; the positive rebel effect only exists if rebellion is something new. The initial rebel effect is driven by respondents who show a positive sentiment toward rebellion in the open-ended question in the between reminder treatment arm.Footnote ²²

We fail to find evidence for the reflection mechanism that suggests seeing rebels repeatedly or in a later vignette causes more critical perceptions of the rebels. Figure 6 shows, first, that respondents hold similar associations with rebels as they do in the first experiment—they are more likely to see rebels as independent and with strong personal convictions (although not as honest) but less likely to see them as loyal to their party (also see Figure 3). It also illustrates that the positive associations with a rebel MP, being independent and with strong personal convictions, as well as the negative association with a rebel MP, being loyal to the party, if anything, increases in frequency from vignette 1 to 4.Footnote ²³

Figure 6. Effect of rebellion on whether respondents attribute a character trait to the MP. Estimates taken from a regression of binary indicator whether a trait is attributed on rebel status interacted with after which vignette those associations were elicited. Note: We report 95 percent (thin line) and 90 percent confidence bounds (thick line) based on clustering standard errors at the respondent level.

With this follow-up experiment, we have set out to explain the diminishing rebel effect. We find that the wearing off of the rebel effect is not due to us using a single- instead of paired-vignette design, nor is it due to different outcome measures. Instead, the rebel effect completely disappears when respondents are reminded of it before the first effect, while their assignment of character traits to rebels becomes more positive with every vignette they see. We conclude that this diminishing effect is, therefore, most likely due to the surprise mechanism; rebellion is rare, and if it remains rare, voters seem to like it. However, once rebellion becomes more frequent, voters still perceive rebellion as a valence signal for the single MP, but do not seem to like the act of rebellion anymore.