1. Explaining party switching in postwar Japan

Party switching is nothing new in Japanese politics because it has occurred on both small and larger scales throughout the postwar period.Footnote ² Examples of smaller-scale switching include the 1960 exit of right-leaning members of the former Japan Socialist Party (JSP) to form the Democratic Socialist PartyFootnote ³ and, in 1977, when another group of right-leaning JSP members left the party to form the Social Democratic League. Examples also include a number of anti-corruption Liberal Democratic Party (LDP) members exiting this predominant party to form the New Liberal Club in 1976.

There are also cases of more significant party switching and new party formation such as the changes that occurred surrounding the 1993 and 1996 lower-house elections. The former contest witnessed two new parties formed by exiting members of the LDP, specifically, the Japan Renewal Party, formed by 44 former LDP incumbents, headed by Tsutomu Hata, and the New Party Harbinger also formed by break away LDP incumbents, led by Masayoshi Takemura. The 1993 election also witnessed another newly formed party founded by Morihiro Hosokawa, who was the former governor of Kumamoto Prefecture and a member of the House of Councilors.Footnote ⁴

This election produced the first non-LDP government in 38 years, but the anti-LDP coalition that allowed this to occur did not last long as party splits, dissolutions, and mergers continued until the 1996 election held in October of that year. New parties competing in this election included the New Frontier Party, a merger of several smaller parties, and the DPJ, which formed a month before the election mostly out of anti-LDP incumbents, including some formerly associated with the New Party Harbinger and some erstwhile Socialists.

The DPJ turned in less than optimal performances in the next three elections of 2000, 2003, and 2005, obtaining 127, 177, but then 113 seats respectively in each. Four years later in 2009, however, the DPJ swept the LDP out of power by obtaining 308 Diet seats to the LDP's 119. While party leader Yukio Hatoyama become Prime Minister with much popularity and fanfare, the DPJ's electoral fortunes quickly soured, and the party began experiencing a secular decline that continued until its dissolution in 2017. The DPJ's decline began with many incumbents exiting and forming new splinter parties, and these changes led the DPJ to enter the 2012 election with only 230 of the 308 incumbents it had in 2009.

These small- and large-scale changes that occurred in postwar Japan have been addressed in the scholarly literature in both descriptive and historical formats as well as in more theoretically informed analyses. The former is represented by such studies as those by Curtis (Reference Curtis1989, Reference Curtis1999) and Hrebenar (Reference Hrebenar2000),Footnote ⁵ while the latter group of scholars – whose study informs the research in this paper – was motivated primarily by the larger shifts that began with the election of 1993. Specifically, Cox and Rosenbluth (Reference Cox and Rosenbluth1995) noted that electoral support considerations were the most important factors that led a substantial number of LDP members to join newly created parties in 1993, while Reed and Scheiner (Reference Reed and Scheiner2003) argued that legislator motives were mixed and included both electoral concerns but more importantly policy interests, particularly, a strong commitment to political reform. Nyblade (Reference Nyblade2012) focused on the DPJ exits that occurred prior to the election of 2012 and found that these exits were a mixture of electoral, office, and policy concerns but also that impacts of these factors were not monotonic.Footnote ⁶

Another analysis that focused on the events leading up to the election of 1993 and beyond was by Kato (Reference Kato1998), who reformulated the exit, voice, and loyalty perspective of Hirschman (Reference Hirschman1970), by arguing that legislators considered switching parties for one additional reason, specifically, concerns with the quality of the public goods their political party produces.Footnote ⁷ Because this is a major part of a political party's reputation, it can lead elected officials to consider exercising an exit option, even in the face of considerable electoral risk.

This scholarship has been informative to be sure, but it has generally not considered how electoral institutions and available party alternatives lead incumbents to weigh their party exit decisions and the benefits they hope to obtain against the risks associated with moving to a new organization.Footnote ⁸ We know from Rosenbluth and Thies (Reference Rosenbluth and Thies2010) that Japan's parallel electoral system put in place in 1994 significantly changed the incentives faced by parties and individual politicians, making Japanese elections less personalistic and more oriented on policy. Nonetheless, we still have not explored how the altered incentives attendant to this parallel system affect party exit decisions, especially how the benefits associated with the factors that motivate legislators' decisions to consider switching political parties are weighed against sources of electoral risk.

This is particularly important since the rules under which this election was held involved a reduced amount of electoral risk for some legislators but not for others. The exit decisions of 88 DPJ incumbents in the 2017 snap election will allow us to accomplish this because we will be able to control for the individual factors that motivated legislators to contemplate switching parties in light of the electoral risks they faced. Our effort begins with a discussion of this 2017 snap election, particularly those features that make it an appropriate contest to calibrate the impact of alternative party choices, electoral institutions, and personal motivations on exit decisions.

2. Japan's 2017 snap election and the affiliations of DPJ incumbents

The 2017 snap election was called by former Prime Minister Abe Shinzo more than 14 months prior to its constitutionally required time, and it involved competition for Diet seats that included both old and new political parties. Indeed, 3 days before the House of Representatives was formally dissolved, former Tokyo Governor, Koike Yuriko, announced the formation of a new political party, the Party of Hope (POH) or ‘Kibo no Toh,’ and 5 days after the House of Representatives was dissolved, Edano Yukio, the DPJ's General Cabinet Secretary, announced that he would form another new political party, the Constitutional Democratic Party (CDP).

These two parties were created to serve as alternative parties for incumbent members of the DPJ, who were in a political party that had essentially reached the end of a long period of electoral decline. The DPJ's decline affected its remaining 88 incumbents, who were forced to confront the decision of how to compete in this election. This is because its leader, Maehara Seiji, announced that the DPJ would not endorse any candidates in this upcoming election, and, as a result, DPJ incumbents were forced to compete as unaffiliated candidates or join either the POH or the CDP.

Given the scheduling of this election, DPJ incumbents had much less time to prepare than if it was scheduled closer to its constitutionally prescribed date, and this is also true for leaders of both the POH and the CDP, who barely had time to help endorsees be ready to compete successfully for the Diet seats they held. Naturally, these DPJ incumbents would enter this contest with different levels of electoral strength, which forced them to consider the risk level attendant to their decisions to join one party or the other or compete without a party endorsement. These considerations would also have to be weighed not only against these members' own policy preferences, but also against the policy preferences of the leaders of these two new alternative parties they might be interested in joining. This is important because it was not enough simply to choose which party they wanted to join based on their electoral chances and their individual policy preferences because Koike, in particular, had clear preferences with respect to the policy profiles of exiting DPJ incumbents she preferred to welcome into the POH.

The former Tokyo governor enjoyed growing popularity up to the election that was arguably greater than former DPJ leader, Edano, which made the POH an attractive alternative, but the former Tokyo governor had specific policy preferences that could raise concerns among at least some DPJ incumbents. Indeed, while she held progressive positions on elevating the status of women and imposing a carbon tax, both to improve the quality of Japan's environment and to reduce its contribution to greenhouse gas emissions, she also held many policy positions that were conservative and nationalist. Among these was her support for official visits to Yasukuni Shrine and history textbook reform, but most notable was her favorable stance on revising Article 9 of the postwar Japanese constitution so that Japan could finally pursue a security policy of collective self-defense.

Her position on Article 9 very possibly complicated the exit decisions of some DPJ incumbents because, while many wanted this party's endorsement for the 2017 election, others did not want to be associated with Koike's advocation of constitutional revision. At the same time, the POH leader was not interested in extending endorsements to those DPJ incumbents who strayed very far from her preferred policy profile. Not having policy preferences that lined up with the POH's leaders did not mean that exiting incumbents had to compete in this election as unendorsed candidates because Edano's CDP was less strict in terms of the policy profiles of former DPJ legislators who would be welcomed into his start-up political party.

What is important here is that we expect decisions to join the POH – or the CDP for that matter – to depend on incumbents' level of opposition to Koike's policy preferences, making strong opponents more likely to seek membership in Edano's CDP. Overall, for DPJ incumbents who were weakly opposed or unopposed to Koike's position, their exit decisions should be less influenced by their policy preferences and, thus, rendered more in obeisance to an office-seeking imperative, that is, how their extant electoral support levels related to the risk of exiting. On the contrary, for those incumbents who were strongly opposed to Koike's position, they would be more likely not to choose, nor be allowed to move to, this start-up party and have to seek a different alternative.

While DPJ incumbents were naturally concerned with both remaining in office and with promoting preferred policies,Footnote ⁹ this does not mean that these two factors shared equal weight across all DPJ incumbents. This is because some candidates were strict policy seekers, others strict office seekers, and still others were motivated by how these influences interacted with each other.Footnote ¹⁰ This means that we must consider both of these individual factors and the potential impact of available alternative parties and electoral institutions. As suggested above, electoral risk would be a function of a candidate's support levels, where high-support level candidates faced less risk compared to weaker candidates, but even electorally weaker candidates could face lower levels of electoral risk compared to others with similar levels of support. This is because Japan's electoral system is a parallel system where office seekers compete for lower house seats in either single-member districts (SMDs) or as a member of a party list a larger proportional representation (PR) district.

In many if not most cases, candidates picked one or the other as the place they would compete for a Diet seat, but there were also incumbents who competed for an SMD seat while simultaneously having their name put on a party's list of PR candidates. This latter group of incumbents had dual or overlapping candidacies, which meant that, if they did not obtain an SMD seat but were high enough on the party list, they could still obtain a seat in their respective PR district. Incumbents who lost an SMD seat but got elected as a member of their party's PR list became known as ‘Zombie Incumbents,’ because they were able to rise from their electoral deaths after losing an SMD seat through the PR portion of Japan's parallel electoral system.Footnote ¹¹

3. Policy, party choice, and electoral risk in exit decisions

To calibrate how individual motivating factors were weighed against the electoral risks associated with Japan's electoral institutions and party alternatives in the exit decisions of DPJ incumbents, we first focus on Japan's electoral rules. Again, Japan's parallel system allows candidates to pursue dual or overlapping candidacies, whereby candidates choosing this option are expected to experience less electoral risk than those who chose to compete strictly in one of the system's SMDs.Footnote ¹² In this snap election, many DPJ incumbents competed for Diet seats with an overlapping candidacy, and these incumbents as well as those who did not are presented in Table 1.

Table 1. Exit decisions of Zombie vs non-zombie incumbents

χ ² = 6.1463.

PR > χ ² = 0.0462.

We observe that 48 out of 88 (55%) of these DPJ incumbents were Zombie winners, while 40 out of 88 (45%) were exclusively SMD winners, and, thus, did not benefit from the reduced electoral risk the Japanese electoral system afforded. We also observe from the data in the table that 67% of Zombie incumbents – 32 out of 48 – decided to join Koike's POH while one-fifth joined the CDP and the remainder (15%) competed without party endorsement. Moreover, 48% of SMD-only incumbents – 19 out of 40 – moved to the POH, 15% to the CDP, with nearly 40% seeking a Diet seat without a party endorsement.

These data suggest strongly that Zombie incumbents were much more likely to join the POH compared to their SMD counterparts, which tells us that Japan's electoral institutions were an important factor influencing DPJ incumbent motivations to join the POH, while SMD-only incumbents were much more likely to compete without a party's endorsement.Footnote ¹³ However, to investigate the statistical significance of this relationship, we performed a χ ² test on the relationship between institutional risk and exit choice, and the results produced a χ ² of P > 0.0462, which indicates a statistically significant relationship at the 95% level of confidence.

These results indicate that the institutional impacts in this election could be a direct influence on exiting DPJ incumbents' decisions to join the POH or seek their Diet seats without a party's endorsement but not to join the CDP. Nonetheless, we need to expand our analysis and explore the possibility that more nuanced relationships exist when exiting legislators weigh the benefits they hope to obtain by switching parties against all sources of electoral risk. To accomplish this, we need to refocus our attention on available party alternatives while controlling for the influence of the individual factors discussed above as well as other control variables.

As mentioned above, the individual factors we consider in this analysis are the strength of an incumbent's support levels entering the 2017 election as well as candidates' policy preferences. Figure 1 presents a theoretical illustration of these relationships as well as our expectations for how they will affect an incumbent's decision of whether to join the POH or the CDP or compete in the election without a party endorsement. Specifically, the x-axis in the figure captures an incumbent's policy preference, in this case, his/her position on constitutional revision (PCR), which was the most important policy factor accounting for a DPJ incumbent's decision to join the POH. The y-axis shows the electoral strength of an incumbent, measured as his/her vote share in the 2014 lower house election. This theoretical representation of the interplay of policy preferences and electoral risk leads to two clear behavioral expectations and two that are less clear but will be investigated more deeply in the empirical analysis we conduct below.

Figure 1. Policy position, electoral strength, and the decision to exit.

The variable of interest here is our expectations about the likelihood that a candidate would move to Koike's POH prior to the 2017 lower house election.Footnote ¹⁴ Here, we categorized 88 DPJ incumbents into four types in terms of their policy preference (x-axis) and electoral strength (y-axis). We hypothesize that, if former DPJ incumbents are purely policy-seekers, then these incumbents will be guided by an issue profile closer to the POH's (i.e., those categorized in quadrants II and IV) and they should be more likely to join the POH, regardless of their electoral strength.Footnote ¹⁵ On the contrary, if former DPJ incumbents are purely office-seekers, then those electorally weaker incumbents (i.e., those categorized in quadrants III and IV) should be more likely to join the POH, regardless of their policy profile.

In reality, however, DPJ incumbents are most likely to be neither pure policy seekers nor pure office seekers, which means that most fall somewhere in between these two pure groupings. As a result, we hypothesize that those electorally stronger incumbents holding policy positions contrary to the preferences of POH leader, Koike Yuriko, and fall into quadrant I should have lower likelihoods of joining the POH. On the contrary, we expect that those electorally weaker incumbents holding pro-POH policy positions, that is, those who fall into quadrant IV, should have a higher likelihood of joining the POH.

For incumbents who noted they oppose revising the Japanese constitution, their exit decisions will be more influenced by concerns with their electoral strength. Specifically, if candidates are against revising Japan's constitution and their vote shares in the 2014 lower house election are small, that is, they are electorally weak, then they are increasingly likely to move to the POH so that they can take advantage of that party's popular leader, increasing their likelihood of winning a seat in this snap election.Footnote ¹⁶ However, if DPJ incumbents are against constitutional revision and their electoral margins in the 2014 lower house election were large, that is, they were electorally strong, they may not be as inclined to move to the POH.Footnote ¹⁷ Specifically, such incumbents may still exit to the POH because doing so will help ensure their winning seats, or they may decide to join the CDP or they may simply to decide to compete without a party endorsement.

While these probabilistic relationships are straightforward, we are less certain about how those incumbents categorized in quadrants II and III in Figure 1 will behave in their decision to join one of the available start-up parties or compete without a party endorsement. We assume that these incumbents will be motivated by mixed incentives in their exit decisions, weighing both their policy preferences and electoral strength in light of these factors' impact on their ability to secure a Diet seat in this snap election. Given that their calculus is based on how these factors interact, we can say that incumbents in quadrant III are likely to have a higher probability of joining the CDP than the POH and that incumbents in quadrant II are more likely to join the POH than those in quadrant III. However, to be more certain of these expected relationships, we explore them empirically in more detail in the next section.

4. The impact of policy, party alternatives, and electoral risk on exit decisions

The statistical analysis we conduct below is designed to disentangle the relationships that exist between how incumbents considering party exits weighed the individual benefits they expected to receive against the electoral risks they encountered from both the types of start-up parties available to them and Japan's unique electoral system. To disentangle these relationships and calibrate the influences of those factors we discussed above, we must first note that exiting DPJ incumbents who decided to compete in the 2017 snap election had three choices before them. They could try and join one of the two new start-up parties,Footnote ¹⁸ Koike's POH or Edano's CDP, or they could enter the 2017 election without a party endorsement.

The multiple choices before DPJ incumbents then make our variable to be explained a multi-valued categorical variable. Because of this, to calibrate the influence of those risk and policy factors we discussed above, as well as any control variables that we include to avoid omitted variable bias, we must employ a multinomial logit analysis. Our analytic effort then focuses on whether a DPJ incumbent joined one of the two available start-up parties (the POH or the CDP) or competed in the 2017 election as an unaffiliated candidate. Because the dependent variable is a categorical variable with three values, we note that exit takes on a value of 1 if a former DPJ incumbent joined the POH, 2 if he/she joined the CDP, or 3 if the former DPJ candidate entered the election without a party endorsement.

As shown in Table 1, 51 out of 88 joined the POH, which as mentioned above, tells us that more than half (58%) of the DPJ's incumbents moved to Koike's POH.Footnote ¹⁹ When defined as what explains an incumbent's decision to join the CDP, we observe that 15 (17%) out of 88 of the DPJ's incumbents moved to the CDP, which means that fewer than one-fifth of exiting incumbents joined Edano's start-up party.Footnote ²⁰

With the variable to be explained defined and categorized for the analysis, we turn next to the variables that we include in our models to capture the impacts associated with electoral strength, policy preferences, and other controls to avoid omitted variable bias in our models. The first of these individual factors we include in the models we estimate below refers to incumbents' policy preferences where we focus on incumbents' views with respect to the following four issues:

1. (1) revising the Japanese Constitution (PCR),

2. (2) collective self-defense (COLLECTIVE),

3. (3) level of Japan's defense capability (DEFENSE), and

4. (4) Japan initiating a preemptive strike (PREEMPTIVE).

Each of these four variables captures an aspect of DPJ incumbents' policy preferences, and they are all calibrated on a five-point ordinal scale. Specifically, a value of 1 refers to an anti-POH preference on these issues; 2 refers to an incumbent's position being moderately anti-POH; 3 indicates an incumbent's position being indifferent with respect to each issue; 4 refers to an incumbent's position being moderately pro-POH; and 5 refers to an incumbent's position being pro-POH. These policy preference variables are based on data collected in the Asahi Today Survey of Diet Members taken in 2014.Footnote ²¹

The second individual-level factor we consider in this analysis is a DPJ incumbent's electoral strength entering the 2017 snap election, what we refer to as that candidate's electoral margin. This indicator is intended to capture the level of risk associated with an individual incumbent's decision to compete as an unaffiliated candidate or with an endorsement from one of the available start-up parties. It is measured as the electoral margin of the exiting incumbent compared to the runner-up in the 2014 House of Representatives Election. The larger a margin, the stronger an SMD incumbent was entering the 2017 election. We calculate candidates' electoral margins for SMD incumbents and Zombie incumbents on the same scale.

For example, let's suppose there are two candidates in an SMD, candidate S (SMD winner) and candidate Z (Zombie winner). Candidate S received 100,000 votes and candidate Z received 90,000 votes. The margin for candidate S is calculated as candidate S's votes divided by candidate Z's votes: 100,000/90,000 = 1.11. The margin for candidate Z is calculated as candidate Z's votes divided by candidate S's votes: 90,000/100,000 = 0.9.

Thus, the SMD winner's margin is always larger than 1, and Zombie winner's margin ranges from 0 to less than 1. The larger the margin, the electorally stronger the incumbent would be entering the snap election. Calculated this way, we can compare incumbent's relative electoral strength, regardless of the two different types of incumbents. The electoral margin used for Zombie incumbents is equivalent to ‘Sekihai ratio’ officially used to select Zombie incumbents in Japan's Lower House elections since 1996. The average 2014 margin captured by DPJ incumbents was 0.97 where this measure of electoral strength ranged from a low of 0.397 to a maximum of 1.556. In our analysis, however, we use both the actual and squared values of this ‘Sekihai ratio,’ because while it may not be monotonic and impact exiting legislators' choices in a nonlinear manner, it may also be linear and including both terms is the appropriate way to capture these possibilities (Nyblade, Reference Nyblade2012: 26).

The multinomial logit models, we estimate below contain two additional control variables, and the first of these is designed to capture an incumbent's level of electoral experience, specifically, how accomplished that incumbent has been accumulating electoral victories. This variable (Previous) then is constructed as an integer capturing the number of past electoral victories an incumbent had prior to entering the 2017 snap election. Electoral victories for DPJ incumbents averaged 3.23 where the minimum value was 0 and the maximum number of electoral victories for a DPJ incumbent was 11.Footnote ²² We included both the integer and squared values of this variable in our models because we wanted to determine if its relationship with DPJ incumbents' exit decisions is monotonic and linear or nonlinear. The second control variable is the age of incumbents competing in this snap election. The average age of these incumbents was 51, and the youngest incumbent was 35 while the most senior was 68. In our analysis, we included both the integer and squared values of an incumbent's age, because we also want to be able to determine if its impact is monotonic or nonlinear.

With all explanatory factors defined, we estimate four multinominal logit models, where model 1's and model 2's baseline category is an exiting legislator choosing the POH and model 3's and model 4's baseline category is an exiting legislator choosing CDP.

4.1 Model 1 (baseline = POH)

$$\eqalign{{\rm ln}\left({\displaystyle{{P( pty17 = CDP) } \over {P( pty17 = POH) }}} \right) =\,\, & \alpha _1 + \alpha _2PCR + \alpha _3COLLECTIVE + \alpha _4DEFENSE \cr & \quad + \alpha _5PREEMPTIVE + \alpha _6Margin + \alpha _7Margin_{sq} \cr & \quad + \alpha _8Age + \alpha _9Age_{sq} + \alpha _{10}Previous + \alpha _{11}Previous_{sq}} $$

4.2 Model 2 (baseline = POH)

$$\eqalign{{\rm ln}\left({\displaystyle{{P( pty17 = IND) } \over {P( pty17 = POH) }}} \right) = & \,\,\alpha _{12} + \alpha _{13}PCR + \alpha _{14}COLLECTIVE + \alpha _{15}DEFENSE \cr & \quad + \alpha _{16}PREEMPTIVE + \alpha _{17}Margin + \alpha _{18}Margin_{sq} \cr & \quad + \alpha _{19}Age + \alpha _{20}Age_{sq} + \alpha _{21}Previous + \alpha _{22}Previous_{sq}} $$

4.3 Model 3 (baseline = CDP)

$$\eqalign{{\rm ln}\left({\displaystyle{{P( pty17 = IND) } \over {P( pty17 = CDP) }}} \right) = & \,\,\beta _1 + \beta _2PCR + \beta _3COLLECTIVE + \beta _4DEFENSE \cr & \quad + \beta _5PREEMPTIVE + \beta _6Margin + \beta _7Margin_{sq} \cr & \quad + \beta _8Age + \beta _9Age_{sq} + \beta _{10}Previous + \beta _{11}Previous_{sq}} $$

4.4 Model 4 (baseline = CDP)

$$\eqalign{{\rm ln}\left({\displaystyle{{P( pty17 = POH) } \over {P( pty17 = CDP) }}} \right) = & \,\,\beta _{12} + \beta _{13}PCR + \beta _{14}COLLECTIVE + \beta _{15}DEFENSE \cr & \quad + \beta _{16}PREEMPTIVE + \beta _{17}Margin + \beta _{18}Margin_{sq} \cr & \quad + \beta _{19}Age + \beta _{20}Age_{sq} + \beta _{21}Previous + \beta _{22}Previous_{sq} + \beta _{24}} $$

Figure 2, which is divided into two sub-figures, shows the results of estimating our four multinomial logit models. Specifically, it contains the log odds of the variables we included in all four models, which because they take on positive and negative values offer a clear way to determine if a factor raises or lowers the odds of an incumbent choosing the CDP or competing without an endorsement over joining the Party of More. Specifically, the estimates in the figure indicate the effect of a unit increase in the value of a predictor variable on the odds of selecting one of the two specified categories on the dependent variable instead of the correct response, which is designated as the base category. In addition to the value of the log odds ratios of each of the models' explanatory variables, the data in Figure 2 also contain error variances around each log odds ratio, which provide a visual method to distinguish those estimates that are statistically significant from those that are not. Our examination of the impact of the policy, risk, and control factors on exit decisions begins with how the security policy preferences of DPJ incumbents affected their exit decisions.

Figure 2. Results obtained by estimating multinominal logit models 1–4. (a) Log odds for models 1 and 2. Note: The details on these results (log odds and odds ratio) are presented in Appendix 5. (b) Log odds for models 3 and 4. Note: The details on these results (log odds and odds ratio) are presented in Appendix 6.