A substantial body of research has found persistent patterns of recruitment bias across a variety of societal spheres. Research on the hiring process has studied discrimination based on identity (i.e., demographic factors) and, in a few cases, ideology (e.g., Bertrand and Mullainathan Reference Bertrand and Mullainathan2004; Booth and Leigh Reference Booth and Leigh2010; Ahmed, Andersson, and Hammarstedt Reference Ahmed, Andersson and Hammarstedt2012; Gift and Gift Reference Gift and Gift2015; Neumark, Burn, and Button Reference Neumark, Burn and Button2016; Adida, Laitin, and Valfort Reference Adida, Laitin and Valfort2016; Vernby and Dancygier Reference Vernby and Dancygier2019). While experimental studies are rarer, there is also a rich body of work on bias in the selection of politicians (e.g., Lawless and Fox Reference Lawless and Fox2005; Dancygier et al., Reference Dancygier, Lindgren, Oskarsson and Vernby2015; Folke and Rickne Reference Folke and Rickne2016; Carnes Reference Carnes2016; Dancygier, Lindgren, Nyman, and Vernby forthcoming; Eriksson and Vernby Reference Eriksson and Vernby2021; Kalla, Rosenbluth, and Teele Reference Kalla, Rosenbluth and Teele2018).
In this article, we employ a field experiment to study selection in the judiciary, a domain that has received less attention than the economic and political spheres. In particular, we focus on the Swedish process of selecting lay judges (citizens who have only minimal legal training and who participate in the judicial decision-making of courts). In this process, private citizens are encouraged by the government to put themselves forward as candidates by contacting local party officials, who control the nominations. This provides an excellent opportunity to test the impact of demographic factors, as well as partisan cues, at this first, and important, step towards becoming a lay judge. If party officials are systematically more responsive to candidates with certain ideological or identitarian traits, this would be troubling from a normative perspective. This is not only a matter of fairness in selection but also a concern for the potential for political bias in court rulings, as previous research shows that judicial outcomes are substantially affected by the ideology and demography of decision-makers (Anwar, Bayer, and Hjalmarsson Reference Anwar, Bayer and Hjalmarsson2012, Reference Anwar, Bayer and Hjalmarsson2014, Reference Anwar, Bayer and Hjalmarsson2019; Martén Reference Martén2015; Boyd Reference Boyd2016; Harris and Sen Reference Harris and Sen2019).
The experiment took place in the midst of a government communication campaign that was carried out by the Swedish National Courts Administration in the spring of 2019. The main aim of the campaign was to encourage citizens from all backgrounds and of all ages above 18 to contact local party branches and put themselves forward as candidates for the next 4-year term of office (Ridal Ceder and Yngve Reference Ridal Ceder and Yngve2020). During this campaign, we sent e-mails from fictitious individuals to local party officials expressing an interest in becoming a lay judge and asking for more information about the position. We employed a factorial design, where the content of the e-mails sent to party officials was randomly varied along a number of dimensions. First, half of the e-mails contained a signal that suggested that the writer was a long-time party sympathizer. This allowed us to test the extent to which political allegiances play a role in recruitment for these supposedly apolitical positions. Second, by randomly varying the name of the e-mailer, we varied signals of supposed gender (male/female-sounding names) and supposed ethnic origin (Arabic/Nordic-sounding names). Third, we included signals of the applicant’s age (25/35/45/55/65). Having sent out these e-mails, we then recorded whether local party branches’ likelihood of responding was affected by these ideological and demographic signals. We also recorded the informativeness of responses, and the degree to which they used a warm and welcoming tone.
The response rate was 42%. While contacting the local party organization through their official e-mail is not the only channel for ordinary citizens to signal their interest in becoming a lay judge, it is the one suggested by the government communications campaign, so this response rate must be considered less than ideal. However, we found that signals of political allegiance had little impact on the likelihood that local party officials would respond, give informative answers, and use a warm and welcoming tone. The same was true for signals of gender. Turning to ethnicity, the results were mixed and inconclusive; for some of the outcomes studied, there was small bias against e-mailers with Arabic-sounding names. The same is true of the results for age; for some outcomes, there appeared to be a slight preference for candidates in their 30s and 40s over both older and younger candidates.
We make several contributions to the literature. First, while there have been experimental studies of bias in jury selection, for example through the use of peremptory challenges (Sommers and Norton Reference Sommers and Norton2007; Norton, Sommers, and Brauner Reference Norton, Sommers and Brauner2007), our study is the first field experimental study of ideological and identitarian bias in the selection of laymen jurors.
Second, our study shows that recruitment in the judiciary by officials of political parties does not always result in demographic and ideological bias. In the Swedish case, the aim of the selection is to generate a demographically diverse set of lay judges with regard to age, gender, and ethnic background who are not necessarily active in political parties (Fritz Reference Fritz2016). From this perspective, our results are reassuring, as there appears to be limited bias with respect to most of these factors at this stage of the selection process. Moreover, and although we only study the first step in the process of becoming a lay judge, our results suggest that political favoritism toward party members or specific age and ethnic groups may not explain the existing discrepancy between societal demographics and the overall composition of lay judges in Sweden. Footnote 1 Instead, alternative explanations should be considered, including the possibility that representational gaps begin already at the supply stage.
Third, we contribute to the broader literature on how descriptive representation manifests itself in different societal spheres. While research in the Swedish context has found age, gender, and ethnic bias in the labor market, as well as in the recruitment of party activists (Ahmed, Andersson, and Hammarstedt Reference Ahmed, Andersson and Hammarstedt2012; Eriksson and Vernby Reference Eriksson and Vernby2021; Vernby and Dancygier Reference Vernby and Dancygier2019), we find little such evidence in our study. A general lesson from these results, then, is that biased recruitment in one societal sphere does not necessarily spill over to biased recruitment in other spheres.
Case and methodology
The Swedish case
In most legal systems, laymen participate in judicial decision-making. In countries that follow the English legal tradition, such as the USA, England, Canada, and Australia, among others, this is institutionalized through the jury system. Other countries, such as Germany, Finland, Sweden, and most of continental Europe, use lay judges. Both types of systems attempt to increase the legitimacy of the judicial system by recruiting an impartial and demographically representative body of laymen as judges. Whereas juries in systems that follow the English tradition are made up of a random selection of citizens, lay judges are randomly drawn from a pool of candidates established by local elections or political nominations (Anwar, Bayer, and Hjalmarsson Reference Anwar, Bayer and Hjalmarsson2019). Footnote 2
In Sweden, lay judges are citizens with only minimal legal training who participate in the judicial decision-making of courts. These lay judges are supposed to be apolitical and, like in a traditional jury system, broadly representative of the population (Ridal Ceder and Yngve Reference Ridal Ceder and Yngve2020). Footnote 3 In principle, any citizen can volunteer to become a lay judge by contacting one of the political parties, which control the selection process. There is no formal requirement for membership in any specific political party to become a lay judge, and the parties are supposed to recruit broadly from outside of their membership (Nilsson, Ahlstrand, and Lyckman Reference Nilsson, Ahlstrand and Lyckman2007).
Recent evaluations show that in 2015 almost all (95%) Swedish lay judges were party members, as well as disproportionately old and native-born compared to the general population (Fritz Reference Fritz2016). Footnote 4 To address these discrepancies, in 2016 the Swedish National Courts Administration was commissioned by the government to carry out a communications campaign, prior to the election of new lay judges in 2019. The aim of the campaign was to reach more people outside of party membership circles, as well as more young people (ages 18–44), and to inform them of the opportunity of becoming lay judges. To this end, the communications were carried out continuously between 2016 and 2019 in dialogue with both the nominating and electing bodies. In addition, an extensive campaign encouraging members of the public to put themselves forward as candidates for the position of lay judge was carried out between March and June of 2019 (Ridal Ceder, and Yngve Reference Ridal Ceder and Yngve2020). Footnote 5
Experimental design
We employed an e-mail correspondence study with a factorial design, where we varied the party sympathy, signals of supposed ethnic background and gender, and the age of fictitious e-mail senders. Footnote 6 Because of the crucial function of political parties as gatekeepers for the selection of lay judges on Swedish courts, all local branches of the eight political parties represented in the Swedish parliament were contacted. Footnote 7 In total, 2,104 e-mail addresses were collected using the parties’ web pages. Footnote 8 Since our sample consisted of the universe of local party branches, its size could not be further increased without sending multiple e-mails to each addressee. Footnote 9
On April 17, 2019, in the midst of the Swedish National Courts Administration’s public campaign (March–June) described above, we launched our experiment: Each local party branch was contacted by a fictitious e-mailer expressing interest in becoming a lay judge, asking about the possibility of becoming nominated by the party and the requirements for becoming a lay judge. Each addressee received only one e-mail. For an example e-mail, see Figure 1. Footnote 10
Figure 1 Example of e-mail sent to the political parties.
Of the e-mail addressees, half were randomly selected to receive an e-mail signed with an Arabic-sounding name. The other half of the e-mails were signed with a traditional Nordic-sounding name. The main reason for focusing on Arabic-sounding names was the possibility of observing discriminatory behavior. Research on ‘ethnic hierarchies’ in other societal spheres shows that people belonging to this group is among the most discriminated against in Sweden (e.g., Vernby and Dancygier Reference Vernby and Dancygier2019), suggesting that if we failed to uncover discrimination against this group when contacting parties about becoming a lay judge, people with names that sounds common to other ethnic minority groups are also unlikely to meet discrimination when doing the same. A second consideration when choosing to focus on Arabic-sounding names was the size of the group. Individuals born in Syria, Iran, Iraq, Somalia, and Afghanistan, and who typically speak Arabic and have Arabic-sounding names, while also belonging to different ethnic groups, account for over a quarter of the foreign-born population in Sweden and 6% of the entire population (Eriksson and Vernby Reference Eriksson and Vernby2021). That said, we are aware that having an Arabic-sounding name by no means necessarily entails being born outside of Sweden but what we are after is a strong signal of an ethnic out-group. Moreover, we randomized whether the letter was signed by a female- or a male-sounding name. Footnote 11 To test if age matters, an age between 25 and 75 was also randomly assigned to the letter, further increasing the number of variations of the letter sender.
Finally, to test the extent to which party political allegiances play a role in recruitment to these supposedly apolitical positions, half of the e-mails contained a signal suggesting that the writer of the e-mail is a long-time party sympathizer. Thus, our factorial design also varied the content of the e-mails by appealing to party sympathy. More precisely, apart from the content shown in Figure 1, some e-mails also contained an additional opening in the second sentence where the e-mailer appealed to party sympathy. To illustrate, e-mails with this appeal had a second sentence that instead of the second sentence in Figure 1 read, “I have sympathized with [party name] a long time and I write because I am interested in being a lay judge” (see Appendix B for more details on the variations of the e-mails).
We analyze a number of outcomes. First, we have coded whether our alias received a reply. Second, to measure the informativeness of the response, we counted the number of words in each response, how many of our alias’ questions were answered and whether the response provided additional information. Third, we coded the tone of the response by noting whether it used the name of our alias and whether it included a phrase that welcomed the e-mail. Finally, we also coded if the response proposed to follow up with our alias, either by invitation to party activities or by inviting continued personal e-mail communication or suggesting a meeting in the future Footnote 12
Results
In Table 1, we show the results for testing whether party officials’ responsivity depended on the supposed ethnicity and gender, and age of the e-mailer, as well as signals of co-partisanship. The results come from regressions where we controlled for all other treatments, the party of the recipient, as well as the municipality of the local party branch. We have coded non-responses as zero rather than missing in order to avoid selection bias (Coppock Reference Coppock2019). Footnote 13 In the table, we have included the mean for each outcome as a point of reference.
Table 1. Treatment Effects by Dependent Variable
Note: N = 1,981. Entries are OLS coefficients from regressions controlling for all other treatments, the party of the recipient, and the municipality of the local party branch. Robust standard errors in parentheses. * p < 0.10, ** p < 0.05, *** p < 0.01.
The overall response rate is 42% and while the results are mixed, there is no strong indication that party officials responded more favorably to e-mailers with Nordic-sounding names when it comes to conveying relevant information about how to become a lay judge. Beginning with the response rate, Arabic-sounding aliases did have a 4.3 percentage-point lower response rate. However, the effect is only barely statistically significant at conventional levels (p < .10), and its size is small to moderate in light of the overall response rate (42%). Turning to the informativeness of the replies, while the results suggest that the replies to Arabic-sounding aliases contained significantly fewer words (p < .05), those aliases were not less likely to receive answers to their questions or receive additional information. It cannot therefore be argued that our Arabic-sounding aliases received less meaningful information than the Nordic-sounding e-mailers regarding how to become a lay judge.
Moving to the tone of e-mail replies, the results show that local party officials were 6.4 percentage-points less likely to use the name of the Arabic-sounding e-mailer in their responses (p < .01), while they were not less likely to welcome the e-mail. Regarding invitations from the party officials to follow up with our e-mailers, both personal and party contact invitations were less likely to occur when the e-mailer had an Arabic-sounding name. More precisely, a future personal contact was 6 percentage-points less likely to be offered to our Arabic-sounding aliases compared to our Nordic-sounding e-mailers, a result which is significant at a 99% level of confidence (p < .01). Furthermore, our Arabic-sounding e-mailers were 2 percentage-points less likely to receive a general offer to connect with the party in the future compared to our Nordic-sounding senders (p < .05).
Turning to the supposed gender of the sender, results consistently showed no differences in the treatment of female- and male-sounding e-mailers. When it comes to age, the results were mixed and inconclusive. The only inclination of favoritism found was toward our e-mailers of age 35. In fact, this group was 10.1 percentage-points more likely to receive a response (p < .01) compared to e-mailers in the baseline category (age 25). The responses were also more likely to contain more words (p < .01), answer all questions (p < .01), and address the alias by name (p < .05).
Perhaps surprisingly, the signal of co-partisanship did not have a consistent impact across outcomes. One the one hand, we found that it was more likely for the party official to use more words in their response (p < 0.01), welcome the e-mail (p < .01), or to extend a general invitation to become politically engaged within the local party branch (p < .01), when co-partisanship is signaled. For all other outcomes in Table 1, however, differences were not significant at conventional levels. While not part of the pre-registered design, we explored this issue further by coding the prevalence of questions posed by the party representative to our alias regarding ideology and party membership. Of the e-mails sent, 9% received a response that asked whether the e-mailer was a party member and 10% were asked about their ideology. In addition, 17% received a response that claimed that party membership was required and 9% were told that one has to share the party’s ideology. Given that the position of lay judge is supposed to be non-partisan, such answers are problematic. However, in light of the reality that almost all lay judges are party members, these figures must be considered surprisingly low. Footnote 14
Robustness and extensions
A number of robustness checks and extensions of the analysis were performed. The results of which are available in Appendix D. First, we show that our main results are similar when we include no or limited sets of controls. Second, we also test whether many small biases add up to more significant bias and find no evidence of such an effect. Third, we utilize the factorial design to perform heterogeneity analyses. In particular, we show that there are no systematic interactions between treatments. Fourth, we analyze whether response rates differ and react differently to treatments across different types of municipalities. We find that they do not.
Since previous research has highlighted the potential connection between party ideology on the one hand, and gender and ethnic bias on the other (e.g. Eriksson and Vernby Reference Eriksson and Vernby2021), we have also performed an analysis where we interact each treatment with the left-right position of the party to which the local official being contacted belongs. The left-right positions are taken from the Chapel Hill Expert Survey of 2019. As can be seen in Appendix D, virtually none of the interaction effects are statistically significant, suggesting that ideology is not an important factor in this particular setting. In addition, we estimate the treatment effects separately for each party. As in the main analysis, most effects are substantively small, and very few are statistically significant. Rather than underscoring the importance of partisanship, our findings suggest that these local party officials are guided by a different set of concerns when responding to our requests (cf. Alizade, Dancygier, and Ditlmann Reference Alizade, Dancygier and Ditlmann2021).
Conclusion
Prior research on recruitment bias in the economic and political spheres provides a relatively clear picture: Identitarian and ideological traits matter and can often result in discrimination. Our study, by contrast, focused on recruitment in the judiciary and found little evidence of such discrimination. The absence of discrimination against female candidates and lack of favorable treatment of co-partisans challenges notions that biased recruitment follows the same logic across different societal spheres. While there was some suggestive evidence of ethnic discrimination in our study of judicial selection, it has often been found to be more pervasive in studies of the economic and political spheres.
We also test whether there is a slight preference for candidates in their 30s and 40s. This behavior would be in line with the expressed desires of the Swedish Government and the National Courts Administration, which have declared that parties should strive to rejuvenate the pool of lay judges. Previous research shows that bureaucrats can be induced to treat underrepresented group favorably when provided with clear administrative guidelines to do so (Arai et al., Reference Arai, Gartell, Rödin and Özcan2021), and we find some evidence, albeit inconclusive, that party officials may also be responsive to such government signals.
Supplementary Material
To view supplementary material for this article, please visit https://doi.org/10.1017/XPS.2022.10
Data Availability Statement
The data and code required to replicate all analyses in this article are available at the Journal of Experimental Political Science Dataverse within the Harvard Dataverse Network, at doi: 10.7910/DVN/64FVHL.
Acknowledgements
We wish to thank Angelica Gradin, who provided excellent research assistance, Martin Klarqvist, who provided invaluable technical assistance, and Linna Martén for valuable comments.
Conflict of Interest
The authors declare no conflicts of interest in this research.
Financial Support
Support for this research was provided by the Swedish Research Council.
Ethics Statement
This research was approved by the Regional Ethical Review Board in Stockholm (see protocol number 2019-02330) and adheres to APSA’s Principles and Guidance for Human Subjects Research. For more details, see the ethics statement in section C of the Appendix.
Open access
