INTRODUCTION

During oral arguments for the recent gerrymandering case Gill v Whitford, Chief Justice John Roberts expressed a concern that if the Court relied on “sociological gobbledygook” (Transcript of oral arguments, p. 40) to settle partisan gerrymandering cases it would “cause very serious harm to the status and integrity of the decisions of this court” (Transcript of oral arguments, pp. 37–38) by making the Court appear political. The public would not understand the social science data and formulas used to decide a case, and thus would assume the decision was driven by partisan politics. While the Chief Justice worries that using social science will harm the legitimacy of the Court, iconoclast jurist Richard Posner argues in favor of using evidence from social science in judicial decisions and, importantly, that the public will respond positively to its use. In his doctrine of legal pragmatism, judges base their decisions on “facts and consequences rather than on conceptualisms and generalities” (Reference PosnerPosner, 1999, p. 227), and social science provides useful evidence concerning those consequences. In addition, Posner argues that the American people are an especially pragmatic people who desire judges concerned more with the consequences of their decisions than the legal theories justifying them, and “(i)f pragmatic reasonableness is what people want of the judiciary, why is it illegitimate for the judiciary to give it to them?” (Reference PosnerPosner, 2003a, p. 350). While Chief Justice Roberts and Posner may be advancing their arguments in order to support their preferred case outcomes, they make competing and empirically testable claims regarding how judicial decisionmaking will affect the legitimacy of the courts.

Unfortunately, because of the lack of empirical research on the public's reaction to judges basing their decisions on social science, these two respected jurists must rely on intuition rather than empirical evidence to support their claims. The studies presented in this article examine whether it is the iconoclast or the Chief Justice who has a better sense of the American public. However, in the end, we cannot offer a simple answer because just as elites are divided on the question of what should influence judicial decisionmaking, so too are the American masses. Our two experiments will show that among legal traditionalists who desire a more formalist court that interprets the law as written using traditional legal theories like original intent, the Chief Justice is correct: the use of social science evidence undermines the acceptance of decisions. However, not all Americans hold the same normative preferences for how judges should make decisions (Reference Bonneau, Kelly, Pronin, Redman and ZaritBonneau et al., 2016; Reference GibsonGibson, 2012; Reference Gibson and CaldeiraGibson & Caldeira, 2009; Reference Parker and WoodsonParker & Woodson, 2020; Reference Scheb and LyonsScheb & Lyons, 2001). Among those who reject traditional legalistic values and instead support the doctrine of popular constitutionalism, it is the iconoclast who is correct: the use of social science evidence increases acceptance of decisions.

While the doctrines of legal pragmatism and popular constitutionalism are not intertwined in legal scholarship, they are related in the minds of the American people, with attitudes toward both doctrines loading onto a single factor in Scheb and Reference Scheb and LyonsLyons (2000). Our results will reinforce this by showing that the same iconoclasts who reject the legal orthodoxy in favor of popular constitutionalism also support legal pragmatism, perhaps reflecting that both doctrines argue for a more flexible interpretation of the Constitution. Importantly, it appears that this group of iconoclasts make up a substantial portion of the population. Reference Scheb and LyonsScheb and Lyons (2001) found using a nationally representative survey that 68% of the American population wants majority public opinion to have at least some impact on the decisions of the U.S. Supreme Court.Footnote ¹ Multiple experiments show a manipulation that varies whether the Court is influenced by public opinion or uses purely legalistic arguments has no aggregate effect on its public support (Reference Bonneau, Kelly, Pronin, Redman and ZaritBonneau et al., 2016; Reference FarganisFarganis, 2012). Those two strands of evidence showing a widespread desire for a representative Court that follows public opinion are reinforced by the high levels of support for judicial reforms meant to bring the Court more in line with public attitudes. Reference GoldeGolde (2019) notes that 72% of the population supports term limits for U.S. Supreme Court justices and 43% support the more extreme option of court-packing. When moving to the state courts, the vast majority of Americans support electing judges (Reference AndersonAnderson, 2003; Reference GeyhGeyh, 2003), with ~75% supporting elections according to nationally representative polls from 2001 to 2008 (American Justice Partnership, 2008; Justice at Stake, 2001). Taken as a whole, the evidence clearly shows the American people are hungry for a more representative judiciary. Gibson in his study on the effect of elections on state high courts came to a similar conclusion and stated that while high and low knowledge individuals may differ in their desired form of representation, “[a]ll agree … that in one form or another, judges should reflect the views of their constituents” (Reference GibsonGibson, 2012, p. 102).

Previous research on normative preferences has shown that a disjuncture between individuals' normative preferences and their actual perceptions of a court's decision-making process causes evaluations of the court to become more negative (Reference Bonneau, Kelly, Pronin, Redman and ZaritBonneau et al., 2016; Reference Gibson and CaldeiraGibson & Caldeira, 2009; Reference Scheb and LyonsScheb & Lyons, 2001). This research proves that normative preferences matter, but understates their impact. Supporting popular constitutionalism involves a more fundamental restructuring of a person's attitudes toward the Court than simply moderating whether being influenced by public opinion increases or decreases the Court's perceived legitimacy. Supporting popular constitutionalism affects the interpretation of virtually every action taken by the Court, even those like using social science in legal decisions that are not directly related to the core concepts of popular constitutionalism. This broad effect occurs because much of what appears as a spasm of politicization to a legal traditionalist morphs into a blossoming of democracy in the eyes of a popular constitutionalist. That shift in interpretation is why dissents decrease legitimacy perceptions for legal traditionalists while increasing them for popular constitutionalists, according to an experiment by Reference Parker and WoodsonParker and Woodson (2020).

In the current paper, we expand upon the understanding of how this group of iconoclasts who reject the legal orthodoxy in favor of popular constitutionalism interprets the actions of the judiciary, and how those interpretations affect their evaluations of it. Because of the almost complete lack of research into the psychology of popular constitutionalists, virtually nothing is known about the criteria they use to evaluate the Court (but see Reference Parker and WoodsonParker & Woodson, 2020). We will show—using an experiment involving the use of social science—that those criteria are different than the criteria used by legal traditionalists, and offer an initial glimpse into the previously-opaque minds of popular constitutionalists who make up a substantial and perhaps even a majority of the American population (Reference Gibson and CaldeiraGibson & Caldeira, 2009; Reference Scheb and LyonsScheb & Lyons, 2001).

SOCIAL SCIENCE AND THE LAW

The Supreme Court has long looked outside the law and turned to expertise in the various social sciences including economics, psychology and sociology to reach their conclusions. The Court has relied on sociological data in Muller v. Oregon (1908) to uphold labor protections for women and has turned to research in psychology in cases like Brown v. Board of Education (1954), Atkins v. Virginia (2002), and Roper v. Simmons (2005). In the area of criminal procedure the Court's reliance on social science has been increasing from 2001 to 2015, (Reference Meitl, Piquero and PiqueroMeitl et al., 2020), with judges appointed by Democrats and those appointed more recently using social science to a greater extent (Reference MeitlMeitl, 2020).

This practice of reaching beyond the law to inform majority opinions is not without controversy. Textualist judges and scholars tend to reject the practice of departing from the text of the Constitution to achieve an outcome. Originalists argue that judges should not depart from the text of the Constitution, and should only engage in analyses that help determine what the words meant at the time they were written. For example, in his concurring opinion in Missouri v. Jenkins (1995) Justice Clarence Thomas argued that the Court erred in Brown v. Board of Education (1954) by basing its decision on psychological factors that are “irrelevant to the question whether state actors have engaged in intentional discrimination.” These criticisms suggest that it is inappropriate to use social science findings that are constantly in flux to evaluate laws that remain textually consistent.

However, a number of scholars have argued that there is an important place for social science in the legal system. Social science can be used to address questions of fact, or the objective reality surrounding a case (Reference FaigmanFaigman, 1989). This is important because legal precedents and remedies often rely on theories about how changes in the law might influence subsequent human behavior. For example, in Elkins v. United States (1960) and Mapp v. Ohio (1961) the Supreme Court expanded the application of the exclusionary rule under the assumption that if illegally seized evidence was deemed inadmissible in trials then there would be an incentive for police officers to respect the protections of the Fourth Amendment so that the criminals they catch do not go free.

In his legal pragmatism approach to the law, Posner proposes a doctrine in which the use of empirical evidence and social science is an integral and central component. While many people may associate Posner most closely with his economic approach to the law in which he applies behavioral economics and economic theory to legal rules (Reference PosnerPosner, 2003b), his overall doctrine of legal pragmatism does not limit the utility of social science to economics alone, but includes any scientific evidence that can be used determine how legal rules and decisions will affect the future behavior of individuals and groups. He describes his vision of pragmatic adjudication as basing decisions on “facts and consequences rather than on conceptualisms and generalities” (Reference PosnerPosner, 1999, p. 227). Legal pragmatism rejects the use of moral, political or abstract legal theories such as fairness, equality and original intent as a guide to judicial decisionmaking, arguing that the while judges may “claim to be engaged in the neutral scientific activity of matching facts to the law” they are actually engaging “in a basically political activity of formulating and applying public policy called law” (Reference PosnerPosner, 2003a, p. 46). Instead, he calls for judges to aim “at the decision that is most reasonable, all things considered, where ‘all things’ includes both case specific and systematic consequences” (Reference PosnerPosner, 2003a, p. 13).

Social science's role—whether through economic theory, psychology experiments, or sociological field studies—is to provide evidence about those consequences. Legal pragmatism does not mean simply deciding cases based on the what produces the best outcome, but it does mean issuing rulings based on contextual judgments of the law combined with empirical evidence regarding the efficacy of government actions (Reference DorfDorf, 1999). Posner argues that this approach is preferable to deciding cases based on constitutional theory, because constitutional theory “lacks the agreement-coercing power of the best natural and social science.” (Reference PosnerPosner, 1998, p. 3). According to this view, not only will it result in better decisions, there will also be greater public acceptance of judicial decisions that are guided by empirical data rather than abstract constitutional theories.

SOCIAL SCIENCE AND ACCEPTANCE OF COURT DECISIONS

Though legal scholars have devoted significant attention to the normative value of social science research to the law, there is very little research on the question of how the use of social science affects attitudes regarding the courts or their decisions. To our knowledge, Reference Redding and Dickon ReppucciRedding and Reppucci (1999) is the only study that examines whether people approve of the use of social science in legal decisions. In a sample of only judges and law school students, they found that those with a more liberal and expansionist view of judicial interpretation had a more positive reaction to using social science when interpreting the law. Because that analysis was only briefly mentioned in an article focusing on a different topic, it is unclear precisely what the authors meant by a more liberal and expansionist view of judicial interpretation or how it was measured, but their evidence suggests that a person's normative preferences for how judges should interpret and apply the law affects how people feel about the use of social science in the law.

In this study, we concentrate on the public's acceptance of court decisions—that is, whether people accept a court decision and move on, or whether they want to challenge it and get it overturned. The conventional wisdom—as stated by the Chief Justice—is that using nonlegal arguments based in social science as the basis of disagreeable court decisions will be perceived by the public as “a bunch of baloney…that is going to cause very serious harm to the status and integrity of the decisions of this Court in the eyes of the country” (Transcript of oral arguments for Gill v. Whitford, pp. 37–38), and the existing literature provides some evidence for his view. Legitimacy perceptions are the main determinant of acceptance judgments (Reference GibsonGibson, 1989, Reference Gibson1991; Reference Gibson, Caldeira and SpenceGibson et al., 2005, Reference Gibson, Lodge and Woodson2014), and multiple studies show that other types of nonlegalistic influences on judicial decisionmaking can lead to a decrease in public support for the court (Reference Baird and GanglBaird & Gangl, 2006; Reference BeneshBenesh, 2006; Reference Scheb and LyonsScheb & Lyons, 2000; Reference TylerTyler, 2006), which will subsequently lead to a decrease in acceptance.

However, this conventional wisdom does not apply uniformly to the entire public because not everyone desires a court that applies the law mechanically without any use of discretion (Reference Bonneau, Kelly, Pronin, Redman and ZaritBonneau et al., 2016; Reference GibsonGibson, 2012; Reference Gibson and CaldeiraGibson & Caldeira, 2009, Reference Gibson and Caldeira2011; Reference Scheb and LyonsScheb & Lyons, 2001). Some prefer that rather than just applying the law mechanically, judges should take into account public opinion (Reference Parker and WoodsonParker & Woodson, 2020). According to the doctrine of popular constitutionalism, the courts are viewed as another political institution, and thus they should similarly reflect the public will (Reference FriedmanFriedman, 2003; Reference KramerKramer, 2004; Reference Kramer and TribeKramer & Tribe, 2001; Reference PozenPozen, 2010). This does not mean that the courts should not have the power of judicial review, or should always support the majority public opinion at a specific time. Rather, precedent and legal rules should evolve with society so that they are consistently updated in accordance with real and enduring changes in the popular will. Those who do not view the law as unchanging and immutable, or do not see the Constitution as a “dead” document, should not be expected to react negatively to nonlegal influences in court decisions. Indeed, using social science research to make sure that legal standards best meet the current needs of society could be seen as a feature, not a bug, in terms of evaluating institutional legitimacy.

For the iconoclasts like Richard Posner who support legal pragmatism, the use of social science is a desired feature in judicial decisionmaking because the empirical evidence provides a foundation on which to rationally base a legal decision, and provides the best evidence available to determine the consequences of that decision (Reference PosnerPosner, 1998, Reference Posner1999, Reference Posner2003a). Posner does not argue for popular constitutionalism, but there is reason to believe that adherents to popular constitutionalism would be more accepting of legal pragmatism than would legal traditionalists. Both schools of thought argue for a more flexible interpretation of the Constitution than do legal traditionalists, and are more accepting of judges using nonlegal information (whether it is public opinion or social science) to guide their decisions. Posner also establishes a link between his doctrine of legal pragmatism and popular constitutionalism by recognizing the importance of acknowledging public preferences. He stated that “judges in a democratic society must accord considerable respect to the deeply held beliefs and preferences of the democratic majority when making new law” (Reference PosnerPosner, 1999, p. 251). The two ideas are also connected in the minds of the public, as attitudes toward popular constitutionalism are associated with attitudes toward legal pragmatism. While using actual decision-making perceptions rather than normative preferences, Reference Scheb and LyonsScheb and Lyons (2000) show in a factor analysis that an item measuring whether judges do “what is good for the public” loads onto the same factor as one that measures whether judges do “what the majority of the public favors.” Importantly, this factor was separate from one measuring the degree of ideological influence and another that measured legal influence.

Supporting popular constitutionalism relates to approval of the judiciary using social science, not necessarily because people believe it will lead to a more representative court, but because support for popular constitutionalism is associated with a fundamental restructuring of a person's attitudes and orientation toward the judiciary. That restructuring should affect not only views concerning the accountability of the judiciary to the public, but also attitudes seemingly unrelated to the versions of popular constitutionalism espoused by legal elites. Among those attitudes is support for the doctrine of legal pragmatism. As a result, those who support popular constitutionalism should also tend to support the use of social science in judicial decisions. The lack of connection between the two doctrines in most normative debates occurring within the confines of academia does not mean they are not associated within mass public opinion. How the doctrine of popular constitutionalism manifests itself in the minds of the public will be dramatically different than how it is articulated in the theories of legal elites, and it is this public manifestation of popular constitutionalism that we examine in this paper.

Our theory argues that normative preferences change the criteria used to evaluate the judiciary, and Reference Gibson and CaldeiraGibson and Caldeira (2009) provide evidence supporting this claim. The study found that exposure to political advertisements concerning the Samuel Alito confirmation hearings decreases support for the Court, but only among those who had legalistic normative preferences. Two other studies have expanded upon that to show that specifically attitudes toward popular constitutionalism affect the criteria used to evaluate the judiciary. Reference Parker and WoodsonParker and Woodson (2020) found that dissents had a negative effect on both legitimacy and acceptance judgments, but only for those who espoused traditional legalistic views. For supporters of popular constitutionalism, the presence of a dissent increased legitimacy perceptions and had no effect on acceptance judgments. Reference WoodsonWoodson (2018) found that a person's view about whether or not the Court was actually influenced by public opinion when making the NFIB v Sebelius (2012) decision changed the effect of displeasing decisions on legitimacy perceptions. For those who thought the Court was influenced by public opinion, a displeasing decision had no effect on the Court's legitimacy. While a displeasing decision increases legitimacy among those who thought the Court used a legalistic process and decreases it among those who thought it used an ideological process. These studies show that reactions to Supreme Court decisions are highly variable based on a person's preferences and perceptions regarding how the judiciary should decide cases. The results from our current study will build upon them to show once again that supporting popular constitutionalism causes “a change in a person's overall orientation toward an institution, which changes the criteria used to evaluate the institution” (Reference WoodsonWoodson, 2018, p. 77).

HYPOTHESES

Our overall hypothesis, which will be tested using two separate experiments, is that support for popular constitutionalism mediates the effect of using social science in judicial decisions on the public's level of acceptance for court decisions. For those who support traditional legalistic values, the use of social science violates the norm of judges doing nothing more than interpreting law to the best of their ability. For this group, evidence from social science is interpreted as an illegitimate nonlegal influence that decreases their level of acceptance for the decisions (H1). While popular constitutionalists are not necessarily adherents to Posner's legal pragmatism, they should prefer judges that incorporate a broader range of information into their decisions. For those who support popular constitutionalism, social science is an important piece of evidence that can be used to understand the effect of judicial decisions, and rather than undermining the decision, it provides both further evidence of its wisdom and objective unbiased evidence on which to base the decision, causing an increase in acceptance (H2). In addition, we hypothesize that support for legal pragmatism is more highly associated with support for popular constitutionalism than any other factor of normative preferences (H3).Footnote ²

STUDY DESIGN

We use two separate studies to examine our overall hypothesis. Each study has a similar design. For both studies, we expose the subjects to either a decision using legalistic reasoning or a decision using a consequentialist analysis supported by social science research. The major difference between them is the type of social science evidence. For Study 1, we use empirical social science studies that examine the effect of a law—in this case “stop and frisk” policies—on criminal activity. In Study 2, we use the behavioral economics approach to the law that relies on theory rather than empirical data. The variable measuring the experimental manipulation is coded as a 1 for the decision using consequentialist reasoning involving social science, and 0 for the legalistic reasoning condition.

Both studies begin by asking the subjects for their position on a political issue. Next, the survey assesses a variety of attitudes including the subjects' knowledge of the court, normative decision-making preferences, evaluation of science, partisanship, and demographic variables. Following this, the subjects read the experimental manipulation, which included a description of the court's decision.Footnote ³ Importantly, the Court's decision always went against the subject's stated preference from the beginning of the study. Research shows that people virtually always accept decisions with which they agree because acceptance is about lacking the desire to challenge and overturn a court decision. The same motivating reasoning processes that cause people to accept without question information that agrees with their prior viewpoint and strongly counter-argue against information that disagrees with their prior viewpoint (Reference Lodge and TaberLodge & Taber, 2013) apply just as equally to court decisions as it does to legislation passed by Congress. Ordinary Americans will rarely have the motivation to take actions to challenge a decision that they agree with (Reference GibsonGibson, 1989, Reference Gibson1991). Because almost everyone will accept any decision when they agree with the policy implications, “the most common research design involves presenting people with a policy with which they disagree and then trying to ascertain whether a court ruling increase the probability of acceptance …” (Reference Gibson, Caldeira and SpenceGibson et al., 2005, p. 187). Of course, all research designs involve some tradeoffs, and asking people about the issue position beforehand is not ideal, but without doing so, we would be unable to separate out those who agree with the policy implications of a court decision from those who disagree, even though we would expect the determinants of acceptance for those two groups to be dramatically different. Reference WoodsonWoodson (2015) used three distinct study designs to examine the effect of politicization perceptions on acceptance—one that split people into groups based on ideology, one that asks directly about issue positions and then presents a disagreeable decision, and a third that asks about acceptance in general rather than about a specific decision—and found similar results across all three. This suggests that the type of study design used here results in similar conclusions as the other potential options.

Because the type of social science evidence differs between the two studies, the issue used must also differ because not all types of social science apply equally well to all legal issues. For Study 1 involving empirical studies, we ask the subjects about “stop and frisk” programs and whether police should be required to have probable cause or just reasonable suspicion to search someone they suspect of criminal activity.Footnote ⁴ Our manipulation involved randomly assigning respondents to read a summary of either a decision that relied on purely legal arguments, or a decision that cited social science research to support the decision. Both decisions involving legal arguments used an interest balancing approach weighing the invasiveness of the search against the government's interest in public safety. The social-science based decision that ruled “stop and frisk” programs were unconstitutional mentions research finding that “stop and frisk” was an ineffective program and reduced public support for and cooperation with law enforcement. For the social-science-based decision upholding “stop and frisk” programs, the decision cites research highlighting the deterrence effect of these programs.Footnote ⁵ The length of the descriptions ranged between 151 and 167 words.

For Study 2 involving behavioral economics, we ask the subjects about a hypothetical situation involving whether the current owner of an oil pipeline should be liable for damages based on environmental contamination caused by the pipeline prior to the current owner purchasing the pipeline. The manipulation involved randomly assigning respondents to read purely legal justifications involving when contract law requires new owners to pay past liabilities, or decisions that relied on an analysis of how economic development would be affected by the case. The length of the descriptions ranged from 99 to 104 words. We chose this scenario because we needed a decision that invoked behavioral economics analysis and, at the same time, be simple enough for the respondents to understand the issue and decision based on a single short paragraph. That did not seem possible with the more salient constitutional issues that normally receive substantial attention from the media and public. Using a statutory rather than constitutional issue has the advantage of showing our theory works in both constitutional and statutory cases. Virtually all existing research examining the acceptance of court decisions involves constitutional questions so examining acceptance across both the constitutional and statutory contexts is a noteworthy contribution.

While using liability for damages as the issue has advantages, it also has a disadvantage. A substantial number of people do not find that issue to be important, and this should affect how people make acceptance judgments. The decision to accept or challenge a court decision is not just about disagreement with the issue but also about the motivation to take efforts to overturn a decision. The motivation to challenge a decision is greatly reduced when people like the policy aspects of a decision, or when they do not care about a policy, causing people to make judgments concerning decisions they like and those they do not care about much differently than displeasing decisions they consider important. As a result, we should expect that people are highly likely to accept a decision if they do not consider it important (Reference Parker and WoodsonParker & Woodson, 2020; Reference WoodsonWoodson, 2015, Reference Woodson2019). For most issues, the percentage of people who do not care about an issue is small enough that it does not affect the results, but for an issue involving the liability over oil pipelines a substantial part of the sample will find the issue entirely irrelevant to their lives. We can see this in the percentage of people in the two studies who said the issue was not important at all. In Study 1 using a decision on “stop and frisk,” 11% of the sample stated the issue was “not important at all.” That percentage rose to 26% in Study 2. In our regression analyses below, we exclude from the sample for both Study 1 and Study 2 those people who deemed the issue “not important at all.”

MEASUREMENT

The wording of the survey questions was similar for both studies. They start by presenting the subjects with a paragraph describing the basic legal arguments for both sides of the case, and then asking which side they agreed with. For the full text of all items used in the survey, see the Supporting Information. For Study 1, the subjects indicated whether they supported the ability of police to “stop and frisk” individuals they suspected may be about to commit a crime or whether police should be required to have probable cause before stopping an individual. For Study 2, the subjects were asked whether the current owner of the oil pipeline “should have to pay damages for the contamination resulting from the prior oil spill?” and were given the answer options of yes or no. In both studies, following these initial dichotomous items the subjects indicated how strongly they held that position and the importance of the issue on four-point scales.

Normative decision-making preferences

Much of the discussion from scholars—and especially pundits and politicians—around how the judiciary makes decisions involves the false dichotomous choice of the Court being either political or legalistic. However, Reference Gibson and CaldeiraGibson and Caldeira (2011) revealed not everyone desires a purely legalistic court. Their analysis argues that while the public does not like strategic and insincere decisionmaking, the public seems amenable to ideological decisionmaking. In addition, both Reference Parker and WoodsonParker and Woodson (2020) and Reference WoodsonWoodson (2018) show that not all types of decisionmaking that are traditionally deemed political are evaluated similarly. The studies show that attitudes toward popular constitutionalism are a separate and independent factor from ideological or politicized decisionmaking. In both studies, two factors of decision-making perceptions emerge that have separate effects on evaluations, with one factor ranging from legalistic decisionmaking at one end to ideological or politicized decisionmaking at the other. The second factor measures attitudes toward popular constitutionalism or the degree to which public opinion influences the Court. While using actual decision-making perceptions rather than normative preferences, Reference Scheb and LyonsScheb and Lyons (2000) also show that perceptions about public opinion affecting the Court load onto a separate factor than both perceptions about ideological and legalistic influences.

The current paper uses a similar measurement strategy as Reference Parker and WoodsonParker and Woodson (2020) to measure normative preferences for the Court's decision-making processes. The subjects are presented with a statement about the Court's decision-making process and asked their level of agreement with the statement on a 7-point scale. The items are designed to measure how people think the Court SHOULD make decisions not how it actually makes decisions. As a result, each statement begins with the phrase “In the ideal world, …” to emphasize the normative nature of the items. Each item then proceeds to describe a potential way in which the Supreme Court could make a decision. See Table 1 for a list of the normative preference items.

Table 1. Normative preference items

While we only expect two factors to emerge, we include four distinct subsets of normative preference items. The first subset measures the degree to which people support the doctrine of popular constitutionalism. These questions do not mention popular constitutionalism directly but rather ask in various ways about what type of influence the public and public opinion should have on the Court. Many, if not most, people who support the concept of popular constitutionalism would not be able to define it if asked. Rather, they have some vague notion that in a democracy the meaning of the Constitution should be influenced by the public and not solely determined by an unelected and unaccountable Court. Study 1 includes four items designed to measure the degree to which people believe the public should influence the Court, with Study 2 including the original four plus an additional five new items.

The three remaining subsets of items are all expected to load onto one factor that ranges from support for legalistic decisionmaking at one end to support for ideological or politicized decisionmaking at the other end. One subset of three items measures the degree to which people believe the Court should do little more than neutrally interpret and apply the law. The other two subsets of items are based on a distinction made by Reference Gibson and CaldeiraGibson and Caldeira (2011) between ideological and politicized decisionmaking. They define the latter as being when judges act in a self-interested and strategic manner like an ordinary politician and the former as when judges are influenced by their personal political opinions or ideology. Because of the literature's previous distinction between these two concepts, we include three items that directly ask about personal political opinions or ideology affecting the Court's decisions and three items that directly ask if judges should act like strategic or self-interested politicians. While we separately measure legalistic, politicized and ideological decision-making preferences, based on prior published results (Reference Parker and WoodsonParker & Woodson, 2020) and our own pre-tests, we expect all three subsets of items to load onto a single factor.

DATA ANALYSIS

Before examining the effect of the experimental manipulations, we first must establish that support for popular constitutionalism is a separate factor and empirically distinct from the spectrum of normative preferences that ranges from legalistic to ideological and/or political decisionmaking. Since the survey items and results for both studies are similar, we analyze the factor analysis from both prior to examining the effects of the experimental manipulations. Table 2 shows the results for both factor analyses. For both studies, only two factors emerged, with the eigenvalues for both the first and second factor larger than 1, and the eigenvalues for all other factors below 1. Because the two factors of normative decision-making preferences are expected to be somewhat correlated, an oblique factor rotation was used.

Table 2. Structure of normative preferences

Note: All factor loadings 0.30 or larger are displayed in bold text.

Table 2 shows that in Study 1 the items measuring support for ideological, political and legalistic decision-making load onto one factor with the items measuring support for popular constitutionalism loading onto a second factor. All nine items that measure support for ideological decisionmaking, politicized decisionmaking or legalistic decision-making load onto the first factor with loadings higher than 0.42, and those items all have low loadings below 0.17 on the second factor. The four items measuring support for popular constitutionalism all load onto the second factor with loadings at or higher than 0.63, with those items also having factor loadings lower than 0.22 on the first factor. These results clearly show that support for popular constitutionalism is a separate and distinct factor from the spectrum of normative preferences that ranges from politicized to legalistic decisionmaking. We call the factor measuring support for popular constitutionalism the public norm scale, and the factor that includes the political, ideological and legalistic items the political-legal norm scale.

We included additional items in the public norm scale for Study 2 to measure different aspects of popular constitutionalism and to provide a fuller view of how the public conceives of the doctrine. As in Study 1, the results clearly show that support for popular constitutionalism is a distinct and separate factor from the political-legal norm scale. All nine items measuring support for popular constitutionalism load onto the first factor with loading higher than 0.56. When examining the second factor, 8 of the 9 items measuring ideological, politicized, or legalistic decisionmaking have factor loadings higher than 0.34 on the second factor.

Two of the items have somewhat different factor loadings in Study 2 when compared to Study 1.The second legalistic norm item that asks whether justices should act as “neutral arbiters of the law” has a relatively low factor loading of 0.27 on the second factor measuring the political-legal norm scale. In addition, the second item measuring the public norm scale, which asks whether the public should have little to no influence on the court's constitutional interpretations and decisions, has a high loading on the factor measuring the public norm scale, but it also has a relatively high loading of 0.3 on the political-legal norm scale.Footnote ⁶ To be consistent with the theoretical expectations and to keep the measurement of key concepts consistent across the two studies, the second legalistic norm item will still be included in the political-legal norm scale, and the second public norm item will be included in the public norm scale for Study 2.Footnote ⁷

For both studies, the political-legal norm scale is created by summing the nine items that measure ideological, politicized, and legalistic normative preferences, with the appropriate items being reverse coded. For that scale, the low end of the spectrum represents support for legalistic decisionmaking while the high end represents support for politicized or ideological decisionmaking. The public norm scale is constructed by adding together all the items measuring support for popular constitutionalism (either 4 or 9 depending upon the study), with higher numbers indicating more support for popular constitutionalism. The two measures are significantly correlated, with r = 0.32 in Study 1 and r = 0.37 in Study 2. As with all other variables used in the following models, these variables are recoded with 0 as their minimum, 1 as their maximum, and the other values ranging in between.

Figure 1 shows the distribution of both variables, and the pattern matches that found in Reference Parker and WoodsonParker and Woodson (2020). The political-legal norm scale is highly skewed toward people preferring a legalistic approach over a political one, with only a very small proportion preferring a political and/or ideological court. The distribution of the public norm scale shows a more consistent distribution with approximately half the sample above the neutral mid-point of the scale and half below it. This suggests that at least among this nonrepresentative sample support for popular constitutionalism is more widespread than support for a political or ideological court.

Figure 1. Distribution of normative preferences

Study 1—Empirical social science study

We now move to examining how the two normative preference scales described above moderate the effect of our experimental manipulations on acceptance. In the models of acceptance in Table 3, we test those moderating effects by including an interaction between the experimental manipulation and both normative preference scales. We also include an interaction between the manipulation and the respondent's knowledge about the Court to control for the possibility that people who are more knowledgeable about the Court may treat using social science evidence differently than those with less knowledge.

Table 3. Normative preferences and social science in legal decisions

Note: SEs in parentheses.

^*p < 0.05. ^**p < 0.1.

The model in Column 1 of Table 3 includes the model for Study 1 using the complete sample with all three interactions and a number of control variables.Footnote ⁸ The coefficients for the interactions of the social science manipulation with both the political-legal norm scale and the court knowledge index are insignificant. The coefficient for the interaction between the social science manipulation and the public norm scale is marginally significant with p < 0.1, and as we predicted, the effect of the social science manipulation on acceptance becomes less negative and more positive as support for popular constitutionalism increases. While in the expected direction, we delay more fully describing it because it is only marginally significant, and we expect the effect of the social science manipulation to differ based on a person's evaluation of science.

The models in Columns 2 and 3 of Table 3 split the sample based on the respondent's evaluation of science. The sample for the model in Column 2 includes everyone at the 50th percentile and below (0.67), and the sample for the model in Column 3 includes everyone above the 50th percentile.Footnote ⁹ As the results show, for those with low evaluations of science there is no interaction between the social science manipulation and any of the three moderating variables, including the public norm scale. When shifting to Column 3, which runs the same model on those with high evaluations of science, the interaction between the public norm scale and the social science manipulation is positive and significant with p < 0.05. The other two interactions are insignificant, indicating that only a person's support for popular constitutionalism moderates the effect of using social science in court decisions on acceptance.

The line labeled Study 1 in Figure 2 displays the marginal effect of the social science manipulation on acceptance for those with high evaluations of science across the range of the public norm scale. The line is solid when the marginal effects are significantly different from 0 at p < 0.05 and dashed when they are not. These marginal effects were calculated holding all other variables at their means. The marginal effect is negative and significant at p < 0.05 for everyone at 0.21 and below on the public norm scale, representing 16% of the sample used in that model. When the public norm scale is 0, the marginal effect of the social science manipulation is −0.19. In the legalistic condition, the average level of acceptance is 0.58, and that decreases to 0.39 in the social science condition. For those at the high end of the public norm scale the effects are in the opposite direction. The marginal effect of the social science manipulation is positive and significant at p < 0.05 for everyone at 0.63 and above on the public norm scale, representing 48 percent of the sample used in the model. At the highest point of the public norm scale (1), the marginal effect of the social science manipulation is 0.21, with the average level of acceptance in the legalistic condition being 0.23 and increasing to 0.45 in the social science condition.

Figure 2. Public norm scale and the marginal effect of the experimental manipulations. Two-tailed significance tests are used for the social science manipulation in Study 1, and one-tailed significance tests are used for the behavioral economics manipulation in Study 2

These results show that the public norm scale moderates the effect of the social science manipulation as expected by H1 and H2, but only among those with high evaluations of science. For those who support popular constitutionalism and approve of science, the social science manipulation increases acceptance, and for those who dislike popular constitutionalism and approve of science, the social science manipulation decreases acceptance. Among those who disapprove of science the manipulation had no statistically detectable effect on acceptance across the whole range of the public norm scale. We now shift to testing our theory in a different context using the behavioral economics approach toward the law as the social science alternative to a traditional legalistic argument.

DISCUSSION

The Chief Justice and the Iconoclast made different predictions about the effect of using social science in decisions because they made different assumptions about the American public. The former assumed that the public sought judges to act as umpires that call balls and strikes as he stated in his confirmation hearing. The latter assumed the public desired practical judges that were concerned with the consequences of decisions. Our two experiments provide evidence supporting both of their predictions, but only among the portion of the public that shared their assumptions and normative preferences about the proper way for judges to make decisions. For the iconoclasts who support legal doctrines like popular constitutionalism and legal pragmatism, social science is seen as a legitimate piece of evidence concerning the wisdom of a decision and thus perceived as a positive feature of a court's decision-making process, which results in the level of acceptance increasing. For the legal traditionalists who support judges acting as umpires calling balls and strikes, the use of social science is an improper and illegitimate nonlegal influence on judicial decisionmaking, and as a result decreases acceptance.

These results do not allow a simple answer concerning whether using social science in decisions helps or hurts the Court's ability to induce compliance and acceptance of its decisions. Gauging the overall effect requires balancing the positive effect among the iconoclasts with the negative effect among the legal traditionalists. Since the magnitude of the positive and negative effects among the two groups was similar, the overall effect among the total population depends upon whether the number of people supporting traditional legalistic ideas is larger than the number of popular constitutionalism supporters. Based on a survey from 2005 about 1/3 of the American population thinks it is “very important” for the court to “represent the majority” (Reference Gibson and CaldeiraGibson & Caldeira, 2009), and thus we can conclude that at least 1/3 of the population would be receptive to the Court using social science in their decisions. Not every remaining person will be a strict legal traditionalist turned off by social science in judicial decisions. Thus, most likely the overall effect of using social science is not to substantially decrease or increase the aggregate level of acceptance among the population, but to change the composition of the people who accept. Whenever the Court uses social science, it increases the number of people who accept the decision among supporters of popular constitutionalism, and decreases it by roughly the same number among legal traditionalists.