Oral argument exposure

Oral argument can serve a variety of purposes. Internally, judges may use oral argument to acquire information for purposes of informing decisions on the merits, coalition formation, opinion writing, and separation of powers considerations (Johnson Reference Johnson2004; Black, Johnson, and Wedeking Reference Black, Johnson and Wedeking2012; Ringsmuth and Johnson Reference Ringsmuth and Johnson2013). Externally, oral argument can enhance institutional legitimacy, increase public awareness, guard against noncompliance, and provide judges with a platform for political position taking (Black et al. Reference Black, Johnson, Owens and Wedekingn.d.; Krehbiel Reference Krehbiel2016; Jacobi and Sag Reference Jacobi and Sag2019).

The external impact of oral argument remains relatively understudied notwithstanding its potential importance. As discussions of judicial legitimacy proliferate, oral argument exposure has been at the forefront. With respect to the Supreme Court, for example, it has been said that “the public spectacle of oral argument assures the parties in the case at hand that their arguments have been heard and considered,” while also “allow[ing] the public to see the Court as an impartial tribunal exploring issues of national importance through a balanced adjudicative process” (Jacobi and Sag Reference Jacobi and Sag2019, 1168).

Institutional maintenance is complicated at the appellate level by a public information deficit. It may be that “to know courts is to love them, because to know them is to be exposed to a series of legitimizing messages focused on the symbols of justice, judicial objectivity, and impartiality” (Gibson, Caldeira, and Baird Reference Gibson, Caldeira and Baird1998, 345). As one federal circuit court judge put it, however, “People simply do not understand what appellate courts are supposed to do” (Medina Reference Medina1961, 155). As the only part of the appellate decision-making process open to the public, oral argument may inform the public’s perception of how these courts operate.

Judges engage in a variety of outreach activities for institutional maintenance purposes (Savchak and Edwards Reference Savchak and Edwards2016; Curry and Fix Reference Curry and Fix2019; Glennon and Strother Reference Glennon and Strother2019), but traveling to conduct oral arguments is a particularly intriguing example because it exposes people to official court proceedings. State and lower federal courts regularly conduct oral argument at law schools.Footnote ¹ Moreover, several state courts have expanded their oral argument outreach initiatives to the public more broadly. Example institutionalized programs include Appeals on Wheels by the Indiana Court of Appeals, Courts in the Community by the Hawaii Supreme Court, and Justice on Wheels by the Wisconsin Supreme Court. Officials hope these initiatives enhance judicial legitimacy and improve public perception (Deits and Keenan Reference Deits and Keenan2004, 243; Nelson Reference Nelson2005, 170), but the extent to which they do so is unclear.

Bringing courts to the people is an old idea. Supreme Court circuit riding may have primarily been a cost-saving measure, but it was also thought to be of “great value in keeping the federal judiciary in touch with the local communities” (Warren Reference Warren1924, 58). When justices lobbied to abolish the practice, legislators resisted in part because of the public-interfacing component. In 1848, for example, one senator argued that abolishing circuit riding would result in the Court “losing… that responsive confidence of the people, which adds so essentially to the sanction of all the acts of the officers of government.”Footnote ² Although circuit riding has long been abolished, calls for renewal regularly emphasize the value of public exposure (Calabresi and Presser Reference Calabresi and Presser2006; Stras Reference Stras2006).

Positivity theory provides a theoretical lens through which one can assess the potential impact of oral argument exposure on public attitudes. The core idea is simple: “[I]ncreased exposure to the judicial process, whatever the circumstances and even when citizens are displeased, results in collateral exposure to the symbols of judicial legitimacy, thereby tending to reinforce rather than undermine institutional support” (Gibson and Caldeira Reference Gibson and Caldeira2009, 4). Symbol exposure “triggers learned associated thoughts, which for most people in the United States have become connected with these symbols largely through socialization processes and experience, and which are typically ones of legitimacy and positivity” (Gibson, Lodge, and Woodson Reference Gibson, Lodge and Woodson2014, 842). As a result, comprehending court proceedings is not a necessary condition for attitude change.

Legitimizing symbols can be specific or abstract. Examples of specific legitimizing symbols that tend to be present at oral argument include “judges’ black robes” (Gibson and Nelson Reference Gibson and Nelson2017, 593), judges occupying an elevated position in the courtroom (Gibson, Caldeira, and Spence Reference Gibson, Caldeira and Spence2003b, 553), and litigants using “honorific forms of address” (Gibson, Lodge, and Woodson Reference Gibson, Lodge and Woodson2014, 838). Experimental evidence indicates that exposure to such symbols can heighten the positive relationship between support and decision acquiescence for low-frequency consumers (Gibson, Lodge, and Woodson Reference Gibson, Lodge and Woodson2014), sever the link between decision disagreeableness and diffuse support for those expressing relatively low levels of disappointment with a court’s judgment (Gibson and Nelson Reference Gibson and Nelson2016), and enhance perceptions of institutional legitimacy (Armaly Reference Armaly2018).

Court proceedings also expose consumers to more abstract legitimizing symbols such as “decorum” (Nelson and Gibson Reference Nelson, Gibson, Howard and Randazzo2017, 134); “judicial objectivity” (Gibson, Caldeira, and Baird Reference Gibson, Caldeira and Baird1998, 345); “emphasizing reliance on the Constitution, precedent, and legal norms” (Nicholson and Hansford Reference Nicholson and Hansford2014, 621); and “impartiality and insulation from ordinary political pressures” (Gibson and Caldeira Reference Gibson and Caldeira2009, 9). Consistent with courtroom settings, studies in other institutional contexts have found legitimizing effects associated with exposures emphasizing transparency (Gottlieb Reference Gottlieb2016; Hawkins et al. Reference Hawkins, Brook, Hansen, Hoopes and Tidwell2019; Hollyer, Rosendorff, and Vreeland Reference Hollyer, Rosendorff and Vreeland2019), deliberation (Tyler, Rasinski, and Spodick Reference Tyler, Rasinski and Spodick1985; Simon and Sulkin Reference Simon and Sulkin2002; Persson, Esaiasson, and Gilljam Reference Persson, Esaiasson and Gilljam2013), legal frames (Baird and Gangl Reference Baird and Gangl2006; Zink, Spriggs, Scott Reference Zink, Spriggs and Scott2009; Farganis Reference Farganis2012), and procedural justice (Gibson Reference Gibson1989; Tyler and Rasinski Reference Tyler and Rasinski1991; Baird Reference Baird2001).

There is little existing empirical evidence on the impact of court exposure on public attitudes. Most relevant for our purpose, Black et al. (Reference Black, Johnson, Owens and Wedekingn.d.) conducted survey experiments exposing observers to short video clips of oral argument exchanges in two state supreme courts. Leveraging variation in audio versus video presentations, contentious versus neutral interpersonal exchanges, and dynamic versus static camera angles, the authors find mixed evidence concerning the impact of exposure on perceptions of legitimacy. With respect to court exposure more broadly, Benesh (Reference Benesh2006) finds that respondents who report having been a juror express more confidence in local courts, while those who report having been a direct party express less confidence. And Grimmelikhuijsen and Klijn (Reference Grimmelikhuijsen and Klijn2015) find that a randomized instruction to watch a television show featuring real trial court proceedings, commentary, and interviews with participating judges yielded increased trust in judges as the self-reported number of shows watched increased.

Building on this literature, we hypothesize that oral argument attendance will increase perceptions of institutional legitimacy and the extent to which judges are thought to base decisions on law versus politics. While there have been relatively few examinations of how court exposure impacts perceptions of decision-making compared to legitimacy, the underlying theory is similar. Legitimizing symbols presumably activate the “myth of legality,” which “holds that cases are decided by the application of legal rules formulated and applied through a politically and philosophically neutral process of legal reasoning” (Scheb and Lyons Reference Scheb and Lyons2000, 929). While so-called “legalist” and “realist” perspectives on judging are theoretically distinct from the concept of legitimacy (Gibson and Caldeira Reference Gibson and Caldeira2011; Cann and Yates Reference Cann and Yates2016), the same psychological processes that lead symbols to enhance legitimacy can reasonably be construed to reinforce perceptions that judges make decisions based on law rather than politics. Evidence indicates, for example, that exposure to a lunch or speech with a Supreme Court justice increases the extent to which decisions are thought to be driven by law relative to ideology (Krewson Reference Krewson2019).

The field experiment

To examine the causal effect of oral argument exposure on perceptions of the judiciary, we conducted a field experiment randomizing attendance at proceedings conducted by the U.S. Court of Appeals for the Tenth Circuit at Utah State University. The Tenth Circuit is one of thirteen federal circuit courts of appeals. It is an intermediate appellate court, sitting between state-based district courts and the Supreme Court in the federal judicial hierarchy. The Tenth Circuit hears appeals arising from district courts in six states.Footnote ³ Oral arguments are heard at the court’s discretion, generally in three-judge panels. According to Tenth Circuit officials attending this event, the court travels for oral argument as an outreach initiative for students and the public.

At this event, a three-judge panel held two oral argument sessions, with two cases in each session and a short break between sessions. Case details are provided in the appendix. As is common with circuit court proceedings, the cases were not particularly salient. Advance session-specific tickets were required for admission to the event, which filled to capacity. Student access was primarily allocated through this study.Footnote ⁴ Prior to the visit, we recruited undergraduate and graduate students to participate in a study concerning courts on event day in exchange for a $10.00 Amazon gift card.Footnote ⁵ In the recruitment email, students were told they would be randomly assigned to attend an oral argument session or participate in a different court-related activity. Participating students selected one of two time slots corresponding to the oral argument sessions. Once students were assigned to a session slot, they were were randomized within sessions at a 50–50 split between treatment and control groups.Footnote ⁶

The treatment and control groups were instructed to appear on event day at separate but nearby campus locations. On event day, control group participants completed a survey immediately upon arrival at their scheduled time and were subsequently dismissed. Treatment group participants were admitted to their respective oral argument sessions, after which they were provided the same survey with a day’s end completion deadline. All surveys were completed electronically on event day.Footnote ⁷ Overall, 248 students participated in the experiment. The first session respectively included 48 and 58 students in the control and treatment groups; the second session respectively included 70 and 72 students in the control and treatment groups.Footnote ⁸ As described in more detail in the appendix, control and treatment groups were balanced within and across sessions by gender, race, ideology, and prior court exposure. Although student samples present obvious generalizability issues, ours is “relevant for theory” (Coppock and McClellan Reference Coppock and McClellan2019, 3) because students are a target audience for traveling courts.

We followed standard practice in measuring our outcomes of interest. To capture perceptions of legitimacy, we used an established battery of statements (see, for example, Gibson, Caldeira, and Spence Reference Gibson, Caldeira and Spence2003a; Gibson and Caldeira Reference Gibson and Caldeira2009; Gibson and Nelson Reference Gibson and Nelson2015) that capture whether institutions enjoy “a widely accepted mandate to render judgments for a political community” (Gibson, Caldeira, and Spence Reference Gibson, Caldeira and Spence2003a). The items include:

• • Judges on the federal courts who consistently make decisions at odds with what a majority of the people want should be removed from their position as a judge.

• • If the federal courts started making a lot of decisions that most people disagree with, it might be better to do away with the federal courts altogether.

• • The right of the federal courts to decide certain types of controversial issues should be reduced.

• • The federal courts get too mixed up in politics.

Given the institutional context of our experiment, we modified these questions as developed for a study concerning the Supreme Court by replacing “Supreme Court” with “federal courts” (cf. Scherer, Benesh, and Steigerwalt Reference Scherer, Benesh and Steigerwalt2010).Footnote ⁹ Response choices were five-point Likert scales ranging from strongly disagree to strongly agree. A principal components analysis on the scores shows that items load on a single dimension.Footnote ¹⁰ We used factor scores from the principal components analysis to create a legitimacy index and normalized scores to the [0,1] interval.Footnote ¹¹

To operationalize our decision-making hypothesis, we used a battery of questions developed by Gibson and Caldeira (Reference Gibson and Caldeira2011) to measure what they call “legal realism.” In essence, this battery captures the extent to which people think judicial decision-making is driven by law versus politics. The items include:

• • Judges always say that their decisions are based on the law and the Constitution, but in many cases, judges are really basing their decisions on their own personal beliefs.

• • Judges’ values and political views have little to do with how they decide cases before the federal courts.

• • Judges’ party affiliations have little to do with how they decide cases before the federal courts.

Again, given the institutional context of our experiment, we modified the questions as designed by replacing “Supreme Court” with “federal courts” (cf. Scherer, Benesh, and Steigerwalt Reference Scherer, Benesh and Steigerwalt2010). Response choices were five-point Likert scales ranging from strongly disagree to strongly agree. We used principal components analysis to create an index concerning decision-making attitudes and normalized scores to the [0,1] interval.Footnote ¹² As detailed in the appendix, the result is similar using an alternative measure of beliefs about judicial decision-making developed by Cann and Yates (Reference Cann and Yates2016).

Analysis and results

Legitimacy

Beginning with the legitimacy hypothesis, Figure 1 plots means, 95% confidence intervals, and underlying data distributions (jittered for clarity) for the control and treatment groups. Between control and treatment, the perceived legitimacy mean increases from 0.48 to 0.60 ( $ p $ $ < $ .001). Substantively, this change indicates that oral argument attendance increased perceived legitimacy by 57% of a standard deviation. As detailed in the appendix, these results hold across sessions. There is evidence of heterogeneous session effects, with perceived legitimacy higher in the first session, but exposure to the second session nonetheless increased perceived legitimacy by 39% of a standard deviation.

Figure 1. The Effect of Oral Argument Attendance on Legitimacy.

Judicial decision-making

Turning to the decision-making hypothesis, Figure 2 plots means, 95% confidence intervals, and underlying data distributions for the control and treatment groups. Between control and treatment, the mean index score increases from 0.34 to 0.46 ( $ p $ $ < $ .001). Substantively, this indicates that oral argument attendance increased the extent to which individuals perceive judicial decision-making to be driven by law versus politics by 61% of a standard deviation. As detailed in the appendix, the result holds across sessions. Here, however, there is no evidence of heterogeneous session effects.

Figure 2. The Effect of Oral Argument Attendance on Beliefs About Judicial Decision-Making.