What is Precedent?

A rich literature across several disciplines engages legal precedent theoretically and empirically (Reference Black and SpriggsBlack and Spriggs 2013; Reference Bueno De Mesquita and StephensonBueno De Mesquita and Stephenson 2002; Reference Clark and LauderdaleClark and Lauderdale 2012; Reference Fowler, Johnson, Spriggs, Jeon and WahlbeckFowler et al. 2007; Reference Fowler and JeonFowler and Jeon 2008; Reference Hansford and SpriggsHansford and Spriggs 2006; Reference Landes and PosnerLandes and Posner 1976; Reference LaxLax 2007; Reference Lax and CameronLax and Cameron 2007; Reference Lax and LandaLax and Landa 2009; Reference Patty, Penn, Schnakenberg, Schofield, Caballero and KselmanPatty, Penn, and Schnakenberg 2013; Reference Spriggs and HansfordSpriggs and Hansford 2001, Reference Spriggs and Hansford2002). Legal scholars define precedent pithily as: “something done in the past that is appealed to as a reason for doing the same thing again” (Reference Landes and PosnerLandes and Posner 1976, 250). Lawyers and judges are taught to reason by analogy: find a precedent case with a similar fact pattern, determine and define the appropriate legal rule or standard from the precedent case, and then apply the rule or standard to the current case (Reference LeviLevi 1948).Footnote ³ Thus, if a precedent is ambiguous, fractious, and promulgates multiple conflicting rules or standards, then lawyers and judges should struggle to apply the precepts of legal reasoning when considering that precedent. Therefore, the doctrinal paradox represents a class of precedents for which the standard model of legalistic deference to precedent breaks down. When paradoxes are cited, these citations may instead indicate judges struggling to clarify the ambiguity inherent in these precedents.

Empirical work measuring precedent typically gives a broad, multifaceted definition of terms like “important,” “significant,” or “authoritative.” For instance, (Reference Fowler and JeonFowler and Jeon 2008: 17) propose that their network-based method (which uses (Reference KleinbergKleinberg's 1999) authority scores), “…indicate[s] the degree to which a case is thought to be important for resolving other important issues that come before the Court.” Importance has been operationalized in prior work as the qualities of a precedent that makes experts regard it as important in a survey (Reference Fowler and JeonFowler and Jeon 2008: 17). Indeed, the fact that their measure correlates with these existing lists of “important” cases is good evidence of construct validity. “In such rankings, legal experts evaluate a case's importance on its historical and/or social significance, its importance to the development of some area of law, its impact on the development of American government, and relatedly, its prevalence in legal textbooks” (Reference Fowler and JeonFowler and Jeon 2008: 20). This definition combines a variety of legal and extralegal concepts. (Reference Fowler and JeonFowler and Jeon's 2008) work seems designed to determine which precedents are most noteworthy in a broad, general sense to the Court itself, and their study represents a sophisticated and thorough approach for accomplishing that goal. Given, however, the more explicitly jurisprudential definition of precedent in legal scholarship, other approaches may be better suited to capturing the legal significance of precedent in the judicial system writ large.

Types of Citations Across a Judicial Hierarchy

The preceding sections imply that when a judge cites a precedent, this citation may or may not be deferential in nature. Even at the Supreme Court, the justices at times arguably enter a “jurisprudential mode.” Justices enter this mode when their preferences over a case's potential outcome are not strong and legalistic considerations (such as stare decisis) are allowed to dominate their decision-making process (Reference Lindquist and KleinLindquist and Klein 2006; Reference PerryPerry 1991). A deferential, or “following,” citation appears jurisprudential in nature, consistent with the definition of precedent in legal scholarship (Reference Landes and PosnerLandes and Posner 1976; Reference MaltzMaltz 1987; Reference SchauerSchauer 1987). Conversely, at times judges cite precedent critically or neutrally.

Why might a judge feel compelled to cite a precedent without deferring to it? The quality of a judicial opinion relates to its future probability of being overruled (Reference Clark and CarrubbaClark and Carrubba 2012). Citations to precedent are the main rhetorical device judges can use when crafting an opinion. As such, a future iteration of the Supreme Court would likely view a precedent that cited no authorities unfavorably.Footnote ⁴ Additionally, judges have multiple goals to satisfy in making their decisions and writing opinions, not all of which are policy-oriented (Reference BaumBaum 1994; Reference Epstein and KnightEpstein and Knight 2013). Judges desire to be respected and taken seriously by their peers and the legal academy. Judges can help satisfy this goal by writing high-quality opinions laden with citations to precedent.

If a precedent is frequently cited in a passing, neutral, or critical fashion, but not a deferential/jurisprudential fashion, can it be said that such a precedent exerts much legal influence? Such a precedent, which would enjoy high citation counts (due to its general relevance), is not guiding legal doctrine. The links between it and future decisions might be the result of shared ideology (homophily), or its general noteworthiness, or the fact that it is so multifaceted and malleable that it can be massaged to justify almost any decision. Thus, the fact that a precedent is well-known enough to be well-cited by judges who have a negligible probability of audit is insufficient evidence to classify a precedent as jurisprudentially influential across an entire legal system. A measurement of the broader jurisprudential influence of precedent should consider both the content and the location of citations across all levels of a judicial hierarchy.

Measuring Precedent

Empirically, how might these different types of citations be analyzed? One avenue lies in the application of network methods. Whenever a case X cites a precedent Y, a “tie” or “edge” is formed between them. These ties combine to form a network amenable to study. The simplest network-based metric of precedential influence is degree centrality. The degree centrality of a given node (opinion) in a network is simply the number of ties it has to other nodes. For the purposes of measuring the significance of a given Court decision, the precedent's indegree centrality (how many cases cite the precedent, rather than the other way around) is the quantity of interest. But there are more sophisticated network-based methods for measuring precedent.

A common method for adding precision to degree centrality involves weighting the centrality of a case according to properties of other cases that cite it.Footnote ⁵ For instance, Reference Fowler, Johnson, Spriggs, Jeon and WahlbeckFowler et al. (2007) suggest Reference KleinbergKleinberg's (1999) hub and authority scores. Kleinberg's method, a variant of the more common eigenvector centrality (Reference BonacichBonacich 2007), is well-described in (Reference Fowler and JeonFowler and Jeon 2008: 17): “The authority score of a case depends on the number of times it is cited and the quality of cases that cite it.” In essence, a case's authority score is its indegree centrality weighted by quality of the cases by which it is cited. “Quality” here means that the citing case also cites a number of other precedents with high authority scores.Footnote ⁶ The authority score method shares much in common with another method, eigenvector centrality.Footnote ⁷ These methods seem promising for measuring the general relevance of precedents. Indeed, as Reference Fowler and JeonFowler and Jeon (2008) show and as I replicate below, Supreme Court cases deemed “important” by experts are highly ranked by these network approaches.

However, precedents are cited in a variety of positive and negative ways, a fact not captured by these measures as applied in prior work. For instance, a judge may cite a precedent while directly criticizing it or overruling it in the text of her opinion. Indeed, Reference Fowler and JeonFowler and Jeon (2008) find that precedents that are later overruled are considered more authoritative. This counterintuitive finding may be at least partially due to the fact their measure does not distinguish between positive and negative treatments of precedent. Further, since Supreme Court justices handle only a tiny fraction of the judicial workload of the United States, and work in an institutional environment that differs considerably from that of lower federal courts, a new approach to measuring precedent is called for—one that both measures the behavior of judges at all levels of the judicial hierarchy, while taking the context of citations into account.

The jurisprudential influence of precedents may be better measured by the number of citations that are “follows,” aggregated across all levels of a judicial hierarchy. In the United States, “follows” are defined according to Shepard's Citations. “Shepard's Citations is a citation index that, among other things, provides a list of all U.S. court opinions that refer to any U.S. state or federal court case decided since the beginning of the U.S. legal system” (Reference Spriggs and HansfordSpriggs and Hansford 2000: 328–329). Shepard's also indicates how a particular court opinion is legally interpreted by the opinions that cite it. Shepard's citation reports classify many different ways in which a precedent can be cited.Footnote ⁸ These classifications (other than distinctions among the more ambiguous neutral classifications) are quite reliable when hand-validated by scholars (Reference Spriggs and HansfordSpriggs and Hansford 2000). Of particular interest is the classification “followed,” which Shepard's uses to indicate “that a citing case's majority opinion ‘expressly’ relied on a cited case as precedent.” According to the manual, “followed” is only to be applied “if the citing opinion contains language that goes beyond a ‘mere going-along’ with the cited case” (Reference Spriggs and HansfordSpriggs and Hansford 2000: 330). Thus, the number of follows a precedent receives may be a good indication of its broader legal influence.

Of course, the three levels of the American judicial hierarchy process caseloads of vastly differing sizes. As such, simply summing the number of times a precedent is followed, at any level of the hierarchy, would disproportionately favor precedents followed in the federal district courts. Instead, a precedent's legal influence should be measured by first standardizing follows at each level of the hierarchy, yielding a z-score for a precedent's follows at each level, then summing the three z-scores. This approach equally weights precedents’ relative influence at each level of the judicial hierarchy. I show in the Appendix that the results are robust to instead aggregating via each precedent's score from an iterated principal factor analysis.

A possible tradeoff arising from using aggregated follows instead of Supreme Court citations is that this metric may measure legal influence without capturing broad, general relevance. Indeed, some precedents set down rules or standards that are frequently followed and applied, but are not of especial interest or prominence to scholars and journalists. Further, the Supreme Court may consider precedents like these to be settled law, so they may not be frequently cited at that level regardless of their broader influence.

To illustrate this tradeoff, consider that of all precedents issued between 1946 and 2010, the one most-followed in district court opinions is 1993's Farmer v. Brennan (511 U.S. 525). Farmer dealt with a transgender woman housed in a male prison. She sued the prison after being beaten and raped in her cell. The Supreme Court ruled that prison officials’ deliberate indifference to a substantial risk of serious harm to the inmate violated the Eighth Amendment. The Court further established a two-part test for lower courts to apply when determining when prison officials may be held liable due to negligence in cases of prisoner-on-prisoner violence.

This precedent has been followed close to 10,000 times in district court opinions. Yet it has only been cited in three U.S. Supreme Court opinions. The mean amount of citations at the Supreme Court for precedents generated between 1946 and 2010 is about 10.7, with a range of 0 to 266.Footnote ⁹ Yet it can hardly be said that Farmer has little significance; while other precedents may enjoy much more noteworthiness, Farmer exerts a sizable influence on the American legal system.

To further illustrate the differences between these approaches, Table 1 shows the top 10 precedents 1946–2010, as measured by their indegree centrality (count of citations in subsequent Supreme Court opinions), and their aggregated count of follows at all levels of the judicial hierarchy.

Table 1. Top 10 Precedents by Supreme Court Citations and Aggregated Follows

Table 1 further demonstrates the tradeoffs inherent in the methodological choices analysts must make when measuring precedent. Many precedents in the top 10 according to Supreme Court citations are broadly well-known constitutional decisions. Conversely, precedents in the top 10 according to aggregated follows tend to be those that set down clear rules, tests, or instructions for lower court judges (Farmer v. Brennan, e.g.). These precedents, which may not be as well-known as high-profile constitutional decisions, nonetheless provide substantial guidance and structure to future judges, who accordingly follow these precedents at extremely high rates.

I continue with a discussion of the data. I then conduct the proposed empirical test: a measurement of precedent that accurately incorporates jurisprudential influence should score precedents that are by definition ambiguous (i.e., doctrinal paradoxes) lower.

Data

For the analysis to follow, I utilize both data from Reference Fowler and JeonFowler and Jeon (2008), graciously made publicly available by Fowler, and original data on the treatment in the federal courts of every Supreme Court precedent promulgated 1946–2010, collected by the author from LexisNexis.Footnote ¹⁰

Interestingly, while the network-based metrics in Reference Fowler and JeonFowler and Jeon's (2008) data (degree centrality, authority scores, and eigenvector centrality) all differ theoretically, they correlate at high levels. A principal components factor analysis shows that these metrics load onto a single factor with an eigenvalue well above one, with no other factor approaching one. The results indicate that these diverse statistics all capture the same underlying phenomenon (Reference KaiserKaiser 1960). As such, for parsimony and simplicity, I conduct the analysis going forward on only indegree centrality. I show in the Appendix that the results below are robust to considering instead eigenvector centrality, authority score, outdegree centrality, or hub scores, as other scholars (Reference Fowler, Johnson, Spriggs, Jeon and WahlbeckFowler et al. 2007; Reference Fowler and JeonFowler and Jeon 2008) show that these network characteristics matter above and beyond degree centrality.

To consider deferential citations and citations in lower courts, I collected citation data from Shepard's Citations via LexisNexis for all cases for which the relevant covariates from Reference Fowler and JeonFowler and Jeon's (2008) data and Spaeth's database were also available. This approach limits the analysis to 1946–2010. Combined with Reference Fowler and JeonFowler and Jeon's (2008) data, there are therefore six court-level variables and two aggregated variables available for analysis. The court-level variables are follows at the Supreme Court, circuit court, and district court level, and all citation types, pooled, at these same three levels. The aggregated variables are follows and citations, separately, aggregated across all three levels. An exploratory factor analysis of the six court-level variables uncovered three factors with eigenvalues above one, suggesting that while certain pairs of variables (such as citations and follows in the district courts) are highly related, the different metrics do not cleanly track a single latent variable.

It is expected that measuring all citations, positive, negative, and neutral, may mistakenly counts some criticisms as legal influence. Thus, measuring all types of citations seems likely to track the general relevance of precedent, while measuring follows (across all levels of a judicial hierarchy) will better meet the theoretical definition of legal influence (Reference MaltzMaltz 1987; Reference SchauerSchauer 1987). Further, it is expected that citations in higher levels of the judicial hierarchy may track judges critiquing and clarifying ambiguous doctrine, while citations at the district court level will be less likely to track this dynamic. I now move to an empirical test of these conjectures.

Distinguishing Between Methods: The Doctrinal Paradox

Doctrinal paradoxes, generalized as the discursive dilemma by Reference List and PettitList and Pettit (2002), are opinions of the Court for which every rationale for the Court's judgment is rejected by a majority. Paradoxes are by definition ambiguous and near-impossible to defer to consistently. I propose that any method for measuring the jurisprudential influence of precedent should score paradoxes lower than other similar precedents. Methods that score paradoxes highly would conversely seem to capture judges critiquing and clarifying the law.

A canonical example of the paradox is National Mutual Insurance v. Tidewater Transfer, 337 U.S. 582 (Reference NashNash 2003; Reference Post and SalopPost and Salop 1992; Reference StearnsStearns 2000). Tidewater centered on whether “citizens” (here a corporation) of the District of Columbia could file lawsuits in federal court against a Virginia business, based on the diversity of their citizenship.Footnote ¹¹ The justices debated two legal premises in the case. First, does the Constitution permit citizens residing in the District of Columbia, which is not a state, to file lawsuits in federal court against citizens of Virginia? Second, if not, can Congress give the federal courts the authority to hear lawsuits from citizens of the District of Columbia against citizens of Virginia? A majority of justices must support at least one of these premises for the Court to find in favor of the D.C. corporation (Reference StearnsStearns 2000).

Only a minority of justices supported either premise. Yet a majority found in favor of diversity, and thus for the D.C. corporation. Had the justices taken a majority vote on the first premise, and then on the second, binding themselves to issue a judgment based on their votes over the premises, they would have found against diversity. Table 2 illustrates the paradox of Tidewater. Two justices support the first premise; three the second, combining for a five to four majority in favor of diversity. A majority held that the Constitution does not permit D.C. citizens to file lawsuits in federal court against citizens of Virginia. A majority also held that Congress lacks the authority to empower the federal courts to hear such lawsuits. Yet a majority ruled that D.C. citizens may file lawsuits in federal court against citizens of the states. Tidewater exhibits the paradox. Because neither premise for this judgment carried a majority, Tidewater subsequently caused confusion in the lower courts (Reference Post and SalopPost and Salop 1992).

Table 2. The Doctrinal Paradox in Tidewater

Notes. Either premise 1 or 2 must have majority support to avoid the paradox.

The paradox was first identified by Reference Kornhauser and SagerKornhauser and Sager (1986) later generalized by Reference List and PettitList and Pettit (2002); cf. Reference EasterbrookEasterbrook (1982). In related work (Reference HittHitt 2013), I identified decisions from 1946–2010 that are likely to be paradoxes based on observable proxy variables.Footnote ¹² I then employed legally trained research assistants (third-year law students) to verify that the identified cases did indeed exhibit the phenomenon. The research assistants read the opinion of the Court and all separate opinions, looking in concurring opinions for explicit rejection of the rationale(s) offered by the opinion of the Court. These assistants constructed a table similar to Table 2 for every decision identified by the proxy variables. This exercise identified 146 doctrinal paradoxes between 1946 and 2010.

Based on the issue area categories of Spaeth's database, about 68 percent of paradoxes fall in one of three issue areas: Criminal Procedure, Civil Rights, and the First Amendment. For contrast, about 45 percent of non-paradoxes fall into one of these three categories. Paradoxes largely derive from the broad category of civil liberties that defines the bulk of the modern court's docket, at even greater rates than consistent decisions. Further, paradoxes occur in every natural Court under analysis.

Additionally, less than one percent of consistent decisions exercise judicial review, while about seven percent of paradoxes exercise judicial review (over Congress). Of all Court decisions analyzed here (1946–2010), about one percent represent an exercise of judicial review. Of these, about 11 percent of these decisions are doctrinal paradoxes. That is, for every instance of judicial review (of Congress) 1946–2010, the Court exercised this power via a paradox just more than once out of every ten overall occurrences. Conversely, only about one and a half percent of decisions not exercising judicial review are paradoxes. The ${\chi }^2$ value of this comparison is 49.47, p < 0.05.Footnote ¹³

Taken together, this discussion shows that doctrinal paradoxes deal with politically and legally meaningful topics, and are not confined to any single narrow issue area.

The test I propose is simple: analyzing every precedent for which both citation data and following data, at all three levels of the judicial hierarchy, could be collected, what is the relationship between the paradox and each metric? The metrics to be tested are: aggregated follows, Supreme Court follows, circuit court follows, and district court follows, aggregated citations, Supreme Court citations, circuit court citations, and district court citations. If the coefficient on a dummy variable for doctrinal paradoxes is negative and significant in a model with a given metric as dependent variable, then that metric discounts confusing and fractious precedents, indicating it tracks broader jurisprudential significance. As such, that metric is to be preferred for measuring legal influence to one without such a result. Conversely, any method that scores doctrinal paradoxes as significantly more influential would therefore track judges grappling with (and perhaps criticizing or clarifying) important, but inconsistent doctrine (Reference SunsteinSunstein 2007).

An Alternative Test: Conflict Cases

While using doctrinal paradoxes as a face validity test makes theoretical sense, there are other options. Reference PerryPerry (1991) noted that one of the key legalistic concerns of Supreme Court justices (when they were in “jurisprudential mode”) was the existence of a intercircuit conflict. Reference Lindquist and KleinLindquist and Klein (2006) go on to show that the Court's merits decisions in conflict cases also appear to be more influenced by legalistic concerns.

As such, precedents stemming from conflict cases would seem to be natural candidates for jurisprudential citations. Therefore, if a precedent metric captures legal influence, its relationship to conflict cases should be the opposite of its relationship to the paradox. That is, the coefficient on conflict resolving precedents should be positive and significant, if a given metric is capturing legal influence.

Control Variables

What observable features of Supreme Court cases might relate to the various ways of measuring precedent, allowing for a well-specified execution of the proposed test? (Reference Benesh and ReddickBenesh and Reddick 2002: 546) find that precedents stemming from complex cases are responded to more quickly in lower courts. They speculate: “Perhaps lower courts take more care in understanding and applying complex decisions, or perhaps these cases are easier to comply with.” Whatever the reason, their findings motivate a control for the relationship between case characteristics and precedent score. Reference Benesh and ReddickBenesh and Reddick (2002) consider a case's issue area to be a case characteristic of importance. In the analysis below, I include fixed effects for all 14 issue area categories in Spaeth's database. Data for this portion of the analysis was taken from the Spaeth Database (http://scdb.wustl.edu/), Reference Fowler and JeonFowler and Jeon (2008), and Reference Martin and QuinnMartin and Quinn (2002), merged with the original data on district court follows described above.Footnote ¹⁴

In studying circuit courts of appeals responses to a sample of plurality opinions, Reference CorleyCorley (2009) suggests several covariates worthy of inclusion. Corley suggests that a precedent's “legal importance,” by which Corley means the precedent either declared a statute unconstitutional, or overturned a previously existing precedent, should be related to that precedent's treatment in the lower courts. In the analysis below, I include separate dummy variables equal to one for precedents that exercise judicial review or alter precedent, respectively.

Reference Corley, Steigerwalt and WardCorley, Steigerwalt, and Ward (2013) show that unanimously decided cases at the Supreme Court in the modern era differ from non-unanimous cases in many important respects—particularly cases in which there is a high degree of legal certainty as to the strongest legal answer as well as nonsalient cases involving economic or government power issues (Reference Corley, Steigerwalt and WardCorley, Steigerwalt, and Ward 2013: 160). I include a dummy variable equal to 1 for non-unanimous cases in the analysis below.

Reference Owens and SimonOwens and Simon (2012) explore the Supreme Court's docket, and find that a significant reason for the decline in the number of cases heard at the Court in recent years is in large part a function of the elimination most of the Court's remaining mandatory jurisdiction. A case heard under mandatory jurisdiction is one in which the Court has no option but to decide. The remainder of the Court's workload (and virtually all of the cases it has heard since 1989) arrives via a discretionary process. As mandatory and discretionary cases may vary in important substantive ways—that is, mandatory cases are less likely to deal with issues of broad national significance—I include a dummy for mandatory cases in the analysis below.

Reference Fowler and JeonFowler and Jeon (2008) use data from the Legal Information Institute, highlighting “important” cases at the Supreme Court, as determined by experts. This variable is equal to one if the panel of experts at the Legal Information Institute labeled a precedent as “important.” If degree centrality tracks noteworthiness, then degree centrality should be positively and significantly associated with case importance.

Finally, Reference Fowler and JeonFowler and Jeon (2008) presented a counterintuitive finding: precedents that are later overturned are found to be more authoritative than other precedents. As reversed precedents by definition no longer influence the American legal system, I include a dummy variable in the regressions below for overturned precedents to investigate whether this finding holds when measuring precedent via other methods.

Results

Tables 3 and 4 present the results of negative binomial and OLS regression models of the above covariates regressed on each of the precedent metrics discussed. Table 3 analyzes follows only, while Table 4 analyzes all citation types together. The unit of analysis in all models is a Supreme Court precedent. The dependent variable of the first model in Table 3 is the count of subsequent Supreme Court follows each precedent received. The dependent variable in the second model in Table 3 is the count of follows in the circuit courts for each precedent. The dependent variable in the third model in Table 3 is the count of follows in the district courts. The dependent variable in the fourth model in Table 3, estimated via OLS, is the aggregated number of follows.Footnote ¹⁵ The models in Table 4 are presented according to the same pattern, using all citation types instead of follows only.

Table 3. Negative Binomial & OLS Estimates of the Relationship of Case Factors to Precedent Follows

^* p < 0.05.

Notes. Robust standard errors in parentheses. Constant and legal issue area fixed effects not reported. Dependent variable in the first three models (estimated via negative binomial regression), is each precedent's count of follows at the respective level of the judicial hierarchy. Dependent variable in the fourth model (estimated via OLS) is the sum of follows across levels, standardized according the mean at each level.

Table 4. Negative Binomial & OLS Estimates of the Relationship of Case Factors to Precedent Citations

^* p < 0.05.

Notes. Robust standard errors in parentheses. Constant and legal issue area fixed effects not reported. Dependent variable in the first three models (estimated via negative binomial regression), is each precedent's count of citations at the respective level of the judicial hierarchy. Dependent variable in the fourth model (estimated via OLS) is the sum of citations across levels, standardized according the mean at each level.

Given that the variance of these metrics are greater than the respective means, the negative binomial model is the appropriate count model specification for the count models.Footnote ¹⁶ Fixed effects for the legal issue area and the constant estimates not reported for brevity. The age of precedents is included as a simple control variable, given that older precedents may be more prone to higher following or citation counts. Results are robust to more complex, non-linear methods of accounting for age, such as including the squared value of age, or controlling for age with a linear or cubic spline. Results are also robust to hierarchical specifications with a random intercept varying by term, as an alternative method of accounting for heteroskedastic effects conditional on precedent age.

Tables 3 and 4 show the results of the proposed test. First, doctrinal paradoxes have a positive and significant association with degree centrality (citation count) at the Supreme Court. From the perspective of measuring jurisprudential influence, this association is in the wrong direction. Conversely, follows at all levels of the judicial hierarchy are negatively and significantly associated with paradoxes, as would be expected given the inconsistent and complex nature of these precedents. Further, citations in the circuit courts have no significant association with doctrinal paradoxes, while citations in the district courts are negatively and significantly associated with doctrinal paradoxes. Metrics that utilize all citation types improve in their ability to track jurisprudential influence when measured in successively lower levels of the judicial hierarchy. Finally, aggregated follows are negatively and significantly associated with the doctrinal paradox, while aggregated citations have no significant association with the paradox.

Tables 3 and 4 also show that circuit conflict cases have no association with follows or citations at the Supreme Court level. Conflict cases are positively and significantly associated with follows and citations at the district and circuit court level, as well as aggregated citations and follows. The results of the tests are clear: jurisprudential influence can be better accounted for if analysts consider only deferential citations, incorporate lower levels of the judicial hierarchy, or, ideally, both. Conversely, measuring all citations exclusively at the Supreme Court level rates ambiguous, fractious precedents highly, indicating this metric captures the justices clarifying or wrangling with difficult and unclear doctrine.

Table 4 also replicates Reference Fowler and JeonFowler and Jeon's (2008) finding: “important” cases are significantly associated with greater counts of citations in Supreme Court opinions. Holding all other variables at their modal values, important cases are predicted to receive about 15 more citations in the Supreme Court, an increase of about one standard deviation.Footnote ¹⁷ Important cases are also predicted to receive about half of a standard deviation more follows across all levels of the judiciary. As such, the magnitude of the relationship between importance and aggregated follows is only a fraction of the size of the association between important cases and degree centrality in the Supreme Court. This finding provides evidence that citations in the U.S. Supreme Court are a better method for measuring a precedent's general relevance.

Interestingly, while declarations of unconstitutionality are significantly more cited in the Supreme Court, there is not a significant association between declarations of unconstitutionality and follows or citations in either the circuit or district courts. Once a statute is declared unconstitutional, it seems likely that less litigation regarding that statute would arise in the lower courts, perhaps explaining this non-significant association.

Unsurprisingly, precedents that alter or overturn prior precedents are themselves significantly more followed and cited at all levels of the judicial hierarchy. Precedents stemming from non-unanimous decisions have a positive and significant association with follows and citations in the Supreme Court, and there is no significant association between non-unanimous cases and follows or citations in the lower courts. Cases that are brought to the Court under its mandatory jurisdiction are significantly more cited (but not followed) at the Supreme Court level, yet these same precedents are followed and cited significantly less in the lower courts. Perhaps because the Court tends to hear cases that have a broad impact on the American legal system under its discretionary jurisdiction (Reference ShapiroShapiro 2006), precedents arising from mandatory cases are less likely to generate much additional litigation, all else equal.

The significant association between reversed precedents and citations at the Supreme Court, reported by Reference Fowler and JeonFowler and Jeon (2008), is replicated in Table 4. There is no significant association between reversed precedents and a precedent's standardized sum of follows across all three levels of the hierarchy. That is, a reversed precedent's jurisprudential influence does not stand out relative to other precedents. This finding makes sense—reversed precedents enjoy a variable period of years in which they are considered good law before being overturned. For those years, they would be followed as much as any other precedent. But, once reversed, follows would naturally decline. Conversely, reversed precedents are positively and significantly associated with citations across all levels of the judicial hierarchy.

Taken together, the findings indicate that the best method for considering a precedent's broad, general relevance likely involves some metric based on citations at the Supreme Court, while jurisprudential influence may be best measured by an aggregation of deferential citations across all levels of the judicial hierarchy.

Conclusions

Measuring the legal influence of precedent using judicial citation data requires considering both the content and context of judicial opinions. Many scholars, such as Reference Fowler, Johnson, Spriggs, Jeon and WahlbeckFowler et al. (2007); Reference Fowler and JeonFowler and Jeon (2008): and Reference Patty, Penn, Schnakenberg, Schofield, Caballero and KselmanPatty, Penn, and Schnakenberg (2013) consider citation context, in terms of a precedent's networked position, at the Supreme Court level. These scholars succeed in their efforts to measure which precedents are most relevant to current disputes at the Court at a given point in time.

Yet courts of last resort may cite precedents in a negative fashion, criticizing older doctrine as they shape the law. Further, courts of last resort may cite fractious, inconsistent precedents that offer minimal legal guidance or constraint, to highlight and refine existing ambiguities in the law (Reference SunsteinSunstein 2007). As such, a precedent that is well-cited in a court of last resort may not receive these citations due to any jurisprudential authority.

Scholars must make two choices when attempting to measure the influence or significance of legal precedents using subsequent citations in judicial opinions. First, at which level of a judicial hierarchy subsequent citations are to be measured from. Second, whether to use all citations, be they positive, negative, or neutral, or to consider only those that express deference (that is, explicitly “follow”) the precedent (Reference Hansford and SpriggsHansford and Spriggs 2006).

The import of these choices is by no means limited to the United States. Measuring precedent in any country which utilizes a hierarchical, common law judicial system seems likely to be susceptible to similar tradeoffs. Scholars should explicitly decide which theoretical construct, general relevance or jurisprudential influence, they aim to measure. Analysts must then consider both which type of citations (only deferential or everything) and which level(s) of the hierarchy to analyze based on their theoretical target.

Using data on the treatment of every U.S. Supreme Court precedent in all levels of the judicial hierarchy 1946–2010, I illustrate the tradeoffs associated with these two choices in the American context. I show that measuring precedent via either following behavior (in any level of the judicial hierarchy), or in federal district courts (even using all citation types) better measures the notion of jurisprudential significance. Further, measuring all citations at the Supreme Court level tracks precedents that may be invoked to criticize or clarify fractious and inconsistent doctrine. Ultimately, it appears that aggregating all deferential citations across a judicial hierarchy tracks the broader jurisprudential influence of precedent quite well. The results suggest that courts of last resort exert broader legal influence most frequently when promulgating rules or standards that apply across a wide range of disputes. These process-oriented precedents may attract less attention than polarizing civil liberties disputes. However, judges across the American judicial hierarchy defer to these precedents at high rates. A worthy future area of study would be analyzing how these highly influential process-oriented precedents impact litigants of various identities. Are the processes they stipulate neutral? Further, measuring precedent by aggregating deferential citations across a hierarchy will likely identify the important process-oriented precedents in other contexts. A comparative study of these precedents would be a worthy aim for future work.

The analysis also shows that measuring precedent using citation counts at the Supreme Court level (which are statistically indistinguishable from more sophisticated options like eigenvector centrality and authority scores) captures the concept of notability, or general relevance.Footnote ¹⁸ That is, important (well-known) cases exhibit more citations. Further, complex, fractious precedents are highly cited in Supreme Court opinions. These decisions are known as doctrinal paradoxes or discursive dilemmas (Reference Kornhauser and SagerKornhauser and Sager 1986; Reference PettitPettit 2001). They are the product of several distinct legal rationales, all of which are rejected by a majority, combined in separate opinions to support a judgment. Though these decisions are difficult to follow in a jurisprudential manner, precedents stemming from paradoxes are positively and significantly associated with citation counts in the Supreme Court. There is no significant association between paradoxes and citations in the circuit courts. Finally, there is a negative and significant association between paradoxes and citations in the district courts. Conversely, there is a negative and significant association between paradoxes and follows in all levels of the judicial hierarchy. Aggregating across levels, there is a negative and significant association between paradoxes and follows, and no significant association between paradoxes and citations.

Considering all citation types together and focusing on the Supreme Court exclusively appears to measure which precedents are broadly relevant nationally. Conversely, the jurisprudential significance of precedent appears to be better captured via measuring deferential citations only, aggregated across all levels of the judicial hierarchy. Analysts therefore should carefully consider whether it is general notability and relevance or jurisprudential influence they wish to measure when studying precedent, and choose the basis for their metric accordingly.