The rapidly increasing role of automation throughout the economy, culture and our personal lives has generated a large literature on the risks of algorithmic decision-making, particularly in high-stakes legal settings. Algorithmic tools are charged with bias, shrouded in secrecy, and frequently difficult to interpret. However, these criticisms have tended to focus on particular implementations, specific predictive techniques, and the idiosyncrasies of the American legal-regulatory regime. They do not address the more fundamental unease about the prospect that we might one day replace judges with algorithms, no matter how fair, transparent, and intelligible they become. The aim of this paper is to propose an account of the source of that unease, and to evaluate its plausibility. I trace foundational unease with algorithmic decision-making in the law to the powerful intuition that there is a basic moral and legal difference between showing that something is true of many people just like you and showing that it is true of you. Human judgment attends to the exception; automation insists on blindly applying the rule. I show how this intuitive thought is connected to both epistemological arguments about the value of statistical evidence, as well as to court-centered conceptions of the rule of law. Unease with algorithmic decision-making in the law thus draws on an intuitive principle that underpins a disparate range of views in legal philosophy. This suggests the principle is deeply ingrained. Nonetheless, I argue that the powerful intuition is not as decisive as it may seem, and indeed runs into significant epistemological and normative challenges. At an epistemological level, I show how concerns about statistical evidence's ability to track the truth can be resolved by adopting a probabilistic, rather than modal, conception of truth-tracking. At a normative level, commitment to highly individualized decision-making co-exists with equally ingrained and competing principles, such as consistent application of law. This suggests that the “rule of law” may not identify a discrete set of institutional arrangements, as proponents of a court-centric conception would have it, but rather a more loosely defined set of values that could potentially be operationalized in multiple ways, including through some level of algorithmic adjudication. Although the prospect of replacing judges with algorithms is indeed unsettling, it does not necessarily entail unreasonable verdicts or an attack on the rule of law.

Philosophical debates over statistical evidence have long been framed and dominated by L. Jonathan Cohen's Paradox of the Gatecrasher and a related hypothetical example commonly called Prison Yard. These examples, however, raise an issue not discussed in the large and growing literature on statistical evidence – the question of what statistical evidence is supposed to be evidence of. In actual practice, the legal system does not start with a defendant and then attempt to determine if that defendant has committed some unspecified or under-specified act, as these examples appear to suppose. Rather, both criminal and civil litigation start with a sufficiently specified act and then attempt to determine if the defendant has committed it. And when we start with a more fully specified act, the statistics look very different, and these prominent examples no longer present the paradox they are claimed to support. Examining the issue of specification, however, does more than simply undercut the prominent examples in a long and extensive literature. The examination also raises normative issues challenging the legal system's traditional reluctance to base liability on the conjunction of probabilities.

Law-enforcement agencies are increasingly able to leverage crime statistics to make risk predictions for particular individuals, employing a form of inference that some condemn as violating the right to be “treated as an individual.” I suggest that the right encodes agents’ entitlement to a fair distribution of the burdens and benefits of the rule of law. Rather than precluding statistical prediction, it requires that citizens be able to anticipate which variables will be used as predictors and act intentionally to avoid them. Furthermore, it condemns reliance on various indexes of distributive injustice, or unchosen properties, as evidence of law-breaking.

The paper explores the use of statistical data and statistical assumptions as evidence in criminal trials. It is suggested that a finding of guilt includes not only its main factual proposition but also additional propositions that support and affirm it. Specifically, it includes not only the proposition that the defendant committed the offence but also the additional affirming proposition that it is this defendant rather than any other potential defendant who committed the offence (the ‘D rather than A’ proposition). Some statistical generalisations provide reasons in defence of the main proposition but not in defence of the affirming proposition, which then remains random or arbitrary. Yet since criminal convictions include a moral judgement, they cannot be justified where some of their propositions are arbitrary. Accordingly, such statistical generalisations cannot justify a criminal conviction.

This paper proposes new grounds for the legal ambivalence about ‘bad character evidence’. It is suggested that errors based on such evidence are profoundly tragic in the Aristotelian sense: the defendant who previously committed crime is likely to reoffend; nevertheless, she beats the odds and refrains from further crime commission – only to then be falsely convicted based on the very odds she has almost heroically managed to beat. It is further proposed that the tragic nature of such false convictions might make them particularly unfair to the defendant. It is, however, submitted that the likelihood of errors based on such evidence is unknown and probably also unknowable. Accordingly, the maximin rule for decision in conditions of deep ignorance is applied, leading to the conclusion that exclusion is to be preferred.