Article contents
Factual Ambiguity and a Theory of Evidence Reconsidered: A Dialogue Between a Statistician and a Law Professor
Published online by Cambridge University Press: 04 July 2014
Abstract
- Type
- Other
- Information
- Copyright
- Copyright © Cambridge University Press and The Faculty of Law, The Hebrew University of Jerusalem 1997
Footnotes
John Henry Wigmore Professor, Northwestern University School of Law.
Associate Professor of Statistics, Iowa State University. The Office of Naval Research, U.S. Department of Defense, is gratefully acknowledged for partially funding Carriquiry's work.
References
1 The paper was published as Allen, , “Factual Ambiguity and a Theory of Evidence”, (1994) 88 Nw. L. R. 604 Google Scholar.
2 See, e.g., Allen, Ronald J., “The Nature of Juridical Proof”, (1991) 13 Cardozo L. R. 373 Google Scholar.
3 Pennington, Nancy and Hastie, Reid, “A Cognitive Theory of Juror Decision Making: The Story Model”, (1991) 13 Cardozo L. R. 519 Google Scholar; Allen, supra n. 2.
4 See, e.g., Dawid, A. Philip, “The Difficulty About Conjunction”, (1987) 36 The Statistician 91 CrossRefGoogle Scholar.
5 Ellsburg, D., “Risk, Ambiguity, and The Savage Axioms”, (1961) 75 Quart. J. Econ. 644 Google Scholar.
6 Berger, J. O., Statistical Decision Theory and Bayesian Analysis (1985)CrossRefGoogle Scholar.
7 Smith, A. F. M., “Present Position and Potential Developments: Some Personal Views About Bayesian Statistics”, (1984) 147 J. Royal Statist. Soc. (A) 245 CrossRefGoogle Scholar.
8 Allen, supra n. 2; Allen, Ronald J., “A Reconceptualization of Civil Trials”, (1986) 66 B.U. L. R. 401 Google Scholar; Bergman, Paul & Moore, Al, “Mistrial by Likelihood Ratio: Bayesian Analysis Meets the F-Word”, (1991) 13 Cardozo L. R. 589 Google Scholar.
9 See, e.g., Friedman, Richard D., “Infinite Strands, Infinitesimally Thin: Storytelling, Bayesianism, Hearsay, and Other Evidence”, (1992) 14 Cardozo L. R. 79 Google Scholar (reconciling Bayesian approaches with the comparative story model). See also Robertson, Bernard and Vignaux, G. A., “Expert Evidence: Law, Practice and Probability”, (1992) 12 Oxford J.L. Stud. 392 (same)CrossRefGoogle Scholar.
10 See generally Popper, Karl R., Objective Knowledge: An Evolutionary Approach (1972)Google Scholar.
11 Fienberg, S. E., and Schervish, M. J., “The Relevance of Bayesian Inference for the Presentation and Evaluation of Statistical Evidence and for Legal Decision-Making”, (1986) 66 Boston L. R. 771 Google Scholar.
12 Allen, R. J., “Rationality, Algorithms, and Juridical Proof: A Preliminary inquiry”, in Int. J. of Evidence and Proof (in press)Google Scholar.
13 Salmon, Wesley C., “Confirmation and Relevance”, in Achinstein, Peter, ed., The Concept of Evidence (1983) 95, at 104 Google Scholar.
14 This reads the probability of -j given h.
15 The probability of -j given i.
16 The probability of -j, given the conjunction of h and i.
17 Salmon, supra n. 13, at 104-105.
18 Cohen, L. J., The Probable and the Provable, (1977)CrossRefGoogle Scholar.
19 See, e.g., Lindley, D. V., Making Decisions, (1971)Google Scholar; Bernardo, J. M., and Smith, A. F. M., Bayesian Theory, (1995)Google Scholar.
20 Cohen, supra n. 18.
21 Dawid, supra n. 4.
22 The legal adequacy of the story is tested by the judge in all cases through the decision to send the case to the jury, except guilt of a criminal charge where the judge theoretically lacks the power to direct a verdict.
23 See Part III, infra. Obviously a fact finder could accept some theory of what happened other then one of the parties'. The process, though, will be one of acceptance of some theory of the case in light of the alternatives advanced by the parties, which is the crucial insight of the relative plausibility theory.
24 Bearing in mind that the judge will act as a legal reality check on the legal adequacy of those stories, save only in criminal cases where the myth of no directed verdicts against the defendant still generally holds sway. But see United States v. Bailey, 444 U.S. 394 (1980); Allen, Ronald J., “Structuring Jury Decision Making in Criminal Cases: A Unified Constitutional Approach to Evidentiary Devices”, (1980) 94 Harv. L. R. 321, 358–60CrossRefGoogle Scholar.
25 See, e.g., generally, Allen, Ronald J., “On the Significance of Batting Averages and Strikeout Totals: A Clarification of the ‘Naked Statistical Evidence’ Debate, the Meaning of ‘Evidence’, and the Requirement of Proof Beyond a Reasonable Doubt”, (1991) 65 Tul. L. R. 1093 Google Scholar; Callen, Craig R., “Cognitive Science and the Sufficiency of ‘Sufficiency of the Evidence’ Tests”, (1991) 65 Tul. L. R. 1113 Google Scholar. See also Callen, Craig R., “Adjudication and the Appearance of Statistical Evidence”, (1991) 65 Tul. L. R. 457 Google Scholar; Shaviro, Daniel, “A Response to Professor Callen”, (1991) 65 Tul. L. R. 498 Google Scholar; Shaviro, Daniel, “A Response to Professor Allen”, (1991) 65 Tul. L. R. 1111 Google Scholar. I elaborate this point below in Part III, pp. 480-485, infra.
26 For example, the fact finder may conclude that the relevant event happened differently than either party asserts, which is perfectly acceptable. The crucial point is that this may benefit any party at trial; it would not induce any systematic biases.
27 Hempel, Carl G., “Aspects of Scientific Explanation”, (1945) 54 Mind 1 CrossRefGoogle Scholar, reprinted in Achinstein, Peter, ed., The Concept of Evidence (1983) 1, at 12 Google Scholar (“… a precise definition of relevance presupposes an analysis of confirmation and disconfirmation.”).
28 At least so far as first order predicate logic is concerned. See, e.g., Copi, Irving M., Introduction to Logic (6th ed., 1982)Google Scholar; Allen, Ronald J., Ferrall, Bard, & Ratnaswamy, John, “The Double Jeopardy Clause, Constitutional Interpretation and the Limits of Formal Logic”, (1991) 26 Val. U. L. R. 281, at 296–7Google Scholar.
29 United States v. Shonubi, (1995) (E. D. New York), reversed on appeal F.3d. (1997).
30 There are other cases, such as newspaper accounts, but all of them, so far as I can see, would be subject to the textual analysis.
31 I now begin an inductive argument that builds to its conclusion at discussion accompanying n. 67, where I articulate the theory of juridical evidence. Those who can't wait may wish to peek ahead.
Inferences about legal rules also cannot be made in a deductive fashion in many, probably all, instances, but that, again, does not mean the inferences drawn are irrational or arbitrary, or that the process is lawless. See Allen, Ferrall, & Ratnaswamy, supra n. 28, at 290-92; Allen, Ronald J., “Evidence, Inference, Rules, and Judgment in Constitutional Adjudication: The Intriguing Case of Walton v. Arizona ”, (1991) 81 J. Crim. L. & Criminology 727, at 734–45CrossRefGoogle Scholar.
32 Reichenbach, H. The Rise of Scientific Philosophy (1958) 37 Google Scholar.
33 Jeffreys, H. Theory of Probability (3rd ed., 1961) 2–3 Google Scholar.
34 Savage, L. J., Subjective Probability and Statistical Practice, in: Savage, L. J. et al. , The Foundations of Statistical Inference (1962)Google Scholar; Jeffrey, supra n. 33.
35 This is an easily verifiable point, of course. The reader can consult his or her own knowledge.
36 See generally the articles by Allen, Shaviro, and Callen, supra n. 25.
37 Allen, R. J., “Burdens of Proof, Uncertainty, and Ambiguity in Modern Legal Discourse”, (1994) 17 Harv. J.L. & Pub. Pol. 627, at 642 Google Scholar.
38 See Harmon, Gilbert, Change in View (1986) 25–27 Google Scholar.
39 Carriquiry, A. L., “Bayesian Legal Decision-Making Impossible Task?”, in Int. J. of Evidence and Proof, (in press)Google Scholar.
40 Clark Glymour, “Relevant Evidence”, in The Concept of Evidence, supra n. 13, at 124, 127-28.
41 And perhaps to Cohen's inductive theory, as well. See Cohen, supra n. 18. The requirement of too much knowledge in his theory resides in needing to know what counts as a test of an hypothesis, what order the possible tests must be run, and how many such tests can be run. See Allen, supra n. 2, at 376, 379.
42 Hempel, supra n. 27, at 12.
43 That the search was doomed from the start in retrospect seems self-evident. In responding to the possibility that there are no objective criteria of confirmation and that maybe all there can be is a “sense of the evidence”, Hempel went on to say: “Clearly the occurrence or non-occurrence of a feeling of conviction upon the presentation of grounds for an assertion is a subjective matter which varies from person to person, and with the same person in the course of time; it is often deceptive and can certainly serve neither as a necessary nor as a sufficient condition for the soundness of the given assertion. A rational reconstruction of the standards of scientific validation cannot, therefore, involve reference to a sense of evidence; it has to be based on objective criteria… [I]t ought to be possible, one feels, to set up purely formal criteria of confirmation …” Ibid., at 17 (footnote omitted) (emphasis added).
This is highly ironic. In a passage discounting “a sense of the evidence,” he ends up appealing to just that sense. Human judgment just does not seem to be extricable from fundamental questions of evidence.
44 Although evidence i confirms hypothesis h and k [the conjunction of A and k], we cannot infer that i confirms h.k. Evidence i may in fact confirm h.k, but to draw that conclusion from the given premises would be another instance of the fallacy of incomplete schematization. Indeed, our investigations have revealed exactly what is missing in the inference. In addition to knowing that i is positively relevant to h and positively relevant to k, we must know what bearing it has on the relevance of h to k. If this is known quantitatively, and if the degrees of relevance of i to h and to k are also known quantitatively, we can ascertain the relevance of i to h.k and to h v k [the disjunction]. Without this quantitative knowledge, we cannot say much of anything. Wesley C. Salmon, supra n. 13, at 122.
45 Compare, e.g., Braithwaite, Richard Bevan, The Structure of a Scientific System in Scientific Explanation (1953)Google Scholar with Hanson, Norwood Russell, “The Logic of Discovery”, (1958) 55 J. Phil. 1073, at 1079 CrossRefGoogle Scholar.
46 Carnap's confirmation functions are another example. See Carnap, Rudolf, Logical Foundations of Probability (2nd ed., 1962)Google Scholar. Compare Salmon, supra n. 13.
47 Salmon, ibid., at 110.
48 See, in particular, Achinstein, Peter, The Nature of Explanation (1983)Google Scholar.
49 Achinstein, Peter, “Concepts of Evidence”, (1978) 87 Mind 22, at 35 Google Scholar.
50 For criticisms of Achinstein, see Bar-Hillel, Maya & Margalit, Avishai, “In Defense of the Classical Notion of Evidence”, (1979) 88 Mind 576 CrossRefGoogle Scholar; Phillips, Hollibert E., “On Appealing to the Evidence”, (1991) 22 Phil. F. 228 Google Scholar; see also Achinstein, Peter, “On Evidence: A Reply to Bar-Hillel and Margalit”, (1981) 90 Mind 108 CrossRefGoogle Scholar.
51 For an interesting discussion of methodological differences and similarities between historiography and law, see Rescher, Nicholas & Joynt, Carey B., “Evidence in History and in the Law”, (1959) 56 J. Phil. 561 CrossRefGoogle Scholar.
52 Hempel, Carl G., “The Function of General Law in History”, (1942) 39 J. Phil. 35 CrossRefGoogle Scholar.
53 Ibid., at 39.
54 Ibid., at 45. Historical work often does not appear on its face to be consistent with Hempel's analysis. This is so, he argues, because:
“First, the universal hypotheses in question frequently relate to individual or social psychology, which somehow is supposed to be familiar to everybody through his everyday experience; thus, they are tacitly taken for granted … Second, it would often be very difficult to formulate the underlying assumptions explicitly with sufficient precision, and at the same time in such a way that they are in agreement with all the relevant empirical evidence available”. Ibid., at 40. Consequently, historians often produce explanation sketches:
“What the explanatory analyses of historical events offer is, then, in most cases not an explanation in one of the meanings developed above, but something that might be called an explanation sketch. Such a sketch consists of a more or less vague indication of the laws and initial conditions considered as relevant, and it needs “filling out” in order to turn into a full-fledged explanation. This filling-out requires further empirical research, for which the sketch suggests the direction. (Explanation sketches are common also outside of history; many explanations in psychoanalysis, for instance, illustrate the point.)” Ibid., at 42.
55 In a technical sense, it seems quite clear that theories such as Hempel's confirmation theory do not apply directly to historical or legal questions. As Hempel observes, “a precise definition of confirmation requires reference to some definite ‘language of science,’ in which all observation reports and all hypotheses under consideration are assumed to be formulated, and whose logical structure is supposed to be precisely determined. The more complex this language, and the richer its logical means of expression, the more difficult it will be, as a rule, to establish an adequate definition of confirmation for it.” Hempel, supra n. 52, at 39. English is a richer, more complex language than those Hempel worked on, which means the ensuing problems are more intractable.
56 See, e.g., Krieger, Leonard, Time's Reasons: Philosophies of History Old and New (1989)CrossRefGoogle Scholar; Krausz, Michael, “History and Its Objects”, (1991) 74 Monist 217 CrossRefGoogle Scholar; Levine, Michael P., “Historical Anti-Realism: Boethian Historians Tell Their Story”, (1991) 74 Monist 230 CrossRefGoogle Scholar.
57 “Scientists tend to seek quantitative hypotheses and general laws.” Achinstein, Peter, The Nature of Explanation (1983) 142 Google Scholar.
58 Historians have not simply shown the past “wie es eigentlich gewesen,” as Ranke insisted they should, nor have they shown clearly “the events which happened in the past,” as Thucydides claimed to do. As Leonard Krieger has demonstrated, historians have almost always smuggled themselves into their accounts, either explicitly by presenting philosophies of history or implicitly by providing coherence through narratives calculated to provide coherence. Kloppenberg, James T., “Objectivity and Historicism: A Century of American Historical Writing”, (1989) 94 Am. Hist. R. 1011, at 1029 Google Scholar (reviewing Novick, Peter, That Noble Dream: The “Objectivity Question” and the American Historical Profession (1988)CrossRefGoogle Scholar).
59 Employment discrimination is the best counter example that I can think of, but disparate treatment is highly atypical of litigation generally.
60 “Science” to me means empirical or hard science, and by data, I mean data like “the meter read 7.3” and not something on the order of “we observed that the photon was annihilated”.
61 History is a halfway house, it seems to me, where there is a concern for universals and data is often problematic.
62 Kuhn, Thomas S., The Structure of Scientific Revolutions (1962)Google Scholar is often cited for such a proposition, and a cite to Kuhn has become standard in the legal literature. Typically, though, the legal literature in this regard is a few decades behind developments in other fields. Kuhn's views were immediately subject to trenchant and powerful criticism, the best in my view being Stephen Toulmin, , Human Understanding (1972) 1–130 Google Scholar, that essentially forced another revolution, this time in Kuhn's views. By 1977, Frederick Suppe would remark that “Kuhn's views have undergone a sharply declining influence on contemporary philosophy of science”. Suppe, Frederick, The Structure of Scientific Theories (1977) 647 Google Scholar.
63 There may be special cases, but I am dealing with typical ones.
64 For example, if the left eye twitches when the witness is slouching fifteen degrees to the vertical and sweat can be seen around the collar, he's lying.
65 Nussbaum, Martha C. and Putnam, Hilary, “Changing Aristotle's Mind”, in Nussbaum, Martha C. & Rorty, Amelie Oksenberg, eds., Essays on Aristotle's De Anima (1992) 27, at 48–49 Google Scholar (focusing on Hilary Putnam's views); see also Putnam, Hilary, Representation and Reality, (1988) 73–105 Google Scholar.
66 Fed. R. Evid. 401, 403. Cohen, , “Freedom of Proof”, in Twining, William, ed., Facts in Law (1983)Google Scholar. Some argue that the rules of evidence do not appear to affect admissibility because they filter out evidence before it is adduced. My experience is to the contrary. Very little is kept out at trial, and most of what is kept out is excluded for efficiency, not reliability or prejudice reasons. The preconceptions of lawyers and parties clearly do filter out large amounts of information precisely because of its irrelevancy.
67 The only exceptions to this that I can think of are motions to disregard and limiting instructions, which consistent with the theory, no one actually thinks have any substantial effect at trial.
68 This description of conventional fact finding can profitably be compared to Stephen Toulmin's description of the progress of science in the first hundred pages of Human Understanding. Toulmin, supra n. 62, at 1-100.
69 Rescher, Nicholas, Rationality (1988) 16–18 Google Scholar. Empirical work on jury decision making largely confirms the obvious proposition that otherwise rational people do not become irrational by being placed on juries. The best of this work is by Nancy Pennington and Reid Hastie, whose story model of jury decision making maps onto Rescher's philosophical speculations quite well. Pennington and Hastie's story model predicts that juror factual findings are determined by the most acceptable of the stories advanced by the parties. Acceptability, in turn, is determined by coverage, coherence, consistency, plausibility, completeness, and uniqueness. Pennington and Hastie, supra n. 3, at 527-28.
70 Jeffreys, supra n. 33; Lindley, supra n. 19.
71 This seems to be true virtually no matter what the intellectual task, perhaps excepting only certain forms of mathematics. See, e.g., Schank, Roger C., Explanation Patterns (1986)Google Scholar; Chi, Michelene T.H. et al. , eds., The Nature of Expertise (1988)Google Scholar.
72 Positive theories quickly become normative when used to criticize deviations from them. My interest is solely in explanation, though, and not criticism.
73 Allen, supra n. 2, at 413-420.
74 “During” — not “at” — because some relevant observations will occur outside of the formal trial setting, during deliberations, for example, or while contemplating the day's events in the evening.
75 Fed. R. Evid. 401, 403.
76 They primarily regulate admission for reasons other than probativeness.
77 Fed. R. Evid. 609, for example. Hearsay, by contrast, is primarily a rule of admission more consistent with the theory of free proof implicit in my theory of juridical evidence than it is with the conventional view. See Allen, Ronald J., “The Evolution of the Hearsay Rule to a Rule of Admission”, (1992) 76 Minn. L. R. 797 Google Scholar.
78 In fact, Wigmore promoted a complex set of evidentiary rules quite consistent with the prediction of the conventional theory of evidence, and it was roundly rejected. Ariens, Michael, “Progress Is Our Only Product: Legal Reform and the Codification of Evidence”, (1992) 17 L. & Soc. Inquiry 213, at 226–237 Google Scholar.
79 The political justifications for juries are implausible today, in my view, except in their jury nullification form in criminal cases, thus leaving the persistence of juries justifiable only in terms of fact finding.
80 Preclusive motions serve efficiency interests, but that is an inadequate justification for them. They obviously are used for other purposes.
81 Nesson's theory of acceptability has it essentially backwards, in other words, in its attempt to force the rules of evidence into the defense of acceptability. See Nesson, Charles, “The Evidence or the Event? On Judicial Proof and the Acceptability of Verdicts”, (1985) 98 Harv. L. R. 1357 CrossRefGoogle Scholar. The rules of evidence promote accuracy by imposing in essence a regime of free proof. Judges, by contrast, serve the interests of acceptability. Judges in addition may serve the traditional concerns about implementing the policies of exclusionary rules. My focus here is on relevancy, however.
82 The theory of evidence also refines the argument over a right not to be tried by a jury. The justification is not that jurors are incompetent, but that it would be too costly to provide them the necessary background information to decide the case. It is no different from not asking jurors to understand a foreign language. We do not assert jurors are incompetent because they do not speak French; we instead provide them translations. Similarly, jurors are not incompetent if they do not understand micro-biology. The relevant question is the cost of the translation.
83 All of which happens to confirm the validity of the theory of evidence.
84 Recent work by Heuer, Larry and Penrod, Steven, “Increasing Jurors Participation in Trials: A Field Experiment with Juror Notetaking and Question Asking”, (1988) 12 Law and Human Behavior 231 CrossRefGoogle Scholar, confirms some of these predictions. The study supports that juror questions aid decision making and disconfirms various objections to the technique. In a related study, Larry Heuer & Steven Penrod, “Trial Complexity: A Field Investigation of its Meaning and its Effects”, (manuscript on file with Nw. L. R.), the authors report that juror questions “have the consistent effect of either reducing the harms of [evidentiary and legal] complexity variables or enhancing their benefits”.
85 Which is why I say “in general”. Those with bad cases often do not want good fact finding procedures.
- 4
- Cited by