NOTE: THE USE OF MATHEMATICAL AND PROBABILISTIC PROOF IN THE TRIAL PROCESS
The Collins case, Cohen's gate-crasher hypothetical, and Problems 20 to 22 raise the issue of what effect, if any, mathematical or probabilistic proof should be given. In application, this issue must be broken down into two main questions. The first question is whether mathematical proof should be admitted at all. If so, there are subsidiary questions: What should the factfinder be told about it? In what form? One might ask this same question from a quite different perspective: Is it possible to utilize any proof that is not probabilistic? The second main question is: To what extent may probabilistic and mathematical models be used as standards on which to base a criminal conviction or civil judgment? This question deals with the sufficiency of proof to sustain a verdict.
In this chapter we are concerned primarily with the first questionthe admissibility or other trial use of a mathematical model or proof without regard for its sufficiency to sustain a verdict, but the sufficiency question is intimately related.
In a powerful article, Trial by Mathematics: Precision and Ritual in the Legal Process, 84 Harv. L. Rev. 1329 (1971), Professor Laurence Tribe argues that the usefulness of mathematical methods in the trial process is greatly exaggerated. In some cases, Tribe writes, such methods clash with other important values of the legal process.
Tribe divides mathematical proof situations into three categories: "(1) those in which such proof is directed to the occurrence or nonoccurrence of the event, act, or type of conduct on which the litigation is premised; (2) those in which such proof is directed to the identity of the individual responsible for a certain act or set of acts; and (3) those in which such proof is directed to intention or to some other mental element of responsibility, such as knowledge or provocation.'' Id. at 1339. Tribe argues that the "significance, appropriateness and dangers of mathematical proof may depend dramatically on whether such proof is meant to bear upon occurrence, identity, or frame of mind.'' Do you agree? Is this division useful? Remember that we are talking about the admissibility of evidence, not its sufficiency to sustain a criminal conviction or civil judgment.
To illustrate this division, Tribe presents several hypothetical uses of mathematical proof (id. at 1339-1343):
1. Occurrence. Consider first the cases in which the existence of the legally significant occurrence or act is itself in question. A barrel falls from the defendant's window onto the plaintiff's head. The question is whether some a negligent act or omission was the cause. Should such proof be allowed and, if so, to what effect?(1)
A man is found in possession of heroin. The question is whether he is guilty of concealing an illegally imported narcotic drug. Evidence exists to support the finding that ninety-eight percent of all heroin in the United States is illegally imported. What role, if any, may that fact play at the defendant's trial?
A man is charged with overtime parking in a one-hour zone. The question is whether his car had remained in the parking space beyond the time limit. To prove that it had not been moved, the government calls an officer to testify that he recorded the positions of the tire air-valves on one side of the car. Both before and after a period in excess of one hour, the front-wheel valve was pointing at one o'clock; the rear-wheel valve, at eight o'clock. The driver's defense is that he had driven away during the period in question and just happened to return to the same parking place with his tires in approximately the same position. The probability of such a fortunate accident is somewhere between one in twelve and one in one hundred forty-four.(2)
Should proof of that fact be allowed and, if so, to what end?(3)
2. Identity. Consider next the cases in which the identity of the responsible agent is in doubt. Plaintiff is negligently run down by a blue bus. The question is whether the bus belonged to the defendant. Plaintiff is prepared to prove that defendant operates four-fifths of all the blue buses in town. What effect, if any, should such proof be given?
A policeman is seen assaulting someone at an undetermined time between 7 p.m. and midnight. The question is whether the defendant, whose beat includes the place of the assault, was the particular policeman who committed the crime. It can be shown that the defendant's beat brings him to the place of the assault four times during the relevant five-hour period each night, and that other policemen are there only once during the same period. In what way, if at all, may this evidence be used?(4)
A man is found shot to death in the apartment occupied by his mistress. The question is whether she shot him. Evidence is available to the effect that, in ninety-five percent of all known cases in which a man was killed in his mistress' apartment, the mistress was the killer. How, if at all, may such evidence be used?(5)
A civil rights worker is beaten savagely by a completely bald man with a wooden left leg, wearing a black patch over his right eye and bearing a six-inch scar under his left, who flees from the scene of the crime in a chartreuse Thunderbird with two dented fenders. A man having these six characteristics is charged with criminal battery. The question is whether the defendant is in fact the assailant. Evidence is available to show that less than one person in twenty has any of these six characteristics, and that the six are statistically independent, so that less than one person in sixty-four million shares all six of them. In what ways, if at all, may that calculation be employed?(6)
3. Intention. Consider finally the cases in which the issue is one of intent, knowledge, or some other "mental'' element of responsibility. A recently insured building burns down. The insured admits causing the fire but insists that it was an accident. On the question of intent to commit arson, what use, if any, may be made of evidence tending to show that less than one such fire out of twenty is in fact accidentally caused?
As in an earlier example, a man is found possessing heroin. This time the heroin is stipulated at trial to have been illegally imported. In his prosecution for concealing the heroin with knowledge that it had been illegally imported, what effect may be given to proof that ninety-eight percent of all heroin in the United States is in fact illegally imported?
A doctor sued for malpractice is accused of having dispensed a drug without adequate warning, knowing of its tendency to cause blindness in pregnant women. Should he be allowed to introduce evidence that ninety-eight percent of all doctors are unaware of the side-effect in question?
Tribe examines and rejects three commonly advanced objections against the use of mathematical proof in such situations: (1) that the use of such techniques is inappropriate for the determination of past events as opposed to the prediction of possible future events; (2) that it is not possible to transform mathematical information from evidence about the generality of cases to evidence about the particular case before the court; and (3) that in very few cases can the mathematical evidence by itself establish the proposition to which it is directed with sufficient strength for it to prevail. Id. at 1344-1350.
Nonetheless, Tribe cautions against the use of mathematical and probabilistic proof at trials because it "would be very likely to yield wholly inaccurate and misleadingly precise conclusions.'' One way in which this happens, Tribe argues, is that the probability proof overwhelms and distorts all the other evidence in the case (id. at 1360-1361):
The problem of the overpowering number, that one hard piece of information, is that it may dwarf all efforts to put it into perspective with more impressionistic sorts of evidence. This problem of acceptably combining the mathematical with the non-mathematical evidence is not touched in these cases by the [probabilistic] approach.
In situations of the sort being examined here, however, when the thrust of the mathematical evidence is to shed light on the probability assessment with which the trier ought rationally to begin, there is at least one way to take the evidence into account at trial without incurring the risk that the jury will give it too much weight when undertaking to combine the mathematical datum with fuzzier information. Let the judge rather than the jury weigh the probabilistic proof in order to determine whether it might not be both equitable and conducive to an accurate outcome to shift to the other side the burden of producing some believable evidence to take the case outside the general rule seemingly established by the probabilities.(7) If one is to avoid a distortion in results, however, any such proposal must be qualified, at least when the question is one of the defendant's identity, by the principle that a party is not entitled to a jury verdict on statistical evidence alone absent some plausible explanation for his failure to adduce proff of a more individualized character.(8)
Tribe also argues that mathematical proof cannot be accurately integrated with nonmathematical evidence because of the phenomenon of the "dwarfing of the soft variables'' (id. at 1361-1362, 1366):
The syndrome is a familiar one: If you can't count it, it doesn't exist. Equipped with a mathematically powerful intellectual machine, even the most sophisticated user is subject to an overwhelming temptation to feed his pet the food it can most comfortably digest. Readily quantifiable factors are easier to processand hence more likely to be recognized and then reflected in the outcomethan are factors that resist ready quantification. The result, despite what turns out to be a spurious appearance of accuracy and completeness, is likely to be significantly warped and hence highly suspect....
One consequence of mathematical proof, then, may be to shift the focus away from such elements as volition, knowledge, and intent, and toward such elements as identity and occurrencefor the same reason that the hard variables tend to swamp the soft. It is by no means clear that such marginal gains, if any, as we may make by finding somewhat more precise answers would not be offset by a tendency to emphasize the wrong questions.
Independent of the above criticisms of mathematical models in the trial process, Tribe argues that the "great virtue of mathematical rigorits demand for precision, completeness, and candormay become its greatest vice, for it may force jurors to articulate propositions whose truth virtually all might already suspect, but whose explicit and repeated expression may interfere with what seems to me the complex symbolic functions of trial procedure and its associated rhetoric.'' Id. at 1371. Tribe has in mind two problems with the use of a mathematical value in criminal cases: the factual presumption of guilt at the start of a trial, arising solely because of the accused's status as the defendant; and the quantification of the probability of guilt at the conclusion of the trial. Use of these mathematical data and conclusions, Tribe argues, conflicts with values served by the presumption of innocence and the reasonable doubt standard. Id. at 1370-1375. But, again, does this argument run against the use of mathematical methods at trial (i.e., admissibility), or does it have more to do with the sufficiency issue?
Finally, Tribe contends that the use of mathematics in the trial process leads to counterintuitive results and thus may "make the legal system seem even more alien and inhuman than it already does to distressingly many.'' Id. at 1376. He continues:
There is at stake not only the further weakening of the confidence of the parties and of their willingness to abide by the result, but also the further erosion of the public's sense that the law's fact-finding apparatus is functioning in a somewhat comprehensible way, on the basis of evidence that speaks, at least in general terms, to the larger community that the processes of adjudication must ultimately serve. The need now is to enhance community comprehension of the trial process, not to exacerbate an already serious problem by shrouding the process in mathematical obscurity.
It would be a terrible mistake to forget that a typical lawsuit, whether civil or criminal, is only in part an objective search for historical truth. It is also, and no less importantly, a ritual--a complex pattern of gestures comprising what Henry Hart and John McNaughton once called "society's last line of defense in the indispensable effort to secure the peaceful settlement of social conflicts."(9)
One element, at least, of that ritual of conflict-settlement is the presence and functioning of the jury--a cumbersome and imperfect institution, to be sure, but an institution well calculated, at least potentially, to mediate between "the law'' in the abstract and the human needs of those affected by it. Guided and perhaps intimidated by the seeming inexorability of numbers, induced by the persuasive force of formulas and the precision of decimal points to perceive themselves as performing a largely mechanical and automatic role, few jurors--whether in criminal cases or in civil--could be relied upon to recall, let alone to perform, this humanizing function, to employ their intuition and their sense of community values to shape their ultimate conclusions.
When one remembers these things, one must acknowledge that there was a wisdom of sorts even in trial by battle--for at least that mode of ascertaining truth and resolving conflict reflected well the deeply-felt beliefs of the times and places in which it was practiced. This is something that can hardly be said of trial by mathematics today.
Michael Saks and Robert Kidd, in Human Information Processing and Adjudication: Trial by Heuristics, 15 Law & Soc. Rev. 123, 125 (1980-1981), contend, contrary to Tribe, that "while certain errors and harm may be inherent even in the proper use of probabilistic tools, even more harm may be inherent in not using them.'' In making this point, Saks and Kidd first apply the research findings of behavioral decision theorists to challenge Tribe's assumptions from an empirical point of view. They conclude that explicit calculation of probabilities will, in most cases, lead a trier of fact closer to the correct conclusion than will reliance on intuitive, commonsense judgments. Id. at 125. This is because most lay decision-makers employ a number of simplifying strategies, known as "heuristics,: to reduce comples information to a point where they can made a decision. Saks and Kidd offer the following examples and analysis of this phenomenon (id. at 127-130):
1. After observing three consecutive red wins, a group of people playing roulette start to switch their bets to black. After red wins on the fourth and fifth spins, more and more players switch to black, and they are increasingly surprised when the roulette wheel produces a red win the sixth, and then the seventh time. In actuality, on each spin the odds of a red win remain constant at 1:1. The shifting of bets to black was irrational, as was the strong subjective sense that after each successive red win, black became more likely.
2. The following description is of a man selected at random from a group composed of 70 lawyers and 30 engineers. "John is a 39-year-old man. He is married and has two children. He is active in local politics. The hobby that he most enjoys is rare book collecting. He is competitive, argumentative, and articulate.'' A large group of respondents was asked to estimate the probability that John is a lawyer rather than an engineer. Their median probability estimate was .95. Another group of respondents was asked the same question, except that they were first told that the group from which John was selected consisted of 30 lawyers and 70 engineers. The second group's median estimate of the likelihood that John is a lawyer was also .95. Information about the composition of the group from which John was selected logically should have affected the estimated probability, but it had no effect at all on the decision makers' judgment. (This problem is taken from Kahneman and Tversky, 1973.) Only at the extremes of the distributions, where the group approaches 100 lawyers and 0 engineers (or the converse) do the decision makers become sensitive to the information about group composition.
3. A cab was involved in a hit-and-run accident at night. Two cab companies, the green and the blue, operate in the city. A witness reports that the offending cab was blue, and legal action is brought against the blue cab company. The court learns that 85 percent of the city's cabs are green and 15 percent are blue. Further, the court learns that on a test of ability to identify cabs under appropriate visibility conditions, the witness is correct on 80 percent of the identifications and incorrect on 20 percent. Several hundred persons have been given this problem and asked to estimate the probability that the responsible cab was in fact a blue cab. Their typical probability response was .80. In actuality, the evidence given leads to a probability of .41 that the responsible cab was blue. (This problem is taken from Tversky and Kahneman, 1980.)
The first example illustrates the simplest and best known of errors in human probability judgment, the "Gambler's Fallacy.'' In a sequence of independent events, outcomes of prior events do not affect the probability of later events. Each event is independent of the other. On the seventh spin, the roulette wheel neither remembers nor cares what it did on the preceding six spins. People know that in the long run, half the wins will be red and half black. They err in believing that a small local sequence of events will be representative of the infinite sequence. "Chance is commonly viewed as a self-correcting process in which a deviation in one direction induces a deviation in the opposite direction to restore the equilibrium. In fact, deviations are not 'corrected' as a chance process unfolds, they are merely diluted'' (Tversky and Kahneman, 1974). Although intuition in this context is out of harmony with reality, we all feel it compellingly, and continue to hear that baseball players who have not had a hit in some time are "due'' for one, and that lightning will not strike twice in the same place. These common-sense judgments are, nevertheless, dead wrong.
The second example illustrates how human decision making tends to be insensitive to base rates when case-specific information is available. Given only the group base rates30 lawyers: 70 engineerspeople rely heavily on this information to make their judgments. They correctly say the probability is .30 that the person selected is a lawyer. When descriptive case-specific information is added, they tend to ignore the numerical base rate and rely instead on the degree to which the description of John is representative of their stereotype of lawyers. Subjects base their estimate of the probability that John is a lawyer on the degree of correspondence between his description and their stereotype of lawyers as argumentative, competitive, and politically aware. Given the base-rate data in this example, it is 5.44 times as likely that John is a lawyer when the group is composed of 70 lawyers and 30 engineers than when the opposite membership distribution holds.
The third example also demonstrates insensitivity to base-rate information, this time in a context where both the base-rate and the case-specific information, this time in a context where both the base-rate and the case-specific information are given numerically. The actual low probability that the cab is blue is due to the fact that the base rate for blue cabs is very low, and the witness is of dubious acuity. Indeed, the base rate is more extreme than the witness is credible. But, fact finders apparently are unable simultaneously to relate the color of the hit-and-run cab to two different concerns, namely, the sampling of cabs from the city's cab population and imperfect color identification by the witness. They ignore the base-rate information and treat the accuracy of the witness as equal to the probability of a correct identification.
These illustrations demonstrate the gap between the judgments people make intuitively and the probabilities yielded by explicit calculation (or by empirical observation of actual outcomes).
Further, Saks and Kidd argue that, contrary to Tribe's assumption, the empirical evidence is that factfinders undervalue quantitative evidence in the face of particularistic or anecdotal information, rather than finding it overpersuasive. The implications of this, they contend, are that statistical data are not so overwhelming as to be prejudicial and that the true problem is getting factfinders to incorporate statistical evidence into their decisions at all. Id. at 149.
Saks and Kidd acknowledge the symbolic values of the trial process. However, "[t]o accept the dilemma posed by Tribe and adopt his preference for intuition is to choose a comforting ritual over accurate decisions, much like a patient who would rather have a human physician make a wrong diagnosis than allow a computer to make a correct one.'' Id. at 146. Not surprisingly, Saks and Kidd refuse to accept the "dilemma'' (id. at 147-148):
Moreover, the choice is not really between computers and people. It is between explicitly presented computing and subjective computing, or between more and less accurate computing. This is not to degrade humans. It is merely to recognize, on the one hand, our information processing limitations and, on the other, our capacity to invent tools that can do the job better.(10) After all, many people trust their pocket calculators and the light meters in their cameras, whose workings they do not begin to comprehend; yet their faith is well placed, because these devices make decisions and judgments faster and more accurately than people do.(11) The comparison is not between humans and mathematics, but between humans deciding alone and humans deciding with the help of a tool.
Our suggestion is modest, and most lawyers should find it comfortingly traditional. Namely, experts ought to be permitted to offer their data, their algorithms, and their Bayesian theorems. The errors that may be introduced will be subjected to adversarial cross-examination. Various formal mathematical models do have room for errorsvariables omitted, poor measurements, and others that Tribe has cogently presented. But so do intuitive techniques. Properly employed and developed, the former can have fewer. It is up to opposing counsel to unmask the errors. Moreover, as a matter of developing and introducing new tools from what might be called decision-making technology, the identification of flaws does not imply that the tools ought not be used. The proper question is whether the tool, however imperfect, still aids the decision maker more than no tool at all.
As an example of how a statistical "tool'' can aid the decisionmaker, Saks and Kidd describe the use of "aggregate probabilities'' or base-rate data in making decisions in specific cases (id. at 149-150):
The problem usually posed is: how can information about a general state of affairs, background information, legislative facts, base rates, serve as evidence about a specific event? Several examples may help to clarify the question (drawn from Tribe, 1971):
(1) A person is found guilty of heroin possession. The next question is whether the drug was domestic or illegally imported. It can be shown that 98 percent of all illegally possessed heroin is illegally imported. May this fact be used in deciding the question in this case?
(2) A physician sued for malpractice is accused of having dispensed a drug without warning of what he knew to be its tendency to cause blindness in pregnant women. Should he be allowed to introduce evidence that 95 percent of all physicians are unaware of that side effect (as evidence that he did not know)?
(3) A plaintiff is negligently run down by a blue bus. The question is whether the blue bus belonged to the defendant who, it can be shown, owns 85 percent of the blue buses in town. What effect may such evidence be permitted to have?
We know from the research described earlier that when a decision involves only simple base-rate data, people make (approximately) the correct probability estimate. The legal question is whether such evidence may be offered as proof. The argument for admitting it rests largely on the contribution such evidence will make to reaching a correct finding based on available information. The argument against it rests on the premise that base rates are uninformative about specific cases. "[I]t has been held not enough that mathematically the chances somewhat favor a proposition to be proved; for example, the fact that colored automobiles made in the current year outnumber black ones would not warrant a finding that an undescribed automobile of the current year is colored and not black, nor would the fact that only a minority of men die of cancer warrant a finding that a particular man did not die of cancer'' (Sargent v. Massachusetts Accident Co., 1940). "[Such cases] are entirely sensible if understood ... as insisting on the presentation of some non-statistical and 'individualized' proof of identity before compelling a party to pay damages, and even before compelling him to come forward with defensive evidence, absent an adequate explanation of the failure to present such individualized proof'' (Tribe, 1971: 1344 n.37).
The assumption in these decisions is that somehow particularistic evidence is of greater probative value, that is, is more diagnostic. The studies we have described can be seen as making some enlightening points about such a seeming distinction. If neither case-specific nor base-rate data are available, the fact finder has no real way to evaluate a witness's statement. In the absence of internal or external contradiction, they probably accept it as credible. When only case-specific information is present, the fact finder regards the probability that proposition X is true as equal to the credibility of the witness. This is a condition which exists only when the base rate is 50:50. If no base-rate data are available, and this is common, the fact finders are doing the best they can; in essence, placing an even bet.
Now consider what is gained when base-rate information is added. The value of the base-rate information is that it provides a context in which the case-specific information has meaning. Once one knows that 85 percent of the buses are blue, and that the witness is 80 percent accurate in the appropriate color identification task, then one can, with the proper tools, evaluate the probative force of the statement "I saw the bus, and it was blue.''(12) Contrary to the speculations of many commentators, the research on heuristics suggests that errors are massively in the direction of being seduced by case-specific information and failing to employ base-rate information to temper belief in a witness's credibility.
Finally, Saks and Kidd challenge Tribe's fundamental distinction between intuitive, anecdotal, or case-specific information on the one hand, and mathematical, aggregate, or base-rate data on the other (id. at 150-154):
Perhaps the most serious error is an epistemological one: the assumption that case-specific information is really qualitatively different from base-rate information. The courts, commentators, and we through most of this article have categorized them separately. And, indeed, it seems obvious that background base-rate information is about other cases while particularistic information is about this case. Whatever meaning the distinction may have, it is not one that pertains to the probability of an accurate decision on the facts. Much of the testimony that is commonly thought of as particularistic only seems so. It is far more probabilistic than we normally allow jurors (or judges) to realize. This includes eyewitness identification, fingerprints, and anything else we could name. This follows not from the nature (and fallibility) of these particular techniques, but from the nature of the logic of classifying and identifying. All identification techniques place the identified object in a class with others (Tribe, 1971: 1330 n.2). There is little, if any, pinpointed, one-person-only evidence in this world. In fairness to Tribe, he notes this non-distinction, then promptly ignores its implications by saying, "I am, of course, aware that all factual evidence is ultimately statistical, and all legal proof ultimately probabilistic, in the epistemological sense that no conclusion can ever be drawn from empirical data without some step of inductive inferenceeven if only an inference that things are usually what they are perceived to be.... My concern, however, is only with types of evidence and modes of proof that bring this 'probabilistic' element of inference to explicit attention in a quantified way. As I hope to show, much turns on whether such explicit quantification is attempted'' (Tribe, 1971: 1330 n.2). The problems of probability do not come into existence only when we become aware of them. Making them explicit does not create the problems, it only forces us to recognize them and enables us to begin dealing with them. Burying them in implicitness is no solution; revealing their existence is not the problem.
Suppose we must decide if a person on trial for possession of heroin is guilty also of possessing illegally imported heroin. And suppose we can learn either that 90 percent of all heroin in the U.S. is illegally imported or that a witness whom we judge to be 80 percent credible (e.g., knows and tells the truth 80 percent of the time) asserts that he (or she) observed the delivery and it was an illegal importation.
The usual argument, recall, is that the particularistic evidence tells us something on which we can base a decision, while the base-rate data are all but irrelevant to the case at hand. But, from the viewpoint of a disinterested fact finder, all information is indirect, distant, abstract, and imperfectly credible. The fact finders, in terms of their truth-seeking role, simply have a set of input information on which to base a judgment, and depending on the characteristics of the evidence and the way it is processed, that finding will have a greater or lesser probability of being correct. The simple fact in this example is that the fact finder can be 80 percent sure of being right or 90 percent sure. Consequently, in this instance it is the base-rate information that is more diagnostic, more probative, and more likely to lead to a correct conclusion.
Making this argument with the relatively concrete images of a case hampers our consideration of the concept. Let us try to make the point with one of those concretely abstract statistical anecdotes. Suppose you are at a state fair and approach a kind of shell game. You are presented with two overturned cups, each hiding a marble. One of the marbles is red. Your task is to bet on which cup is covering the red marble. You learn that under one cup is a marble drawn randomly from a bag containing 90 percent red marbles. A bystander, whom you know to tell the truth 80 percent of the time tells you, "I saw the marble placed under the other cup, and it was red.'' Placing your bet, the base-rate vs. case-specific character of the evidence is irrelevant. The odds of betting correctly, of maximizing the likelihood of winning, are dictated only by the content of the information. The question for the decision maker is which is more informative, an imperfectly credible witness, or an imperfectly pinpointed set of base-rate information. One choice offers a .90 probability of being correct, the other only .80. The diagnostic value of the information is not affected by whether it appears to report background facts or "case-specific'' facts. Even so-called particularistic evidence is probabilistic. Invariably, all information is really probability information. Only if we neglect to uncover, or otherwise conceal from a fact finder the base rates of witness (or other evidence) reliability, will the case-specific information seem more informative. Only if we conceal from the bettor the fact that the witness who says "I saw the marble and it was red'' is only .80 truthful or .50 accurate in color perception, will the assertion seem to have special probative force. The distinction between what one can learn from case-specific as opposed to base-rate information is more imaginary than real. In terms of accurate fact finding, it is a difference that makes no difference.
Similarly mistaken are distinctions between certain kinds of identifications. Descriptions which lead to a probability of correct classification of a person (e.g., "a completely bald man with a wooden left leg, wearing a black patch over his right eye and bearing a six-inch scar under his left, who flees from the scene of the crime in a chartreuse Thunderbird with two dented fenders'') are treated as different from the "particularistic'' type where a witness says, "Yes, that's the person.'' Some have argued that evidence that the above description fits only one person in 64 million ought not to be used in the trial of a person fitting that description, because it merely specifies the class to which he belongs and its size; it does not identify him. The latter identification would be more welcome, because it singles out a unique individual. The identifying witness may be confident that the identification is correct, but the fact finder ought to appreciate the inherently probabilistic nature of perception, storage, recall, and identification. Apparently, fact finders (like legal commentators) fail to appreciate this point. They act as though the eyewitness identification is highly accurate, when in reality it may be far more likely than once in 64 million to be in error. Indeed, the probability of correct eyewitness identification has been found to be far lower than commonly assumed.
The most meaningful difference between these two kinds of identification is that in one we allow the identifying witness to make the decision instead of letting the fact finder do so. But to think we have here evidence that is somehow uniquely diagnostic is only to conceal from ourselves the probabilistic nature and limited accuracy of the identification process. In both kinds of identifications we are dealing with classes containing more than one person, and there is no guarantee that the "particularistic'' approach yields smaller classes.
Saks and Kidd's point of view has serious implications for our second question, concerning the sufficiency of probability evidence to support a verdict. In the end, Saks and Kidd are outspokenly critical of the traditional, nonquantitative, ritualistic approach (id. at 156):
Tribe advocates, in short, the maintenance of a fantasyland of apparent certainty in a world of patent uncertainty. Regarded from only a mildly different angle, such a deliberate turning away from reality may serve neither the law nor the defendant. First of all, the symbolism is so at variance with the objective reality as well as with the conceptualizations of legal scholars (certainly including Tribe himself) and the subjective experience of judges and jurors, that this may be one more of the legal fictions that tend to undermine the law's own credibility. An institution that would so deliberately ignore real, measurable doubt and assert not that it has made the best decision it was able to but that it is "certain'' it is correct, is unlikely to keep the masquerade going forever or to fool everyone.
Note how many of the advocates of probability evidence analyze problems of evidence and inference by analogy to bets. In our view, this obscures the difference between a trial and a wager. The essential quality of a wager is that the truth is going to be revealed. This is why people betthe revealed truth will resolve the bet and there will be winners and losers. A trial, however, is fundamentally different. In a trial there is no revealed truth. Instead, the system in effect asks us to accept the verdict as a surrogate for revealed truth. In this situation, an important objective of the litigation process is to provide an acceptable surrogate for revealed truthan acceptable conclusion about what happenedand, by so doing, put to rest our uncertainty about the event that is being tried.
If it is an objective of the trial system to generate an acceptable conclusion about what happened, the problem with probability evidence is that it presents uncertainty in stark form, the starkest form being an explicit numerical probability statement that undermines this objective. When uncertainty can be starkly quantified, there can be no acceptance of the result as "the truth.'' Many aspects of the litigation system function to promote the acceptance of the jury's verdict as a surrogate for the truth. For example, the secret deliberation of the jury, the inarticulateness of the verdict, and the peculiar acceptability of eyewitness testimony (credibility proof) may all be explained in terms of their role in making it possible for the system to provide a surrogate for truth. Such acceptance enhances the functioning of the system. If government is to perform some serious action based on some past event, such as jailing or executing a person or ordering a transfer of wealth, the government's action is much more likely to be accepted if the determination about the past event is accepted as a surrogate for the truth. Probability statements about what happened, with their quantified possibility of error, do not provide as strong a basis for confidence in the acts of the system as do inarticulate, unquantified expressions of belief.
Is there a real difference between beliefs and probability statements? We think so. Watch the man at the carnival manipulating the pea and the three cups, or the con artist in the park playing three-card monte. You watch the cup with the pea (or the red card), and when the dealer finishes moving the cups or cards around and asks you to point to the cup with the pea or the red card, you know that the one cup not to point to is the one you think the pea is under.
It is essential to the operation of the trial system that people not believe that the system is trying to manipulate themthat it be as conducive as possible to the formulation of beliefs based on evidence. When we look at a trial result, we must have the confidence that, although we were not there, people just like us looked closely at all the evidence and decided by a fair procedure that the defendant is guilty. We can then accept the jury's judgment as ours and believe that the system has functioned properly. The "odds'' cases don't fit this model. Not only is there a quantifiable risk of error, but there is no reason to defer to the judgment of the jury in these cases. If the evidence is so stark that there is a probability statement before the case goes to the jury, there is still a probability statement after the case goes to the jury. The jury doesn't see anything that we can't see, and we are still in doubt to the extent that the probabilities present the possibility of an error.
What emerges is a principle of complexity, an idea that says, in order to get a case to a jury, the evidence has to be sufficiently complex so that it does not present us with an explicit probability statement. There must be some nonquantifiable evidence and the jury must operate, to some extent, as a black box. Credibility testimony is ideally suited to this purpose. It is not possible to make an explicit probability statement with respect to it, though some have tried. Its evaluation is so subjective that each person's assessment of the appropriate probability could be different. For this reason, eyewitness testimony, which depends so much on the credibility of the observer, is highly favored.
This also explains the difference between using probability evidence in discrimination cases and using probability evidence in connection with identifications in criminal cases. It helps explain the important distinction between admissibility and sufficiency. Problem 22, the prisoners case, is more troubling than the blue bus or license plate roulette. Yet, probability evidence is used often in criminal cases. For example, in State v. Garrison, 120 Ariz. 255, 585 P.2d 563 (1978), the defendant was charged with the strangulation murder of Verna Martin. Her body was mutilated by bite marks. The state called Homer Campbell, Jr., a dentist, as an expert witness. Campbell testified that the wounds in Martin's breasts had ten points of similarity with Garrison's teeth and that "the probability factor of two sets of teeth being identical in a case similar to this is, approximately, eight in one million.'' The court affirmed the use of this evidence over a strong dissent. It distinguished Collins on the grounds that Dr. Campbell's probability figure was not from "personal mathematical calculations'' (as in Collins), but from articles in forensic journals and books. In fact, probabilistic evidence forms the basis for many kinds of forensic proof commonly accepted in criminal cases (e.g., fingerprint evidence, DNA analysis, blood testing, hair and tissue comparisons), sometimes with explicit probability explanations for the strength of the proof. As the Cole case, infra at page 127, indicates, courts are not always adept at the use of such evidence. Why is the admission of such evidence more acceptable than the correct use of the product rule on the issue of identification? Would the more commonly accepted forms of forensic proof based on probability theory be sufficient to sustain a conviction?
1. A sensible, and now quite conventional, approach to this question is "to treat the probability as the fact if the defendant has the power to rebut the inference." Jaffe, Res Ipsa Loquitur Vindicated, 1 Buff. L. Rev. 1, 6 (1951). On this theory, if the defendant produces a reasonably satisfactory explanation consistent with a conclusion of no negligence, and if the plaintiff produces no further evidence, the plaintiff should lose on a directed verdict despite his mathematical proofunless (1) he can adequately explain his inability to make a more particularized showing (a possibility not adverted to in id.), or (2) no specific explanation is given, but there is some policy reason to ground liability in the area in question on a substantial probability of negligence in the type of case rather than to require a reasoned probability in the particular case, thereby moving toward a broader basis of liability. It will be noticed that no such policy is likely to operate when the mathematical evidence goes to the question of the defendant's identity and the plaintiff does not explain his failure to produce any more particularized evidence, for it will almost always be important to impose liability on the correct party, whatever the basis of such liability might be.
2. If tires rotated in complete synchrony with one another, the probability would be 1/12; if independently, 1/12 x 1/12, or 1/144.
3. A Swedish court, computing the probability at 1/12 X 1/12 = 1/144 on the dubious assumption that car wheels rotate independently, ruled that fraction large enough to establish reasonable doubt. Parkeringsfragor, II. Tilforlitligheten av det s.k. locksystemet f;auor parkernigskontroll. Svensk Juristidining, 47 (1962) 17-32. The court's mathematical knife cut both ways, however, for it added that, had all four tire-valves been recorded and found in the same position, the probability of 1/12 X 1/12 X 1/12 X 1/12 = 1/20,736 would have constituted proof beyond a reasonable doubt. Id.
4. Note that in this criminal case, as in the preceding civil one, a fact known about the particular defendant provides reason to believe that the defendant is involved in a certain percentage of all cases (here, cases of being at the crucial place between 7 p.m. and midnight) possessing a characteristic shared by the litigated case.
5. . In this case, unlike the preceding two, it is a fact known about the particular event that underlies the litigation, not any fact known about the defendant, that triggers the probabilistic showing: a certain percentage of all events in which the crucial fact (here, the killing of a man in his mistress' apartment) is true are supposedly caused by a person with a characteristic (here, being the mistress) shared by the defendant in this case.
6. This is, of course, People v. Collins, 68 Cal. 2d 319, 438 P.2d 33, 66 Cal. Rptr. 497 (1968), minus the specific mathematical errors of Collins and without the interracial couple. One special factor that can lead to major mathematical distortions in this type of case is the ``selection effect'' that may arise from either party's power to choose matching features for quantification while ignoring non-matching features, thereby producing a grossly exaggerated estimate of the improbability that the observed matching would have occurred by chance. See Finkelstein & Fairley [A Bayesian Approach to Identification Evidence, 83 Harv. L. Rev. 489, at] 495 n.14. This difficulty may well have been present in People v. Trujillo, 32 Cal. 2d 105, 194 P.2d 681, cert. denied, 335 U.S. 887 (1948), in which an expert examined a large number of fibers taken from clothing worn by the accused and concluded, upon finding eleven matches with fibers taken from the scene of the crime, that there was only a one-in-a-billion probability of such matching occurring by chance.
7. For example, in line with the suggested approach, the judge might decide to employ the doctrine of res ipsa loquitur, or any of a variety of rebuttable presumptions. See, e.g., O'Dea v. Amodeo, 118 Conn. 58, 170 A. 486 (1934) (presumption of father's consent to son's operation of automobile); Hinds v. John Hancock Mut. Life Ins. Co., 155 Me. 349, 354-67, 155 A.2d 721, 725-32 (1959) (presumption against suicide). One of the traditional functions of the use of presumptions, at least those rebuttable by any substantial contrary evidence, is ``to make more likely a finding in accord with the balance of probability.'' Morgan, Instructing the Jury upon Presumptions and Burden of Proof, 47 Harv. L. Rev. 59, 77 (1933).
8. If the statistical evidence standing alone establishes a sufficiently high prior probability of X, and a satisfactory explanation is provided for the failure to adduce more individualized proof, there seems no defensible alternative (absent believable evidence contrary to X) to directing a verdict for the party claiming X, for no factual question remains about which the jury can reason, and directing a verdict the other way would be more likely to lead to an unjust result. If, however, more individualized proof is adduced, and if the party opposing X has discharged the burden (created by the statistical evidence) of producing believable evidence to the contrary, the question remains whether the risk of distortion created by informing the trier of fact of the potentially overbearing statistics so outweighs the probative value of such statistics as to compel their judicial exclusion. If this situation arises in a criminal case, see, e.g., the heroin hypotheticals, the police hypothetical, and the mistress hypothetical, the added threats to important values should probably suffice, in combination with the danger of a distorted outcome, to outweigh the probative value of the statistics. But if the situation arises in a civil case, as in the barrel hypothetical, or in the bus hypothetical, all that I am now prepared to say is that the question of admissibility seems to me a very close one.
9. Hart & McNaughton, Evidence and Inference in the Law, in Evidence and Inference 48, 52 (D. Lerner ed. 1958). I do not exclude the possibility that, in extraordinary cases, and especially in cases involving highly technical controversies, the ``historical'' function may be so dominant and the need for public comprehension so peripheral that a different analysis would be in order, laying greater stress on trial accuracy and less on the elements of drama and ritual.
10. That people can invent tools that do a better job than the humans who invented them should come as no surprise to people who have used such devices as radios, light meters, or hammers. Indeed, the adversary process is just such a tool. It seems intuitively wrong to many people, but it is capable of accomplishing certain purposes that intuitive individuals cannot.
11. Trust in the pocket calculator is based on experience with it. People who acquire experience with mathematical decision making in management, operations, planning, science, economics, and so on, develop a similar trust in these other computational aids.
12. If 95 percent of the buses are blue, and the witness is 80 percent accurate, when a witness reports seeing a blue bus, this yields a .98 probability that the bus was, indeed, blue.