Teachers' Manual to Green, Nesson & Murray, Problems, Cases and Materials on Evidence, 3rd Edition.


Go to: Teacher's Manual Introduction Chapter 1 Contents Next Section


F. Probability and Statistical Proof (93)

People v. Collins, 68 Cal. 2d 319, 438 P.2d 33, 66 Cal. Rptr. 497 (1968)(93)

The Collins case raises the issue of the admissibility of probabilistic proof at trial. The Smith case, which follows, raises the question of the extent to which probabilistic and mathematical models may be used as standards on which to base a criminal conviction or civil judgment -- the sufficiency question. The two questions are obviously related although for an evidence course, the admissibility question is primary.

Collins is in virtually every evidence and torts book. There is no need to say much more here about the particular case. When pressing students on the questions following the case (103) you can ask if probabilistic evidence should be admitted if the foundational problems are eliminated and there is additional probabilistic evidence relating to the following identifying characteristics: defendant was wearing a blue Dodger cap; the defendant had one arm; four of the six digits on defendant's automobile were identified. The point is, it is easy to make the probability evidence very persuasive, assuming that the foundational problems can be eliminated, and in most cases they probably could be.

The role of probability in judicial fact finding is not a subject on which there are clear right answers. Some of the most active disputes in the evidence literature involve deep disagreements about this subject. With some hesitance, we suggest our view here in a brief way.

The advocates of probability evidence have a strong tendency to analyze problems of evidence and inference by analogy to bets. This obscures the difference between a trial and a wager. The essential quality of a wager is that the truth is going to be revealed. This is why people bet -the revealed truth will resolve the bet and there will be winners and losers. A trial, however, is fundamentally different. In a trial there is no revealed truth. Instead, the system in effect asks us to accept the verdict as a surrogate for revealed truth. In this situation, an important objective of the litigation process is to provide an acceptable surrogate for revealed truth -an acceptable conclusion about what happened -- and by so doing, put to rest our uncertainty about the event that is being tried.

If it is an objective of the trial system to generate a conclusion about what happened, the problem with probability evidence is that it presents uncertainty in stark form, the starkest form being an explicit numerical probability statement that undermines the objective of providing an acceptable conclusion about what happened. When uncertainty can be starkly quantified, there can be no acceptance of the result as "the truth." Many aspects of the litigation system function to promote the acceptance of the jury's verdict as a surrogate for the truth. For example, the secret deliberation of our jury, the inarticulateness of the verdict, the peculiar acceptability of eyewitness testimony (credibility proof), may all be explained in terms of their role in making it possible for the system to provide a surrogate for the truth. Such acceptance enhances the functioning of the system. If government is to perform some serious action, such as jailing or executing a person or ordering a transfer of wealth, based upon some past event, the government's action is much more likely to be accepted if the determination about the past event is accepted as a surrogate for the truth. Probability statements about what happened, with their necessary and numerical possibility of error, do not provide as strong a basis for confidence in the actions of the system as does inarticulate, unquantified expression of belief.

Is there a real difference between beliefs and probability statements? We think so. Watch the man at the carnival manipulating the pea and the three cups, or the con artist on the Commons playing three card monte. You watch the cup with the pea (or the red card), and when the dealer finished moving the cups or the cards around and asks you to point to the cup with the pea or the red card, you know that the one cup not to point to is the one you think the pea is under.

It is essential to the operation of the trial system that people not believe that the system is trying to manipulate them -- that it be as conducive as possible to the formulation of beliefs based on evidence. When we look at a trial result, we must have the confidence that, although we were not there, people just like us looked closely at all the evidence and decided by a fair procedure that the defendant is guilty. We can then accept the jury's judgment as ours and believe that the system has functioned properly. The "odds" cases don't fit this model. Not only is there a quantifiable risk of error, but there is no reason to defer to the judgment of the jury in these cases. If the evidence is so stark that there is a probability statement before the case goes to the jury, there is still a probability statement after the case goes to the jury. The jury doesn't see anything that we can't see and we are still in doubt to the extent that the probabilities present the possibility of an error.

What emerges is a principle of complexity, an idea that says, in order to get a case to a jury the evidence has to be sufficiently complex so that it does not present us with an explicit probability statement. There must be some nonquantifiable evidence and the jury must operate, to some extent, as a black box. Credibility testimony is ideally suited to this purpose. It is not possible to make an explicit probability statement with respect to it, though some have tried. Its evaluation is so subjective that each person's assessment of the appropriate probability could be different. For this reason, eye witness testimony which depends so much on the credibility of the observer, is highly favored.

This also explains the difference between using probability evidence in discrimination cases and using probability evidence in connection with identifications in criminal cases. For teacher who are particularly interested in this topic, see State v. Garrison, 120 Ariz. 255, 585 P.2d 563 (1978), and State v. Sneed, 76 N. Mex. 349 (1966).

In Garrison, defendant was charged with the strangulation murder of Verna Martin. Her body was mutilated by bite marks. The state called Homer Campbell, Jr., a dentist, as an expert witness. Campbell testified that the wounds in Martin's breasts had ten points of similarity with Garrison's teeth and that "the probability factor of two sets of teeth being identical in a case similar to this is, approximately, eight in one million . . . ." The court affirmed the use of this evidence over a strong dissent. It distinguished Collins on the grounds that Dr. Campbell's probability figure was not from "personal mathematical calculations" (as in Collins), but from articles in forensic journals and books.

In Sneed, defendant was convicted of murder. The prosecution presented a professor of mathematics, Dr. Thorp, to testify that applying the "product rule" to certain identifying features, there was a chance of 240 billion to one that the defendant was the person who purchased the murder weapon from a pawnshop on August 17, 1954 in Las Cruces, New Mexico. The court held that admission of this evidence was reversible error because the estimates of the probabilities of the individual variables that were utilized to produce the 240 billion to one figure were entirely speculative.

There are a number of interesting scholarly treatments of this subject. See Ball, The Moment of Truth: Probability Theory and Standards of Proof, 14 VAND. L. REV. 897 (1961); Brilmayer & Kornhauser, Review: Quantitative Methods and Legal Decisions, 46 U. CHI. L. REV. 116 (1978); Brook, Inevitable Errors: The Preponderance of the Evidence Standard in Civil Litigation, 18 U. TULSA L.J. 79 (1982); L. Cohen, The Probable and the Provable (1977); Cullison, Probability Analysis of Judicial Fact-Finding: A Preliminary Outline of the Subjective Approach, 1969 U. TOL. L. REV. 538; Finkelstein & Fairley, A Bayesian Approach to Identification Evidence, 83 HARV. L. REV. 489 (1970); Kaye, The Laws of Probability and the Law of the Land, 47 U. CHI. L. REV. 34 (1979); Lempert, Modeling Relevance, 75 MICH. L. REV. 1021 (1977); Nesson, Reasonable Doubt and Permissive Inference: The Value of Complexity. 92 HARV. L. REV. 1187 (1979); Saks and Kidd, Human Information Processing and Adjudication: Trial by Heuristics, 15 LAW & SOC. REV. 123 (1980-81); Schum, Book Review, 77 MICH. L. REV. 446 (1979); Tribe, Trial by Mathematics: Precision and Ritual in the Legal Process, 84 HARV. L. REV. 1329 (1971).

Smith v. Rapid Transit, Inc. 317 Mass. 469, 58 N.E. 2d 764 ( 1945) (103)

This classic evidence case is the inspiration for "The Blue Bus" problem, which follows. It poses the question of when, if ever, purely probabilistic evidence can be sufficient for a finding. While it is clear that evidence of the routes and times of defendant's busses would be admissible on the issue of whether it was defendant's bus which caused the accident, the court is unwilling to allow the jury to base a finding that it was one of defendant's buses in the absence of some evidence making a direct connection. Because of the opacity of jury verdicts, and the general reluctance of courts to take matters of fact from juries by directed verdicts and judgments n.o.v. , there is relatively little discussion in the American law of proof of how evidence should be used to reach a conclusion and how much or what kind of evidence is sufficient to support a given conclusion. This is one area where we see "sufficiency" as a meaningful "second screen" in fact determination.

The question is whether the distinctions in Smith and "Blue Bus" hold up as we consider cases in which purely probabilistic identifications are provided by fingerprint or voiceprint, or ultimately DNA evidence?

Problem- License Plate Roulette (104)

Problem- Blue Bus (105)

License Plate Roulette and Blue Bus pose the same problem: the admissibility of probability evidence vet non. The Blue Bus problem is a variant on Smith v. Rapid Transit. Inc., 317 Mass. 469, 58 N.E. 2nd 754(1945), suggested in Tribe, Trial by Mathematics: Precision and Ritual in the Legal Process, 84 Harv. L. Rev. 1329, 1344-45 (1971). It is a challenging problem on not only the admissibility. but also the sufficiency of statistical evidence.

How should a case be decided in which it is not possible to form a belief about what actually happened, but it is nonetheless probable that the defendant was responsible for the plaintiff s injury?

According to a probabilistic model, the plaintiff would seem entitled to win, having shown that it is considerably more probable than not that it was a Blue Bus that ran her off the road. By contrast, one who regarded an important objective of the system to be the projection of a believable account about what happened would conclude that plaintiffs evidence was insufficient, notwithstanding the high probability that she demonstrated.

Probabilists have had enormous trouble with the Blue Bus problem because they intuitively agree that the plaintiff should lose. Their problem has been to explain why without having to junk their theory. (Brook comes closest to biting the bullet and saying that the plaintiff should win.) They attempt to rationalize the conflict between outcome apparently called for by the probabilities and their intuition by emphasizing the subjective element of the probability assessment. They argue that while the plaintiffs objective proof indicates an 80% likelihood of a Blue Bus, a juror could well include as an element in his subjective assessment a substantial degree of skepticism about the plaintiffs case based on its thinness -- "Why was that all the proof she had?" -- enough skepticism to reduce the subjective probability assessment of the defendant's liability below 50%. The juror thinks, "If it really was a Blue Bus, the plaintiff would have had better proof than this; therefore I think the odds are less than fifty-fifty that it was a Blue Bus." See Brook supra.

This argument explains how a juror might find against the plaintiff, but that is not the problem posed by the case. The plaintiff loses by directed verdict: the case never reaches the jury. The argument based on subjective probability asserts that a juror is not bound to assume the probability of 80%, and could, rationally, assess the probability at less than 50%. The argument does not assert that the jurors are bound to assess the probability at less than 50%. It explains, perhaps, why the plaintiff is not entitled to a directed verdict in her favor, but not why she should automatically lose by directed verdict. The jurors could, after all, arrive at a subjective probability higher than 50%, even if they discount somewhat from 80% because of the thinness of the plaintiffs case. Nothing objective compels a juror to drop his subjective probability so drastically that the plaintiff must lose. The logic of the argument, in other words, suggests that the case should reach the jury, and that the jury's verdict should be upheld, no matter which way it comes out.

Another popular attempt by the probabilists at rationalizing why the plaintiff should lose by directed verdict is that if the plaintiff could recover in the circumstances of the Blue Bus case, then the defendant company would lose not only that case, but all similar cases, thereby causing the company to pay for all such accidents when it was responsible for only 80% of them. To some this seems unfair to the defendant company, although the apparent alternative is to award the plaintiff nothing in all cases, creating a windfall for the Blue Bus company in the cases in which it is at fault. (See Brook, supra at 102.) To others the problem is economic. As Richard Posner sees it, for example, allowing an award to the plaintiff, and to all similarly situated plaintiffs, would dislocate the market: it would overburden the Blue Bus company, subsidize its smaller competitors, lead them to be less careful, and thereby increase accident rates. Posner uses this argument to justify a special rule of proof for cases like the Blue Bus -namely a requirement that there be some proof of liability which goes beyond background statistical information. Posner, having seen that the simple application of a high probability acceptance rule (more probable than not) would produce results which would not maximize utility, introduces a further proof requirement beyond mere high probability. He thus adds a kind of epicycle to his probabilistic approach to keep his model in harmony with what seems right. (He seems not to notice the economic subsidy (and attendant dislocating effects) that his approach provides the bus industry at the expense of innocent bystanders.)

No matter how the probabilists rationalize it, admitting that bare statistical proof is not sufficient to warrant a plaintiffs verdict reveals a fundamental weakness in the conceptual foundation of their model. A conclusion that it was 80% probable that a Blue Bus ran the plaintiff off the road is all too evidently nothing more than a restatement of the frequency information about the ratio of buses that travel down the road, not a statement about the specific occurrence, and therefore not a basis for forming a belief about what actually happened. Ball, The Moment of Truth: Probability Theory and Standards of Proof, 14 VAND. L. REV. 807 (1961). The proposition about the past which is understood to justify a judicially imposed damage award is that the defendant negligently ran the plaintiff off the road. The statement of probability does not permit an assertion about the truth of the justifying proposition. From the vantage of one who sees the projection of a believable account about what happened as an essential part of projecting the rule of law, it is a natural prerequisite to allowing a jury to decide for the plaintiff that there must be proof adequate to support a belief about the justifying historical proposition. The jurors need not necessarily form such a belief. They are quite properly asked to decide by a preponderance test. It is nonetheless a condition to allowing them to decide the case that the evidence be such that their resultant verdict is capable of being accepted as a statement about what happened.

Does this analysis hold up when one is considering DNA evidence, which produces an identification to a probability of not 80%, but 99.99%? See the problems and discussion below.

Problem - Conjunction (106)

The Point:

When a finding depends on more than one inference, and each inference is probabilistic, then (depending on the degree of independence of the two inferences) the probability of the finding will be less than the probability of either inference.

Answer and Analysis:

This problem illustrates the difficulty of probabilistic proof when a finding depends on a series of related inferences. The mathematical probability of the finding (assuming that the inferences are independent) is the product of the probabilities of the inferences, in this case .36, or considerably less than 50%. Many judicial findings are complex in this sense.

Problem - Prison Yard (107)

The point:

Evidence is insufficient to support a verdict in a criminal case if it is based on "pure" probabilities that reveal a quantifiable risk of error.

Answer and Analysis:

In the prosecution of Prisoner # 1, when the only evidence is that of the distant witness, the best that we can say is that the odds that the defendant is guilty are 24 in 25, or 96%. There remains a clear plausible hypothesis consistent with innocence which the fact finder cannot eliminate either by making credibility judgments or by making reasonable assessments of the significance of the circumstantial evidence. Therefore the defendant is entitled to a directed verdict.

If Prisoner #2 testifies that it was he who hid in the shed, then the defendant is no longer entitled to a directed verdict. The case goes to the fact finder because it is possible for the fact finder to make assessments of credibility that eliminate the hypothesis of innocence.

To take this point further, suppose the evidence is that 12 or 13 of the prisoners withdraws from the attack. At the trial of Prisoner #1, there is evidence that the Prisoner had an argument with the victim the day before in which threats were exchanged. Is Prisoner #1 entitled to a verdict of acquittal in this case? In which case is a guilty verdict more likely to be correct, in this hypothetical or in the original problem?

DNA Evidence (111-123)

George F. Will, DNA, the Death Penalty and Horrifying Mistakes

Peter Neufeld & Barry Scheck, Better Ways to Find the Guilty

In the last decade evidence based on analysis of DNA from crime victims and suspects has cast an entirely new light on probabilistic proof. Although probabilistic evidence of identification such as fingerprint and voice print analysis has long admissible, and in many cases sufficient to generate positive identifications in civil and criminal cases, the widespread use of DNA analysis has prompted a new look at how this powerful form of evidence, based solely on mathematical probability, should be used in criminal prosecutions and civil cases and ultimately presented to the jury.

The material included in the 3rd Edition comprises two newspaper articles reporting on the use of DNA evidence to exculpate persons facing execution based on convictions obtained with the use of conventional evidence, including eyewitness identification. In several cases analysis of DNA in blood or sperm on the victim showed that the material was not from the defendant, thus giving rise to a very strong inference that the actual attacker was someone other than the defendant. These articles contrast the scientific reliability of DNA evidence with the fallibility of other forms of evidence on which convictions are often based.

The articles are followed by an excerpt from the "Reference Guide on Forensic DNA Evidence" (114) published by the Federal Judicial Center and regularly consulted by federal and state judges in connection with issues surrounding the use of DNA. This material gives an idea of how DNA analysis "works" and points out some of the potential problems and pitfalls in the gathering and analysis of DNA evidence.

One must keep in mind that the probative force of DNA evidence is not quite symmetrical. DNA analysis can provide scientifically specific proof that two samples are not alike, and hence that the two samples did not originate from the same person. If one accepts the validity of the science and the analysis is properly performed, the DNA evidence alone can furnish proof positive that, for example, the defendant was not the person who deposited DNA-containing material (e.g. blood or semen) associated with the crime.

However proof of identity by DNA evidence is only probabilistic. That is, DNA analysis of two samples which show like characteristics, can only demonstrate a probability (albeit usually a high one) that the two samples originated from the same person. In this sense DNA evidence shares the same infirmities of other forms of probabilistic proof.

Despite the climate of widespread acceptance of DNA evidence as proof identity as well as non-identity, it continues to pose certain questions, which sometimes seem reminiscent of the concerns raised by the court in Collins. Prominent among these is how inherently probabilistic DNA evidence of identity will be presented to the jury so that the jury will have a fair understanding of the strength as well as the limitations of this kind of proof.

The three excerpts from recent state court decisions discussing the admissibility, use and presentation of DNA evidence (123-130) illustrate some of these problems. In Commonwealth v. Daggett, 416 Mass 347, 622 N.E.2d 272 (1993) (123) the Massachusetts Supreme Judicial Court rejected DNA evidence that it was "highly likely" that the blood found in the defendant's car came from the victim because the statistical probability analysis used to reach the "highly likely" formulation was not generally accepted. In Minnesota v. Bloom, 516 N.W. 2d 159 (Minn. 1994) (124) the Minnesota Supreme Court wrestled with the question of whether an expert witness whose opinion is based on probabilistic DNA analysis should be allowed to tell the jury that, "to a reasonable scientific certainty" the defendant is the source of the analyzed material. In People v. Barney, 8 Cal. App. 4th 798 (Cal. App. 1992)(126) the California appellate court notes the lack of general baseline agreement on the proper method to analyze and present probabalistic DNA evidence and refuses to admit the evidence in the absence of a generally accepted approach.

In many of these cases the issue of the reliability and sufficiency of inherently probabilistic DNA evidence, and how such evidence should be communicated to the jury is merged with issues as to the scientific reliability of such evidence, or given procedures to collect and analyze it under standards for presentation of Expert Testimony. See Chapter 8 below.

Go to: Teacher's Manual Introduction Chapter 1 Contents Next Section