Tag Archives: transposition

Another expert succumbs to the transposition fallacy

A book that attempts to inform defense lawyers on how to handle DNA cases is Dealing with DNA Evidence: A Legal Guide (London: Routledge-Cavendish 2007). In this short primer, Andrei Semikhodskii, Director of Medical Genomics, Ltd., explains that “[u]nderstanding how DNA evidence is obtained and evaluated helps lawyers to find pitfalls in evidence and in data interpretation … .” (P. xi).

Fair enough, but the burden on a book whose purpose is to provide accurate explanations is a heavy one. A common mistake in DNA and other statistical testimony is transposition — mistaking the probability of the evidence given a hypothesis, P(E|H), for the probability of the hypothesis given the evidence, P(H|E). (See the blog of January 18, 2010, on McDaniel v. Brown.) A variation on the transposition fallacy occurs in parentage tests. Dr. Semikhodskii’s laboratory advertises the “world’s most accurate  paternity testing,” but Dealing with DNA Evidence is less than pellucid when it explains that

DNA testing does not give a 100 per cent probability of confirming parentage. When biological parentage is possible, its likelihood is estimated by the CPI [Combined Parentage Index] The value of the CPI indicates how many more times the alleged parent is likely to be the true biological parent of the child than in comparison to an untested unrelated individual from the same population.  (P. 45).

Apparently, the book is referring to a likelihood ratio for the hypothesis that the tested man is the father as against the hypothesis that an unknown man (with no close genetic relationship to the accused) is. But a likelihood ratio that takes on some value x does not mean that the tested man is x times more likely to be the father than is the untested man. It means that the genetic data are x times more likely to arise if he is the father.

Not clear? Well, suppose a ridiculously limited genetic test indicated that a child is 10 times more likely to inherit a genotype from his mother and the putative father than from his mother and a randomly selected man (of equal fertility). Does this mean that the putative father is ten times more likely than Mr. Random to be the biological father? It cannot mean this (in general). After all, if the putative father were up in the International Space Station (and the mother was not) during any plausible date of conception, the likelihood ratio would still be 10. Geneticists can compute the chances of a child’s inheriting various alleles if and when a given man is the father. Even with the best paternity test in the universe, the laboratory cannot compute the chance that the man is the father just by knowing the alleles the child inherited from his father.

Therefore — and contrary to this expensive guide for lawyers — the likelihood ratio does not “show how many times more plausible the prosecution hypothesis is given the DNA evidence.” (P. 76). The ubiquitous transposition fallacy is at work here, as it is in the case law. (I discuss some cases involving such transposition in the likelihood ratio in The Modern Wigmore on Evidence: Expert Evidence.)

This confusion between a “likelihood” P(E|H) (the probability of data given a hypothesis) and a “posterior probability” (that the hypothesis is true given evidence in support of that hypothesis) infects a later discussion of the rule that “[t]he expert should not be asked his opinion on the likelihood that it was the defendant who left the crime stain … .” R. v. Doheny [1997] 1 Cr. App. R. 369. Dr. Semikhodskii thinks that “in contravention of this ruling, almost every DNA report submitted to courts does contain the verbal expression of how much support is to be given to the prosecution hypothesis and in most cases this is allowed to be admitted and aired in front of the jury.” (P. 60). But if “what is admitted and aired” is merely a likelihood ratio and a characterization of its magnitude in English, the expert is not giving “an opinion on the likelihood that it was the defendant who left the crime stain.” An expert who states that it is, say, 100,000 times more likely for certain evidence to arise when the defendant really is the source than otherwise and that this means that the evidence gives “very strong support” to this hypothesis is avoiding rather than offering a statement about the source probability.

Somehow or other, the expert must explain the strength of the evidence to the jury, and classifying it as weak or strong is one way to do it. Indeed, a committee of the U.S. National Academy of Science recently recommended that forensic scientists use such standardized terminology to characterize evidence. The problem with this recommendation is not that it invades the province of the jury by directly expressing an opinion on an ultimate issue, but that the verbal predicate is superfluous. If the expert can state the numerical value of the likelihood ratio — the quantity that measures the strength of the evidence rather than the probability of the hypothesis — then what does adding an arbitrary but standard adjective accomplish?

Let’s hope there is a better guide for lawyers.

McDaniel v. Brown: Prosecutorial and Expert Misstatements of Probabilities Do Not Justify Postconviction Relief — At Least Not Here and Not Now

In The Double Helix and the Law of Evidence (pp. 173-176), I briefly described the Ninth Circuit Court of Appeals’ muddled opinion upholding a writ of habeas corpus in Brown v. Farwell, 525 F. 3d 787 (9th Cir. 2008). Much has happened since then. First, the Supreme Court granted a writ of certiorari to review whether the Ninth Circuit used the correct legal standard and whether it should have considered a letter written by a geneticist at the behest of defense counsel eleven years after Brown’s trial. Second, the Court received a slew of briefs, including one on defendant’s behalf from “20 Scholars of Forensic Evidence.” Third, after scheduling oral argument, the Court decided that it could dispose of the case on the briefs alone. Finally, on January 11, 2010, the Court issued its unanimous per curiam opinion (sub nom. McDaniel v. Brown).

The case arose from the brutal rape in 1994 of a nine-year-old girl in Nevada. A jury convicted Troy Brown on evidence that included a DNA profile (at VNTR loci) that had an estimated population frequency of 1 in 3 million. On redirect examination, however, the prosecutor induced its DNA analyst, Renee Romero, to accept his mischaracterization of this number as the probability that someone unrelated to the defendant was the source of the rapist’s profile. In addition, Ms. Romero testified that the probability of a VNTR match to “the very next child” of the same parents would only be 1/6500 when the actual probability is less than 1/1024. (She did not mention other tests she had done that would have brought the probability closer to her figure. See “False, But Highly Persuasive”: How Wrong Were the Probability Estimates in McDaniel v. Brown, 108 Mich. L. Rev. First Impressions 1 (2009).) Defense counsel neither objected to nor corrected her testimony even though the legal and scientific literature at the time made it indisputable that the prosecution was misconstruing the 1/3,000,000 figure and that the 1/6500 figure was miscomputed.

After losing various appeals and state postconviction petitions, Brown argued for the first time in federal court that the probabilities were incorrectly computed or interpreted and that without the DNA evidence, no reasonable juror could have found him guilty beyond a reasonable doubt. He also argued that trial counsel’s representation of him was so poor as to amount to ineffective assistance of counsel. The district court agreed with both claims. The Ninth Circuit affirmed on the ground that without the “false and highly misleading” DNA evidence, there was insufficient evidence for the conviction and hence a violation of due process under Jackson v. Virginia, 443 U. S. 307 (1979). It did not reach the question of effective assistance of counsel.

The Supreme Court reasoned that the Jackson claim fails because Jackson merely holds that when the evidence against the defendant — whether or not properly admitted according the rules of evidence or the constitution — is insufficient, then, as a matter of due process of law, the conviction cannot stand. In Brown, however, there was “no suggestion that the evidence adduced at trial was insufficient to convict unless some of it was excluded … thus dispos[ing] of [the] Jackson claim.”

As explained in The Double Helix, the more applicable due process claim is that the misstatements about probabilities rendered the conviction fundamentally unfair. The Court barely discussed this “DNA due process” claim, as Brown denominated it. Instead, it insisted that “[r]espondent has forfeited this claim, which he makes for the very first time in his brief on the merits in this Court.”

Although the Court thus avoided the colorable due process issue posed by the admission of the DNA evidence, it is hard to see how Brown could have prevailed even on that belated claim. Admitting the mischaracterized random-match probability may well have been plain error, but it did not rise to the level of a due process violation. See “False, But Highly Persuasive,” supra. The error regarding the probability of a random match to a sibling is sufficiently technical as not to amount to plain error, let alone constitutional error. Moreover, the prosecution could have produced a correctly computed sibling-match probability close to 1/6500. See id. Therefore, the trial court’s failure to exclude the statistics — to which defendant did not object — hardly seems like the type of error that rendered his trial fundamentally unfair.

In any event, having determined that Jackson was of no assistance to Brown and that Brown had “forfeited” the better due process claim, the Supreme Court remanded the case to the Ninth Circuit to consider whether Brown’s trial counsel had performed so dismally as to deprive him of due process of law. McDaniel v. Brown is thus a narrow, procedural holding regarding the scope of federal habeas corpus claims of insufficient evidence.

Even as to the procedural issue, however, the per curiam opinion raised the hackles of two Justices. The Scholar’s Brief importuned the Court to seize the opportunity to condemn misinterpretations of DNA evidence at trial. It argued that the DNA analyst’s testimony was badly flawed and assured that the Court that the defense expert was entirely correct. (The author of the brief, Bill Thompson, is a University of California-Irvine colleague of the letter’s author, Larry Mueller; the two have been called part of the “combine from Irvine.”) Apparently, the brief did not persuade the Justices that Mueller’s computation of the sibling-match probability was correct. In the Court’s jaundiced eyes, the letter’s “claim that [the state’s expert] used faulty assumptions and underestimated the probability of a DNA match between brothers indicates that two experts do not agree with one another, not that [the state’s] estimates were unreliable.” Yet, as the one scientific authority cited in the Court’s opinion — a 1996 Report of the National Academy of Sciences report — indicates, Romero plainly transposed the random-match probability and she miscomputed the sibling-match probability — even on her own assumptions! See”False, But Highly Persuasive,” supra. On these matters, the Scholar’s Brief was correct. Rather than acknowledge this fact, however, the Court only treated Mueller’s two criticisms as hypothetically true. Accepting them solely for the sake of argument, the Court observed that they did not justify exclusion of the DNA evidence in its entirety.

Given the Court’s interpretation of the Jackson claim, however, this discussion of the probabilities is superfluous. If Jackson only means that when the totality of the evidence — admissible or otherwise — is sufficient for conviction, why talk about whether the admissible evidence alone is sufficient? Was the Court giving some credence to the possibility that a modified Jackson claim would be tenable? That is, could due process require a federal habeas court to excise unfounded exaggerations and then to determine whether the reduced corpus of evidence could permit a reasonable juror to convict?

In a concurring opinion, Justices Thomas and Scalia took the Court to task for considering the implications of the Mueller letter on the admissibility of the DNA evidence. Unequivocally rejecting any possibility of a modified Jackson standard like the one that the Ninth Circuit entertained and then misapplied, these Justices

disagree[d] with the Court’s decision to complicate its analysis with an extensive discussion of the Mueller Report. … [T]he report’s attacks on the State’s DNA testimony were not part of the trial evidence and have no place in the Jackson inquiry. … [E]ven if the report had completely undermined the DNA evidence … the panel still would have erred in considering the report to resolve respondent’s Jackson claim. The reason, as the Court reaffirms, is that Jackson claims must be decided solely on the evidence adduced at trial.

The concurring Justices are correct in describing the per curiam opinion’s analysis of the Mueller letter as dicta. But the reason is not that the letter itself was “not part of the trial evidence.” It is that Brown raised the pseudo-Jackson claim rather than the straightforward due process claim about unfair exaggeration in the presentation of DNA evidence. Even if defense counsel had never presented Mueller’s letter to the trial court, he could have relied solely on sources subject to judicial notice to argue on appeal, in state postconviction proceedings, and then again in the federal habeas court proceedings that the trial judge’s failure to correct the prosecution’s mistakes sua sponte deprived him of due process. But Brown did not make this “DNA due process claim” in state court, and the prosecution’s indisputable errors are not relevant to the claim that he did make.

I shall address more fully the pretermitted claim that transposition of a probability can constitute a violation of due process in a later installment. Several courts of appeals considered such claims in the years before DNA testing, and their conclusions are instructive.