Another expert succumbs to the transposition fallacy

A book that attempts to inform defense lawyers on how to handle DNA cases is Dealing with DNA Evidence: A Legal Guide (London: Routledge-Cavendish 2007). In this short primer, Andrei Semikhodskii, Director of Medical Genomics, Ltd., explains that “[u]nderstanding how DNA evidence is obtained and evaluated helps lawyers to find pitfalls in evidence and in data interpretation … .” (P. xi).

Fair enough, but the burden on a book whose purpose is to provide accurate explanations is a heavy one. A common mistake in DNA and other statistical testimony is transposition — mistaking the probability of the evidence given a hypothesis, P(E|H), for the probability of the hypothesis given the evidence, P(H|E). (See the blog of January 18, 2010, on McDaniel v. Brown.) A variation on the transposition fallacy occurs in parentage tests. Dr. Semikhodskii’s laboratory advertises the “world’s most accurate  paternity testing,” but Dealing with DNA Evidence is less than pellucid when it explains that

DNA testing does not give a 100 per cent probability of confirming parentage. When biological parentage is possible, its likelihood is estimated by the CPI [Combined Parentage Index] The value of the CPI indicates how many more times the alleged parent is likely to be the true biological parent of the child than in comparison to an untested unrelated individual from the same population.  (P. 45).

Apparently, the book is referring to a likelihood ratio for the hypothesis that the tested man is the father as against the hypothesis that an unknown man (with no close genetic relationship to the accused) is. But a likelihood ratio that takes on some value x does not mean that the tested man is x times more likely to be the father than is the untested man. It means that the genetic data are x times more likely to arise if he is the father.

Not clear? Well, suppose a ridiculously limited genetic test indicated that a child is 10 times more likely to inherit a genotype from his mother and the putative father than from his mother and a randomly selected man (of equal fertility). Does this mean that the putative father is ten times more likely than Mr. Random to be the biological father? It cannot mean this (in general). After all, if the putative father were up in the International Space Station (and the mother was not) during any plausible date of conception, the likelihood ratio would still be 10. Geneticists can compute the chances of a child’s inheriting various alleles if and when a given man is the father. Even with the best paternity test in the universe, the laboratory cannot compute the chance that the man is the father just by knowing the alleles the child inherited from his father.

Therefore — and contrary to this expensive guide for lawyers — the likelihood ratio does not “show how many times more plausible the prosecution hypothesis is given the DNA evidence.” (P. 76). The ubiquitous transposition fallacy is at work here, as it is in the case law. (I discuss some cases involving such transposition in the likelihood ratio in The Modern Wigmore on Evidence: Expert Evidence.)

This confusion between a “likelihood” P(E|H) (the probability of data given a hypothesis) and a “posterior probability” (that the hypothesis is true given evidence in support of that hypothesis) infects a later discussion of the rule that “[t]he expert should not be asked his opinion on the likelihood that it was the defendant who left the crime stain … .” R. v. Doheny [1997] 1 Cr. App. R. 369. Dr. Semikhodskii thinks that “in contravention of this ruling, almost every DNA report submitted to courts does contain the verbal expression of how much support is to be given to the prosecution hypothesis and in most cases this is allowed to be admitted and aired in front of the jury.” (P. 60). But if “what is admitted and aired” is merely a likelihood ratio and a characterization of its magnitude in English, the expert is not giving “an opinion on the likelihood that it was the defendant who left the crime stain.” An expert who states that it is, say, 100,000 times more likely for certain evidence to arise when the defendant really is the source than otherwise and that this means that the evidence gives “very strong support” to this hypothesis is avoiding rather than offering a statement about the source probability.

Somehow or other, the expert must explain the strength of the evidence to the jury, and classifying it as weak or strong is one way to do it. Indeed, a committee of the U.S. National Academy of Science recently recommended that forensic scientists use such standardized terminology to characterize evidence. The problem with this recommendation is not that it invades the province of the jury by directly expressing an opinion on an ultimate issue, but that the verbal predicate is superfluous. If the expert can state the numerical value of the likelihood ratio — the quantity that measures the strength of the evidence rather than the probability of the hypothesis — then what does adding an arbitrary but standard adjective accomplish?

Let’s hope there is a better guide for lawyers.