I’ve been getting some good feedback through the FaceBook regarding the idea of “dispositive null” as good new jargon; so let’s consolidate the discussion and near consensus here (see that post for a synopsis of the contributions from others to this idea).
A “null detection” or “non detection” is the absence of a detection; it often means that you have shown that there is no signal down to some upper limit. In a statistical sense, we often write that we have ruled out signals of strength K or stronger within certain parameters with some confidence (X% or n-sigma).
This is a logical prerequisite to ruling out a specific model, but not its rhetorical equivalent. In the title of a paper, or its abstract, or in a press release, we want to write that we have disproved someone’s claim, or some conventional wisdom, or some popular model of some effect. Saying that we have “failed to detect” it makes it sound like the experiment was a failure, when in fact the experiment may be definitive.
A great example is the Michelson-Morley experiment which disproved the existence of “ether”, the medium in which light travels. If the ether existed, as most physicists assumed, then the motion of the Earth through the ether (as it rotates and revolves around the Sun) would cause light to travel at different speeds depending on its direction: in the direction of Earth’s motion it would seem to be going more slowly, the same way that a car moving down the highway in the lane next to you seems to me moving more slowly than 55 mph. Light traveling opposite Earth’s motion would seem to be going quite quickly, the way that cars moving in the other direction seem to go past you very fast.
Michelson and Morley showed that the speed of light was independent of direction to a precision far beyond the signal expected from the motion of the Earth. Their purpose was not to determine if the speed of light was direction dependent, but to determine if that directional dependence had a particular value. They showed that if it did have a nonzero value, it was much smaller than the ether hypothesis predicted, so there was no ether. Their null detection of the ether was dispositive.
Now, in principle they actually had an upper limit and only ruled out the ether to some very high confidence, but the term “dispositive null” is a qualitative conclusion, not a strictly quantitative assessment of their confidence. The term “statistical significance” plays a similar role in statistics: it has no universally-agreed-upon quantitative meaning, but indicates that the authors are claiming a result is real, and not the result of a statistical fluke.
We had a similar experience with HD 149382 b; we were ruling out a specific claim, and we did so beyond any “reasonable doubt”. We also put strong upper limits on the existence of any planet in that system, but only subject to a complex set of parameter space (short period orbits except those commensurate with one sidereal day… etc. etc.) So we had an “n-sigma upper limit” for many kinds of planet, but a dispositive null for Geier et al.’s claim.
I also linked to two other cases of similar results: HD 97657, whose transit was erroneously reported, and Fomalhaut b, whose thermal emission remains undetected. In the former case, the transits were claimed, but this paper shows that if they exist are not at the claimed depth. In principle, this is a strong upper limit, but with respect to the earlier claim this is a dispositive null.
With respect to Fomalhaut b, things are a bit more subtle. The detection of that object in optical emission implied that it was a planet of a certain mass and age. Models say that if that is true, then there should be a certain level of thermal emission, which Janson et al. would see, but do not. This is thus a dispositive null with respect to a certain class of model explaining the HST detection of Fomalhaut b: that it is a young-Jovian mass planet. Something is there with a lot of optical emission, but it’s not what Kalas et al. postulated (this is mildly unfair to Kalas et al., though: they knew that the infrared emission was below the expected value, and suggested reasons that might be so. The new Spitzer observations are an order of magnitude more sensitive than Kalas et al.’s upper limit, ruling out most of those reasons.)