The Cosmic Haystack

I taught the first Penn State graduate course in SETI last spring, and I thought it went really well. We read a lot of papers, had great discussions about Schelling Points and all the different ways to search, and got to visit Green Bank to conduct original SETI observations. You can read all about the course at the class website here, along with student reactions to the papers (tweeted out by Sofia Sheikh here).

One of the cuter terms in the syllabus was that any student whose final project got published in a refereed paper would get a retroactive ‘A’. Well, the first automatic ‘A’ has arrived!  Shubham Kanodia and Emily Lubar, following a suggestion by Jill Tarter, calculated rigorous volumes of the Cosmic Haystack, and the paper is now on the arXiv.

Some highlights:

The first instance of the term “Cosmic Haystack” to mean the space through which SETI searches to find alien technology is apparently from this article in the Christian Science Monitor in 1977, as tracked down by @spacearcheology:

It appears in a paper 4 years later in Wolfe et al. (1981) in a NASA report, which includes this figure (drafted by Jill Tarter):

The Cosmic Haystack here is 3 dimensional, “compressed” down from many more for clarity. The rectangles inside the haystack represent relatively narrowband searches that were either sensitive or included many targets (it’s hard to do both).

Today, wide-field instruments and broadband receivers and backends can search the haystack for alien “needles” much faster. The question is: how much have we searched? Some have gotten the wrong idea about the whole endeavor, as we explain in our abstract:

Many articulations of the Fermi Paradox have as a premise, implicitly or explicitly, that humanity has searched for signs of extraterrestrial radio transmissions and concluded that there are few or no obvious ones to be found. Tarter et al. (2010) and others have argued strongly to the contrary: bright and obvious radio beacons might be quite common in the sky, but we would not know it yet because our search completeness to date is so low, akin to having searched a drinking glass’s worth of seawater for evidence of fish in all of Earth’s oceans.

Shubham and Emily decided to construct a radio Cosmic Haystack that included transmitters anywhere, not just around nearby stars, akin to NASA’s old SETI Sky Survey from the 1980’s. We also tried to capture the way that very broadband signals might be detected, and not just the narrow-band signals considered by some of the earliest searches. This required us to include transmission bandwidth as a haystack dimension, and calculate a rough sensitivity function in terms of it. We also wanted to make sure that our haystack boundaries were well defined, and that we were calculating only search space within a well-defined range of parameters, which led to some interesting integrals with lots of cases to consider:

Sensitivity as a function of transmission bandwidth and central frequency for an idealized broadband instrument, with a “ceiling” set by the haystack boundary.

Case six, for very broadband, very bright transmissions.

The bottom line was that we got to calculate what region of parameter space has been searched for transmitters in the Milky Way.  It’s tiny.  Jill Tarter once estimated it as a glass of water out of all the oceans in the world; we got more like a bathtub (despite a much different haystack construction):

We found that the MWA low frequency searches dominated our haystack search volumes, because of their incredible étendu: a very wide field and very good sensitivity. And they did it in just a couple of hours of searching!

We also construct another one-parameter way to calculate search completeness: what density of uniformly spaced megawatt transmitters in the Milky Way can we rule out?  We get about one per 0.27pc for the latest Breakthrough Listen paper (for L-band transmitters). The Galaxy could be filled with more transmitters than stars, and we wouldn’t have found them yet!

So we haven’t done much searching, and people should not use the “failure” of radio SETI to date as evidence that we should stop looking.

But it’s also important not to let the pendulum swing to far the other way! One might see numbers like -18 in the exponent and declare the entire project hopeless.  On the contrary:

  1. New radio technology allows us to search through haystack volumes much faster than before; the Breakthrough Listen program is quickly catching up to the historical NASA programs by our metric, and MWA needed only 2 hours to dominate our calculation.
  2. If we expect to find transmitters near stars, our completeness is many orders of magnitudes higher, because we have concentrated our efforts there
  3. We don’t need to search the entire haystack unless there is nothing to find.  Another way to look at it: you don’t have to drain the oceans to find marine life, unless they are sterile and you are trying to prove that.  A bathtub’s worth of water is probably not enough to find a fish if you randomly sample the ocean, but if you look in the right places it’s about the right order of magnitude of search required.

Hopefully our paper will be a useful extension of Jill Tarter’s work along these lines (see here and here).

We also had a lot of fun figuring out where the phrase “needle in a haystack” comes from which I detailed here. (it’s not Cervantes!)

Finally, a note on how just as every journey begins with a single step, every search begins with comically weak upper limits.  We quote a critical referee on Bachall & Davis’s first neutrino search paper (an upper limit):

Any experiment such as this, which does not have the requisite sensitivity, really has no bearing on the question of the existence of neutrinos. To illustrate my point, one would not write a scientific paper describing an experiment in which an experimenter stood on a mountain and reached for the moon, and concluded that the moon was more than eight feet from the top of the mountain. (Bahcall & Davis 1982, p. 245)

which I also discussed back here. Waste not want not!

[Update: @SpaceArcheology finds an earlier attestation, probably the source of the Monitor article and perhaps the origin of the term by—who else?—Frank Drake, and @astrocrash finds earlier non-SETI attestations from Shapely and Hubble:



Leave a Reply

Your email address will not be published. Required fields are marked *