Category Archives: science

Planck Frequencies as Schelling Points in SETI

Early when I was learning about SETI I was reading about “magic frequencies” and the “Water Hole.”

Back in the early days of radio SETI, instrumental bandwidths were pretty narrow, so Frank Drake and others had to guess what frequencies to observe at to find deliberate signals. One wants a high frequency to avoid interference from the Earth’s ionosphere and background noise from the Galaxy. But one also wants to observe at a high frequency to avoid lots of emission from Earth’s atmosphere.  There is a “sweet spot” between these problems, in the range of 1–10 GHz:

Figure showing background levels as a function of frequency, with a minimum between 1-10 GHz


In this broad minimum are two famous and strong astrophysical emission lines: the spin-flip hyperfine transition due to hydrogen (the “21 cm line”), and the emission from the hydroxyl radical (OH-).  Since these two species combine to form water, and since water is essential to life-as-we-know-it, the region between these two lines is known as the “water hole”. This has a nice pun to it too as a reference to a place where savannah animals (or barflies) gather.  As Barney Oliver put it “Where should we meet? The watering hole!”

Trying to determine exactly which frequency in the Water Hole to search became a game of guessing “magic frequencies” (I think the term is due to Jill Tarter, though I could be wrong) to tune one’s telescope to.

When I was learning about all of this, I was reading the Wikipedia article on the Water Hole and I saw this intriguing link:

Screenshot of the Wikipedia page on the Water Hole showing a link to "Schelling Points"

Clicking on that last link sent me down a nifty rabbit hole and eventually got Schelling points introduced into the SETI literature.

I wrote a while back on Schelling points and their relevance to SETI.  Go there for the full story, but briefly: Thomas Schelling was an economist and game theorist who considered games where players must cooperate (everyone wins or everyone loses) but cannot communicate. His example was finding someone else who is also looking for you in New York City.  The prospects for winning such a game seems hopeless, but Schelling’s insight was that it was actually pretty easy if you could correctly guess the other person’s strategy, since some strategies are clearly bad (a random search) and others are plausibly good (go to a major landmark).

These optimal strategies are characterized by what we now call Schelling points: in New York City, it’s the Empire State Building and noon are good ones.

Amazingly, ABC News Primetime got people to actually play that game and they won! In hours!

When introducing this idea to the world in his book The Strategy of Conflict, Schelling wrote:

[A good example] is meeting on the same radio frequency with whoever may be signaling us from outer space. “At what frequency shall we look? A long spectrum search for a weak signal of unknown frequency is difficult.  But, just in the most favored radio region there lies a unique, objective standard of frequency, which must be known to every observer in the universe: the outstanding radio emission line at 1420 megacycles of neutral hydrogen” (Giuseppe Cocconi and Philip Morrison, Nature, Sep. 19, 1959, pp. 844-846). The reasoning is amplified by John Lear: “Any astronomer on earth would say ‘Why, 1420 megacycles of course! That’s the characteristic radio emission line of neutral hydrogen.  Hydrogen being the most plentiful element beyond the earth, our neighbors would expect it to be looked for even by tyros in astronomy’” (“The Search for Intelligent Life on Other Planets,” Saturday Review, Jan. 2, 1960, pp. 39-43). What signal to look for? Cocconi and Morrison suggest a sequence of small prime numbers of pulses, or simple arithmetic sums.

So the water hole is a Schelling point!  Or it could be—we need to guess the mind of ET and ask: what frequencies would they guess we would guess, and perhaps water and ionospheres and radio just aren’t their thing?  Schelling’s players can win the game only because they have a common cultural heritage and know about the Empire State Building being famous. If we played that game with aliens, we’d probably lose.

So what do we know we have in common with aliens?  Max Planck had an idea.

Image of Max Planck

Max Planck

Max Planck is one of the most important figures in modern physics, famous for many key insights but among them his constant, h, and his eponymous “natural units.”  Planck realized that there is a fundamental length scale of the universe, set by the nature of space and time, gravity, and quantum mechanics. We call it the “Planck length” and it is given by:

Planck Length formula

Very roughly and heuristically, it is the wavelength of a photon so energetic that its wavelength is equal to its Schwartzchild radius (that is, a photon so dense with energy that it would be a black hole).  Today, we recognize this as the scale on which quantum mechanics and General Relativity give different answers or become mutually incompatible.  Dividing by the speed of light, one defines a fundamental timescale of the universe, which could be interpreted as an observing frequency.

In his famous paper on the topic written in 1900, he wrote:

It is interesting to note that with the help of the [above constants] it is possible to introduce units…which…remain meaningful for all times and also for extraterrestrial and non-human cultures, and therefore can be understood as ’natural units’

and that

…these units keep their values as long as the laws of gravitation, the speed of light in vacuum, and the two laws of thermodynamics hold; therefore they must, when measured by other intelligences with different methods, always yield the same.

So he imagined that these units would be known to extraterrestrial physicists, unlike, say, kilograms and seconds which are completely arbitrary and anthropocentric. Since what we’re looking for is a frequency that we know they would know that we know, this seems like a good Schelling point!

The problem is that the Planck time is way way too short—photons at those frequencies are little black holes (or something—we don’t have physics for it) so don’t exist (or can’t be produced, anyway.)  So how could we use them?

Well, there is another fundamental physics constant, another fundamental unit in nature, which is the fundamental unit of charge. Aliens would have to know that!  Combining the charge of the electron with the speed of light and Planck’s constant h one gets the fine structure constant:

Formula for the fine structure constant

which has a value near 1/137 and is dimensionless.  This is a constant of nature that we do not have a way to calculate purely mathematically or from first principles—it measures the degree to which electrons “couple” to photons, and so governs how all of electromagnetism and light works.

So, can we get another frequency if we multiply the inverse of the Planck time by the fine structure constant. That frequency turns out to still be too high to observe, but there’s no reason we couldn’t keep multiplying by the fine structure constant until we get a useful frequency. This process would actually generate a large number of frequencies—a frequency “comb” with many “teeth.”  Some of the frequencies generated this way are at interesting frequencies: 610.3 nm in the optical, and 26.16 GHz in the microwave, for instance, are both easily observed from Earth and in fact the sorts of frequencies we might use for communication.  These are Planck’s Schelling points!

But there are some caveats here.  Are Planck’s units really that universal?  Look at those equations above: they both have a factor of 2π in them.  Where did that come from?

Well, Planck liked to define things in terms of angular frequency, meaning the time it takes the phase of an oscillator to change by 1 radian. The 2π goes away if you choose to define frequency in terms of cycles per unit time (as astronomers do for light or engineers do for AC electricity).  It’s arbitrary!  So, we could also define both constants without the 2π. Maybe aliens like it better that way?  So we can build a frequency comb that way, and that’s another potential Schelling point.

Also, maybe we’re overcomplicating things.  If we’re going to choose a base to raise to a power then maybe the fine structure constant isn’t the natural one to use—any mathematician will tell you that the natural base to use is the base of the natural logarithm, e! (Different e from the one above, though). Then you can do it all with only 3 physical constants instead of 4, so maybe more obvious? So that’s another potential Schelling point.

Or maybe you want to make sure the frequencies have physical meaning akin to the 21 cm line Shelling mentioned, and as long as you’re thinking about light you might like to use the mass of the electron instead of the gravitational constant G.  In that case you could define your base unit of energy as half of its rest energy of the electron and use the fine structure constant to make your comb.  Seems a bit arbitrary, at first, but the energies defined by that comb are

(m_e c^2)/2 \alpha^n

and when n=2 we have an important unit in physics, the Rydberg, related to the energy it takes to ionize hydrogen (in reality there’s a small correction factor because protons are not infinitely massive, but this is the fundamental unit).  This unit was known even to classical physics and so is a very “natural” way to define a universal energy or frequency.  So there’s yet another frequency comb.

We could surely define more. The truth is, Planck’s insight isn’t all that helpful for guessing exactly which frequencies we should look at because we still need to make lots of choices and we don’t have any guide beyond what seems natural.

But still, it’s a useful illustrations of both the power and limitations of Schelling’s idea. Also, we can add the frequencies that appear in these frequency combs to the lists of “magic frequencies” we check for—more ideas about places to look can’t hurt, because today modern radio observatories can search billions of frequencies at once, so it costs nothing to check a few more more that they observed.

But there may be another insight here: these frequency combs generate multiple frequencies, and perhaps we should look for signals at all of them.  After all, unlike looking for someone in New York, there is little preventing us from looking in more than one channel at once, or from their signals being at more than one frequency at once.  Perhaps we should be looking for combs of signals, or at multiple wavelengths simultaneously!

Anyway, this idea of Schelling points has gained a lot of traction since I made passing reference to it in a review article a while back, but it has no proper, refereed citation in a SETI context (beyond Schelling’s offhand remark in his book).  So I’ve written up the idea formally, including the Planck Frequency Comb as a case study, in a new paper for the International Journal of Astrobiology. You can read it on the arXiv here.


Thanks to Sabine Hossenfelder and Michael Hippke for these translations of Planck’s 1900 paper.

On Meeting Your Heroes

Freeman Dyson died on Friday. He was a giant in science, possibly the most accomplished and foundational living physicist without a Nobel Prize. He was 96.

Tania/Contrasto, via Redux

He had a big influence on my turn to SETI. I’ve written about him several times on this blog, including about his “First Law of SETI Investigations”, his role in the development of adaptive optics, how that intersected with Project Orion and General Atomic, and of course his eponymous spheres that I’ve spent some time looking for.

I got to meet him twice. Once was when Franck Marchis invited him, Jill Tarter, Matt Povich, and me to talk about Dyson spheres on a Google Hangout for the SETI Institute:

The second time was a at UC San Diego. I was there to give a talk, and walking down the hallway of the astronomy part of the physics department I saw “F. Dyson” on one of the doors. I asked, and was surprised to learn that he spent his winters in San Diego where his grandchildren lived, and that he had an office in the department.

And he was there that day.

And he’d be at my talk.

About Dyson spheres.

Indeed, his face was on the second slide.

The talk went well and afterwords he invited me to lunch to discuss it. He asked if I was free. I looked at my schedule: of course I had a lunch appointment. “Yes, it looks like I’m free!” I said, then briefly excused myself to explain the change to my host.

Freeman Dyson and me after my talk at UCSD

Freeman Dyson and me after my talk at UCSD

I asked where we should go and he said “I like Burger King.” So he walked me to the student union where he got a hotdog, and we sat at a table for four, next to a slightly annoyed undergraduate looking at his phone.  We talked about Dyson spheres and SETI, I’m sure. I also could not resist and asked embarrassingly naïve questions about experimental tests of the vacuum energy and the like. “I don’t think that’s a promising line of research” he politely deflected.

I have a list of bands I’ll see if they come to town, and a shorter list of bands I’ll see if they come within driving distance. It’s not a list of my favorite bands, it’s a list of bands that might be on their last tour that I want to have seen at least once. I’ve seen Dylan (twice!), Springsteen (twice!), The Who (Quandrophenia in Philadelphia), Bob Seeger, Rod Stewart, Elton John, Metallica (twice!), the Rolling Stones, Paul McCartney, and more. Cher caught a cold and so they canceled the State College show (I really bought the tickets for Pat Benitar’s opening act, though).

I missed Prince. You never know.

I’ve twice missed talking to my heroes because they were old and I dallied. I invited Nikolai Kardashev to this summer’s SETI Symposium, but I got a decline from someone managing his email account, and then we learned last August that he had passed away at 87.

When I was organizing the letter writing campaign for a prize from the AAS for Frank Kameny, I got his contact information (the phone number at his house). I wanted to call him to tell him what we were doing, but I decided to wait until the prize was official so I could tell him the good news. On August 1, 2011 I learned the AAS was officially going to consider the prize. On October 12, Kameny passed away at 86. On October 15, the AAS announced the prize, which had to be posthumous.

I never called. I’m not sure Frank knew about the effort at all, that his old professional society was finally honoring him.

I’m glad I met Freeman. I’m sad I won’t get his feedback on the big review article on Dyson Spheres that I’ve written that will be published this summer. I probably should have sent it to him earlier.

Observers and Theorists Being Wrong

Once upon in time, probably in graduate school, someone told me an aphorism, which went something like this:

A theorist only has to be right once to garner a reputation as a good scientist, but an observer only has to be wrong once to ruin theirs.

Alex Filippenko—spreader of the aphorism

I asked about it on Twitter and Facebook, and multiple people pointed to Alex Filippenko as the originator (which may also be where I heard it, when I TA’d for him at Berkeley). I asked Alex, and he wrote that he heard it from other grad students at Caltech, perhaps Richard Wade. I asked Richard, and he wrote “I think it was a fairly common expression around Caltech when I was a grad student, so Alex could heard it from me. I probably heard it from other grads.”

So, I’m not sure where it comes from, but it’s a great quote!

Some of Facebook and Twitter objected to the sentiment it expresses— ” I’d say it’s best forgotten. No good comes from playing it safe the whole damn time… 😉” quipped David Kipping. Brian Metzger writes “I think one has to make an important distinction: theory that in principle is well-motivated and has a sound physical basis but just turns out to be the wrong explanation (but might still lead to progress by posing new questions), versus theory that e.g. employs bad physics or already disproven assumptions and couldn’t in principle have been correct. ”

But I think it’s got a kernel of truth worth discussing.

As I’ve drifted into theory from observation I’ve been struck by how much more comfortable theorists are being wrong than observers (sometimes I call this Steinn’s bad influence on me ;).

But it makes sense. Theorists are expected to work on hypotheses that might turn out to be wrong, and there is no discredit in one’s theory turning out to be wrong if it was interesting and spurred work that eventually turned up the right answer. There’s no real reason for an equivalent “you’re doing a good job even if you’re wrong” land for observers.

I think David Kipping and Alex Teachey’s laudable and cautious approach to their exomoon candidate illustrates the divide. As an observation project, especially a high profile one, they must be extra careful not to overstate the evidence, careful to call things “candidates” and not “discoveries”, and careful to emphasize the uncertainty inherent in the problem. Their peers, journalists, and the public will scrutinize their verbiage and they will get blowback if it turns out to not be an exomoon and their presentation of the evidence, in retrospect, was overstated.

But a theoretical analysis of the abundance of exomoons (or exoplanets!) that turns out to be off by orders of magnitude can still get cited favorably a decade later if it included novel and important components. After all, everyone understands that theory is hard and that we build theories up piece by piece, and so we’ll get it wrong many times before we get it right. And so such work rarely includes the careful hedging that Kipping and Teachey used in their work.

Or, to give a more dramatic example: If inflation turns out to be completely wrong, the theorists who dedicated their careers to it will still be considered good theorists, but the BICEP2 team that got a subtle issue with dust wrong have a whole book about the very public and embarrassing debacle that followed their (incorrect) detection of sign of inflation in the CMB.

I’m not saying this dichotomy in unfair or inappropriate—on the contrary, I think it’s appropriate!—I’m just pointing out how the aphorism resonates because it identifies something real and tacit about the way we judge science.

Freeman Dyson’s First Law of SETI Investigations

It’s come up a few times, so let me state here for the record the origin of Freeman Dyson’s First Law of SETI Investigations:

It’s from an email he sent me. We were discussing a paper of mine in anticipation of an outreach event we were planning:

and he remarked of the Ĝ strategy:

I am happy to see that your plan is consistent with the First Law of SETI investigations: every search for alien civilizations should be planned to give interesting results even when no aliens are discovered.

I asked permission to repeat this, and he agreed. It’s consistent with his general approach about SETI, searching for the physical limits of technology in a way that also generates ancillary science and makes minimal assumptions about agency.

I think Freeman himself means this as a counterpoint to radio or laser SETI, which has the benefit of working against low natural background but this apparent disadvantage that they are unlikely to discover new natural phenomena in the course of their searches. I think this perspective is often overstated—radio SETI is closely aligned with pulsar and FRB astrophysics, and generates great science along the way, and there are natural sources of very brief optical flashes, too.

Justifying Science Funding in an Unjust World

I was asked recently a stock question by interviewers about how to justify spending SETI when “some people would say” that we have so many problems needing solving , so many better places to spend the money.

There are a few answers to this.

One is that it’s a false choice: certainly if I had to choose between NSF funding for basic research, including SETI, and feeding starving people, I choose feeding the starving people every time. But that is not the choice we face: humanity produces more than enough food to feed the planet and cutting the NSF budget won’t feed any starving people.

Similarly, if I had to choose, I’d rather our government guarantee all people the basics of modern life—shelter, health care, safety, clean water and nutritious food, life, liberty, and the pursuit of happiness, and all that—than search the skies for technosignatures. But that’s not the choice Congress makes every year when it makes up its budget—we can easily afford all of those things.

Another answer is beautifully illustrated by classic Congressional testimony by R. R. Wilson, director of Fermilab, justifying building the lab’s first accelerator in a time when the national defense dominated the budget conversation:

R. R. Wilson

SENATOR PASTORE. Is there anything connected in the hopes of this accelerator that in any way involves the security of the country?

DR. WILSON. No, sir; I do not believe so.

SENATOR PASTORE. Nothing at all?

DR. WILSON. Nothing at all.

SENATOR PASTORE. It has no value in that respect?

DR. WILSON. It only has to do with the respect with which we regard one another, the dignity of men, our love of culture. It has to do with those things.

It has nothing to do with the military. I am sorry.

SENATOR PASTORE. Don’t be sorry for it.

DR. WILSON. I am not, but I cannot in honesty say it has any such application.

Senator John Pastore, dunkee

SENATOR PASTORE. Is there anything here that projects us in a position of being competitive with the Russians, with regard to this race?

DR. WILSON. Only from a long-range point of view, of a developing technology. Otherwise, it has to do with: Are we good painters, good sculptors, great poets? I mean all the things that we really venerate and honor in our country and are patriotic about.

In that sense, this new knowledge has all to do with honor and country but it has nothing to do directly with defending our country except to help make it worth defending.

“It helps make [America] worth defending”—quite the rhetorical dunk there.

But more broadly, basic science is an essential part of culture, civilizations, and humanity. We have more than enough labor and wealth to do SETI and be safe. Indeed, we spend hundreds of billions per year on national security—a future that President Eisenhower warned us against—and against this, basic science expenditures are a rounding error.

The third answer is: because we can easily afford it.

I once heard a figure that America spends more on doggie treats than on publicly funded science.  I threw this figure out in that interview, but worried I had it wrong.  So I looked it up.

The US pet food and treat market is almost $30 billion. Of this, dog and cat “treat” sales reached $4.39 billion in 2017. The FY17 appropriation for NASA’s science mission directorate was $5.76  billion.

So, I was wrong: NASA spent 30% more on science in 2019 than America did on dog and cat treats than. But we spend far more on dog and cat food.

But looking at some other points of comparison:

Since America spends over $2.5 billion on dog treats each year, its probably safe to say that Americans spend about twice as much on doggie treats than their federal taxes do on astronomy.

The point, obviously, is not that we spend too much on doggie treats or that there is some obviously more correct ratio between these two expenditures—it’s that America is a very, very rich country (even ignoring the “1%” that doesn’t spend much on pet treats) and the amount we spend on things as important to culture as basic science is actually quite small, comparable to niche consumer markets like pet treats.

I’m not arguing we should spend less on basic human needs, pet treats, or any of these other things that define modern life. Regardless of whether we fund science with new taxes or by cutting other expenditures like the military budget, we can easily afford to spend a lot more on a lot of those things, including SETI.


The hats astronomers wear

Once upon a time science departments in universities often had draftsmen on staff that would produce figures for scientific publications. Today, those positions are much rarer (and called “graphic designers”) except in very large institutions because scientists themselves are expected to do much of that work. Other work routinely done by administrative staff in the past—like travel reimbursements—are now done by faculty themselves.

Part of this is that computerization and other technologies have made these tasks easier and so it’s not unreasonable to expect a typical scientist to do the job quickly and competently. But it also means that the modern astronomer (queue Gilbert & Sullivan “I am the very model of a…”) has to do a wide variety of tasks outside of their training.

Today in group meeting we tried to make a list. This whole exercise was inspired by this Jonathan Fortney tweet thread about “scientists” vs. “engineers”:

and about giving yourself permission to not be interested in certain parts of the job of scientist that other people find endlessly fascinating

My punchline is that it’s fine to love and get really good at a few of the aspects of being a scientist, and that it’s OK to not be good at or enjoy other aspects.

Indeed, I sometimes chafe when people say things like “every scientist has an obligation to communicate their science to the public” or “all astronomers should learn Python” and such. I think its fine and good that astronomers specialize in different parts of the job, and that collaborations consist of a group of people that, together, have all the pieces needed to do great science.

Also, part of my definition of a good job is one where you spend most of your time doing things you both like and are good at, and a minimum of time doing things you dislike and are bad at. Enumerating the parts of the job can help you find the job you like (or turn your current job into that). So it’s OK not to be good at everything.

Here’s the list of “hats” astronomers wear that we came up with.  What did we miss?

  • Teacher / instructor
  • Science popularizer
  • Public ambassador of science
  • Salesperson
  • Writer
    •  proposals
    • research articles
    • emails
    • popular materials
    • journalism (e.g. press releases)
  • Copyeditor
  • Graphic(s) designer
  • Examiner (tests, defenses)
  • Peer reviewer
  • Mentor
  • Research adviser
  • Manager
  • Administrator
    • Travel
    • Grants administration
    • Budgeting
  • Computer programmer
    • Team coding
    • Public code
    • “Private” code
  • Computer systems administrator
  • Web developer
  • Marketer
  • Engineer
  • Physicist
  • Theorist
  • Observer
  • Data analyst
  • Statistician
  • Philosopher of Science
  • Ethicist

[Update: good suggestions from Twitter:




Atmospheric scientist and meteorologist


A Second Pleiades in the Sky

Astronomers have discovered a second Pleiades in the sky.

Wait, what?  The Pleiades are an obvious feature of the night sky, known to ancient peoples around the world and a common test of visual acuity.  In a telescope they look like this:

The Pleiades

Four or five bright stars make them obvious in even a moderately dark sky, and the nebulosity adds a nice touch that makes them interesting. Astronomers like them because they were all formed from the same birth cloud, so they share composition, age, and distance. This makes for a great stellar laboratory, since it lets us isolate and focus on the few differences among the stars, like their mass and whether they have binary companions.

So if these are so obvious, how could there be another one?  Answer: if they’re so spread out across the sky that no one noticed they were related!  Back in February Meingast et al. announced that they had spotted a group of hundreds of stars all moving in the same direction—but they were spread all out against the sky.  If you plot most of the sky out flat, here’s what they look like (in red):

The Pisces-Eridanus Stream. The Milky Way, which spans 360 degrees of the sky, appears as a circle in this projection. The stream is over 120 degrees across.

Co-moving stars like this are like clusters but more spread out. It took the precise measurements recently announced by the Gaia team to notice that these stars were all moving in the same direction.

Many of these stars, like the Pleiades are bright enough to see with the naked eye:

When Meingast et al. published, I got excited because they guessed that the stream was 1 billion years old—if correct, that would make it one of the closest older clusters in the sky. Jason Curtis had spent a whole PhD dissertation and more proving Ruprecht 147 was a true cluster, 3 billions years old and only 300 parsecs away. Eunkyu Han worked hard to check out another claimed nearby old cluster, Lodén 1.  Here was one 3 times younger and 3 times closer: another important discovery! I got excited and tweeted at Jason about it.

We started wondering what to call it, and also getting suspicious of the reported age. So we pulled in the expert on “moving groups” and stellar ages, Eric Mamajek:

Jason, Eric, and I then took the conversation offline. First, it needed a name.  Eric came up with the “right” answer:

Next, the stream seemed like it had to be younger than 1 billion years to us, but how old was it?  Then Jason Curtis went to town with TESS, the all-sky planet hunting telescope. It had already hunted for planets around many of the stream’s stars, and Jason was able to quickly measure the stars’ rotation periods.  He tells the story here:

I encourage you to read the whole tweet thread!

Basically, Jason was able to show that the stars in the stream are spinning way too fast to be 1 billion years old. In fact they were spinning just as fast as the Pleiades—so they are probably almost exactly the same age.  They’re also the same distance, and there are just as many of them!

I was actually kind of disappointed:

Chris Lintott was grumpy at my framing of the cluster as a “second Pleiades” because the public would understand that to mean “another thing I can see with my eyes in the sky that wasn’t there before”, which is misleading:

His point is well taken, but Heidi Hammel would say that good science communication means linking to things people know about.  Eric explained well what I meant and why the discovery is a big deal:

This discovery came very fast, and was only possible because of the hard work of the teams that made Gaia, Kepler, and TESS possible.  Because those are all-sky surveys committed to making data public as fast as possible, unexpected gems like this can go from tweets to papers in a matter of weeks.  Jason Curtis pointed out that most of the actual science only took him hours:

It’s a new era of stellar astronomy. So exciting!

In Defense of Magnitudes

Astronomical magnitudes get a bad rap.

Hipparchos 1.jpeg

Hipparchus, supposedly

The Greek astronomer Hipparchus famously mapped the sky and assigned each star a “magnitude” (or size) based on its apparent brightness.  The human eye is a surprisingly precise photometer (you can with just a little effort estimate brightnesses differentially to about 0.1 magnitude; I’m sure dedicated amateurs can do better) So Hipparchus could have been thorough about this, but he was actually quite general: he just lumped them into 6 categories: “stars of the first magnitude” (the brightest), “stars of the second magnitude” and so on.

But while the human eye is precise it’s not linear: it’s actually closer to being a logarithmic detector. This gives it a great dynamic range but it means that what seems to be “twice as bright” is actually much, much brighter than that.

In 1856 Norman Pogson formalized this in modern scientific terms by proposing that one magnitude be equal to a change in brightness of the fifth root of 100, with a zero point that roughly aligned with Hipparcus’s rankings so that “first magnitude stars” would have values around 1. This captured the logarithmic scale and spirit of the original system, and has frustrated astronomers ever since.

Astronomers regularly complain about this archaic system. A lot of this comes from trying to explain it in Astronomy 101 or even Astronomy 201 where our students expect a number attached to brightness to increase for brighter objects, and where we have to teach them a system literally no other discipline uses. Especially at the Astronomy 101 level, where we are loathe to use logarithms, we often skip the topic altogether.

But I think astronomers don’t realize how good we have it.

First of all, the scale increases in the direction of discovery: there are very few objects with negative magnitudes (the Sun and Moon, sometimes Venus, a few stars in certain bands) but lots of objects up in the 20’s where the biggest telescopes are discovering new things. Big numbers = bigger achievements is much better than “we’re down to -10 now!”, in my opinion.

Secondly, the numbers have a nice span. The difference between 6 and 7 is just enough to be worth another number. This is because the fifth root of 100 is only 8% larger than the natural logarithmic base e, which is the closest thing we have to a mathematically rigorous answer to the question “how much is a lot?”.

But most importantly, the system is a beautiful compromise between simplicity and precision that allows for very fast mental math and approximations for any magnitude gap.

This is because we long ago settled on base-10 for our mathematics, and the magnitude system is naturally in base 10. 15 magnitudes is a factor of 100,000, because every 5 magnitudes is exactly 100.  2.5 magnitudes is a factor of exactly 10.

It doesn’t take much practice to get very fast at this. If we used, say, e as the base instead, the 8% difference would compound with each magnitude.  exp(15) is 3.6 times larger than 100,000.

Finally, and most importantly IMO, because this interval is very close to a factor of e, we get the lovely fact that very small magnitude differences translate pretty well to fractional differences.  So, a change of 0.01 magnitudes is almost exactly 1% (only 8% off, actually). That’s so useful when trying to do quick mental estimates.  For instance: a transiting planet with a 10 mmag depth covers 1% of the star, so it has 10% of the star’s radius (since sqrt(0.01) = 0.1). A 1 mmag transit therefore corresponds to 10x less surface covered, so it has 3% of the star’s radius. Easy!

I think of it as akin to the twelfth-root-of-two intervals on an equal-tempered instrument. No interval on such an instrument produces the mathematically perfect 3:2, 4:3, or 5:4 harmonic, but they’re all close enough and in exchange you can transpose music and shift keys with ease and without loss of musical fidelity. The pedants may complain, but it’s worked great for centuries.

Do NASA and the NSF support SETI?

Does the federal government support SETI?  We usually say it does not, but in the 2019 audit of the SETI Institute, there is a letter from NASA protesting this characterization.  It contains this language:

The OIG’s statement on the absence of NASA’s funding for SETI research is misleading and the finding incorrect.1 NASA has funded the development of several instruments that enable such searches

Michael New made a similar point at the Houston NASA Technosignatures workshop: NASA has funded some SETI work since 1993 (including that workshop itself).

The footnote in the text above mentions 3 grants explicitly, but I think they missed one. Working with Jill Tarter and others I’ve tried to count every NASA and NSF grant for SETI work since 1993. I don’t know of any from the ’90’s, but, as the report states, there are some in the past 15 years.

Here’s my list:


  • “A 2 Billion Channel Multibeam Spectrometer for SETI” 2 years, $398,040 (PI: Marcy, NRA-01-OSS-01-ASTID)
  • “Arecibo Multibeam Sky Survey for Direct Detection of Inhabited Planets,” 4 years, $485,642   (PI: Korpella, Exobiology 2008 NNH08ZDA001N-EXOB) Money funded running the SERENDIP IV survey and SETI@Home
  • “Detection of Complex Electromagnetic Markers of Technology,” 3 years (+ 1 year no-cost extension), $660,079 (PI: Jill Tarter NRA-OSS-01-ASTID). Money funded studies by Cullers, Stauduhar, Harp, Messerschmitt, and Morrison on using autocorrelation and other methods for detecting broadband SETI signals.
  • “Instrumentation for the Search for Extraterrestrial Intelligence,” 3 years, $590,589 (PI: Werthheimer, ASTID 2011 NNH11ZDA001N-ASTID) Money funded building an instrument at Arecibo/GBT.


  • AST-0808175 :  Radio Transient and SETI Sky Surveys Using the Arecibo L-Band Feed Array  $362,624.00 (PI: Wertheimer, NSF-AST 2008 )
  • AST-0838262:  Collaborative Research: The Allen Telescope Array: Science Operations
    $310,000.00 (PI: Tarter, NSF-AST 2008 )
  • AST-0540599:  Collaborative Proposal: Science with the Allen Telescope Array
    $300,000 (PI: Tarter, NSF-AST 2005)
  • AST-0243040 :  Multipurpose Spectrometer Instrumentation for SETI and Radio Astronomy
    $704,080.00 (PI: Marcy, NSF-AST 2002)
  • OAC-0221529:  Research and Infrastructure Development for Public-Resource Scientific Computing
    $911,264.00 (PI: Anderson, NSF-OAC 2001)

Total since 1993: $2,587,968 (NASA) + $2,134,350 (NSF) = $4,722,318

Wow! $4.7 million!  That’s a lot, right?

Well, not really. That means that since 1993, the entirety of federal grant spending on the topic is about $180,000/yr, which, after indirect costs, supports 1-ish FTE (i.e. one scientist/engineer).  So one person at a time.

Now maybe that’s not fair, and we should count from 2001, when the first of these grants began.  Then it’s $262,000/yr, so we’re up to maybe 1.5 FTEs.

So, while it’s technically true that NASA has supported SETI for decades, the amount we’re talking about is so small that it’s not really a rebuttal to the reality of the situation, which is that the government doesn’t adequately fund SETI.  Why not?  The letter in the audit gives a reason:

NASA sets its priorities by following the recommendations of the National Academies of Science, Engineering, and Medicine while simultaneously implementing national priorities established by the President and Congress. SMD will continue to evaluate technosignatures research in the context of the Directorate’s overall portfolio through its standard scientific prioritization process.

This is mealy-mouthed, but the bottom line is that SETI funding is not a high priority in the NASA authorization bills or in the 2000 or 2010 Decadal reviews, so NASA doesn’t feel that it needs to fund it.

Now, this isn’t really a great excuse—the Decadal reviews do say that SETI is good and worth pursuing (even if they don’t recommend funding), and there’s nothing preventing NASA from including SETI under the astrobiology umbrella (which is a field it’s required to pursue).

Indeed, NASA is very inconsistent about whether SETI is allowed to be funded via grants—contrast its protest above that yes it does too fund SETI with Jill Tarter’s exploration of how SETI is/isn’t allowed in various NASA calls through the years here.

The bottom line is that what SETI needs is an explicit recommendation for funding in this upcoming Decadal process and/or explicit mention of technosignatures as an authorized expenditure for NASA and the NSF by Congress.  Here’s hoping that the winds really are changes and that we’ll get both in the next couple of years!

Galactic Settlement and the Fermi Paradox

The Fermi Paradox is the supposed inconsistency between the ease with which a spacefaring species could settle the entire Milky Way given billions of years and the fact that they are not obviously in the Solar System right now.

This, original form of the paradox was formulated most trenchantly by Michael Hart (more on him in Section 2.2 here) who called the lack of extraterrestrial beings or artifacts on Earth today “Fact A”. He showed that most objections to his conclusion stem from a lack of appreciation for the timescales involved (it takes a small extrapolation from present human technology to get interstellar ships, and even slow ships can star-hop across the Galaxy in less than its age) or what I’ve called the monocultural fallacy (positing a common behavior to all members of all extraterrestrial species, forever).

William Newman and Carl Sagan wrote a major rebuttal to Hart’s work, in which they argued that the timescales to populate the entire Galaxy could be quite long. In particular, they noted that the colonization fronts Hart describes through the Galaxy would move much more slowly than the speed of the colonization ships. They also argue that long-lived civilizations are anti-correlated with rapidly-expanding ones, and so they conclude that civilizations with very slow population growth rates are necessarily very slowly expanding. They conclude the Galaxy could be filled with both short-lived rapidly expanding civilizations that don’t get very far and long-lived slowly expanding civilizations that haven’t gotten very far—either way, it’s not surprising that we have not been visited.

We rebutted many of these claims in our paper on the topic. In particular, we argued that one should not conflate the population growth in a single settlement with that of all settlements. In particular, there is no reason to suppose that colonization is driven by population growth, resource depletion, or overcrowding, or that a small, sustainable settlement would never launch a new settlement ship. One can easily imagine a rapidly expanding network of small sustainable settlements (indeed, the first human migrations across the globe likely looked a lot like this).

Jonathan Carroll-Nellenback

Once this constraint is lifted, a second consideration makes Newman & Sagan’s numbers smaller. Most of the prior work on this topic exploit percolation models, in which ships move about on a static substrate of stars, but real stars move. Many of these papers also assume that the entire network of settlements have a similar behavior, and some posit they all might suffer a simultaneous culture shift away from settlement.

Jonathan Carroll-Nellenback at the University of Rochester with Adam Frank, and in collaboration with Caleb Scharf and me, has just finished work on analytic and numerical models for how a realistic settlement front would behave in a real gas of stars characteristic of the Galactic disk in the Solar Neighborhood.

The big advances here are a few:

  1. Jonathan has worked out an analytic formalism for settlement expansion fronts and validated it with numerical models for a realistic gas of stars
  2. Jonathan has accounted for finite settlement lifetimes, the idea that only a small fraction of stars will be settle-able, and explored the limits of very slow and infrequent settlement ships
  3. Jonathan has not assumed that settlement lifetimes or settlement behaviors are correlated. Rather, he assumed a simple, conservative set of parameterized rules for settlement and explored settlement behavior as a function of those fixed parameters.

In particular, the idea that not all stars are settle-able is important to keep in mind. Adam calls this the Aurora effect after the Kim Stanley Robinson novel in which a system is “habitable, but not settle-able.”

The results are pretty neat. When we let the settlements behave independently, Hart’s argument looks pretty good, even when the settlement fronts are pretty slow.  In particular, one can have very limited range (no faster than our own interstellar ships but lasting a million years, or faster ships that can only travel about 1pc) and still settle the entire Galaxy in less than its lifetime because the front speed becomes limited by the speed of the stars, which carry settlements into range of new stars regularly and naturally diffuse throughout the Galaxy.

Jonathan explores a few regimes where Earth would not have been settled yet. He finds that it doesn’t take much—just a single settlement front with modest ship ranges and launch rates—to populate the entire Galaxy in much less than a Hubble time.

Also neat, is that Jonathan explores regimes where they have been here, but we just don’t notice because it was so long ago.  Adam and Gavin Schmidt explored this possibility in their Silurian Hypothesis paper, and I did something similar in my PITS paper. The idea is that “Fact A” only applies to technology that has visited very recently or visited and then stayed permanently. Any technology on Earth or the Solar System that is not actively maintained will eventually be destroyed and/or buried, so we can really only explore even Earth’s history back in time for of order millions of years, and not very well at that.

So really, the question isn’t “has the Solar System ever had a settlement” it’s “has it been settled recently”.  Jonathan shows that there is actually a pretty big region of parameter space where the Solar System is amidst many settled system but just hasn’t been visited in the last 10 million years.

Of course, there are still lots of other reasons why we might not have been permanently settled by a Galactic network of settlements—as we note in the paper:

Hart’s conclusions are also subject to the assumption that the Solar System would be considered settleable by any of the exo-civilizations it has come within range of. The most extravagant contradiction of this assumption is the Zoo Hypothesis (Ball 1973), but we need not invoke such “solipsist” positions (Sagan & Newman 1983) to point out the flaw in Hart’s reasoning here. One can imagine many reasons why the Solar System might not be settleable (i.e. not part of the fraction f in our analysis), including the Aurora effect mentioned in Section 1 or the possibility that they avoid settling the environment near the Earth exactly because it is inhabited with life.

In particular, the assumption that the Earth’s life-sustaining resources make it a particularly good target for extraterrestrial settlement projects could be a naive projection onto exo-civilizations of a particular set of human attitudes that conflate expansion and exploration with conquest of (or at least indifference towards) native populations (Wright & Oman-Reagan 2018). One might just as plausibly posit that any extremely long-lived civilization would appreciate the importance of leaving native life and its near-space environment undisturbed.

So our results are a mixed bag for SETI optimists: Hart’s argument that settlement fronts should cross the whole Galaxy—which is at the heart of the Fermi Paradox—is robust, especially because of the movements of stars themselves which should “mix” the Galaxy pretty well, preventing simply connected “empires” of settlements from forming.  If Hart is correct that this means we are alone in the Galaxy, this is actually very optimistic for extra-galactic SETI, because it means other Galaxies with even a single spacefaring species should rapidly become endemic with them. Indeed, our analysis did not even include any effects like halo stars or Galactic shear which will make settlement timescales even faster.

On the other hand, there are a lot of assumptions in Hart’s arguments that might not hold, in particular that if the Sun has ever been in range of a settled system that “they” would still be here and we would know it. Perhaps Earth life for some reason keeps the settlements at bay, either because “they” want to keep it pristine or it’s just too resilient and pernicious to permit an alien settlement from surviving here. Is Earth Aurora?

The paper is here.

SETI is a very young field (academically) Part II

In a previous post, I discussed the five PhD dissertations focused on SETI (ever!) and mentioned that I could not track what had become of one of their authors, Darren Leigh.  Well, it turns out I should have just asked!

Darren was kind enough to email me with the details of his degree and his thoughts on the merits of a degree in SETI, Paul Horowitz as an adviser, and his career path since then.

I’ve updated my previous post to reflect his input. Below is his email to me, which he kindly allowed me to reproduce here.

Darren Leigh, the first person to write a doctoral thesis focused on their search for extraterrestrial intelligence.

Hi Jason,

A friend stumbled onto this post of yours and sent me the link.

I didn’t think I would be that hard to find. :-)

At the time I did my dissertation, I was told that it would be the world’s first on the subject of SETI. A couple of previous astronomy dissertations had contained a chapter on SETI, but did not have it as the main topic. The fact that I had done a bachelor’s and master’s in EE at MIT (with some physics background) probably made this easier than it would have been for a real physics major looking for a career track in academic astronomy. (Note that my PhD says “Applied Physics”, and is from the Division of Engineering and Applied Sciences, and not the Physics Department).

The real pull of doing SETI was working for Paul Horowitz at Harvard. I was actually in the early stages of a PhD program at MIT when I met Paul and decided to move up the street to work with him. Paul always prided himself on being a generalist, rather than a narrowly-focused academic. Note the wide range of things that he works on, including the amazing “Art of Electronics”. Those of us in the Horowitz lab were amused when Ernst Mayr complained about what a waste SETI was, both in terms of resources as well as in terms of the professional lives of Paul’s students. I think Paul’s students have all done pretty well, taking a more generalist approach than many doctoral recipients.

I’ve been doing corporate-type R&D since I defended, and my SETI background has served me well in areas from electronics to signal processing to satellite communications to marketing and public relations. [I spent a lot of time with camera crews and the press around 1995 due to the SETI work and the (then recent) discovery of 51 Pegasi b.]

Jonathan Weintroub, another of Paul’s PhD students who defended the same year that I did and also an EE, was doing actual astronomy, looking for highly red-shifted hydrogen. A lot of the work we were doing overlapped. He now works for the Harvard-Smithsonian Center for Astrophysics on the Submillimeter Array.

Ian Avruch was a doctoral student of Bernie Burke, but hung around the Horowitz lab a lot because he was also looking for highly-redshifted hydrogen and could actually get stuff built there. He’s a real physicist and has done a lot of professional astronomy since. I believe that he is at the European Space Agency now.

Chip Coldwell (on your list) was a physics major, but has spent most of his professional life doing software/computer stuff, and is now apparently moving into RF hardware. You can check with him yourself, but I don’t think he was doing astronomy research after his PhD, even though he has worked for such astronomers. He spent a lot of time at Red Hat and is now at MIT Lincoln Lab.

Of the other Horowitz students on your list, Andrew Howard had been a physics major and got a physics PhD and is now a professor of astronomy at CalTech. Curtis Meade was (I believe) an EE, who got his PhD in “Applied Physics” at the School of Engineering and Applied Sciences, like I did. I don’t know what he’s up to now.

I can’t think of any of Paul Horowitz’s doctoral students who has had professional problems. I guess Mayr was used to narrowly-focused grad students who could be ruined if they weren’t trained exactly right for academia. Paul took in both EEs and physicists and made us all better at both of those things, as well as turning us into skilled and pragmatic researchers.

As far as wasted money and resources go, SETI is cheap. I think people believe that it is expensive because they associate it with “space” and that with NASA and it’s enormous budgets. There’s a good chance that the press spent more money covering our SETI work than we spent actually doing it.

Me? I’m currently a VP at (and one of the founders of) Tactual Labs. We do advanced human-machine interaction, especially high-performance capacitive sensing systems. I’ve been working in R&D shops for my entire professional career. After finishing my doctorate, I spent ten years at Mitsubishi Electric Research Labs, coming up with new IP and product ideas. That lab was magical and very influential, and many alumni went off to professorships at MIT, Harvard and other prestigious universities, as well as to corporate R&D labs at Microsoft and Google.

SETI is a very young field (academically)

[Note: This is a “living” post which I update periodically as I learn about people who have done graduate work in the field. If I’m missing a name please email me.]

SETI is not a field that has a large presence in academia, especially in terms of graduate education. Indeed, there are only two regularly numbered graduate courses in the world on the topic that I’m aware of (at Penn State and UCLA).

Because of this, it’s hard to get a PhD while having the primary focus of your dissertation be searching for technological extraterrestrial life. In fact, so far as I can tell (speaking with many of the people in the field) it’s only been done eight times:

  1. Darren Leigh (1998, Horowitz, thesis)
  2. Stephen Brown (2000, Dixon & Kraus, thesis)
  3. Charles Coldwell (2002, Horowitz, thesis)
  4. Andrew Howard (2006, Horowitz, thesis)
  5. Andrew Siemion (2012, Bower & Werthimer, thesis)
  6. Curtis Mead (2013, Horowitz, thesis)
  7. Ian Morrison (2017, Tinney, thesis)
  8. Emilio Enriquez (2019, Falcke)

Paul Horowitz, SETI PhD adviser extraordinaire.

Until 2017, Paul Horowitz was responsible for supervising 2/3 of all doctoral SETI dissertations! Thanks, Paul! Of these eight, four are professional astronomers today, Mead is at Apple, Coldwell works in a astronomy-related industry, Brown is apparently a scientist at Harris Corporation, and Darren Leigh describes his career here.

I’m also aware of some terminal master’s degree on the topic (many are EE degrees related to the Argus SETI array):

  1. Dennis Cole (1976, Dixon & Kraus, thesis)
  2. Jim Bolinger (1988, Dixon & Kraus)
  3. Hyung Joon Kim (1999, Ellingson & Burnside)
  4. Tom Alfernik (2000, Ellingson & Burnside)
  5. Emarit Ranu (2000, Ellingson & Burnside)
  6. Amy Reines (2002, Marcy & Cool)
  7. Mikael Flodin (2019, Mattsson, thesis)
  8. Andreea Dogaru (2019, Kerins & Breton, thesis)

This is not to say that no other graduate students have done work on the topic. Here are a few of the (presumably many) theses that had a significant SETI component:

  1. Maggie Turnbull
  2. Jayanth Chennamangalam
  3. Hayden Rampadarath
  4. Kimberley M. S. Cartier
  5. Branislav Vukotic

And there has also been a lot of doctoral work in the social sciences studying SETI itself, for instance in this thesis by Daniel Romesberg and the ongoing work of Claire Webb.

I’m also aware of three current graduate students who have or have planned for major (50-100%) components of their dissertation work to be searching for intelligent life in the universe:

  1. Sofia Sheikh (J. Wright)
  2. Paul Pinchuk (Margot)
  3. Bryan Brzycki (Siemion/dePater)

And three more with at least a portion of their thesis about SETI:

  1. Gerry Zhang (Siemion/dePater)
  2. Maren Cosens (S. Wright)
  3. Neda Stojkovic
  4. Daniel Giles (Walkowicz)

So the number of thesis is poised to go up by at almost 100% in the next few years! This is (weak) evidence of what certainly feels like a resurgence in the field. Still, these numbers are tiny compared to the perception of the amount of SETI work being done, and illustrate how young the field really is, despite the nearly 60 years that have elapsed since its inception.

‘Oumuamua, SETI, and the media

Avi Loeb

Avi Loeb is the chair of the astronomy department at Harvard, a distinguished and well cited astronomer (he has an h-index of 87), and the chair of the Breakthrough Starshot initiative. He’s a strong proponent of making sure that science doesn’t succumb to groupthink and champion of outré ideas.

He also has been making headlines recently for articles he has co-authored, interviews he has given, and popular media columns he has written about the possibility that fast radio bursts, and now ‘Oumuamua, are artificial in origin. This has created a great deal of buzz in popular culture and a lot of hand-wringing and criticism on social media by scientists who find his actions irresponsible. Many have asked my opinion, so I’m collecting my many thoughts on the topic in this post.

I am happy to defend Avi on these grounds:

  • He is driving us to have an important conversation about what “acceptable” SETI research looks like, and in this conversation I’m mostly on his side. He’s essentially moving the scientific equivalent of the “Overton Window” towards SETI, and that’s a good thing. These are exciting and interesting questions and we should not let the face-on-Mars/Ancient-Aliens/UFOlogy types prevent us from discussing them.
  • He is using tenure and his stature the way we all imagine it’s supposed to be used: as a shield so that he can explore potentially unpopular research avenues without fear of retribution or ostracism. We all imagine that’s what we would do in his position (I hope!) but too often it ends up just being a club to get junior scientists to conform to one’s vision for what “proper” science looks like and what “good” problems are.
  • The papers he and his postdocs are writing are important first steps in making Solar System and other forms of SETI a serious academic discipline.
  • He is being a role model for how scientists can explore outré ideas and spend an appropriate amount of their time on potential breakthroughs.
  • He is putting SETI in the public eye and doing a lot of outreach.

Avi wouldn’t be pushing the envelope hard enough if he weren’t getting some pushback, and indeed there is plenty of fair and good-faith criticism that can be made about his approach (not all of which I agree with):

  • The degree of certainty he expresses in ‘Oumuamua being artificial does seem unwarranted to me (though to be fair I’ve always been an ‘Oumuamua-might-be-artificial skeptic.)
  • Given the way we know the press (especially the yellow press) will handle any story about “aliens”, one can argue that the “extraordinary claims require extraordinary evidence” maxim is especially applicable to SETI (I’ve made this argument strongly when discussing my own research in the press.) Avi could hew more closely to this maxim.
  • The tone of his papers and his public comments are quite divergent. The body of the paper on ‘Oumuamua-as-lightsail, for instance, has a brief mention about the potential of the artifice of ‘Oumuamua at the end, but most of it is about the perfectly general problem of thin objects in interstellar space. Snopes highlights this divergence well pointing out that the paper is quite sober and restrained compared to some of the media coverage. (It’s true that the title and abstract of the paper are about ‘Oumuamua specifically, and that it serves as the case study for the whole analysis.) Avi’s public statements are much less conservative and equivocal.
  • He is not just quietly following the evidence; he is using his platform to have a very public and high-visibility discussion about his research. I will concede that Avi is an exception to my earlier (somewhat petulant) protest that SETI scientists are not in it for the attention. That said, I will object to anyone who would claim Avi is only in it for the attention, or that such attention is inherently a bad thing.
  • Many of his papers are de novo explorations of topics like the fate of comets in interstellar space, with little connection to the substantial amounts of work that has already been done on the topic, and his papers would be better and less naive if they had a closer connection to this prior work rather than starting from scratch.

More broadly, let’s look at two threads on Twitter criticizing Avi. I’ll start with this one by Bryan Gaensler:

Bryan makes the rather Popperian argument that if your model is too flexible then it can’t be falsified, so you’re not doing science.  The implication is that since we don’t have a good model for aliens, we can always play the “aliens of the gaps” game and so SETI isn’t good science unless it’s looking for unambiguously artificial signals like narrow-band radio waves.

This argument isn’t as tight as it seems. Most interesting new theories start without concrete predictions—General Relativity was so hard to use that even Einstein wasn’t sure what it predicted (he got the deflection of starlight wrong the first time he calculated it; he wrote a paper saying gravitational waves don’t exist). Theories don’t spring fully-formed from theorists’ heads; many important breakthroughs start with something less than quantitative or precise (“maybe we need to modify gravity”; “maybe there is a new subatomic particle involved”) and let the data guide the theories’ details.

This is the normal progression of science. SETI is no different, and so no less scientific.

Then there is this one, by Eric Mamajek, which I mostly agree with:

It’s mostly fine through tweet #9, but then he conflates things in the last tweet using an unwarranted leap of logic.

Up until then he had been criticizing the Holmesian logic of how ‘Oumuamua must be alien because we had ruled out natural explanations. I quite agree with him.

But in the last tweet he jumps to criticizing even bringing up the hypothesis of ETI’s in general, implying that scientists who do are pulling a Giorgio Tsoukalos. (There’s also the assertion at the end such anomalies will “inevitably” turn out to be not just natural, but mundane, which is obviously not strictly true.)

But Tabby and I weren’t pulling a Tsoukalos when we submitted our proposal with Andrew Siemion to NRAO to study Tabby’s Star. We really weren’t. I have clarified the actual events with Eric, so I’m pretty sure that’s not what he meant to imply here, but that is how this tweet reads.

Bryan makes a similar (but softer) implication in his final tweets:

We all would! Indeed, Avi Loeb suggested that Breakthrough Listen point Green Bank at ‘Oumuamua1 because he understands very well that the proof of alien technology is something like the bullets on Bryan’s list.

But the implications of these tweets aren’t just wrong, they’re harmful to the field of SETI. A very plausible path to SETI success will be that we will see something strange (not “Eureka!” but “That’s funny…” as the old fortune quip goes) and eventually, after lots of follow up, we might find the smoking gun, or perhaps it will just end up being a proof by exclusion.  As I wrote in 2014:

Artifact SETI can thus proceed by seeking phenomena that appear outside the range that one would expect natural mechanisms to produce. Such phenomena are inherently scientifically interesting, and worthy of further study by virtue of their extreme nature. The path from the detection of a strange object to the certain discovery of alien life is then one of exclusion of all possible naturalistic origins. While such a path might be quite long, and potentially never-ending, it may be the best we can do.

Communication SETI, on the other hand, shortcuts this path to discovery by seeking signals of such obviously engineered and intelligent origin that no naturalistic explanation could be valid. Together, artifact and communication SETI thus provide us with complementary tools: the most suspicious targets revealed by artifact SETI provide the likeliest targets for communication SETI programs that otherwise must cast an impossibly wide net, and communication SETI might provide conclusive evidence that an extreme but still potentially naturalistic source is in fact the product of extraterrestrial intelligence (Bradbury et al. 2011).

Bryan’s thread and Eric’s final tweet could easily be read to foreclose this sort of research, essentially saying “it’s not worth thinking about the aliens hypothesis until it’s so unavoidable that you’ll get no flak for it” (radio signals à la Contact, the proverbial saucer on the White House lawn, etc.). They certainly make it clear that they won’t hesitate to chastise you on Twitter for going down this road.

But if we want to get to the end of that road, we’ve got to start walking down it at some point, and when the media very reasonably asks what we’re doing so they can report on it to a very understandably curious public, we should be allowed to answer their questions without having our motives (or scientific credibility) questioned by our peers.

In short: your mileage may vary on Avi’s particular style of public communication and conclusions on ‘Oumuamua, but when making your critique please be mindful that you are not slamming the whole endeavor. SETI as a serious science will make hypotheses, explore anomalies, and discuss the possibility of alien technology as the cause, and we need to be able to do so without obloquy from our peers, and without them policing which kinds of SETI we’re “allowed” to work on or talk about in public.

If I seem touchy about this, it’s actually not because I’m smarting from these Twitter threads or anything like that (which I don’t actually disagree with much—in particular I’m friends with Eric and I know I have his respect). As I wrote at the top, I’m glad we’re having this conversation and I hope it continues!

But another purpose of this post is that Avi and I (and other SETI researchers) have advisees that work on SETI and these sorts of messages are not lost on them: these tweets imply that senior people in your field will disapprove of you because of the topic of your research, and they will police what you’re allowed to say to the press, regardless of how good a scientist you are. Keep in mind, “Avi’s” paper on ‘Oumuamua that is being criticized has a postdoc as first author.

So in closing: I pledge to keep the SETI real and well grounded in science, to be responsible in my interactions with the media about it, and to train my students to do the same.

And, I hope my peers will pledge to create a welcoming environment for my advisees as SETI (hopefully!) comes back into the astronomy fold (even when—especially when—they are complaining about Avi).

[Updates: Bryan responds in this thread (click to expand):


1= privately, Bryan clarified to me his tweet was referring to his team’s MWA search for signals, not the search by Green Bank, as I suggested in my post. I should have read Bryan’s tweet more carefully and followed link before critiquing his tweet.

Also, I’ve changed the language about who suggested that GBT observe ‘Oumuamua; Joe Lazio informs me that the observations were made with WVU time following discussions with Breakthrough Listen that preceded Avi’s recommendation. In spite of both errors on my part in the original post, my point that Avi appreciates the importance of dispositive evidence stands.

Also, Avi touches on his motives in this interview:

But the search for intelligent life remains outside the mainstream. I am trying to change that in two ways. First, by speaking out in the way that I did on ‘Oumuamua.



AstroWright Group and NEID science at #AAS233

Good morning!  Here are some abstracts at the 233rd meeting of the AAS to be sure to grab:


2pm-3:30pm Room 305: Special splinter session for NEID! Come hear all about this new facility precise RV instrument for the community.


140.27: Mark Giovinazzi of Penn presents his work with Cullen Blake on the characterization and operation of the gigantic NEID CCDs.

140.28: See my poster on the science we’ll do with our Guaranteed Time for Observations with NEID


146.02: Come see Emily Lubar’s work on the design and performance of NEID’s amazing, state-of-the-art environmental control system.



9:15am and all day:

Come to the Technosignatures Decadal Writing Workshop in room 202!

See how you can:

  • co-sign a technosignatures white paper
  • contribute to a technosignatures white paper
  • find co-signers for your white paper
  • find new science cases for your white paper
  • recommend citations to your white paper

Kickoff meeting is at 9:15am after the plenaries.

Planet-Planet Tides in TRAPPIST-1

Tides are complicated and hard.  When a planet orbits a star, the star raises a bulge on the planet’s surface.  As the planet rotates, this bulge moves across the surface, and because rocks and oceans and atmospheres are viscous this dissipates energy.  The energy comes out of the planet’s rotation, which spins it down.  This is likely why Venus and Mercury rotate so slowly.

Diagram from wikipedia

The planet can also raise tides on the star (as can a moon on its planet), and that tidal bulge gets carried around by the central object’s rotation.  This spins down the central object, and can make the planet (or moon) move inward or outward in its orbit.

Finally, if the planet is in an eccentric orbit, then the magnitude of the tides change.  This dissipates energy, which ultimately comes from the planet’s orbit. This makes the planet slowly circularize its orbit.

Put it all together, and this is why the moon is synchronously rotating around the Earth, moving in a (nearly) circular orbit, slowly moving away from the Earth, and why the Earth’s rotation is slowly slowing down.

Things can get really complicated if you have lots of moons.  The moons of Saturn and Jupiter tug on each other gravitationally, exciting eccentricities that drive tides and keep the bodies warm inside, leading to liquid oceans and volcanism despite the very cold temperatures they would have if they were only heated by sunlight.  Solving all of the resonances and tidal effects in these systems is still an active area of research.

TRAPPIST-1 is a system of at seven close-in planets tightly packed in orbital resonances, much like the moons of Jupiter.  In fact, they have more in common in terms of scale with the Galilean satellites than the Solar System planets:

Comparison of the TRAPPIST-1 system with the inner Solar System and the Galilean Moons of Jupiter

From European Southern Observatory

I think that the TRAPPIST-1 system is exhibiting yet another effect that we haven’t seen before: planet-planet tides. The planets in this system are very tightly packed, and quite massive (much more than the Solar System moons are).  It’s likely that eccentricity tides have made their orbits circular (they have very low eccentricity) and that they are synchronously rotating (always keeping the same face towards the star, or else stuck in a spin-orbit resonance of some kind, like Mercury is).

But if that’s true, then when neighboring planets lap each other, the strain caused by their mutual tides are actually of similar magnitude to the strain caused by the eccentricity tides with the star. In fact, if our estimates of the planet masses and eccentricities are right (they’re pretty uncertain, so this is not necessarily a safe assumption) then planet f causes significant tides on planet g that are actually more important than the very small variations in that planet’s tides raised by the star.

This isn’t the case for any moons I know of in the Solar System: although they are more tightly packed than the TRAPPIST-1 system, the Galilean satellites are so much less massive that their tides just can’t compete with that of Jupiter.  The mass ratio for the TRAPPIST-1 system is much more favorable, so the tides may matter.

Now, planet-planet tides can’t matter too much.  If they did, the planets would not be synchronously rotating with their orbits, which would make the stellar tides orders of magnitude stronger, and then the planet-planet tides would be a minor effect.  But if the planets are in a synchronous rotation state, then I think the planet-planet tides at the very least have to be included in the calculations.

After doing some rough reality checks with Eric Ford and Andrew Shannon, I fired up the old RNAAS template at Overleaf and shot another Research Note on the topic off.  You can find it here.

I hope this is right!  This is way outside my training so I’m worried I missed something obvious…


The Cosmic Haystack

I taught the first Penn State graduate course in SETI last spring, and I thought it went really well. We read a lot of papers, had great discussions about Schelling Points and all the different ways to search, and got to visit Green Bank to conduct original SETI observations. You can read all about the course at the class website here, along with student reactions to the papers (tweeted out by Sofia Sheikh here).

One of the cuter terms in the syllabus was that any student whose final project got published in a refereed paper would get a retroactive ‘A’. Well, the first automatic ‘A’ has arrived!  Shubham Kanodia and Emily Lubar, following a suggestion by Jill Tarter, calculated rigorous volumes of the Cosmic Haystack, and the paper is now on the arXiv.

Some highlights:

The first instance of the term “Cosmic Haystack” to mean the space through which SETI searches to find alien technology is apparently from this article in the Christian Science Monitor in 1977, as tracked down by @spacearcheology:

It appears in a paper 4 years later in Wolfe et al. (1981) in a NASA report, which includes this figure (drafted by Jill Tarter):

The Cosmic Haystack here is 3 dimensional, “compressed” down from many more for clarity. The rectangles inside the haystack represent relatively narrowband searches that were either sensitive or included many targets (it’s hard to do both).

Today, wide-field instruments and broadband receivers and backends can search the haystack for alien “needles” much faster. The question is: how much have we searched? Some have gotten the wrong idea about the whole endeavor, as we explain in our abstract:

Many articulations of the Fermi Paradox have as a premise, implicitly or explicitly, that humanity has searched for signs of extraterrestrial radio transmissions and concluded that there are few or no obvious ones to be found. Tarter et al. (2010) and others have argued strongly to the contrary: bright and obvious radio beacons might be quite common in the sky, but we would not know it yet because our search completeness to date is so low, akin to having searched a drinking glass’s worth of seawater for evidence of fish in all of Earth’s oceans.

Shubham and Emily decided to construct a radio Cosmic Haystack that included transmitters anywhere, not just around nearby stars, akin to NASA’s old SETI Sky Survey from the 1980’s. We also tried to capture the way that very broadband signals might be detected, and not just the narrow-band signals considered by some of the earliest searches. This required us to include transmission bandwidth as a haystack dimension, and calculate a rough sensitivity function in terms of it. We also wanted to make sure that our haystack boundaries were well defined, and that we were calculating only search space within a well-defined range of parameters, which led to some interesting integrals with lots of cases to consider:

Sensitivity as a function of transmission bandwidth and central frequency for an idealized broadband instrument, with a “ceiling” set by the haystack boundary.

Case six, for very broadband, very bright transmissions.

The bottom line was that we got to calculate what region of parameter space has been searched for transmitters in the Milky Way.  It’s tiny.  Jill Tarter once estimated it as a glass of water out of all the oceans in the world; we got more like a bathtub (despite a much different haystack construction):

We found that the MWA low frequency searches dominated our haystack search volumes, because of their incredible étendu: a very wide field and very good sensitivity. And they did it in just a couple of hours of searching!

We also construct another one-parameter way to calculate search completeness: what density of uniformly spaced megawatt transmitters in the Milky Way can we rule out?  We get about one per 0.27pc for the latest Breakthrough Listen paper (for L-band transmitters). The Galaxy could be filled with more transmitters than stars, and we wouldn’t have found them yet!

So we haven’t done much searching, and people should not use the “failure” of radio SETI to date as evidence that we should stop looking.

But it’s also important not to let the pendulum swing to far the other way! One might see numbers like -18 in the exponent and declare the entire project hopeless.  On the contrary:

  1. New radio technology allows us to search through haystack volumes much faster than before; the Breakthrough Listen program is quickly catching up to the historical NASA programs by our metric, and MWA needed only 2 hours to dominate our calculation.
  2. If we expect to find transmitters near stars, our completeness is many orders of magnitudes higher, because we have concentrated our efforts there
  3. We don’t need to search the entire haystack unless there is nothing to find.  Another way to look at it: you don’t have to drain the oceans to find marine life, unless they are sterile and you are trying to prove that.  A bathtub’s worth of water is probably not enough to find a fish if you randomly sample the ocean, but if you look in the right places it’s about the right order of magnitude of search required.

Hopefully our paper will be a useful extension of Jill Tarter’s work along these lines (see here and here).

We also had a lot of fun figuring out where the phrase “needle in a haystack” comes from which I detailed here. (it’s not Cervantes!)

Finally, a note on how just as every journey begins with a single step, every search begins with comically weak upper limits.  We quote a critical referee on Bachall & Davis’s first neutrino search paper (an upper limit):

Any experiment such as this, which does not have the requisite sensitivity, really has no bearing on the question of the existence of neutrinos. To illustrate my point, one would not write a scientific paper describing an experiment in which an experimenter stood on a mountain and reached for the moon, and concluded that the moon was more than eight feet from the top of the mountain. (Bahcall & Davis 1982, p. 245)

which I also discussed back here. Waste not want not!

[Update: @SpaceArcheology finds an earlier attestation, probably the source of the Monitor article and perhaps the origin of the term by—who else?—Frank Drake, and @astrocrash finds earlier non-SETI attestations from Shapely and Hubble:



What do SETI terms mean? A committee weighs in.

Inspired by a paper by Iván Almár’s paper on SETI terms:

I wrote down my opinions on the topic and recommended a taxonomy of SETI in this paper, which I describe in my blog post hereSofia Sheikh presented my suggestions at the Decoding Alien Intelligence workshop:

One of my recommendations was to organize SETI in a taxonomy like this:

After the talk, Frank Drake(!) stood up and recommended that there be an ad hoc committee to discuss SETI terms and recommend what they mean (thanks to Michael Oman-Reagan for providing me the audio!). So, in anticipation of the upcoming NASA Workshop of Technosignatures, in Houston next week, I convened such a committee, including Sofia and me, Jill Tarter, Iván Almár, Kathryn Denning, and Steven Dick.

We didn’t agree on everything, but we came up with a lot of good recommendations and clarifications that I hope NASA will find useful. We did not go with “Communication SETI” because the committee thought it was too reminiscent of METI, and we did not recommend against “civilization” or “colonize” (mostly because both are in wide use and we did not want to endorse major changes without very good reason) but we did note the problems with those terms.

Our final report is here:

and Sofia will present it at the workshop on Wednesday!

Measuring Rocky Exoplanet Compositions with Webb

Back in June 2016 I advertised a postdoctoral position between me and Steve Desch at ASU for someone to work on the problem of exoplanet interior compositions.

Normally, the way we determine the interior composition of exoplanets is a combination of inference, measurement, and guesswork.  In the Solar System we can study the surface compositions of planets directly, get the bulk density by dividing masses by radii cubed, and a sense of internal structure by looking at how the surfaces have changed or what a body’s gravitational field says about the interior mass distribution. It’s amazing how much we can determine about, say, the interior of Europa!

Evidence from NASA’s Galileo mission suggested that there might be a liquid water ocean underneath Europa’s icy crust. Image credit: NASA/JPL

On Earth we can also use seismology to directly probe the density and composition of the Earth’s interior.  Some things we still don’t know to great precision (the water content of Earth’s mantle is still pretty uncertain, though it’s small) but in general we have a good sense of what’s down there, despite having a very limited set of actual samples from far below the surface (from volcanos, mostly).

Based on this we have a good sense of how the planets formed, and so we can make good inferences and guesses about the parts we don’t know about yet. For instance, we suspect that Jupiter formed around a rocky core of about ten times the mass of the Earth; one of the purposes of Juno is to see if we can better determine Jupiter’s inner composition and perhaps check this model prediction.

This artist’s concept shows the pole-to-pole orbits of the NASA’s Juno spacecraft at Jupiter. Image credit: NASA/JPL-Caltech/SwRI

Exoplanets are much harder.  We generally can measure their masses from radial velocities and radii if they transit, and for planets for which we have both we have bulk densities. By analogy with Solar System planets we can then guess that they have similar compositions to those, and from the compositions of their host stars we can make inferences about how they might differ. But getting anything like a mantle or surface sample seems impossible.


There is a whole class of planet recently discovered by Kepler (Saul Rappaport and Roberto Sanchis Ojeda did a lot of the pioneering work here) that appears to be evaporating. The transit signatures of these things is amazing; they vary in depth (sometimes disappearing entirely!) and don’t have the usual shape of transiting planets. Apparently, these are small rocky bodies  (which have orbital periods of less than a day and which are too small to see transit at all) have gigantic, variable tails of rock that is condensing from a plume of surface material evaporated by the intense heat of their parent star.

Screen shot 2013-03-09 at 5.24.42 PM.png
Transits of KIC 12557548, from Fig. 2 of Rappaport et al. 2012
Here’s an artist’s impression:

Exoplanet KIC 1255b orbits its parent star followed by a comet-like dust tail. Image credit: Maciej Szyszko.

So by now maybe you’ve spotted the opportunity: that background star is a great lamp to pass light through that material so we can figure out what it’s made of!  Is it mantle material? Is it core material?  It it hydrated?

Studies of white dwarf “pollution” can measure the bulk, relative elemental abundances of material from presumably rocky planets that have crashed into the white dwarf, but here we have the opportunity to study chemical composition of one particular layer of a distant rocky exoplanet—we can’t even do this for the Earth!

But will it work? Are the transits deep enough? Can we distinguish different minerals with spectroscopy? Are there any stars bright enough for this?

Yes to all.  ASU/NExSS postdoc Eva Bodman has all the details in her latest paper. It’s an exciting time!

Eppur si muove

We sometimes read in the history of astronomy that it was Bessel that finally proved the Earth moves around the sun with the measurement of the parallax of 61 Cygni, one of the brightest and closest stars to Earth in the Northern Hemisphere.

This is because people early on realized that if the stars are different brightnesses because they are at different distances, and if the Earth really does move around the Sun, then we should see the nearby ones appear to move with respect to the background ones annually as our light of sight towards them changes.  The effect is quite small, and it was challenging for 18th and 19th century astronomers to measure the tiny effect using only their eyes to make measurements (the biggest parallaxes are like a part in a million).

And so astronomers put a lot of effort into this, and indeed eventually Bessel pulled it off. But the clinching observational proof actually came about 100 years earlier, based on those same observations!

Astronomers were expecting to see a “reflex” motion: when the Earth moved in one direction the nearby stars would appear to slightly move in the opposite direction, so when the Earth was at one extreme of its orbit they would appear to be a bit closer to the center of Earth’s orbit (i.e. the Sun) than they should be.  Instead, astronomers kept measuring a much larger than expected motion (around 20 arcseconds instead of less than 1 arcsecond) in the wrong direction: the stars seemed to move towards a point about 90 degrees away from the Sun, towards the ecliptic.

This is what was actually happening, but it was actually pretty confusing at the time.  One way they were measuring positions was by using the zenith as an absolute direction (you can use a plumb or a liquid to determine which direction is straight up) and a telescope to see how close stars got to the zenith as they passed overhead.  So they could only measure one component of the star’s motion, so all they knew is that it was large and had the wrong phase (90 degrees from what was expected).

What was actually going on is that the stars were suffering from aberration. Imagine you have a trash can an you want to collect as much rain as possible.  If there is no wind, you should just keep the can vertical.  But, if you are on the bed of a pickup truck moving at 10 mph, then some of the rain that would have fallen into the can will hit the side instead.  To maximize rain collection, you have to tip the bucket towards the front of the truck (i.e. in the direction of the truck’s motion) to get the rain to go straight down into the can.

Wikicommons illustration of the aberration of starlight. Original here. By Brews ohare, CC BY-SA 3.0.

Similarly with telescopes: to get starlight to go straight down the optical axis of the telescope, you actually have to “lead” the motion of the Earth slightly by pointing in the direction of Earth’s motion.  This effect is equal in radians to the speed of the Earth’s orbit divided by the speed of light, or about 20″.  This is what was being observed, and this is what Bradley finally figured out in 1727.

You can learn more about this history here. Interestingly, Bradley’s original paper describing the aberration and proving the Earth moves has only one citation in ADS!

So while hunting for the proof of Earth’s motion, astronomers actually discovered a much easier to find proof of Earth’s motion!  But it took decades to understand it.

Today, it’s tricky to repeat these measurements with modern equipment. Finding the zenith to a precision of 1″ is not something most observatories are set up to do; we almost never use high precision, absolute positions with respect the ground in astronomy any more (our instruments tend to change their pointing with temperature, humidity, and other factors, so we calibrate them on actual star positions every now and again to keep our pointing models accurate and precise).

But interestingly, there is a way most college observatories and many amateur ones can find an absolute position: star trails!  By turning off tracking and letting the stars trail, one can identify the center of the trails as the true Celestial Pole.  As the Earth goes around the Sun, one can measure star positions with respect to that and detect the aberration, thus proving the Earth orbits the Sun.

Of course, there are lots of other ways to do this: you could build an R~10,000 spectrograph and feed it light from stars on the ecliptic and measure the Doppler shift caused by Earth’s motion, or you could measure the parallax directly with differential astrometry, like Bessel did. But this is a novel solution that actually doesn’t require special equipment of good seeing, just an ordinary camera.

I was going to try this someday, and even wrote up a whole blog post about it, but finally decided that if I haven’t started by now I probably never will, so I’ve “given the idea away” in the form of a Research Note to the AAS (my sixth!).  One reason I never started is that although the equipment needed isn’t special, there are lots of complications.  One is that the Poles move on their own (due to Earth’s axial precession) so you have to remove that effect first.

As part of my plan to learn Python and Astropy I tried to make a figure showing how the apparent position of the true Celestial North Pole moves with respect to the background stars, showing the precession and the aberration.  It was surprisingly tricky!  Plotting things near coordinate singularities is not something most plotting software does well, so in the end I cheated and just plotted things as a plane chart in ecliptic coordinates and drew the Celestial coordinates in by hand.  I think it came out really well:

Figuring out how to calculate the motion of the Celestial Pole (also affected by nutation) was tricky; it turns out Astropy does not expose those functions to the user so I had to cheat.  In the end, I did it two ways that gave the same answer: I asked for the ICRS (astronomical) coordinates of the zenith at the North Pole of Earth.  Astropy converts from geocentric coordinates to barycentric coordinates by correcting for the orientation of the Earth (the axial motion) and the aberration, so this yields the green curve above.  Almost equivalently, one can ask for the Celestial Pole in CIRS coordinates (the intermediate coordinate system between the Earth and Celestrial Sphere) at many times and then ask Astropy to convert these positions to the ICRS frame.   The latter is faster.

To learn more, you can read my research note describing the experiment here.  And if you try this, please let me know!


A Needle In A Haystack

Where does the old idiom “finding a needle in a haystack” come from?

According to my physical copy of Bartlett’s Familiar Quotations, the phrase originates with Cervantes in Book III Chapter X of Don Quixote, and indeed most searches online state so with authority.

But I just checked my physical copy and the Project Gutenberg version and not only is there no  Book III, the phrase does not appear!

Some sleuthing of other phrases that do appear reveals that “Book III” refers to what is normally called “Part II”, and indeed there in Chapter X of my copy it reads:

…tracking Dulcinea up and down El Toboso will be as bad as looking for a needle in a haystack or for a scholar in Salmanca.

just as promised.  So why isn’t it in the Project Gutenberg version? There it reads:

looking for Dulcinea will be looking for Marica in Ravena, or the bachelor in Salamanca.

which is not the same thing at all. Indeed, the Spanish original seems to read:

buscar a Dulcinea por el Toboso como a Marica por Rávena, o al bachiller en Salamanca.

So the phrase is actually not in the original!  It seems to be due to Cervantes’ English translators who used the phrase as a more familiar [to English ears] version of “to find Maria in Ravena”.  Bez Thomas helped me to figure this out on the Twitters:

So where does the phrase “needle in a haystack” originate?  The OED has two attestations that predate Cervantes:

c1530   T. More Let. Impugnynge J. Fryth in Wks. (1557) 837/2   “To seke out one lyne in all hys bookes wer to go looke a nedle in a medow.”
1592   R. Greene Quip for Vpstart Courtier sig. Ev   “He…gropeth in the dark to find a needle in a bottle of hay.”

Where a “bottle” here means “bundle”.  Apparently the translators were using a 100+ year old phrase!   The “haystack” version is from later:

1779   W. Rogers in J. Sullivan Jrnls. Mil. Exped. (1887) 262   “But agreeably to the old adage it was similar to looking for needles in a hay stack.”

And there you have it.  Even an authority as solid as Bartlett’s occasionally gets things wrong, so it’s good to check!


[Update: Apparently Bartlett’s is full of these errors with respect to Don Quixote: see this article here which details the “haystack” mistake and many more.]