What good are blogs?

The LIGO collaboration recently made headlines, made an epoch, and earned itself an all-but-certain Nobel Prize, for the detection of gravitational waves.

Recently, an unrefereed paper appeared on the arXiv questioning the strength of the detections. In brief, LIGO uses two detectors separated by 2,000 miles to filter out noise by looking for coincident signals: real gravitational waves will affect both sites equally, but local sources of vibration should be uncorrelated between the sites.  The Danish authors, who include a scientist name Andrew Jackson, took some public LIGO data of the detection, did some analysis of it, and found that the noise appeared to be correlated between the two sites.  At the very least, they argue, this means that the LIGO collaboration has overestimated the strength of their signal.

It’s a provocative claim: that a major milestone in physics could be a mistake, revealed by a relatively straightforward analysis that any physicist could understand. When I heard about it, I thought “that’s probably wrong, but I’m curious why it is that the sites have correlated noise and how the LIGO team deals with that.”

Sabine Hossenfelder is a theoretical physicist, blogger, and freelance science journalist. She apparently had similar thoughts to mine, and used them as the basis for a column in Forbes on the topic. It’s a nice piece of science popularization, that explains the issue in an accessible way.  She, very responsibly, went to the LIGO team for a response:

Jackson is no unknown to the LIGO collaboration. Upon my inquiry with a member of the LIGO collaboration what to make of the paper, I got the annoyed reply that the collaboration’s management recommends to “respectfully respond that we have talked at some length with the group in the past and do not agree on the methods being used and thus with the conclusions.” Another let me know that a response is not planned.

She also walks the reader through some of the problems with the claim.  I found the column illuminating, but wished I had a better explanation of the issue. I was intrigued!  Hossenfelder concludes:

Making sense of somebody else’s data is tricky, as I can confirm from my own experience. Therefore, I think it is likely the Danish group made a mistake. Nevertheless, I would like to see a clear-cut explanation and “they did something wrong” is too vague for my comfort. This is a Nobel-worthy discovery and much is at stake. Even the smallest doubt that something is at odds should be erased.

Right! But even beyond the issue of how if they want their Nobel they should make sure there is no doubt, there’s also a general outreach angle: gravitational waves are a very popular topic, and this paper, however annoying, had the effect of raising interest in a particular aspect of the science. This paper provided the LIGO collaboration with an opportunity to cash in on that level of interest and explain this particular aspect of the science.

And they did. A member of the collaboration, with the blessing of the collaboration, wrote a guest post on Sean Carroll’s popular blog.  It was just what I hoped to read: an accessible (to me, anyway) discussion of why the Danish group’s analysis is almost certainly wrong and naive, along with a quick tutorial on how LIGO makes sure it doesn’t make similar mistakes.

The whole thing to me was a good example of how various levels of science communication can work: the arXiv (for better or for worse) provided a formal forum for a team to make a scientific claim of high visibility before peer review; the science column provided a way for a professional scientist to engage the public in the issue; the science blog provided a way for the team to make an informal but quick and almost definitive response to what was apparently a straightforward mistake by the Danish group (thus illustrating why one shouldn’t make provocative, unrefereed claims on the arXiv: you’ll usually end up being very publicly wrong).

Then there was a blog post by Mark Hannam on the whole episode.  Now, I get that the LIGO team is understandably frustrated by this sort of high profile sniping by a team that apparently didn’t know what they are doing, and annoyed that they have to spend time putting out these PR fires. The mature response is to turn it into a teaching moment, and they did with the Carroll guest blog post. Now the broader community understands better, the Danish team’s mistake is laid bare, and everyone knows more physics than they did before.

But Hannam doesn’t reserve his fire for the Danish group.  He actually says the thing that annoys him most about the whole thing is Hossenfelder writing it up in Forbes! He refers to her “sloppy journalism” and “the frenzied sensation-driven nature of mainstream publishing”. Did he read the same article the rest of us did?  Take a look for yourself.

Hannam is annoyed that the “controversy” played out it real time and not at the pace of peer reviewed paper. I expressed my eye-rolling at Hannam’s post on Twitter, and got some pushback, mostly because in the confines of 140 characters it looked like I was attacking the LIGO collaboration. It went over to Facebook, and a lot of people disagreed with me.  Some excerpts of rebuttals:

“the methods used for the discovery have been laid out in excruciating detail”

Well, yes, but saying that interested people could figure out the problem by reading your papers is terrible popular science communication; it’s much faster and more efficient for someone to take some time to explain it to everyone briefly than to expect everyone else to take a couple of days to digest the papers and figure out for themselves what the Danish team did wrong.

Of course, you don’t have to communicate your science well in general, but it’s sort of an obligation when your project is on the front page of the New York Times and in the running for a Nobel.

“Younger scientists may think it is cool to have open discussion with non-peer-reviewed arxiv-postings but that’s busy work that takes away from more important analysis and when I was a student/postdoc that’s what conferences were for”

The quick answer is that blogs are cheaper, more far-reaching, and faster than conferences, so why prefer conferences for this particular item?  And it is only “busy work that takes away from more important analysis” if you think communicating your science is not important. If you think it’s part of your job, then you know you have to do it anyway, so it makes sense to focus your efforts on items that already have the public’s attention, like the Danish team’s paper.

Ok, let’s consider an analogy. Recently, we did a press release for a paper that got picked up by a few people, and there were several articles, some with open online comments. And some of those comments (trying to think of a non-“scornful” way to say this) strongly disagreed with our work while exhibiting a lack of familiarity with the subject. Would Jason Wright and James Guillochon suggest that I “missed a teaching opportunity” by not engaging with the comments?

Well, yes, obviously, but not a very big one. Responding to every comment on every article is obviously inefficient and doesn’t scale. But when the whole world is watching your team and your results are influencing in billion-dollar space missions decisions and a Nobel Prize hangs in the balance, it doesn’t seem unreasonable to have some kind of public response to news stories on an accusation from your colleagues that you’ve made a huge mistake.

This is actually a great argument for science blogs: it’s a way to quickly, very publicly make an arbitrarily-detailed response to things like an unrefereed arXiv post. They let you balance the time you put into the response to the claim you’re responding to.

For instance, instead of putting out fires as they come up on Twitter and Facebook as a result of my off-the-cuff tweet, I can have a blanket response to all of them in one big post I can link to.  It’s the same reason that people hold press conferences instead of answering phone calls from reporters all day (but on a much smaller scale, of course).

As I said, if I were the LIGO team I would be annoyed by the episode, but the things that would annoy me the least are that the world showed an intense interest in my work, that I had to explain my science to that interested audience, and that I got to show up a gadfly on a big stage.

[Edit: More responses and details in my next post]

[Edit: Hannam responds (click to expand):

Poor papers are a dime a dozen. Experts recognise them as such and ignore them. (Life is too short.) 1/n https://t.co/He52JNP7j4

— Mark Hannam (@MarkDHannam) June 23, 2017

]