Journalism has been in trouble since… well maybe since forever but it certainly feels more precarious since the World Wide Web destroyed most print journalism as a viable stand-alone business. The New York Times and Washington Post have hung on with quality journalism, but not without moving to a heavy online presence and worrying a lot about generating “clicks” (at least on the business side).
One way to survive is to generate lots of those clicks, and that means “clickbait”—provocative headlines that dare you to ignore them. Whether you find this to be outrageous or just a fact of business probably depends on how exposed you have been to it. I imagine most journalists take it as a given that headlines and ledes must generate clicks and scrolls in order for them to have jobs, but that what really matters is the meat of the article.
It would be nice if it weren’t so, especially when it’s your science being hyped. Yesterday I wrote about Sabine Hossenfelder’s Forbes article and along the way defended it against accusations that it was “sloppy journalism” and representative “the frenzied sensation-driven nature of mainstream publishing” mostly by linking to it, quoting it, and being incredulous at those descriptions. Some have disagreed with that assessment:
Creswell paper didn't contain claim that 'it could just be noise', rather they said 'noise around this signal is not entirely understood'.
— TomD_HD (@TomD_Hann) June 23, 2017
As I believe I said, the lede is quite misleading. A set up "big experiment may be wrong", followed by "but probably not" is risky sci comm
— Jonathan Pritchard (@jr_pritchard) June 22, 2017
First of all let me sympathize: it is mortifying to see your research in print in a way that sensationalizes it.
So maybe it’s because I’ve seen a lot worse and I’ve written for popular outlets and my student Kimberly Cartier wrote a thesis about science communication that I’ve become inured to the way that headlines beg for clicks, even if they don’t represent the meat of the article well.
But I think in this case the Forbes article is actually not in that camp. With the benefit of hindsight or insider knowledge, it might look overblown, but consider:
- At the time it was written, there was no LIGO response to go on.
- Hossenfelder contacted LIGO and they had no formal response—and said they planned none!
- Hossenfelder then got a third party to comment, and they found “the results “quite disturbing” and hope[d] that the collaboration will take the criticism of the Danes to heart. ‘Until LIGO will provide clear scientific (!) explanation why these findings are wrong, I would say the result of the paper to some extent invalidates the reliability of the LIGO discovery.’”
- We all know that the scientific literature is often understated, especially when suggesting someone is wrong. An abstract that concludes “A clear distinction between signal and noise therefore remains to be established in order to determine the contribution of gravitational waves to the detected signals.” has a pretty clear meaning: the GW detections are in jeopardy.
- After all, the strongest signal was 5.1-sigma. If there is any reason to think the noise is underestimated then the signal drops below the 5-sigma level. Now I appreciate that the difference between “significant” and “not significant” is not Itself statistically significant,” but the “New York Times threshold” is actually the one place where which side of 5-sigma you are on absolutely matters!
Given all these items, I think the headline and fInal introductory sentence of the Forbes article is fine:
Was It All Just Noise? Independent Analysis Casts Doubt On LIGO’s Detections
…But what if there wasn’t a signal at all, but rather patterns and correlations in the noise that fooled us into believing we were seeing something that wasn’t real? A group of Danish researchers just submitted a paper arguing that the celebration might have been premature.
Especially given the 5.1 sigma issue, I think that’s totally fair. The paper did cast doubt, none of Hossenfelder’s followup investigations dispelled that doubt, and “celebration might have been premature” is an accurate description of the implications if the detection was actually only, say 2.5-sigma (and Creswell et al. implied it could be much lower than that, IMO).
Anyway, YMMV on this. We can agree that it would be nice if popular journalism were as sober as The New Yorker and as popular as cat memes, but I think we can also agree that that will not happen.
The only practical solutions I see to the larger problem are:
- Get scientists better at science communication.
- Appreciate those that are good at it and see them and the work they do as an important part of our profession (the aforementioned Kimberly Cartier included a chapter on this as part of her PhD thesis in Astronomy & Astrophysics—I hope in the future this will not seem novel).
- Make sure that when our work pops up in the media we know how to manage it. Especially a large, high-profile project like LIGO should have1 team members (or access to professionals) that know how to quickly manage stories.
After all, if LIGO had told Hossenfelder that it was planning a response that completely addressed the Creswell et al. paper and showed it was based entirely on poor analysis, her article would have had a different conclusion and different tone (if it had been written at all!). They had an opportunity to get in front of it and shape the Forbes article, but they did not.
But it all turned out OK: their eventual (though unofficial) response is solid and succeeded because they put it on Sean Carrol’s popular and respected blog instead of waiting for the peer review process. Now the story can be “nothing to see here, move along” and, as a bonus, the referee of the Creswell et al. paper has a great template for their review. Double bonus: now the world understands LIGO better!
1For the kids: Bentsen-Quayle.
[Edit: Hannam responds (click to expand):
Poor papers are a dime a dozen. Experts recognise them as such and ignore them. (Life is too short.) 1/n https://t.co/He52JNP7j4
— Mark Hannam (@MarkDHannam) June 23, 2017