Bare-knuckle boxing in PLOS ONE

A paper published recently in the open-access journal PLOS ONE entitled “Getting a Grip on Memory: Unilateral Hand Clenching Alters Episodic Recall,” which, as hinted at by the pun, is about the supposed tie between making a single fist and remembering an event, has been the center of controversy lately.

The abstract of the paper states:

 It was hypothesized that right hand clenching (left hemisphere activation) pre-encoding, and left hand clenching (right hemisphere activation) pre-recall, would result in superior memory.

And they say their hypothesis was proved to be true. One commenter, however, disagrees—and with a pun of their own in the title: “Fist-clenchingly poor science.” The comment attempts to discuss the inherent flaws, starting by:

There are so many flaws it is difficult to know where to begin, but I’d be hard pressed not to fail this paper if it was presented to me as an undergraduate project report.

The climax of the critique showed the main failing of the report:

..the authors admit [the difference between the control and sample groups] was not significant despite repeating throughout the title, abstract, discussion, and press release…

The critique on the whole is pretty funny and informative, and I admit to always loving a controversy (as long as I’m not directly involved). Science controversies are a real favorite of mine, because each side thinks they know the truth and rarely back down or concede. And then when someone does admit to maybe being slightly incorrect, no one lets them forget it. There’s a surprising amount of ego in a group that’s supposed to be made of dispassionate observers.

A mini-argument has also erupted in the comments on the comment.

Woof, quite a long-winded and rhetorical way to state the hypothesis is weakly supported or even disputed, and sample groups too small and biased. The rest of the “comment” steeped in ego.

To which, someone replied.

Did you veer over from a place where ‘woof’ is vigorous rejoinder- where ad hominem is appreciated, and comments on technical subjects are held to two sentences?

With another sub-commenter, commenting on the harshness of the critique:

My problem isn’t with commentary or criticism of the science itself, it’s with how it is expressed. It’s one thing to say “These results need to be treated with caution because the study was under-powered and the analyses didn’t correct for multiple comparisons”. It’s another to say “This is fist-clenchingly, shockingly poor science that wouldn’t even have been acceptable as an undergraduate project”. One is a factual comment on the science, the other is just an emotion-laden and potentially hurtful attack.

All-in-all it’s a pretty good read. The comments that is, not the actual paper, which is arguably garbage.

2 thoughts on “Bare-knuckle boxing in PLOS ONE

  1. “Arguably garbage”- now you’re bein’ a bat of a meany there, airen’t ya?

    It may be a perfectly useful set of data, but given the study structure, it’s really tough to know. That’s the crux of the controversy. As I understand it (i.e., incompletely), the knot had to do with the sampling method and a few other study details. The effect sizes were fairly large and consistent, but the confidence intervals were somewhere between double- and triple-‘orreebow, thanks to rather teeny samples (they divided up a group of about 50 people into subgroups they evaluated for different tasks). In other words, the effects were well within the bounds of what could be considered reasonable for a random set of results.

    For a bit more than the record, the initial “woof” quote was pulled off subsequent to your post here, as was (per reasonable policy) the response you note here (mine), I got a leetter that said they did it because the “woof” note wasn’t contributing to the science. Good for them. And anyway, the writeup isn’t rendered any less enlightening and lively at the loss of the bullshit side splat. On the contrary, the rather emotional, yet reasoned, initial response by Jon Simons was responded to politely and clearly by the communicating author of the paper, and there were some other intelligent contributions that stuck quite close to the knitting. All of these were enlightening, focused on rationale, and distinctly science-centric. Bravo, I say.

    I have since responded with a followup in the comments to an original post by me to the authors. My initial bleat, regarding the rather racy tone of the research writeup, wasn’t responded to by the study authors. A few others piled on with more discussion of how rude is too rude, and such. My main beef was a bit more practical: I read the writeup abstract and results and, before seeing the important critiques, spent a few hours carefully clenching my fists in the right order, hoping to remember, for once, what to bring home from the grocery store after work. I had to look to Ibuprofin to relieve the resultant ache in my modest forearms. They are glad of the respite, while we await replicatons that prove how garbage the science is. Actually, their writeup resulted in a fair slug of breathless media articles. That was at the heart of the issue for me: that there are exploratory and confirmatory studies, and the writeup of this study had more of the flavor of the latter. The Results section should have been written in the spirit of being exploratory results, with large effects and lousy confidence intervals noted together.

    The whole notion of scientist comments on journal articles is so pink-baby-skin new that any such pointed discussion in efficient, communicative terms like this should be heartily applauded. I look forward to more scientists opening themselves up to such input through submission to open journals. I especially look forward to the day when scientists actively critique on them like Dr. Simons did, and when many other paper authors are as courageous as Dr. Propper here, in responding politely and effectively.

    Do hope for an answer to me comment, though. May be a a bridge too far to ask for in the year 2013. That’s OK.

    • I think the open critique is quite interesting as well. It opens up a whole new level of scientific discussion — although discussion will probably never be as polite as they could be, especially if anonymous comments (or a pseudonym) are allowed. It is scary ground for new student publishing their first paper. I imagine I’d feel pretty bad if a lot of people came in and ripped my work to shreds. Personally, I’d probably never be involved (on my own will) in a comment war like that. I’d just be too scared that I was wrong. But there are a lot of confident people out there who aren’t afraid to publicly discredit someone else. It’s pretty fun to read the comments though.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s