Don’t believe everything (peer-reviewed) you read on the web

Recently, my attention was drawn to an article the same way I find many articles: through Facebook. Several of my many science-minded friends referenced a recent article from sciencemag.org entitled “Who’s Afraid of Peer Review?”, where a spoof paper with clearly bad controls was submitted to 304 open access journals and was accepted by 157. After reading a variety of comments with tones ranging from outrage to, “damn, I need to start writing some fake papers,” there was no way I wasn’t going to check out the article.

peer review image

Image used with permission from http://strange-matter.net/screen_res/nz060.jpg

Basically, some writers from Science generated a fake paper the claimed a chemical extracted from a lichen had shown anticancer properties. While the article claimed that there was a strong dose-dependent effect of the drug on the cancer cells, the effect barely varied over 5 orders of magnitude. The paper claimed that the chemical was dissolved in large amounts of ethanol before being added to cells, but the control cells were given no ethanol, meaning that likely what was killing the cells was not in fact the chemical, but the ethanol itself. Testing controls with the same solvent as the other conditions is standard, especially when large amounts of the solvent itself can have toxic effects. The spoof paper also went on to make large claims about how the molecule tested has potential as an anticancer drug.

It would seem likely, given the inherent flaws in the article, that the academics reviewing this paper would immediately raise a red flag about its content. However, the majority of the journals accepted the paper, including 45% of the journals in the Directory of Open Access Journals (DOAJ), which is meant to identify the credible open access journals. Many of the journals that offered the authors any feedback ignored the glaring scientific mistakes and simply made suggestions for changes in formatting.

As a grad student, I read peer-reviewed papers nearly every day. And while I’ve always known that I should be critical of everything I read, I was still shocked that this spoof article with such glaringly bad science was accepted by so many publishers. While I’m not as tempted to submit my own fake articles as some of my Facebook friends, this sting operation performed by the writers at Science is making me much more skeptical of papers I find on the web.

Amanda Levy is a doctoral student in the materials science and engineering laboratory of Peter Searson, director of INBT.

For all press inquiries regarding INBT, its faculty and programs, contact Mary Spiro, mspiro@jhu.edu or 410-516-4802.

Comments

  1. Perhaps the reviewers checked the statistics of the experiments directly and did not bother to read and understand the essence of the study

Leave a Comment

*