By Poncie Rutsch
BU News Service

As pop culture would tell you, scientists are old white guys with crazy hair. While that perspective is heavily biased (my hair is crazy, but not white), it isn’t totally unfounded.

The people who make science share their knowledge through academic journals, which traditionally take their contents very seriously. The journals accept science by peer review, meaning that the most prestigious, whitest haired, top-of-the-line scientists make sure the contents of the journal are up to snuff. And you can only read the journal if you, as part of the scientific elite, choose to pay for access.

But this model is outdated…or so would say the open access journals, which sprung to popularity about a decade ago. Open access journals claim their goal is to remove legal, financial, and technical barriers between people and their science. The only thing keeping people from reading the contents should be access to the internet itself.

The problem is, open access journals don’t have quite as spiffy a reputation as traditional journals. And this was what inspired the recent efforts of John Bohannon.

John wrote a spoof paper and sent it to hundreds of open access publishers. 157 published it. And then Science published him.

“Any reviewer with more than a high-school knowledge of chemistry and the ability to understand a basic data plot should have spotted the paper’s short-comings immediately,” John writes. “Its experiments are so hopelessly flawed that the results are meaningless.”

John submitted a paper that proclaimed a new wonder drug. He set up the paper with a simple formula: “Molecule X from lichen species Y inhibits the growth of cancer cell Z.” He substituted each variable with molecules, lichens, and cancer cell lines to create hundreds of papers. Each was unique enough to not attract attention, but the structure was similar enough to be used as a constant in John’s investigation. He submitted the paper using false names and institutions that he generated randomly from databases of common African names, words in Swahili, and African capital cities.

He included the same flaws in each paper – data that showed the opposite of his conclusions, an obvious lapse in the methods, and a control group that didn’t receive one of the constant level of radiation as the others.

Over 150 open access journals accepted the fake paper. John writes that over 250 of his papers went through an editing process, but that 60% showed no sign of peer review.

The final verdict is that open access journals have a long way to go. There may in fact be some merits to the peer-review system…even if you have to pay to get in.

You can also see John’s paper at BMJ. But of course, you’ll have to pay to get in.

Also, here’s a history of open access journals, courtesy of the wise people of tumblr: