ith all the fake news out there these days, it's arguable that we're starting
to see the death throes of the mainstream media. But fake news is just part of a
bigger issue: the unidirectional model, where static bits of new information are
transmitted from a producer to a consumer, is increasingly inadequate.
Scientists need not feel left out: the flow of information in science has always been unidirectional. Blogs and open access journals address this to some extent, but these days the economic model of open access journals seems to be under stress: to obtain enough content to survive, they're starting to relax standards. More and more I find myself declining to referee articles from journals I never heard of on topics like alternative medicine and plant biology that I know little about.
Conditioning apparatus from the Rouleau paper. An Arduino is visible in the front. The blue disks are not capacitors, but bits of colored bread dough. EEG leads are attached to the larger lump of blue dough in the background.
Sometimes weird articles slip by even in good online journals. A good example is “Experimental Evidence of Classical Conditioning and Microscopic Engrams in an Electroconductive Material” (Rouleau N, Karbowski L, Persinger M, PLoS One, October 2016 (DOI: 10.1371/journal.pone.0165269) which reports on the discovery of associative learning in a lump of bread dough.
I've read it twice, and I'm still not sure whether it's a real article or a hoax. They hooked up a pile of dough, described as 46% flour, 31% water, 15% proton donor (i.e. lemon juice), 7% salts, and 1% ‘other,’ to an EEG machine, and claimed to have demonstrated classical conditioning, a form of associative learning, between an LED (the conditioned stimulus or CS) and an electric shock (the unconditioned stimulus or US; 1–5 volts). And not just ordinary CS-US conditioning, but trace conditioning! After the training session they sliced the bread dough up, stained it with toluidine blue, and did histology on it.
Now, I'm not saying this paper is fake: it could turn out to be really important. You never know when the knowledge of whether bread can be classically conditioned may determine the ultimate fate of the universe. You never know what people will discover by just using their loaf in an unusual way.
But one tip-off to fake research is when you see statistics like correlation coefficients or z-scores instead of measured values plotted against time. Another red flag is when graphs don't start at zero, but at some carefully chosen arbitrary point, say 17.5. There are a few things like that in this paper, but I don't want to embarrass the authors too much, so let's just leave it at that.
As Han Solo said, I've seen a lot of strange stuff. I once saw a talk where a respected researcher did a computer simulation that showed that virtually every known psychological phenomenon could be carried out by a single neuron. It was impressive, but it struck me as a good demonstration of how computer simulations can lead people down the garden path.
The cumulative effect of millions of papers going down thousands of garden paths is enormous. Findings that don't make any sense or findings unsupported by hard evidence can be ignored (even if potentially valuable), but the problem of good experiments that are misinterpreted is harder to deal with.
Last week I found one where they did a very clever experiment using an old technique from the days of radioisotopes called pulse-chase. Everyone in my field cites this paper, but after struggling for two days trying to extract data from their blurry figure, I realized their results were analyzed incorrectly and the conclusion was almost certainly wrong. A wrong answer can set back a cure for a disease by years. So this issue is not just holding back science, but potentially costing lives.
Unlike in chemistry, where they say what they did and then stop talking, biomedical papers tend to babble on forever. Again it's the peer review process at work: reviewers demand that we say this and that, and that we cite all the big cheeses in the field, which is to say we have to cite the reviewer. Routinely we struggle to stay within the word limits, and we create gigantic 20-part composite figures to get around the journals' limits on figures.
What can be done? We still need open access journals, but the traditional journal format is inadequate. The examples above show they don't adequately protect us against going off in false directions, and they can't prevent the publication of fake science.
Outside of science, in the so-called <air-quotes>real world</air-quotes>, the traditional unidirectional formats like newspapers and TV broadcasts have shown themselves unable to prevent the spread of fake news. Perhaps a blog-like format, where people could comment on articles, ask questions, correct mistakes, and make suggestions, would be better.
There are, of course, many blogs already. The challenge is to prevent them from degenerating into social media, where the most obnoxious, pushy, pompous, self-aggrandizing authors (like me, for instance) have a disproportionate impact.
Here are some off-the-top-of-my-head ideas:
- All journals should be online and free. Arxiv is a good model.
- Open comments, made anonymously.
- Any form of popularity ranking, like that seen on slashdot, is strictly forbidden.
- Rigorous peer review, possibly with the referees' reports published along with the paper, so readers can see what was missed in peer review.
- Criticism of comments is forbidden, to prevent flame wars and politics.
- Articles should be as short as possible, with any material not directly relevant to the experiment put in an appendix.
- Articles should have a version number and should be updateable in case an error is found.
Information cannot just flow one way, or the information provider will find itself increasingly insulated from the real world, and its credibility will become suspect. When we find ourselves defending against the charge of fake science, it will be too late.
Revised Jan 13 2016; last edited Jan 13 2016 12:42 pm
