eople say there are no jokes in scientific papers. The public's perception of
scientists is that they are always somber and serious.
But in 2005 a guy named John P.A. Ioannidis wrote an article titled “Why Most Published Research Findings Are False.” The article, published in an open-access journal, came with a disclaimer in small print that it was an opinion piece, but to the untrained eye the mathematical equations, graphs, and computer simulations made it look scientific. It is not. It is that rarest of birds: a sophisticated piece of science humor.
The author considers some plausible but unproven assertions about science, like “The greater the financial and other interests in a scientific field, the less likely the research findings are to be true” and “The hotter a scientific field (with more scientific teams involved), the less likely the research findings are to be true.”
He then defines a ‘bias’ factor u and creates a number of graphs plotting post-study probability vs. pre-study odds as a function of u. The graphs are a family of curves that approach 100% as u decreases and the number of replications increases. Unfortunately, since u is unmeasurable, there's no way of guessing whether u is close to zero or to one, so the graphs are meaningless.
Next there's a box describing how a gene study with 100,000 genes that gets a p-value of 0.05 gives a meaningless result. It is as though he believes biologists and biostatisticians are unfamiliar with this issue and that therefore their results are questionable.
Then the punch line: the number of possible hypotheses is enormous, so any given hypothesis has an infinitesimal probability of being true. This, of course, is itself a hypothesis. So the paper is a reductio ad absurdum of itself. My hat is off.
Okay, so it's not a thigh-slapper like “Outside of a dog, a book is man's best friend. Inside of a dog it's too dark to read!” or even “Why did the chicken cross the road?” But it's as close to satire as you'll ever find in the scientific literature. My reaction when I first saw this paper was that it must be an April Fool's joke. I know nobody who takes it seriously.
The problem with a subtle joke is that some people might take it seriously. Indeed, just last week someone on a religious website called First Things did exactly that, using the above paper to justify taking a whack at science.
Why do people attack science?
We like to think of science as benevolent because it does things like curing diseases and such. But science has enemies. They used to be confined to a fringe group of leftists who thought science was all about building weapons and bombs. But that all changed when environmentalists hijacked the field of climatology. Environmental sciences are now a playground of climate activists who will use science or any other tool at their disposal to achieve their goal of crippling industrial capitalist society. And this has harmed science immeasurably.
Science has gotten associated in the public mind with big corporations and highly political things like obamacare, the gender wars, and global warming. These are serious issues. It's no wonder people are up in arms.
And of course the creationism/Darwinism dispute is still simmering. Some prominent atheists adopt the mantle of science to bash religion, and religious people want to strike back at science.
There are scandals in science, to be sure: the cholesterol scare, the alar scare, the acid rain scare, and many others. The reasons for these scandals need to be analyzed. But people have glommed on to bogus criticisms like the one above, and that can only lead to bogus solutions.
Other critics of science
There are others. I've already discussed to death the flaws in C.G. Begley's infamous claim that cancer research is not reproducible here, here, and here. Just recently there was another study claiming that behavioral psychology research is also hard to replicate. Psychology is an easy target because it's a soft science, which means psychologists have to tease out differences between what people say and what they do. This is an extraordinarily difficult task. So if there is variability it simply means there's variability in how people behave. Hard to believe, but it's true.
For industry, where Begley came from, there's a perception that the authoritarian structure of corporations weakens the credibility of their science. So when industry people criticize academic research there's more than a little defensiveness there. In industry, research is already highly bureaucratized. So there is also a sense of unfairness: if we have to have it, why not them? They don't see the obstacles, only the unfairness.
And who can blame them, when government throws the weight of its power of coercion against any company that dares to say something the government doesn't want them to say?
Another batch of anti-science critics are sports fans who are unwilling to accept the possibility that American football can cause brain trauma. Unfamiliar with how science works, they bash science as a whole, as if it will change the facts.
Here is the truth: there are articles in the literature that suggest that brain injuries are occurring. There are others that are skeptical. This does not mean it is an established fact. To claim otherwise is to politicize science. In science, the evidence prevails.
What science really does
A scientific paper describes the idea being tested, the methods, the result, and the authors' interpretation. Getting those four to work together is tough work.
But we should make no apologies for the fact that there are some theories out there that turn out to be wrong, some published methods that don't work well, some studies that are statistically underpowered, and some results that aren't interpreted correctly. To fix all that would require beefing up peer review, and the current trend is to weaken it in the interest of increasing public access by reducing costs. Maybe that needs to be re-evaluated.
My colleagues would probably say it just points to the need for more funding. There's some merit to that, but some studies are done half-heartedly because the investigator doesn't really believe it will work. Sometimes wild shots in the dark pay off, but when they don't you look foolish if you spent millions of dollars testing them.
Part of the task for a first-year grad student in the sciences is learning to discriminate between good and bad papers. A skeptical approach is essential in trying to interpret the scientific literature. It is not a repository of truth. It is a record of our attempts to reach the truth. Anyone treading there who accepts it uncritically—unskeptically—deserves their fate, which is to get bogged down in falsehoods and contradictions. Anyone who can't understand this should stay away from it.
Updated May 02, 2016; edited sep 04, 2018
