randombio.com | commentary
Saturday, April 23, 2016
Science Under Siege, Part IIPeople say there are no jokes in scientific papers. But I found one.
eople say there are no jokes in scientific papers. The public's perception of scientists is that they are always somber and serious.
But in 2005 a guy named John P.A. Ioannidis wrote an article titled “Why Most Published Research Findings Are False.” The article, published in an open-access journal, came with a disclaimer in small print that it was an opinion piece, but to the untrained eye the mathematical equations, graphs, and computer simulations made it look scientific. It is not. It is that rarest of birds: a sophisticated piece of science humor.
The author considers some plausible but unproven assertions about science, like “The greater the financial and other interests in a scientific field, the less likely the research findings are to be true” and “The hotter a scientific field (with more scientific teams involved), the less likely the research findings are to be true.”
He then defines a ‘bias’ factor u and creates a number of graphs plotting post-study probability vs. pre-study odds as a function of u. The graphs are a family of curves that approach 100% as u decreases and the number of replications increases. Unfortunately, since u is unmeasurable, there's no way of guessing whether u is close to zero or to one, so the graphs are meaningless.
Next there's a box describing how a gene study with 100,000 genes that gets a p-value of 0.05 gives a meaningless result. It is as though he believes biologists and biostatisticians are unfamiliar with this issue and that therefore their results are questionable.
Then the punch line: the number of possible hypotheses is enormous, so any given hypothesis has an infinitesimal probability of being true. This, of course, is itself a hypothesis. So the paper is a reductio ad absurdum of itself. My hat is off.
Okay, so it's not a thigh-slapper like “Outside of a dog, a book is man's best friend. Inside of a dog it's too dark to read!” or even “Why did the chicken cross the road?” But it's as close to satire as you'll ever find in the scientific literature. My reaction when I first saw this paper was that it must be an April Fool's joke. I know nobody who takes it seriously.
The problem with a subtle joke is that some people might take it seriously. Indeed, just last week someone on a religious website called First Things did exactly that, using the above paper to justify taking a whack at science.
We like to think of science as benevolent because it does things like curing diseases and such. But science has enemies. They used to be confined to a fringe group of leftists who thought science was all about building weapons and bombs. But that all changed when environmentalists hijacked the field of climatology. Environmental sciences are now a playground of climate activists who will use science or any other tool at their disposal to achieve their goal of crippling industrial capitalist society. And this has harmed science immeasurably.
Science has gotten associated in the public mind with big corporations and highly political things like obamacare, the gender wars, and global warming. These are serious issues. It's no wonder people are up in arms.
And of course the creationism/Darwinism dispute is still simmering. Some prominent atheists adopt the mantle of science to bash religion, and religious people want to strike back at science.
There are scandals in science, to be sure: the cholesterol scare, the alar scare, the acid rain scare, and many others. The reasons for these scandals need to be analyzed. But people have glommed on to bogus criticisms like the one above, and that can only lead to bogus solutions.
There are others. I've already discussed to death the flaws in C.G. Begley's infamous claim that cancer research is not reproducible here, here, and here. Just recently there was another study claiming that behavioral psychology research is also hard to replicate. Psychology is an easy target because it's a soft science, which means psychologists have to tease out differences between what people say and what they do. This is an extraordinarily difficult task. So if there is variability it simply means there's variability in how people behave. Hard to believe, but it's true.
For industry, where Begley came from, there's a perception that the authoritarian structure of corporations weakens the credibility of their science. So when industry people criticize academic research there's more than a little defensiveness there. In industry, research is already highly bureaucratized. So there is also a sense of unfairness: if we have to have it, why not them? They don't see the obstacles, only the unfairness.
And who can blame them, when government throws the weight of its power of coercion against any company that dares to say something the government doesn't want them to say?
Another batch of anti-science critics are sports fans who are unwilling to accept the possibility that American football can cause brain trauma. Unfamiliar with how science works, they bash science as a whole, as if it will change the facts.
Here is the truth: there are articles in the literature that suggest that brain injuries are occurring. There are others that are skeptical. This does not mean it is an established fact. To claim otherwise is to politicize science. In science, the evidence prevails.
A scientific paper describes the idea being tested, the methods, the result, and the authors' interpretation. Getting those four to work together is tough work.
But we should make no apologies for the fact that there are some theories out there that turn out to be wrong, some published methods that don't work well, some studies that are statistically underpowered, and some results that aren't interpreted correctly. To fix all that would require beefing up peer review, and the current trend is to weaken it in the interest of increasing public access by reducing costs. Maybe that needs to be re-evaluated.
My colleagues would probably say it just points to the need for more funding. There's some merit to that, but some studies are done half-heartedly because the investigator doesn't really believe it will work. Sometimes wild shots in the dark pay off, but when they don't you look foolish if you spent millions of dollars testing them.
Part of the task for a first-year grad student in the sciences is learning to discriminate between good and bad papers. A skeptical approach is essential in trying to interpret the scientific literature. It is not a repository of truth. It is a record of our attempts to reach the truth. Anyone treading there who accepts it uncritically—unskeptically—deserves their fate, which is to get bogged down in falsehoods and contradictions. Anyone who can't understand this should stay away from it.
Everyone in science knows of a lab head who has abused power, forced subordinates to work on their own project, taken credit for their work, or sabotaged their grants and papers. There are scientists who cherry-pick their results and apply bogus statistical tests—what one editor called ‘statistical hallucinations.’ They're often surrounded by dishonest and ambitious yes-men.
These are the things that undermine the integrity of science. But people aren't angry about that. They are angry because they perceive science as taking sides with political causes that are abhorrent to them.
And that's a clue. The attacks on science are really a struggle for power. Environmentalists sue people for expressing ideas they don't want the public to hear. Government uses its coercive power to suppress ideas that interfere with its plans to gather more revenue. Religion has very little coercive power in our society, but it too depends on its power to convince people. Science has accumulated vast power because of its success, and competing interests feel threatened.
Thus anti-sciencers seek to neutralize the threat by imposing more ‘accountability’ on science, which really means more bureaucracy and regulation.
But what is really needed is changes at the administrative level, by making life better for junior scientists. If you want to see where bureaucratization leads, visit a GLP lab that does analytical testing for clinical trials. For every movement those poor devils make, there is a corresponding sheet of paper with somebody's signature on it. Bringing that culture into the research lab would be the fastest way to kill it.
When a branch of science is taken over by activists, its credibility can disappear overnight. When it becomes politicized, skeptics can simply dismiss anything scientists say.
It's not enough to dismiss the critics as scientifically illiterate or as having an inadequate grasp of statistics. Their war on science is really a war against the activists. But it has a devastating effect on the respect that ordinary people have for science.
It is too late to save some fields. Sociology and climatology, for instance, have been badly tarnished. But we can learn one thing from their misfortune: the activists are coming, and if they're allowed to gain a foothold, no branch of science, not biology and not even physics, will be safe.
Update: Conservative columnist Daniel J. Flynn has written an entire book attacking science for investigating whether football can cause brain trauma. The blurb on his book says “people are more than willing to trash a great sport in hot pursuit of a buck.” It is typical of an adversarial approach based on a lack of understanding of how science works. Nobody wants to take away football. But if football is harming people, we need to know about it.
Updated May 02, 2016
Science under siege, Part I
Bogus claims about the reproducibility of scientific research will not die on their own. We must give them a push.
Corrosive claims about academic research
Industry scientists often claim that academic research is not reproducible. But they never show their data.
An autopsy of the late global warming movement
Why are drugs so expensive?
The reason is not what you might think.
How to identify bad science
Here are some tips on how to identify a scientific snowjob.
If biology is co-opted by the Left, our future could be written by ideologues.
The Hydrogenated Bomb: Science and the Cholesterol Scandal
Saturated fat and cholesterol are now good for you again. There will be a quiz.