science commentary

How to identify bad science

Here are some tips on how to identify a scientific snowjob.

by T.J. Nelson


B y now most people know there's a lot of bad science out there. But how can the average person tell the difference? Even with an advanced degree in a hard science there's no sure-fire way. But there are clues that even a non-scientist can use. They're based on the assumption that people who do bad science have certain shared personality characteristics.

What pseudoscientists all have in common is that they aren't interested in understanding the truth. They're interested in convincing you that something is true because it benefits them in some way. Some are motivated by financial gain, and others by the prospect of political power. Still others are interested in fame and glory. They use science because it gives them credibility. Here are some of the things to look out for.

  1. Speaking in absolutes, never expressing doubt or uncertainty
    The only certainty in science is that there's no certainty in science. Einstein once said that the second law of thermodynamics was the only theory that would never be overthrown. If someone says without qualification “x causes cancer” or “y is caused by z” it's a clue they may not be speaking scientifically. Look for their evidence.
  2. Self-benefiting claims
    Sometimes scientists, in their endless quest for more funding, start proselytizing. Whenever someone makes a claim that, if true, would mean you have to give them money, the only thing you can be certain about is that you will lose money.
  3. Claims based on politics
    These days lots of people go into science not to improve our state of knowledge but because they want to change the world. But changing the world is the realm of politics and religion, not science, and these people have done enormous damage to the reputation of science. That's one reason why climate science is said to have so much trouble attracting talented people these days. There are a few good scientists there, but nowadays it's taking on more and more of the characteristics of a religion.

    Perhaps the clearest example of this is the attempt by twenty by climate scientists asking the government to use RICO laws to prosecute scientists who disagree with them. It's hard to see much difference between this and the persecution of thousands of heretics including scientist/philosophers Cecco d'Ascoli, Giordano Bruno, and Galileo by the Catholic Church. This was such an injustice we still talk about it 500 years later. I wonder what people will think of those climate activists in 500 years.
  4. Single factor causality
    You may have seen those posters showing biochemical pathways, with literally thousands of molecules interacting with each other. It takes years to understand the complex and intricate workings of the cell. So if anyone tells you that one molecule is the only important one, they are almost certainly wrong. Some diseases, it is true, can be traced back to a defect in a single molecule or a single microorganism, but if you spend your whole career just studying one molecule, there's a tendency to think it's the most important one. Lots of scientists fall into this trap.
  5. Conclusions based on a computer model
    Computer models are tremendously useful in science. They can be used to generate hypotheses and test theories. But it is impossible to get new information about the world from a computer model. A computer-generated result always has to be tested against real evidence before it's worth believing. Anyone who tries to convince you something is true because their model says so is not doing science.
  6. Conclusions based on statistics
    Statistics are almost as bad as models. A statistical test, properly calculated, only tells you the probability that the result was due to chance. But statistics are often abused. I've seen people try one test after another, jumping from t-tests to non-parametric tests and ANOVAs until they get a result that makes their results statistically significant. I've also seen people throw out data points because they mess up their statistics.

    I've even seen people draw a curved line to separate two populations—or apply a mathematical transform to the data to produce the same effect. There may be rare times when this makes sense, but statistical fantasies are so common that any result that relies solely on statistics should be regarded with suspicion.

    And, as everybody knows, there is also no statistical test that can determine which is the cause and which is the effect. Whoever invents one will make a fortune. There's a good business opportunity here.
  7. Appeal to authority
    Science is probably the world's only example of a functional anarchy. There can never be a central authority in science; any attempt to create one would destroy science. That's why the best science is done in non-authoritarian countries and in non-authoritarian labs. So anyone who says something like “99% of all scientists agree with this; therefore it must be true!” is trying to deceive you. It is manifestly impossible ever to get more than three scientists to agree on anything.
  8. Claims produced in an unfree lab
    A scientific statement is only meaningful if the scientist is saying it freely. When you see a lab where all the scientists agree, it can only mean one thing: they are being coerced. Most people know how this happens in corporations, but it can happen anywhere a person who questions the boss's wacky theory gets canned. So another tip-off is if the lab keeps getting smaller.

    Incidentally, that's why tenured academics often do the best science. There may be good arguments for eliminating tenure, but if you do it without providing an alternative way of maintaining academic freedom, you will destroy science.
  9. Attacking competitors
    The only way scientists can get their ideas listened to is through their professional reputation. Thus, a common tactic of pseudoscientists is to attack others or to claim that others are unfairly attacking them. If your conclusions are sound, they are based on solid evidence. The beauty of science is that the evidence speaks for itself; the scientist can remain silent. A real scientist will merely point to the evidence and ask the reader to decide for him- or herself. If you see someone claiming to be persecuted, the reason is probably that their evidence is weak or nonexistent. That doesn't mean they're not being persecuted. But you can't persecute evidence.

Fake scientists are parasitical: they benefit from the hard work of real scientists, who actually do real work and think critically about how to solve problems that plague mankind. These days there are lots of people trying to infiltrate science to promote their political and financial goals. But not all bad science comes from phony scientists—sometimes even good scientists with decades of research in top institutions fall prey to it. If they're really as good as they think, they figure it out and, it is hoped, repent and come back to the fold.

See also:

Science commentary

Denying Biology
If biology is co-opted by the Left, our future could be written by ideologues.

Fraud in Science
Where there's misconduct in science, there's invariably a deep reservoir of social and managerial pathology that creates it.

What is the role of consensus in science?
The myth that science seeks to achieve a consensus has been debunked many times. But activists continually revive it.

An autopsy of the late global warming movement

What is the value of computer modeling?
If mathematical models are done badly, they will discredit an entire branch of science. It's happened before.

On the Internet, no one can tell whether you're a dolphin or a porpoise
oct 13, 2015; updated oct 14, 2015


to top