randombio.com | commentary
Monday, March 30, 2020

The four classes of science

We are awash in unfalsifiable computer predictions. But if a prediction can never be proven wrong, is it still science?


T his week, an article in Science described how physicists are “brawling” over dark matter. Here's how the brawl started: a group of astrophysicists led by one Benjamin Safdi claimed to have been unable to detect a 3.5 keV signal from sterile neutrinos, which are hypothetical particles thought to constitute dark matter. Another physicist, a Doctor Alexey Boyarsky, brutally savaged Safdi, saying: “I think this paper is wrong.”

Okay, it's not exactly WWE. But we can make a mountain out of this baby. Just hang on.

Another story in Nature says there is now a vast ozone hole, maybe the biggest ever recorded, over the Arctic. According to the dogma, it shouldn't be there: the 1987 Montreal Protocol banning chlorofluorocarbons did away with the cause, declared to be CFCs, over 30 years ago.

2020 Arctic ozone hole
2020 Arctic ozone hole. Source: Nature

The new ozone hole is being ignored in the press, partly because they're too busy keeping us in a state of panic about some virus, but also because it doesn't fit any narrative. How to explain something that doesn't fit the theory? Simple: somebody must be doing something bad. Maybe China is manufac­turing CFCs, or maybe there's vast amounts of them leaking from old air conditioners somewhere. The alternative, that the theory needs a tiny bit of tweaking, is unthinkable.

Now, don't panic: the Arctic ozone hole is no more dangerous than the Antarctic ozone hole was. But it raises the question: what constitutes science? It seems to me there are four classes of science.

  1. Causation The goal of science is to establish cause and effect. Scientists design experiments that intervene in some natural process in some way, creating a counterfactual that illuminates cause and effect. Only when causation is established by interventionist experiments is something scientifically solid.
  2. Description Science is based on observations. Any repeatable observation is science, provided it's consistent with our current theories. For instance, the sun comes up every morning; fish usually swim underwater.
  3. Guys in white coats Science is anything that looks and sounds scientific. The harder it is to understand and the more abstruse the math, the more scientific it is.
  4. Instrumentalism Science is technical information that leads to some desired result. If it scares people into doing the right thing, it's science. Postmodernists claimed all science was instrument­alist because it was a narrative designed to support the power structure.

Class 4 science is rightly dismissed as junk. But a few years ago one physicist proposed that theoretical physics, which is regarded as a Class 1 science, should become a Class 3 science. In a talk about string theory, which proposes the existence of items that are far too small ever to be observed experimentally, he suggested that we abandon the demand for experimental verification altogether. It led to a brutal knock-down, drag-out fight: his colleagues generally agreed to stick with empirical validation for now.

Damn, it's hard to make these things exciting.

Hard sciences like physics or chemistry are in Class 1. Those in category 2 and 3, including climate studies and sociology, have shakier reputation. In a hard science, calling someone's paper “descrip­tive” is the equivalent of body-slamming them and smashing a folding chair over the author's head.

Where does the ozone hole theory stand? It can never be tested empirically, only with models. This is a problem: the only value of a model is to falsify a hypothesis. A model is like those logical proofs that philosophers use: if it leads to a contradiction, it means your theory is wrong. If it doesn't, it means your reasoning could explain the observation. It doesn't mean your reasoning is sound, and it's not true until confirmed by experimentation.

In a real sense, models are unfalsifiable; they only predict what might happen if the assumptions are valid. The mathematics may or may not model the real mechanism on any deep level; increasingly, all that matters is whether the model predicts the effect. A model cannot give us new knowledge, only new hypotheses.

Theoretical physicists recognize the risk: they live with the knowledge that all their beautiful equations can be flushed into the sewer by a single observation. Models also lead to claim inflation, where only those models that predict bigger effects are publishable. This has brought entire branches of science into disrepute.

Predicted COVID-19 deaths per day
Predicted COVID-19 deaths per day (from Ferguson et al., 2020, Fig. 1a)

Last month, epidemiologist Neil Ferguson[1] created a model that predicted 56,000 deaths per day and a total of 2.2m deaths in the USA and 500,000 in the UK from the Wuhan coronavirus. The authors said that ‘mitigation’ (targeting at-risk individuals) would be ineffective and ‘suppression’ (targeting everybody), would be needed. It was telling that, despite the fact that the R0, risk factors, and mortality rates aren't known with any degree of certainty, no confidence limits were on the curve (see figure), a tip-off that it was not intended to be taken as a real prediction but as an instrument of persuasion: a warning, not a prediction, and thus immune from scientific rebuttal.

One prediction was that by sequestering the population until a vaccine is available, which could take 18 months or more, the number of deaths could be reduced by 97.8 to 98.6% depending on whether R0 was 2.6 or 2.0. Just to scare us even more, they emphasize that it's possible nothing may work in the long term.

It had the intended effect: the paper was passed around in medical schools by terrified administrators and physicians, and in government offices. When observations showed the actual deaths to be unlikely to ever reach the predicted number, the response was not that the model was wrong, but that they proved it right: the government-mandated social isolation inspired by the model prevented the catastrophe. Champagne all round.

Maybe it was right, for some values of right. And no doubt, given the parameters, the calculations are correct. But the gnawing question is: if it can never be wrong, is it still science?


1. Ferguson NM et al. (2020). Impact of non-pharmaceutical interventions (NPIs) to reduce COVID-19 mortality and healthcare demand. Link


Correction An earlier version of this article said there were only three classes of science. We don't apologize and we're not sorry about it one little bit.


mar 30 2020, 6:50 am


Related Articles

Be skeptical about overly-pessimistic epidemiological models
Epidemiologists and their computer models cannot predict how big an epidemic will be.

The standard model of sock physics
Socks actually tell us a great deal about quantum mechanics. Unfortunately, most of what they tell us is wrong.

The futility of modeling
Mathematical modeling is a form of metaphysics.


On the Internet, no one can tell whether you're a dolphin or a porpoise

back
science
book reviews
home