science commentary

The futility of modeling

Mathematical modeling is a form of metaphysics.

by T.J. Nelson

science commentary

M y scientific career has recently led me into the unholiest branch of science: model-building. We're trying to develop a drug against one particularly unpleasant brain disease. My scientific intuition tells me this particular drug will probably not kill too many of our patients, if we're lucky. These days that's about as optimistic as I ever get about these things. But we have to try.

Getting the pharmacokinetics right was easy—PK/PD, as it's called, is a particularly simple field of study—but when it came to calculating what happens in the brain, the model predicted the wrong time course.

It's tempting to just tweak the parameters to make the numbers come out right. That would make my boss happy, yet it would make the model a failure. The ultimate success of a model only happens if we get an unexpected—or ‘wrong’—result. That's the only time you ever discover anything worthwhile.

Infrared photo of trees
Tweaking the model until you get the result you know must be true is what scientists disparagingly call “curve-fitting.”

Methodo­logically there is one rule that anyone calling himself a scientist must uphold: knowledge must be traceable to empirical observation. This principle is the basis of all scientific knowledge; any knowledge not so based may or may not be true, but it is not scientific.

This is not just some academic argument. If you define truth as correspondence between what you say and what is really happening, then to accept a theory that is contradicted by evidence would be to accept a falsehood. Underneath it is the metaphysical idea that the universe is knowable and what we see and hear bears some relationship, albeit a noisy one, to reality.

Okay, so maybe it is an academic argument then. Our philosophers have argued for ages about whether truth is a real thing or a metaphysical abstraction.

I used to swear I'd never cite Paul Feyerabend: I was aghast at how badly this philosopher of science understood us. But I'm going to.

Feyerabend viewed scientific education as a form of indoctrination. It's not: you cannot do science productively unless you're familiar with what's been done before. That's what we teach our students. The alternative is for them to risk rediscovering something that's already known. We leave the indoctrination to those hippies on the other side of campus, and we keep our doors secured 24/7 so they don't get in and steal our transgenic mice.

Feyerabend was also wrong when he said there can be no methodological rules in science. Reason, he claimed, interferes with with scientific progress. The Copernican view, he said, replaced witchcraft because reason was overruled at some time in their past.

This is what he meant when he called science an anarchy: a mental anarchy quite different from the usual meaning, which is that there can be no central authority in science.

It was not so much that he was wrong, but that what he advocated was almost totally useless, and had very little to do with how science was really done. It was a form of curve-fitting, trying to squeeze his view of science into a narrow idea of how people should think. Being ‘against method’ is simply an admission that we don't know how to reduce thinking to a set of rules.

The ultimate aversion to method is to imitate nature by putting what we know into a computer. If what comes out differs from our empirical data, we know we missed something. But if we don't believe it, and change our theory instead, it means we were lying to ourselves all along, which means—no offense to political readers—we were just doing politics.

Let's take a real-world example—oh, I don't know, say climate modeling. Suppose your model predicted one thing—say, off the top of my head, as a purely hypothetical example, warming—and instead the climate remained unchanged for, oh, let's say, 18 years or so. If you were a competent scientist your natural response would be to get excited. There's something we don't understand! For a scientist, it's a visceral thrill. We'd expect to see, over the next few years, hundreds—maybe thousands—of papers breathlessly proposing new theories, along with new data and reasoning. It would be an exciting time for science.

Monty Python airspeed velocity graph
Aerodynamic simulation

Yet here we are, 18 years into the ‘pause’, and we're hearing the scientific equivalent of crickets. (Update: There were lots of papers trying to explain it away, though, but that's not what I mean here.) Last week Science magazine had a special issue on global warming, with the predictable handwringing of how bad things are. The latest discovery that biological processes in coral reefs contribute more to ocean acidification than atmospheric carbon dioxide was not even mentioned.

Computer models predict that the reefs will start to dissolve when atmospheric CO2 reaches 560 ppm. But Yeakel et al., writing in PNAS, report that it's not so simple. Reef alkalinity has seasonal variations, with drawdowns in alkalinity coinciding with dissolved inorganic carbon (CO2, bicarbonate, and carbonate) hitting a maximum in the summer months. We all know that bacteria and other living things produce acid as a byproduct of respiration. And it turns out that's exactly what's happening in the reefs: there were large-amplitude swings in pH, with the primary driver being the seasonal phytoplankton blooms.

They observed no trends in seawater CO2-carbonate chemistry resulting from atmospheric CO2, which they attributed to the short 5-year study period. Yet large depletions of alkalinity and dissolved carbonates occurred each year, while reef pH remained constant.

Sunlight, they're saying, causes phytoplankton to draw CO2 out of the ocean, raise the alkalinity, and increase carbonate deposition, while respiration in corals and zooplankton releases CO2 into the ocean and decreases calcification. It's another factor that complicates the simplified narrative that human industrial activity is killing the reefs. Reef biogeochemical processes completely swamp anthropogenic factors, at least over the measurable short term. Empirical research trumps computer models every time.

The models have failed. But that is great—it means we don't understand the system! Anyone who thinks that's a problem is not a scientist, but a dogmatist. Those dogmatists need to stop dictating what conclusions scientists must reach and let them find out the truth.

As the heretic said in Dante's Inferno:

“Like one who has imperfect vision, we see things that are distant from us ... but when they approach, or come to be, our intelligence is wholly void, and we know nothing of your human state, except what others tell us.”

This might as well be our computers talking, if only we would listen to them.

See also:

Related Articles

An autopsy of the late global warming movement

What is the value of computer modeling?
If mathematical models are done badly, they will discredit an entire branch of science. It's happened before.

Name and address
nov 15, 2015


book reviews