veryone who's done computer modeling knows the dilemma: what do you do when
the model doesn't give you the right answer? Should you “tune”
the model by changing the parameters, or should you report that your model,
which you created with weeks of sweating out code, doesn't work?
A while back I created a pharmacokinetic model to predict the concentration of some drug my boss was interested in. To do that, you have to know a bunch of parameters like the elimination rate, absorption rate, and the rate constants for how the drug travels from one compartment to another. These numbers are different for each drug.
I did all that, pulling numbers from papers where we lacked them, and built the model. It predicted the first forty-eight hours pretty well. At that point, they gave the patient a second dose. The model went one way and the patient—we only had one at the time—went another. The system had already started to adapt to the drug, “downregulating” some proteins and upregulating others. Each time that poor patient got another dose, the system changed more. I found that not only was the biochemistry changing, but by the tenth dose the elimination rate had dropped by a factor of ten, which meant this drug (not its metabolite—we checked that)—would eventually accumulate to toxic levels.
Needless to say, my boss didn't like that result. But without the model we never would have noticed that the elimination rate was changing. If I'd tuned it, I would have been telling the computer to produce the answer the boss wanted, and we would have thought everything was going just as we originally guessed. In effect I would have thrown away—denied, in contemporary parlance—everything the model was telling me, which was that we did not fully understand what was happening.
That is what we build models for. We don't build them to tell us what we want to hear, but to tell us whether we understand the system. If they give us the wrong information, that is valuable information. It takes confidence in your own programming skills to know there were no mistakes and therefore the theory is incomplete. More than that, it takes courage to tell your boss, who might be a virtual boss or some government funding agency, that the model is telling him that the computer can't give him the answer he wants.
Models are only as good as the programmer is honest. When an astronomer calculates the trajectory of an asteroid, or an engineer calculates the stability of a bridge, the program is not “tuned.” It doesn't need to be. It uses mathematical equations based on basic principles. If those equations predict that your building will move too much or not enough, the engineer jolly well needs to know about it.
If you predict a drug is safe and patients start dropping like flies, you have a problem. If you predict the opposite and your boss acts on it, your company has a problem.
Of course the computer always gives some answer. In the case of those infamous global circulation models, maybe it says the Earth should warm by fifty degrees, or maybe that the globe should be cooling when you know it must be warming. So you tune it by doing what every first-year physics student knows is an invalid procedure: by throwing in a fudge factor to make the answer come out correct.
The model must be tuned, for without it the model cannot reproduce past measurements. If the model should predict no warming, there is no problem to solve, you would immediately be rendered unemployable, and your field would once again become a backwater. But how long can you go on tuning your model more and more, making the parameters more and more disconnected from any physical laws, knowing you're only discrediting your profession, before you realize you're not predicting anything at all but doing curve-fitting?
Range of global warming predictions in the IPCC Fifth Assessment Report (Climate Change 2014: Impacts, Adaptation, and Vulnerability Part A), p.179. Dashed lines are different simulations with different “storylines.” Heavy lines are RCP (Representative Concentration Pathways), i.e. different “mitigation” schemes. Source
Proof that this strategy is inadequate can be seen in the widely varying estimates of warming of different models in the image at right. Estimates of error, which might indicate some attempt at scientific rigor, if present in the simulation, are not published by the IPCC. Clearly the science, if it can be called that, is not “settled.”
The graph shows that the modelers are not really modeling the atmosphere at all, but society. Their goal is not to understand what's happening, but to get the computer to tell them they are allowed to change society.
To a climate modeler it's a no-win situation. In academia there are two things you must never do: criticize a colleague by name, and deprive your colleagues of funding. If you broke either of these two rules you'd instantly become a persona non grata. They would call you a denier, burn your books, and prevent you from ever publishing again.
You know using the Stefan-Boltzmann law and a few basic parameters won't give you the correct answer. So you convince yourself that the planet is just too complicated and the problem is too urgent and we need a model even if it's flawed. And lo and behold your program predicts a catastrophe.
Then some people believe your predictions and start gluing themselves to famous paintings like John Constable's The Hay Wain or throwing tomato soup on priceless art masterpieces. (So far no one has thrown tomato soup on Andy Warhol's Tomato Soup. At least doing that would be ironic. By no means am I suggesting that anyone should do it.) When fanatics are on your side, it is never a good sign.
The loss of credibility of a major branch of science is no small problem. If a real climate catastrophe should someday be on the horizon, will anyone believe the next prediction? Or will they say, as one cartoon character in Futurama said, “Fool me seven times, shame on you. Fool me eight or more times, shame on me”?
Maybe we should look at the bright side: at least the Just Stop Oil folks are bonding.
oct 28 2022, 5:40 am. last updated oct 30 2022, 6:49 am
