Another misleading observational study on HCQ and CQ

he news media are celebrating the results of a new study in The Lancet ^[1] claiming that hydroxychloroquine and chloroquine not only don't work, but increase death rates in patients. See here and here for examples.

I've been skeptical about both drugs, and there is certainly reason to worry about giving it to COVID-19 patients, because the disease attacks the cardiovascular system as well as the lung, which means cardiac problems that might be minor in healthy patients could be fatal in sick ones. But what I or anyone else thinks doesn't matter. What matters is whether these drugs are effective.

The mechanism of action of these drugs isn't really clear. Most researchers think they raise the pH in the endosome, which would slow down the rate of protease enzymes that cleave and activate the virus. This is a reasonable hypothesis because that's how they work against malaria. In this case, they should be given early or even prophylactically.

Other researchers think they act as anti-inflammatory agents, in which case they might inhibit cytokine storm but have no effect on the virus. If so, they would work best on late-stage patients to inhibit cytokine storm. There is evidence for and against both hypotheses.

Didier Raoult

Let's leave aside for the moment that the media have mainly reported the trials that purported to show that HCQ doesn't work and is dangerous, while remaining mostly silent on the ones that claim to show efficacy. Let's also leave aside the previous data-mining studies that have come out—such as the VA study—which were badly done. Is this study any good?

Before we get to that, let me talk about Dr Raoult. Most people know Didier Raoult as the hydroxy­chloro­quine guy. Some people try to portray him as a quack—a slur that's becoming increasingly popular in the press. But he's no quack. There's a reason doctors got excited about his claim about HCQ: Raoult is famous—a revered figure in virology.

It was Raoult's team who discovered a new mimivirus in a cooling tower in Paris that turned out to be one of the biggest viruses ever found—over 400 nm. His team also discovered virophages: viruses that infect other viruses. Their mimivirus had a small satellite virus attached to it, which they called Sputnik. Sputnik kills other viruses by causing them to create defective capsids. This was a discovery of enormous significance.

Science doesn't run by authority, but making a big discovery means your intuition is good, and that means something in science. Of course, it doesn't mean he's right about HCQ.

Zinc

There are also anecdotal stories out there that zinc is necessary for HCQ/azithromycin to work. I'm sceptical about these claims, as they seem to be based on misinterpretations of in vitro studies. But many people don't realize that ACE2, the receptor for SARS coronaviruses, is a zinc metalloprotease, which means (1) it requires zinc and (2) it cuts proteins. As far as I know, no one has ever tested whether ACE2 might cleave the spike glycoprotein in the virus or what the effects of zinc on its activity might be. Zinc might help or it might be harmful. Maybe that's worth testing sometime.

The Lancet article

Okay, so on to the article^[1]. It's not a clinical trial at all. It's a registry analysis—another observational study. The authors downloaded deidentified patient records from 671 different hospitals in the FDA Surgical Outcomes Collaborative Database, which had records from 96,032 patients hospitalized for COVID-19.

Patients who did not have a virus test or who tested negative were excluded. Patients were also excluded if the treatment started >48h after diagnosis, if it started while on ventilator, or if they got remdesivir. The authors tried to adjust the results by doing complicated statistical treatments by defining subgroups from their gigantic control population that matched the treatment group within 10% on a number of factors, including SPO2, ACE inhibitors, age, BMI, sex, race, statins, and comorbidities.

Here are the results they reported.

Mac = macrolide, which means azithromycin or clarithromycin. In the above table, I reanalyzed the p-values from the authors' table using a chi-square test with Yates correction, not that it would make much difference with an N of 81,000. All the treatments were associated with increased mortality with a high level of statistical significance.

Looking at the table you might conclude that HCQ and CQ are very bad for the patients. The patient demographics look reasonably well matched. But here's the problem: the doses of CQ and HCQ were all over the map. The mean doses used in the 671 hospitals were:

This is a big standard deviation. Below is a graph generated in my statistics program that shows what a 765±308 SD distribution looks like with 1868 samples. To get their RSD of 40.3%, some patients would have had to receive 1300 mg and others as little as 230. So considering the variance in the time on the drug, their total dosage could have ranged from 550 to to 14,027 mg. That puts it well into the toxic range (12,000 mg over 10 days) seen in the Brazil study. This doesn't even consider variations in patient weight, which nobody seems to do, but which typically can vary by a factor of two. Thus the deaths might not have been randomly distributed among the patients, but in patients who got more drug. Based on the narrow therapeutic index of these drugs and their propensity to accumulate due to a slow elimination rate, that's almost certainly what was happening.

Artificial data of 1868 samples with mean=765 and standard deviation=308

The authors are well aware of the limitations of observational studies. They write:

Nevertheless, a cause-and-effect relationship between drug therapy and survival should not be inferred. . . . Randomised clinical trials will be required before any conclusion can be reached regarding benefit or harm of these agents in COVID-19 patients.

You might also remember the infamous VA study study where they gave sicker patients HCQ—nothing wrong with that—but then reported that HCQ was killing the patients, something that was absolutely not truthful. We don't have enough information to assess that possibility here.

What this study shows is that it's possible for people to die from getting too much HCQ or CQ. Of course, this was already well known. What we needed to know was: how much is too much?

With 1800 patients, the authors could have easily created a dose-response curve, which would have told us that. The authors overlooked the single most important piece of information in that database. Instead they averaged all the doses, which varied by as much as 25.5-fold, and then told us that their results have no causal significance.

The study suggests what we already knew: HCQ and CQ aren't a miracle cure. But honestly, everybody and his aunt ought to know how to run a randomized placebo-controlled study by now. Heaven knows I've ranted about it enough times, and I'm far from alone.

I have little hope for HCQ (even Raoult's study didn't show a significant effect for HCQ alone), but I'm tired of seeing data-mining studies reported as if they were clinical trials. In this paper, statisticians lumped incompatible data together and buried a disclaimer that their study had little value. Instead of estimating the maximum safe dose, which would have been valuable and easily derived from the database, they told us what we already knew, and told the editors (and the press) what they wanted to hear.

Moral: you can have a million patients in your study, but if you analyze the data wrong, you'll still end up with garbage.

Update, Jun 04 2020 The authors of this study have retracted the paper from Lancet after an investigation into ‘suspicious data.’

1. Mehra MR, Desai SS, Ruschitzka F, Patel AN (2020). Hydroxychloroquine or chloroquine with or without a macrolide for treatment of COVID-19: a multinational registry analysis. https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(20)31180-6/fulltext Published: May 22, 2020 DOI:https://doi.org/10.1016/S0140-6736(20)31180-6

may 23 2020, 11:23 am. minor edits may 24 2020, 9:46 am