randombio.com | science commentary
Saturday, July 04, 2020. Updated Sunday, July 05, 2020

Hydroxychloroquine is great again

Two higly publicized papers—one on HCQ and one on dexamethasone—show the dangers of relying too much on statistics

O ver the past few months, hydroxychloroquine (HCQ) has gone from being a killer drug banned by the FDA to the greatest thing evah. This week another retrospective study[1] came out claiming that HCQ reduced fatalities from COVID-19, reducing the hazards ratio by 66%. Some are hailing this as vindication for President Trump. They need to be more cautious.

In the study, data from 2541 patients in multiple hospitals were analyzed. The primary end point was mortality. The results are as follows:

Group   Died   Survived   Percent died   P value  
Untreated 108 30126.4
HCQ 162104013.5 <0.0001
Azithromycin 33 11422.4 0.377
HCQ + azithromycin 157 62620.1 0.0154

As the table shows, HCQ reduced mortality by almost half. It was a highly statistically significant effect. (In the table, I re-calculated the authors' stats using Fisher's exact chi-square test.) HCQ is also significantly different from HCQ + azithromycin at p<0.0001.

The evidence from in vitro studies[2] and the Gautret / Raoult study[3] suggest a synergistic effect. Based on the structure, there's no suggestion that the two molecules would form a complex or react with each other, so any synergism would probably happen on a cellular level. The results in this study don't support a synergistic effect.

However, the subject population was highly unbalanced. There were significantly more patients in the untreated group who were younger than 65, male, and white, and who had lower BMI, normal O2 saturation, and less chronic lung disease. This means that the study was, on average, strongly biased toward not seeing a beneficial effect. This is similar to many other previous retrospective studies and it means that HCQ was preferentially given to the sicker patients, as often happens.

Why then would the HCQ treated patients have a better survival? One possible answer is that they also got other treatments. In the table below, 78.9% of the HCQ patients also got steroid (methyl­pred­nisolone and/or prednisone). I also added a column showing that 13.8% were on a ventilator compared to 8.3% of controls—both are significant at p<0.001.

Group   % Given steroid   % On ventilator  
Untreated 35.7 8.3
HCQ 78.913.8
Azithromycin 38.8 9.5
HCQ + azithromycin 74.329.9

To correct for these confounding factors (which are quite remarkable: a whole table showing p<0.001 differences for almost every baseline parameter), the authors did some sophisticated statistical analysis called propensity score matching. The study is certainly encouraging, and there are many other studies that confirm a benefit but also many that find no benefit. There are also some studies that used suspicious data, faulty design, or faulty analysis to claim that HCQ increased the death rate; to my knowledge, all of those have been thoroughly discredited.

The FDA clearly erred in revoking the emergency use authorization. According to what we know about viruses, HCQ should work, at least a little, as it raises the pH in the endosome where the virus gets processed. But none of these factors have any bearing on whether the drug is effective in the real world, and even with this study at hand we cannot say that has been established yet.

My advice: don't get too excited about this finding. It's not a randomized controlled clinical trial. It's a retrospective study with a very unbalanced population. The fact that the HCQ group differs from HCQ + azithromycin at p<0.0001 is a big red flag that something could be very wrong with these numbers.

There are times, as in image analysis, when statistics can pull signals out of the noise. But this only works when the human brain can confirm the result (a landscape of Mars, for instance). In a clinical trial we can't. Trying to rebalance a population after the fact doesn't work. I know people in the drug industry who have tried it; ultimately, they were forced to run another trial.

The general rule is: if something goes wrong in your experiment, or if it's badly designed, you can't just fix it with statistics.

Dexamethasone protects you against doctors with ventilators


The results are consistent with a highly publicized non-peer-reviewed preprint by Hornby et al., who claimed that dexamethasone, a corticosteroid, reduces mortality in COVID-19 patients receiving oxygen or on ventilators, but not in less severe patients[4]. Some others[5] are urging caution about this, and several authors previously recommended against corticosteroids, as steroids will suppress the antibody response and T-cell response that are necessary to fight the disease. As every med student knows, cortico­ster­oids also need to be administered with antimicrobials to prevent infection.

Both Science magazine and Nature had high praise for the dexamethasone finding because of the large number of patients. Here is the table of 28-day mortality from the paper (I recalculated their statistics using Fisher's exact test):

  Dexamethasone Usual care
   Died  Alive  Total  %Died  Died  Alive  Total  %Died  p-value 
No oxygen 85 431 501 17.0 137 897 103413.4 0.091
Oxygen 275 1004 1279 21.5 650 1954 260425.0 0.018
Ventilator 94 230 324 29.0 278 405 68340.7 0.0004
All patients 454 1650 2104 21.6 1065 3256 432124.6 0.0065

Large populations are better, but there's a risk: you can get a high level of statistical significance from an effect that's so small it's not medically significant. Also, statistics can be misleading. If eight more patients had died—283 instead of 275—in the dexamethasone + oxygen group, the difference wouldn't have been statistically significant.

This effect looks impressive, but what does it mean? Dexamethasone mainly benefited patients who were on a ventilator. The authors would probably say dexamethasone is suppressing lung inflam­mation. But you could also say it protects you from doctors coming at you with their ventilators.

See you next year, same time, same station.

1. Arshad S, Kilgore P, Chaudhry ZS, Jacobsen G, Wang DD, Huitsing K, Brar I, Alangaden GJ, Ramesh MS, McKinnon JE, O'Neill W, Zervos M (2020). Treatment with Hydroxychloroquine, Azithromycin, and Combination in Patients Hospitalized with COVID-19 Int J Infectious Diseases 28 May 2020 DOI: https://doi.org/10.1016/j.ijid.2020.06.099

2 Andreani J, Bideau ML, Duflot I, et al. In vitro testing of combined hydroxychloroquine and azithromycin on SARS-CoV-2 shows synergistic effect. Microbial Pathogenesis. 2020;145:104228. doi:10.1016/j.micpath.2020.104228.

3. Gautret P, Lagier JC, Parola P, Hoang VT, Meddeb L, Mailhe M, Doudier B, Courjon J, Giordanengo V, Vieira VE, Dupont HT, Honoré S, Colson P, Chabrière E, La Scola B, Rolain JM, Brouqui P, Raoult D (2020). Hydroxychloroquine and azithromycin as a treatment of COVID-19: results of an open-label non-randomized clinical trial. Int J Antimicrob Agents. Mar 20, 105949. doi: 10.1016/j.ijantimicag.2020.105949. PMID: 32205204

4. Horby P, Lim WS, Emberson J, Mafham M, Bell J, Linsell L, Staplin N, Brightling C, Ustianowski A, Elmahi E, Prudon B, Green C, Felton T, Chadwick D, Rege K, Fegan C, Chappell LC, Faust SN, Jaki T, Jeffery K, Montgomery A, Rowan K, Juszczak E, Baillie JK, Haynes R, Landray MJ (2020). Effect of Dexamethasone in Hospitalized Patients with COVID-19: Preliminary Report
doi: https://doi.org/10.1101/2020.06.22.20137273.
(non-peer-reviewed preprint)

5. Theoharides TC, Conti P (2020). Dexamethasone for COVID-19? Not so fast. J Biol Regul Homeost Agents. Jun 4;34(3). doi: 10.23812/20-EDITORIAL_1-5. PMID: 32551464

jul 04 2020, 11:19 am. updated jul 05 2020, 6:23 am. minor error in table fixed jul 05 2020 5:41 pm. figure added jun 05, 6:00 pm. last edited jun 07 2020, 5:14 am

Related Articles

Another misleading observational study on HCQ and CQ
The news media are celebrating a new study on hydroxychloroquine and chloroquine in COVID-19 that was badly mis-analyzed.

Underground virologists question the origins of SARS-CoV-2
Sophisticated structural and sequence analyses of the Wuhan coronavirus are popping up on the Internet

Does COVID-19 cause a type of immunodeficiency?
A new theory may explain why disastrous effects of COVID-19 are seen in some patients but not others

Here's what we know so far about clinical trials on COVID-19 (Updated)
Chloroquine results questioned; new system for starting trials rapidly is needed (Updated)

On the Internet, no one can tell whether you're a dolphin or a porpoise

book reviews