Prophylaxis against covid-19: living systematic review and network meta-analysis

Dear Editor,

This living systematic review and meta-analysis reported by Bartoszko et al. (1), and the associated WHO guideline on “Drugs to prevent COVID-19” (2), contain serious flaws in data interpretation and judgement regarding hydroxychloroquine chemoprophylaxis. This was pointed out in our Rapid Response to the WHO guideline on the 29th of March (2). Both articles describe “high certainty” evidence from randomised controlled trials (RCTs) that hydroxychloroquine prophylaxis neither reduces mortality nor reduces hospital admission, and “moderate certainty” evidence of poor tolerability from a significantly increased rate of adverse events (AEs) leading to drug discontinuation (1-3). Yet in the three pre-exposure prophylaxis RCTs evaluated (4-6) there were no deaths, and there were only two COVID-19-related hospital admissions. Furthermore, there is an error in the analysis of study drug discontinuations. After correction there is no longer a statistically significant difference between those taking hydroxychloroquine and the controls (7).

Antivirals, if they are effective, are expected to provide greatest benefit early in the course of an individual COVID-19 infection, when viral burdens are greatest, and not in severely ill hospitalised patients when inflammatory processes dominate. There is no doubt that hydroxychloroquine is ineffective in hospitalised patients with COVID-19 (8,9), but results from hospitalised patients should not be pooled with results from ambulant people who are either well or mildly ill (10,11). The GRADE reporting framework requires authors to consider the similarity of the trials included in their synthesis in terms of the population, interventions, cointerventions, outcomes, and study methods (12). The evidence base for hydroxychloroquine prophylaxis in COVID-19 assessed in the systematic review (1) is a heterogeneous assortment; three RCTs in post-exposure prophylaxis (PEP- which approximates to early COVID-19 treatment), and three in pre-exposure prophylaxis (PrEP- true COVID-19 prevention) (1-3). The largest PrEP study (>75% of the PrEP data) used a low dose of hydroxychloroquine, similar to antimalarial chemoprophylaxis, rather than the rheumatoid arthritis doses used in most other COVID-19 prophylaxis trials (5). The six RCTs, conducted either in North America or Europe, enrolled 6,059 participants, but they provided few endpoints. In the three PrEP trials there were only 26 confirmed COVID-19 cases in total (15 of 1,197 [1.25%] randomised to hydroxychloroquine versus 11 of 687 [1.6%] randomised to placebo).

A previous meta-analysis of the hydroxychloroquine prevention studies, which evaluated the appropriately adjusted primary endpoints (13), estimated a meta-analytic risk-ratio of 0.86 (95% confidence interval [CI] 0.70 to 1.06) in the direction of benefit from hydroxychloroquine. Since then, a large cluster randomised trial has been reported from Singapore in which there was a borderline significant protective benefit from hydroxychloroquine prophylaxis (14). Whilst there is certainly not enough evidence to recommend hydroxychloroquine for COVID-19 prophylaxis (there never has been), how can the small and heterogeneous evidence base that contributed to this systematic review and to the formulation of the WHO guideline justify the statement that “Hydroxychloroquine did not reduce the rate of SARS-CoV-2 infection, admission to hospital or mortality” (1) when, in the PrEP studies, there were slightly fewer cases (15 of 1,197 versus 11 of 687), only two COVID-19 admissions and no deaths? This was described as “high certainty evidence”. A much more sensible conclusion would have been that there remains substantial uncertainty.

We have previously contrasted this assessment based on few or no end-points (1) with the earlier assessment, by the WHO COVID-19 guidelines group, of dexamethasone and hydroxychloroquine in hospitalised COVID-19 patients (7, 15, 16). Those judgements in September 2020 were based on the very large RECOVERY platform RCT (17) in which there were 980 deaths with an odds ratio for death in dexamethasone recipients receiving respiratory support of 0.82 (95% CI 0.72 to 0.92). For hydroxychloroquine, lack of efficacy was concluded from the outcomes of 10,859 mainly hospitalised patients (almost half from the RECOVERY trial) with over 2,000 deaths. The stratified meta-analytic estimate for mortality when combining data from the RECOVERY (8) and SOLIDARITY (9) trials (these data amount to 60% of all hospitalised patients in RCTs, and both trials used the same hydroxychloroquine dosage) resulted in a 95% CI for the risk ratio of 0.98 to 1.21 (9). Yet both of these landmark results were graded only as “moderate certainty evidence” (defined as: “the true effect is probably close to the estimated effect”) with serious risk of bias (15,16). It makes no sense that these effect estimates (and thus the certainty of the treatment recommendations) in severe COVID-19, based on thousands of deaths in well conducted RCTs, are considered less certain (i.e. less reliable) than a mortality estimate for hydroxychloroquine post-exposure prophylaxis derived from 13 deaths (5 hydroxychloroquine versus 8 controls), and for pre-exposure prophylaxis based on no deaths at all (1).

The meta-analysis (and as previously highlighted, the WHO guideline on “Drugs to prevent COVID-19”) miscoded the number of study discontinuations in the study by Grau-Pujol et al. (6). After correction, there is no longer a significant difference between hydroxychloroquine and placebo recipients in premature study drug discontinuations (we used an inverse variance method to pool estimates as implemented in the R package meta; the fixed-effects model estimates a 95% CI for the odds ratio of 0.83-3.29). Thus, the statement that more people discontinued hydroxychloroquine because of adverse effects is also flawed. The recent Singapore study confirmed the generally good tolerability of hydroxychloroquine chemoprophylaxis (14).

It is already clear that hydroxychloroquine does not provide high efficacy in COVID-19 prevention or early treatment. At best it might have modest protective efficacy, but to say with “high certainty” that it does not do this is unwarranted and unjustified (1), and it devalues the important and impartial processes that should be used to synthesise evidence and underpin therapeutic guidelines.

REFERENCES
1. Bartoszko JJ, Siemieniuk RAC, Kum E, Qasim A, Zeraatkar D, Ge L, Han MA, Sadeghirad B, Agarwal A, Agoritsas T, Chu DK, Couban R, Darzi AJ, Devji T, Ghadimi M, Honarmand K, Izcovich A, Khamis A, Lamontagne F, Loeb M, Marcucci M, McLeod SL, Motaghi S, Murthy S, Mustafa RA, Neary JD, Pardo-Hernandez H, Rada G, Rochwerg B, Switzer C, Tendal B, Thabane L, Vandvik PO, Vernooij RWM, Viteri-García A, Wang Y, Yao L, Ye Z, Guyatt GH, Brignardello-Petersen R. Prophylaxis against covid-19: living systematic review and network meta-analysis. BMJ. 2021 Apr 26;373:n949. doi: 10.1136/bmj.n949. PMID: 33903131; PMCID: PMC8073806.
2. World Health Organization. Drugs to prevent COVID-19: A WHO living guideline. https://www.who.int/publications/i/item/WHO-2019-nCoV-prophylaxes-2021-1. accessed 11 March 2021.
3. Lamontagne F, Agoritsas T, Siemieniuk R, Rochwerg B, Bartoszko J, Askie L, et al. A living WHO guideline on drugs to prevent covid-19. BMJ. 2021;372:n526.
4. Abella BS, Jolkovsky EL, Biney BT, et al, Prevention and Treatment ofCOVID-19 With Hydroxychloroquine (PATCH) Investigators. Efficacy and Safety of Hydroxychloroquine vs Placebo for Pre-exposure SARS-CoV-2 Prophylaxis Among Health Care Workers: A Randomized Clinical Trial. JAMA Intern Med 2021;181:195-202.
5. Rajasingham R, Bangdiwala AS, Nicol MR, Skipper CP, Pastick KA, Axelrod ML, et al. Hydroxychloroquine as pre-exposure prophylaxis for COVID-19 in healthcare workers: a randomized trial. Clin Infect Dis. 2020: doi: 10.1093/cid/ciaa1571.
6. Grau-Pujol B , Camprubí D, Marti-Soler H, Fernández-Pardos M, Carreras-Abad C, Velasco de Andrés M et al. Pre-exposure prophylaxis with hydroxychloroquine for COVID-19: initial results of a double-blind, placebo-controlled randomized clinical trial. PREPRINT (Version 1) available at Research Square [+https://doiorg/1021203/rs3rs-72132/v1+]. 2020.
7. Schilling WHK, Callery J, Chandna A, Hamers RL, Watson JA, White NJ. The WHO guideline on drugs to prevent COVID-19: small numbers- big conclusions. Wellcome Open, in press.
8. RECOVERY collaborative group, Horby P, Mafham M, Linsell L, Bell JL, Staplin N, et al. Effect of Hydroxychloroquine in Hospitalized Patients with Covid-19. N Engl J Med. 2020; 383: 2030-20403.
9. SOLIDARITY, Pan H, Peto R, Henao-Restrepo AM, Preziosi MP, Sathiyamoorthy V, et al. Repurposed Antiviral Drugs for Covid-19 - Interim WHO Solidarity Trial Results. N Engl J Med. 2021; 384: 497-511.
10. White NJ, Strub-Wourgaft N, Faiz A, Guerin PJ. Guidelines should not pool evidence from uncomplicated and severe COVID-19. Lancet, Mar 22: S0140-6736(21)00469-4. doi: 10.1016/S0140-6736(21)00469-4.
11. Paules CI, Fauci AS. COVID-19: The therapeutic landscape. Med (N Y). 2021 Apr20. doi: 10.1016/j.medj.2021.04.015. Epub ahead of print. PMID: 33899041; PMCID:PMC8057546.
12. Shepperd S, Straus S. GRADE: complex decisions. J Clin Epidemiol 2011; 64: 120-1.
13. García-Albéniz X, Amo Jd, Polo R, Morales-Asencio JM, Hernán MA. Systematic review and meta-analysis of randomized trials of hydroxychloroquine for the prevention of COVID-19. medRxiv. 2021:2020.09.29.20203869.
14. Seet RCS, Quek AML, Ooi DSQ, Sengupta S, Lakshminarasappa SR, Koo CY, So JBY, Goh BC, Loh KS, Fisher D, Teoh HL, Sun J, Cook AR, Tambyah PA, Hartman M. Positive impact of oral hydroxychloroquine and povidone-iodine throat spray for COVID-19 prophylaxis: An open-label randomized trial. Int J Infect Dis. 2021;106:314-322. doi: 10.1016/j.ijid.2021.04.035. Epub ahead of print. PMID:33864917; PMCID: PMC8056783.
15. World Health Organization. Corticosteroids for COVID-19. https://www.who.int/publications/i/item/WHO-2019-nCoV-Corticosteroids-20.... Accessed 11 March 2021.
16. World Health Organization. Therapeutics and COVID-19. https://www.who.int/publications/i/item/WHO-2019-nCoV-therapeutics-2020.1. Accessed 11 March 2021.
17. RECOVERY collaborative group, Horby P, Lim WS, Emberson JR, Mafham M, Bell JL, et al. Dexamethasone in Hospitalized Patients with Covid-19. N Engl J Med. 2021;384: 693-704.

William HK Schilling1,2, James Callery1,2, Arjun Chandna1,3, Cintia Cruz1,2, Raph L Hamers1,4, James A Watson1,2, Nicholas J White1,2
1. Centre for Tropical Medicine & Global Health, Nuffield Department of Medicine, University of Oxford, UK
2. Mahidol Oxford Tropical Medicine Research Unit, Faculty of Tropical Medicine, Mahidol University, Bangkok, Thailand
3. Cambodia Oxford Medical Research Unit, Angkor Hospital for Children, Siem Reap, Cambodia
4. Eijkman Oxford Clinical Research Unit, Jakarta, Indonesia