Highly biased meta analysis. Authors originally wrote a highly biased meta analysis that avoided statistical significance on individual outcomes with extreme exclusions [Popp]
, although efficacy was still seen when looking across all outcomes. Authors modified the protocol published a short time before, thereby performing a retrospective analysis, clearly designed to produce a desired outcome.
Authors indicated they would update the analysis but did not for a very long time. Authors would have been unable to maintain the lack of statistical significance with the protocol. In this new meta analysis, authors invented a new method to exclude most studies, thereby producing another retrospective analysis, again clearly designed to produce the desired outcome.
Authors included only 14 of 60 studies in the original analysis. For the update, they include even fewer studies, 11 of 87. A key method used was excluding studies without confirmation of prospective registration, however authors violate this to include the Together Trial [twitter.com]
, which accounts for 40% of the patients in the extreme subset selected.
In the companion article for the new exclusion methods, authors note that one indicator for exclusion is if the observed risk reduction is too large [medrxiv.org, ]
. The trial resulting in Paxlovid approval would be excluded on this basis.
As just one example of extreme bias, authors classify the Together Trial as low risk of bias. In fact, this trial has refused to release data despite pledging to, has reported multiple impossible numbers, and had blinding failure and randomization failure, along with many other issues [Reis]
The analysis is also very out of date, including trials only up to April 2022, and including only trials with >1,000 patients since Dec 16, 2021 (yet another cherry-picking mechanism).
With regards to ivmmeta:
- authors claim ivmmeta "states the FLCCC and BIRD as its resources". This is false, there is no relationship with FLCCC or BIRD.
- author's discussion of pooled estimates is disingenuous. ivmmeta reports individual outcome results which are the first item discussed in the abstract. The advantages and disadvantages of pooled estimates are clearly discussed.
- authors statement that there is no prospective protocol is highly disingenuous. The ivmmeta protocol was published in November 2020, is unchanged from the same protocol published in October 2020 used for another medication, and the same protocol is used for 42 treatments. The ivmmeta analysis has been updated regularly with the same protocol. In contrast, authors have published their meta analysis only twice, both times changing the protocol creating a retrospective analysis. Further, authors have created a new unique protocol for this treatment.
- authors claim that "there is no assessment of the risk of bias or the certainty of evidence". This is false, studies are evaluated and 29 are excluded in exclusion analyses. Authors could note that ivmmeta focuses on actual bias as opposed to theoretical risk of bias. While authors assess risk of bias, their assessment is implausible, as shown with the example of the Together Trial above. Note that not only does the Together Trial have extreme actual bias, the theoretical risk of bias is also extremely high due to the conflicts of interest and trial design.
for many other issues.
Currently there are
for COVID-19 studies
, showing 51% [36‑62%]
lower mortality, 31% [15‑45%]
lower ventilation, 45% [22‑61%]
lower ICU admission, 33% [20‑44%]
lower hospitalization, and 78% [67‑86%]
Popp et al., 21 Jun 2022, preprint, 10 authors.