Many major issues including multiple impossible numbers,
blinding failure, randomization failure, and many protocol violations,
as detailed below.
Submit
Updates or Corrections
Private comments:
"There is a clear signal that IVM works in COVID patients..
that would be significant if more patients were added..
you will hear me retract previous statements where I had been previously negative"
— Ed Mills, Together Trial co-principal investigator
[stevekirsch.substack.com].
"I’m not interested in this question as its not the correct way to interpret the outcome"
— Ed Mills, responding to a per-protocol death count request [pierrekory.substack.com].
"F*ck you" "F*ck off" "Glory to Satan" "You are one of these f*ckers.." "You f*cking *sshole"
— Ed Mills, responding to
various emails on the trial and ivermectin research [pierrekory.substack.com].
Public comments:
“There was no indication that ivermectin is clinically useful”
—
Ed Mills, Together Trial co-principal investigator.
"..the question of whether this study was stopped too early
in light of the political ramifications of needing to demonstrate that the
efficacy is really unimpressive.. really could be raised.."
— Frank
Harrell, "I totally agree with Frank" — Ed Mills
[vimeo.com].
Severity | Issue (most
recent update 19 days ago) | Author response |
CRITICAL | 1. Blinding failure |
- |
CRITICAL | 2. Randomization violation, major confounding |
- |
CRITICAL | 3. Data pledge violation, unavailable over 294 days from protocol, over 89 days from publication |
- |
CRITICAL | 4. DSMC not independent |
- |
CRITICAL | 5. Extreme conflicts of interest |
- |
CRITICAL | 6. Three conflicting death counts |
- |
CRITICAL | 7. Patient counts for reported period impossible |
- |
CRITICAL | 8. Placebo arm counts vs. fluvoxamine arm not possible |
- |
CRITICAL | 9. Conflicting adverse event counts |
- |
CRITICAL | 10. 3-day dosing patients before March 23 missing (27 days ago) |
- |
CRITICAL | 11. Multiple false statements by investigators (29 days ago) |
- |
CRITICAL | 12. Investigators not responding to concerns (29 days ago) |
- |
CRITICAL | 13. ICODA reports never having the data (19 days ago) |
- |
CRITICAL | 14. Placebo tables may not match treatment tablets (19 days ago) |
- |
SERIOUS | 15. Widespread community use of ivermectin |
conflicting responses |
SERIOUS | 16. Team selected dose below what they believe is required |
- |
SERIOUS | 17. Side-effect prevalence consistent with treatment error |
- |
SERIOUS | 18. Screening to treatment delay unknown |
- |
SERIOUS | 19. Unknown onset patients included |
- |
SERIOUS | 20. Conflicting comorbidity counts |
- |
SERIOUS | 21. Unexplained >6 month delay |
- |
SERIOUS | 22. Major imputation error |
- |
SERIOUS | 23. Incorrect conclusion |
- |
SERIOUS | 24. Missing age information |
- |
SERIOUS | 25. Mid-trial protocol changes |
- |
SERIOUS | 26. COI: designed by Cytel |
- |
SERIOUS | 27. COI: analysis company works closely with Pfizer |
- |
SERIOUS | 28. Unexpected differences in missing data |
- |
SERIOUS | 29. Out of funding claim contradicted by funder (29 days ago) |
- |
SERIOUS | 30. Misrepresentation of dosing recommendation (29 days ago) |
- |
MAJOR | 31. Unknown onset results dramatically better |
- |
MAJOR | 32. Mean delay likely excluding unknown onset |
- |
MAJOR | 33. 3-dose placebo much more effective |
- |
MAJOR | 34. Multiple conflicting randomization protocols |
- |
MAJOR | 35. Dominated by Gamma variant, no discussion |
- |
MAJOR | 36. Viral load protocol violation, high Ct may hide efficacy |
- |
MAJOR | 37. Conflicting dosing, previously unheard of weight limit |
- |
MAJOR | 38. Conflicting target enrollment and reasons for termination |
- |
MAJOR | 39. Soft primary outcome easy to game, selected after single dose arm |
- |
MAJOR | 40. Conflicting futility thresholds, reported terminated due to futility, but threshold not reached |
- |
MAJOR | 41. Subgroup analysis protocol violations |
- |
MAJOR | 42. Many pre-specified outcomes missing |
- |
MAJOR | 43. Single-dose recruiting continued after change |
- |
MAJOR | 44. Funding list incorrect, missing Gates Foundation and Unitaid |
- |
MAJOR | 45. Statistical analysis plan dated after trial start |
- |
MAJOR | 46. Imputation protocol violation |
- |
MAJOR | 47. Expected analyses missing |
- |
MAJOR | 48. Single dose results missing |
- |
MAJOR | 49. Conflicting reasons for dose change (27 days ago) |
- |
MAJOR | 50. Details of placebo unspecified |
- |
MAJOR | 51. Vaccine inclusion changes, likely confounding |
- |
MAJOR | 52. Bayesian probability of superiority, featured for FLV, hidden in appendix |
- |
MAJOR | 53. Two different per-protocol counts |
modified w/o explanation |
UNKNOWN | 54. Source of ivermectin unspecified (fluvoxamine source specified) |
- |
UNKNOWN | 55. 100% adherence reported for 3-day placebo (29 days ago) |
- |
MINOR | 56. Per-protocol conflict with fluvoxamine arm |
required by journal |
Apr 5: The paper was silently updated, with no
indication or explanation of the changes. Changes include: age range, placebo
description, per-protocol count, and death counts (details below).
May 5: The paper was silently updated again. A
new summary notes that authors attempted to screen for previous
ivermectin use, contradicting both the discussion section, where authors claim
they ensured no use for COVID-19, and the exclusion criteria and
interview forms, which do not specify ivermectin use.
May 30 update: there is still no response to data requests,
and the authors continue to maintain radio silence on the many serious issues
[twitter.com].
Delayed >6 months. The paper was
delayed over 6 months with no explanation. The companion fluvoxamine arm,
completed at the same time, was published Aug 23, 2021. The paper was
submitted to NEJM in Sep 2021
[vimeo.com (B)].
COI forms suggest that additional authors were added after submission and the
corresponding author changed from Prof. Mills to Dr. Rayner
[],
whose conflicts include Pfizer, Merck, the Gates Foundation, and the
Australian Goverment.
No response to data request. The trial
registration states that data was to be available at termination and upon
request
[clinicaltrials.gov], however
authors have not responded to a request for the data. Even funders of the
trial have been unable to access the data
[odysee.com].
Requests can be sent to thetogethertrial@gmail.com, let us know the outcome.
ICODA reports never having the data.
Investigators report that the data is available via ICODA:
"The final trial
dataset will be accessible by written request to the study principal
investigators (G Reis or EJ Mills). There are no contractual agreements to
limit access to final trial data. All data collected by the TOGETHER Trial
will be shared with the International COVID-19 Data Alliance". Not only
has there been no reports of successful access to the data, but an ICODA
manager reports that they have never had the data
[t.me].
Placebo tablets may not match the
treatment tablets. Authors do not specify the appearance of the
placebo tablets, suggesting that they may not match the treatment tablets,
providing an additional reason for blinding failure. A Brazilian investigator
reports that, at the time of the trial, there was only one likely placebo
manufacturer, and they reportedly did not receive a request to produce
identical placebo tablets
[].
They also report that compounded ivermectin in Brazil is considered
unreliable.
Three different death counts. In the
original paper, Table 3 shows 21 and 24 deaths, while Table S6 shows 20 and 25
[]. In
Table 3, death and grade 5 events showed the same 21/24 numbers, but different
effect sizes, with 0.81 being closer to the 20/25 counts and the previously
reported number. This is consistent with one death being moved between arms
after manuscript generation, but not updated in Table S6 or the Table 3 AE RR.
This cannot be explained by the safety population excluding patients with zero
doses because the AE control deaths are higher.
In email, a co-principal investigator suggested that the discrepancy was due
to one being COVID-19 deaths and the other being all-cause deaths
[stevekirsch.substack.com].
That explanation does not fit the data because one arm increases while the
other arm decreases.
Both co-principal investigators report in the paper that
"they had full
access to all the trial data and vouch for the accuracy and completeness of
the data and for the fidelity of the trial to the protocol."
A third set of death counts, 20 and 24, with RR 0.84, was
presented by a co-principal investigator on Mar 18, 2022
[vimeo.com (C)].
In total, 4 different death relative risks have been presented: Mar 18, 2022
presentation: 0.84, Mar 30 paper: 0.88 (T3), 0.81 (T3 AE), and 0.80 (TS6,
presented as 20 and 25 only without group sizes).
6 days after publication, the paper was updated, with no information given on
what was changed.
In this version, a "respiratory, thoracic and mediastinal disorders" death was
removed from the control arm and an "infections and infestations" death was
added to the ivermectin arm. The paper still indicates RR 0.81 for death AE.
Trial was not blind.
Ivermectin/placebo blinding was done by assigning a letter to each group that
was only known to the pharmacist. If a patient received a 3-dose treatment,
investigators immediately know that the patient is more likely to be in the
treatment group than the control group, because 3-dose placebo was relatively
rare (~46% from PP). If a patient received non-3-day treatment, investigators
immediately know that the patient is not an ivermectin treatment patient.
Moreover, by observing the frequency of allocations, investigators can easily
determine which letter corresponds to active ivermectin 3-day treatment,
thereby removing all blinding. For example, consider 3-dose-ivermectin and
3-dose-placebo being identified by the letters G and K. If allocations to date
have been G:11 and K:20, there is a very high probability that K is
ivermectin. Note that this blinding failure is only obvious because the
journal required the authors to restrict to the 3-day placebo group. Also note
that it would have been trivial to avoid if desired, for example by using a
unique identifier for all medication bottles. Note that there may be
additional reasons for blinding failure, for example the paper specifies
identically shaped bottles, but does not appear to specify identical
appearance tablets
[].
Patient counts do not match
previously released enrollment graph. Authors claim the ivermectin and
control patients were all from on or after March 23, 2021, however independent
analyses of the enrollment graph (contained in this presentation
[dcricollab.dcri.duke.edu])
require including patients prior to this date to reach the reported numbers
[doyourownresearch.substack.com (C), longhaulwiki.com].
The enrollment graph shows much higher enrollement to ivermectin near the
start of the trial. The only way that the number of placebo patients can be
the same as the number of treatment patients is if placebo patients were taken
from an earlier period
[Marinos], which creates a nonconcurrent
control group
[nejm.org] and
substantial
confounding by time as below.
Conflicting placebo arm counts
across IVM/FLV arms. The IVM placebo arm has 679 patients and the FLV
arm has 756. The 679 should be shared between the arms, with 77 extra patients
for FLV. For FLV, there were 34 placebo patients requiring mechanical
ventilation, for IVM there was only 25, indicating that 9 of the extra 77
placebo patients for FLV had mechanical ventilation, a much higher percentage
during a period that had lower deaths and CFR (and included vaccinated
patients). Placebo all-cause hospitalization shows 95/679 for IVM and 99/756
for FLV, i.e., only 4 of the extra 77 patients were hospitalized, but the
paper reports an additional 9 patients with mechanical ventilation.
For FLV, there were 11 grade 1 AEs, for IVM there were 12, with
77 less patients.
For FLV, there were 50 grade 3 AEs, for IVM there were also 50,
meaning the 77 extra patients had 0% grade 3 AEs vs. an expected 7.4%
For FLV, there were 54 CKD patients according to Figure 3 and
eTable1 (2 according to Table 1). For IVM there was 5.
DSMC not independent. Reviewer 1 of the
protocol notes that the DSMC is not independent
[gatesopenresearch.org]. Prof. Thorlund is Vice
President of the contract research organisation (CRO, Cytel), professor at the
sponsoring university, and an author of the protocol. Dr. Häggström is an
employee of the CRO.
[doyourownresearch.substack.com (C), twitter.com (C)]
reveals many other conflicts. Prof. Thorlund has written >100 papers with
Prof. Mills. Prof. Singh has written 29 papers with Prof. Mills. Prof.
Orbinski has written 9 papers with Prof. Mills. The first version of the web
site showed Prof. Mills and Prof. Thorlund as joint leads. Emails pointed to a
company MTEK Sciences, founded by Prof. Mills and Prof. Thorlund (MTEK is
hypothesized to stand for Mills, Thorlund, Edward, Kristian). MTEK received
grants from the Gates Foundation. MTEK also employed Dr. Häggström. MTEK was
acquired by Cytel in 2019. Dr. Häggström works for the Gates Foundation. Two
members of the DSMC have published a paper with members of a well known
anti-ivermectin research group
[Thorlund] and Dr. Hill, whose meta
analysis has reports of external influence
[c19ivermectin.com, c19ivermectin.com (B), ].
The trial protocol reports that
"an independent DSMC will be established,
composed of scientists of unrivalled reputation and expertise, without
involvement with this research protocol."
Unequal randomization, significant confounding
by time. The trial reports 1:1:1:1 randomization, however independent
analysis shows much higher enrollment in the ivermectin treatment arm towards
the start of the trial
[c19ivermectin.com (C), longhaulwiki.com].
This introduces very significant
confounding by time due to the major change
in the distribution of variants.
[Zavascki] show dramatically higher
mortality for Gamma vs non-Gamma variants (28 day mortality from symptom onset
aHR 4.73 [1.15-19.41]). Many more patients were randomized to ivermectin vs.
placebo in the first few weeks, for example the first week shows 82 ivermectin
vs. 28 placebo patients, 2.9x higher. The period of excess ivermectin
enrollment coincides closely with a period of significantly higher deaths and
CFR in Brazil.
Missing time from onset patients
show statistically significant efficacy. For the known time since
onset subgroups, both groups show worse results than the overall results
[], with the
missing 317 patients showing significant efficacy RR 0.51, p = 0.02 (compared
to 1.00 and 1.14 for known patients).
Unknown onset patients were enrolled,
subgroup results opposite of previous trials. After imputation, the
percentage of patients in the late treatment subgroup went from 46% to 56%.
87% of the unknown patients were predicted to be in the late group. This is
reasonable and expected — patients that do not recall when the onset was
are more likely to have had onset further in the past. What is not clear is
how these patients could be enrolled in the trial, how many of these patients
had onset >7 days, how this very late 317 patient subgroup could show much
greater efficacy as above, and why authors did not report this result, analyze
this in greater detail, or recommend further research.
Side effect profile consistent with many
treatment patients not receiving authentic ivermectin and/or control patients
receiving ivermectin. The side effects (e.g., gastrointestinal side
effects were lower in the ivermectin arm) suggest that many ivermectin
patients may not have received authentic ivermectin, or that placebo patients
may have taken ivermectin. For comparison, there was a 3.6 times greater
incidence of diarrhea in the treatment arm in
[Lim].
A local Brazilian investigator reports that, at
the time of the trial, there was only one likely placebo manufacturer, and
they reportedly did not receive a request to produce identical placebo tablets
[].
They also report that compounded ivermectin in Brazil is considered
unreliable. The protocol reports that "the study medication used will come
from pharmaceutical plants that hold a commercial authorization for their
production, already approved by ANVISA."
Vaccine inclusion changes. The trial
changed from including vaccinated patients to excluding them on Mar 21, 2021
[clinicaltrials.gov (B), twitter.com (F)],
and on Jul 5 the exclusion was changed to specify >14days. As above, meeting
the reported placebo counts likely requires taking placebo patients from the
earlier period, which has very significant confounding due to variant changes,
and additionally confounding due to vaccination. Both of these favor the
placebo group. The original vaccine inclusion criterion is unusual, but is
shown in both the protocol and the clinicaltrials.gov record
[clinicaltrials.gov (C), togethertrial.com].
This may have been chosen to reduce the potential for efficacy. For more on
vaccination inclusion see
[].

Incorrect conclusion. The
conclusion states that ivermectin "did not result in a lower incidence of
[hospitalization] or of [ER observation >6hr]". This is incorrect,
hospitalization was 17% lower, which is not statistically significant with the
sample size and typical statistical analysis. For the Bayesian analysis the
authors use, the ITT probability of superiority for ivermectin was 79.4%,
which is a positive result, the opposite of the conclusion.
Ivermectin use widespread in the
community. Recent ivermectin use was not in the exclusion criteria.
Ivermectin was available OTC, was recommended by the government for COVID-19,
and had nine times higher sales
[twitter.com (H)]. Authors
claim they ensured patients did not use ivermectin via "extensive screening",
but do not explain why this was not an exclusion criterion, or how this
unwritten exclusion was ensured even though there is extensive missing data
related to written exclusion criteria. Similar unwritten exclusions were not
mentioned for other arms
[], a primary
investigator previously stated such an exclusion should not be an issue
[twitter.com (J)], and it
is not mentioned in the interview sheets
[osf.io]. After
publication, a co-principal investigator reportedly wrote that
"even if
some patients did access IVM, the fact that it is blinded should still
maintain balance", which is incorrect, placebo patients taking ivermectin
are expected to improve, treatment patients that already have significant
tissue distributions may have positive, neutral, or negative responses to
additional treatment.
Single-dose recruiting continued after
change. The trial had requested moving to 3-dose treatment by Feb
15/19, when only 19 patients had been recruited, however the trial continued
recruiting an additional 59 patients to single dose treatment
[twitter.com (K)].
Per-protocol population different to the
contemporary fluvoxamine arm. Table 2 per-protocol numbers show 92%
per-protocol patients for ivermectin and only 42% for control. This appears to
be a post-hoc change selecting only 3-day placebo patients, while similar
selection does not appear to have been done for the companion fluvoxamine
trial (showing 74% and 82% per-protocol patients for fluvoxamine and control)
[Reis].
Multiple conflicting randomization
protocols.
[twitter.com (L)] reviewed
the randomization protocol, finding three different algorithms, and
conflicting versions in the papers.
Time of onset, required for inclusion, missing for 317
patients. For the companion fluvoxamine arm, 24% of patients had an
unknown time from onset, including 179 of the control patients
[Reis]. In this trial, 0 patients have an unspecified time from onset
in Table 1, due to imputation. However, Figure 2 reveals that the time from
onset is unknown for 317 patients, similar to the fluvoxamine paper. However,
time from onset is required for the inclusion criteria. According to Figure 2,
age and BMI also show missing values.
Conflicting comorbidity counts.
The companion fluvoxamine arm ran from Jan 20 to Aug 5, 2021, while this trial
ran from March 23 to Aug 6, 2021 — most control patients should be
shared, with an additional 10% for fluvoxamine from the earlier start. The Aug
6 presentation, which has a date of 9:38am Aug 6 local trial time
[],
shows 678 placebo patients, indicating that either 0 or 1 placebo patients
were randomized on Aug 6. Zero patients should have been randomized on Aug 6,
because authors cannot add patients after unblinding.
The fluvoxamine control arm shows 16/756 control patients with
asthma. The ivermectin control arm has a subset of these patients (679), but
shows a much higher prevalence of asthma (60 patients). This might be possible
due to imputation if there was a very high percentage of missing data, however
imputation does not appear to be a good explanation. For example, placebo CKD
goes from 2 to 5 (FLV->IVM). First, it is not logical to impute CKD on
patients based on the other variables. Second, the protocol specifies
imputation only with up to 20% missing data, making it unlikely that
imputation would add 150% of CKD patients. Third, the degree of change between
FLV and IVM varies dramatically, with IVM reporting 666%, 275%, 150%, and 43%
more patients for CPD, asthma, CKD, and CCD, without any clear explanation for
similar differences in the percentage of missing data (all were collected on
the same interview form).
Conflicting target enrollment.
There are conflicting target enrollment numbers. The protocol showed 800
patients per arm as of Mar 21, 2021 (after the trial started)
[static1.squarespace.com, twitter.com (M)],
the co-principal investigator reported 800 per arm in an interview published
June 14, 2021
[halifaxexaminer.ca],
and the protocol changed to 681 on June 22
[static1.squarespace.com (B)].
However, the trial record from Jan indicates 2724 (681*4) patients
[clinicaltrials.gov (C)],
suggesting that the 800 goal was later, and was kept for fluvoxamine but reverted for
ivermectin.
The fluvoxamine arm which started two months earlier was terminated at the
same time, and was terminated due to superiority
[Reis] after 741/756
patients.
Note that Gamma was declining significantly around the termination point,
which likely favors improved efficacy if the trial continued, given the late
treatment and dosage used.
The co-principal investigator reports three different reasons for stopping the
trial
[twitter.com (N)]: a) because
they ran out of money, b) because third parties were not supportive, and c) it was done
by the DSMC and was out of their control.
Reportedly terminated for futility although
futility threshold not reached. The trial was reportedly terminated
due to futility
[twitter.com (O)], however
the futility thresholds were 20%, 40% and 60%, and all published probabilities
are >60% (ITT 79.4%). Additionally, the fluvoxamine arm did not have the
higher 60% threshold, only using 40%. Note the DSMC was not independent as
below.
Screening to treatment delay.
Most Together Trial protocols show an additional day delay in already late
treatment for most patients. The Aug 5, 2021 protocol published with the
metformin paper
[ars.els-cdn.com],
shows treatment administration one day after screening, baseline, and
randomization (Table 2, schedule of study activities). This can also be found
in the protocol dated Mar 11, 2021
[drive.google.com].
The protocol attached to the ivermectin paper, dated Feb 15, 2021, shows a
different schedule, stating that the treatment
should be administered
on the same day of randomization.
There is no explanation of when this change was made, how the overlapping
metformin and ivermectin arms could use different schedules, or how this
change was implemented (there are many tasks in the screening and baseline
visits). There is no reporting for how many patients received treatment on the
same day. The form for the first treatment visit asks if there were clinical
events including >6hr ER visits since the baseline visit, which would not be
possible if this visit was immediately after randomization. Time of first
treatment was recorded
[osf.io], but no information has
been reported. According to
[Forrest], WhatsApp messaging and video
was used for recruitment, raising the question of how medication was delivered
in cases where recruitment was done online.

Mean delay. The reported mean
number of days from symptoms to randomization likely only includes known onset
patients and therefore is likely to significantly underestimate the actual
average, in addition to not including the time between randomization and
treatment.
Viral load not reported. The
protocol has change in viral load as an outcome, however only viral clearance
is reported, and without any details (for example, using a high Ct value would
have limited relevance).
Incorrect dose reporting, many
patients at higher risk due to BMI may have received lower per kg doses, and
show lower efficacy. The paper reports 400μg/kg for 3 days, however
the protocol indicates that this was only up to 90kg, meaning that the dose
received for higher-risk high BMI patients was even further reduced from
dosage which is already far below clinician recommendations for the dominant
variant
[twitter.com (P)].
50% of patients had BMI ≥30. Much greater efficacy was seen in the low BMI
subgroup (RR 0.77 vs 0.98).
Plasma concentration below known effective
value. [Krolewiecki] show an antiviral effect only with plasma
concentrations above 160ng/mL. Figure S5 shows that the authors expected the
mean concentration to be well below this level
[]. Dosage
requirements are likely to vary significantly depending on many factors
including the variant encountered, time of administration, mode of
administration, patient genetics, concomitant medications, SOC, and the
distribution of the infection in different tissues. However, the dose used is
far below what is recommended by clinicians for post-infection treatment with
the Gamma variant — about 2.5 - 6.5x lower, depending on the
recommendation and which estimate of fasting/fed administration is used. The
trial used fasting administration, however Merck's product information reports
that
"administration of 30mg ivermectin following a highfat meal resulted
in an approximate 2.5-fold increase in bioavailability relative to
administration of 30mg ivermectin in the fasted state."
[merck.com].
Moreover Dr. Craig Rayner, a senior investigator on the trial, previously
published research indicating that a higher dose is required
[sciencedirect.com],
raising the question of why the dose and fasting administration was chosen,
especially for the initial single dose regimen.
Primary outcome easy to game, selected after
ivermectin one dose arm. The subjective "emergency room visit for >6
hours" criterion shows higher risk (RR 1.16), while hospitalization is lower
(RR 0.83 all-cause, RR 0.84 COVID-19). The primary outcome results were set on
March 21, 2021, after the single dose ivermectin arm. Given the known public
biases of some investigators, this may have been specifically chosen to reduce
efficacy. Authors claim that the 6hr threshold did not include waiting time,
however the emergency visit form has no mention of waiting time, only
recording presentation and discharge times
[osf.io].
Including contraindicated chronic kidney disease
patients. "Stage IV chronic kidney disease or on dialysis" was an
inclusion criterion, however ivermectin is contraindicated with kidney disease
[Arise, en.wikipedia.org, Nunes] (not always
recognized, and may be less critical with very low dose use for other
conditions). According to Table 1 there were only 7 CKD patients, however two
conflicting numbers are provided in the fluvoxamine paper
[Reis]:
Table 1 reports 2 CKD placebo patients - it's not clear how CKD was imputed
for 3 more patients in the smaller IVM group. Moreover, Figure 3 shows 54
placebo CKD patients for FLV.
Antigen test requirement. The
protocol indicates that patients with a negative test may be included if they
become positive a few days later, potentially resulting in a long unreported
delay between randomization and treatment, depending on how investigators
interpreted the protocol. The requirement for a positive antigen test excludes
the possibility of early treatment in many cases - tests have very high false
negative rates in the early stages of infection, and symptoms may appear
before the test becomes positive.
Inconsistent subgroup analysis.
The presented subgroup analysis is inconsistent with plans and with the
fluvoxamine paper, including not presenting pre-specified subgroups,
presenting subgroups that were not pre-specified, presenting different
subgroups to the contemporary fluvoxamine paper, and modifying subgroup
definitions
[twitter.com (R)].
Missing analysis. Authors do
not provide time from onset analysis for either mortality or hospitalization,
only the combined measure including the ER visits where anomalous results are
seen. Authors do not provide per-protocol or mITT results for mortality or
hospitalization. Per-protocol mortality results were provided for the
companion fluvoxamine trial.
Missing outcomes. Many
outcomes specified in the protocol appear to be missing, including the
co-primary outcome of COVID-19 mortality (only all-cause mortality is
provided, specific AE details not provided), time to clinical failure, days
with respiratory symptoms, mortality due to pulmonary complications,
cardiovascular mortality, COVID-19 symptom scale assessment, WHO clinical
worsening scale assessment, and 14 day mortality.
Imputation error. In the
paper authors use imputation in Table 1 but not in Figure 2. Authors also
released a version of Figure 2 with imputation
[togethertrial.com (B)], where the numbers
for age and BMI now match the imputed numbers in Table 1. However, the time
from onset numbers are very different, with the treatment arm showing 302
patients for 0-3 days, and the imputed version of Figure 2 showing 367
[].
Missing age information.
According to Figure 2, 98 patients are missing age information
[].
Out of funding claim contradicted by
funder. A co-principal investigator has reported that the trial was
stopped because they ran out of funding, however this is contradicted by
the Rainwater Foundation, which reported that they would have given more money to
finish the trial if the investigators had asked
[pierrekory.substack.com (B)].
Misrepresentation of dosing
recommendation. Investigators have misrepresented an email from the
FLCCC regarding recommended dosing
[pierrekory.substack.com (B)].
Unexpected differences in missing
data. Age is unknown for 98 patients, however according to Figure 2,
BMI is missing for only 11 patients, smoking status is unknown for only 2
patients, lung disease is unknown for only one patient, and cardiovascular
disease is known for all patients.
Mid-trial protocol changes. There
were several mid-trial protocol changes on July 5, 2021
[clinicaltrials.gov (D)].
The number of patients for viral load analysis was reduced, only for the
ivermectin arm. All-cause, cardiovascular, and respiratory death outcomes were
deleted (all-cause was reported). Exclusions were modified to allow enrolling
patients vaccinated within the last 14 days. Inclusion criteria were modified
to allow enrolling healthy young people — the criterion
"fever >38C
at baseline" was added, allowing enrollment independent of increased
risk.
Statistical analysis plan dated after trial
start. The statistical analysis plan appears to be dated after the
trial started
[twitter.com (T)].
Per-protocol placebo results very
different. The 3-dose placebo appears to have been much more effective
[Marinos, twitter.com (U)].
This
could be consistent with placebo patients accidently receiving
treatment.
Imputation protocol violation.
The protocol specifies multiple imputation with up to 20% of missing data,
however imputation was done with time from symptom onset, which has >23%
missing data
[twitter.com (V)].
Two different per-protocol counts.
Figure 1 shows 228 per-protocol for the control arm, while Table 2 shows 288.
This was modified in the Apr 5 update without explanation.
Conflicting adverse event counts.
Table 3 and Table S6 adverse event counts do not match for any grade, e.g.,
grade 1/2 in Table S6 shows 82 for IVM, while Table 3 shows 65
[]. The
Apr 5 update changed the grade 5 events without explanation, however the other
grades remain conflicting.
3-day dosing patients before March 23
missing. The co-principal investigator wrote on March 6 that 3-day
dosing was being administered, and that the clinicaltrials entry was out of
date at that time
[twitter.com (X)]. This
earlier start of the 3-dose arm would resolve an oustanding major
inconsistency. Analysis of the trial randomization shows that reaching the
3-day placebo count requires patients from March 4
[doyourownresearch.substack.com (F)],
and it would be logical for the 3-day placebo and 3-day active arms to have started
on the same day.
This reinforces existing concerns as to which patients were included in the
analysis, and adds additional questions regarding what happened to the
patients prior to March 23, and if patients were treated prior to ethics
approval. Ethics approval for the dose change was received on March 21
according to the paper
[twitter.com (Y)],
with the regulator document dated March 15
[twitter.com (Z)].
Multiple false statements by
investigators. There has been multiple false statements by
investigators raising questions about their ethics and the reliability of
their work
[pierrekory.substack.com].
Investigators not responding to
concerns. After details of major data errors and protocol violations
became known, investigators appear to have stopped responding to all
researchers regarding serious concerns with the trial
[pierrekory.substack.com, twitter.com]
(and have still not responded to us).
Possibly the largest financial conflict of
interest of any trial to date. Disclosed conflicts of interest
include: Pfizer, Merck, Bill & Melinda Gates Foundation, Australian
Government, Medicines Development for Global Health, Novaquest, Regeneron,
Astrazeneca, Daichi Sankyo, Commonwealth Science and Research Organization,
and Card Research. Many conflicts of interest appear unreported. For example,
Unitaid is a sponsor
[Harper, togethertrial.com (C)].
Analysis done by a company that receives
payment from and works closely with Pfizer. All analyses were done by
Cytel. Cytel is a statistical modelling company that helps pharmaceutical
companies get approval — they work very closely with Pfizer
[cytel.com].
Cytel's software and services are used by the top 30 pharmaceutical companies
[cytel.com (B)].
A co-principal investigator works for Cytel and the Gates
Foundation
[empendium.com]:
"The majority of the time I work for a company called Cytel, where I design
clinical trials, predominantly for the Bill & Melinda Gates
Foundation".
Reportedly, the first author's center is funded by
pharmaceutical companies, and independent investigators tried to participate
in the trial but were denied
[odysee.com (B)].
The Gates Foundation is a founding partner of GAVI, which took
out Google ads telling people not to use ivermectin
[], and a
major funder of Unitaid, which may have modified the results of the Hill meta
analysis in a way that prevented adoption
[c19ivermectin.com, c19ivermectin.com (B), ].
Associated with MMS Holdings. The trial is
associated with MMS Holdings
[dcricollab.dcri.duke.edu],
whose mission includes helping pharmaceutical companies get approval and
designing scientific studies that help them get approval. One of their clients
is Pfizer
[mmsholdings.com].
Certara. One of the senior investigators was Dr.
Craig Rayner, President of Integrated Drug Development at Certara - another
company with a similar mission to MMS Holdings. They state on their website
that: "Since 2014, our customers have received over 90% of new drug and
biologic approvals by the FDA." One of their clients is Pfizer
[certara.com].
Local variant shows very different
characteristics. The trial took place in an area of Brazil where the
Gamma variant was prominent. Brazilian clinicians report that this variant is
much more virulent, and that significantly higher dosage and/or earlier
treatment is required, as may be expected for variants where the peak viral
load is significantly higher and/or reached earlier
[Faria, Nonaka].
Funding list incorrect. The paper
does not include the Bill and Melinda Gates Foundation or Unitaid as funders,
however the protocol shows the Gates Foundation
[gatesopenresearch.org (B)] and the web site
shows Unitaid
[togethertrial.com (C)].
Single dose arm results missing.
Results for the single dose ivermectin arm have not been reported.
Anomalous results from the same region. A local
Brazilian investigator reports that the study was conducted in almost the same
time and location as the Brazilian component of the molnupiravir trial.
Notably, molnupiravir's EUA relied on the unusually higher efficacy observed
in Brazil.
Designed by Cytel. The trial was
designed by Cytel, a company that helps pharmaceutical companies get approval
and that works very closely with Pfizer
[cytel.com, cytel.com (C)].
Cytel's software and services are used by the top 30 pharmaceutical companies
[cytel.com (B)].
Bayesian probability of
superiority hidden in appendix. The bayesian probability of
superiority figure, featured in the main paper for FLV, MET, HCQ, was hidden
in the appendix for IVM
[twitter.com (AB)].
Conflicting reasons for dose
change. Conflicting reasons have been given for the change from 1-day
to 3-day dosing. In email from March 6, the co-principal investigator says the
change was "based on emerging trials from Andrew Hill's synthesis"
[twitter.com (AC)].
The paper says the change was made "on the basis of feedback from advocacy
groups". Neither of these match the report that the dosing change was made at
the request of one of the trial funders
[pierrekory.substack.com].
Placebo unspecified. The
placebo appears to be unspecified in the paper and protocol. The initial trial
announcement indicated the placebo was vitamin C
[cytel.com (C)],
which would be an active treatment according to the
results of
45 studies (mortality RR
0.72
[0.59-0.88]).
The metformin arm reports using talc, however fluvoxamine and ivermectin do
not appear to report details of the placebo, which could potentially be
different, for example based on manufacturer limitations for matching active
treatment tablets.
Previous protocol changes. There are two previous
published protocols, both are called "version 1", we refer to them as 1A
(3/11/21
[drive.google.com] )
and 1B (8/5/21
[gatesopenresearch.org]. 1B
deletes subgroup analysis by treatment delay, and deletes a statement
requiring prior approval for amendments. 1B adds the statement: "we
hypothesize that younger patients will benefit more than older
patients."
Patients 50 years old were assigned to different groups in
Table 1 and Figure 2 (≤50 vs. <50).
Greater efficacy was seen for patients >50 (RR 0.77) vs.
patients ≤50 (RR 1.01).
Source of ivermectin unspecified.
Authors do not specify the source of the ivermectin used in the trial, whereas
they do specify the source for the fluvoxamine arm (Luvox, Abbott). Depending
on the source, Ivermectin has been reported to be of unreliable quality in
Brazil.
The following comments are prior to the publication. We note
that authors claim they have not included patients prior to the time period
for the 3 dose ivermectin patients, however this conflicts with previously
reported data as per the analyses above.
The trial randomization chart does not match the protocol,
suggesting major problems and indicating substantial
confounding by time. For
example, trial week 43, the first week for 3 dose ivermectin, shows ~3x
patients assigned to ivermectin vs. placebo
[reddit.com].
Treatment efficacy can vary significantly over time, for example due to
overall improvement in protocols, changes in the distribution of variants, or
changes in public awareness and treatment delays.
[Zavascki] show dramatically higher mortality for Gamma vs non-Gamma
variants (28 day mortality from symptom onset aHR 4.73 [1.15-19.41]), and the
prevalence of the Gamma variant varied dramatically throughout the trial
[ourworldindata.org].
This introduces
confounding by time, which is common in COVID-19 retrospective
studies and has often obscured efficacy (many retrospectives have more
patients in the treatment group earlier in time when overall treatment
protocols were significantly worse).
According to this analysis
[reddit.com],
the total number of patients for the ivermectin and placebo groups do not
appear to match the totals in the presentation (the numbers for the
fluvoxamine arm match) — reaching the number reported for ivermectin
would require including some of the patients assigned to single dose
ivermectin. Reaching the placebo number requires including placebo patients
from the much earlier ivermectin single dose period, and from the early two
week period when zero ivermectin patients were assigned. If these earlier
participants were accidently included in the control group, this would
dramatically change the results in favor of the control group according to
the changes in Gamma variant prevalence.
An investigator from Brazil notes that the gamma variant became
prevailing in the state of Minas Gerais later than in the rest of the country,
with the time when gamma prevailed for the trial locations being more closely
aligned with the start of the ivermectin arm
[ufmg.br].
Due to the substantial differences in disease course and risk between the
variants, authors need to consider only patients recruited during the same
time period.
Treatment delay is currently unknown, however the protocol
allows very late inclusion and a companion trial reported mostly late
treatment. Overall mortality is high for 18+ outpatients. Results may be
impacted by late treatment, poor SOC, and may be specific to local variants
[Faria, Nonaka, Sabino].
Treatment was administered on an empty stomach, greatly reducing expected
tissue concentration
[Guzzo] and making the effective dose about
1/5th of current clinical practice. The trial was conducted in Minas Gerais,
Brazil which had substantial community use of ivermectin
[otempo.com.br],
and prior use of ivermectin is not listed in the exclusion criteria.
This trial uses a soft primary outcome, easily subject to bias
and event inflation in both arms (e.g., observe >6 hours independent of
indication). There is also an unusual inclusion criteria: "patients with
expected hospital stays of <= 5 days". This is similar to "patients less
likely to need treatment beyond SOC to recover", and would make it very easy
to reduce the effect seen. This is not in either of the published
protocols.
RCTs have a fundamental bias against finding an effect for
interventions that are widely available — patients that believe they
need treatment are more likely to decline participation and take the
intervention
[Yeh], i.e. RCTs are more likely to enroll low-risk
participants that do not need treatment to recover (this does not apply to
the typical pharmaceutical trial of a new drug that is otherwise
unavailable). This trial was run in a community where ivermectin is widely
known and used.
The same trial's results for a previous treatment were
initially reported as RR 1.0 [0.45-2.21]
[ajtmh.org], while
the final paper reported something very different — HR 0.76 [0.30-1.88]
[jamanetwork.com].
Trial design, analysis, and presentation, along with previous
public and private statements suggest investigator bias. Design: including
very late treatment, additional day before administration, operation in a
region with high community use, specifying administration on an empty
stomach, limiting treatment to 3 days, using soft inclusion criterion and a
soft primary outcome, easily subject to bias. Analysis: authors perform
analysis excluding events very shortly after randomization for fluvoxamine
but not ivermectin, and report viral load results for fluvoxamine but not
ivermectin. Presentation: falsely describing positive but not statistically
significant effects as "no effect, what so ever"
[Amrhein, odysee.com (C)].
Prior statements:
[odysee.com (C)].
The local Brazilian investigator also reports that nitazoxanide
was tested in the same location, however very few patients reportedly
experienced urine discoloration, while 100% are expected to experience this
side effect; and that 6-hour observation is a poor choice because it is almost
impossible to stay less than 6 hours in Brazil.
For additional issues see:
[covid19criticalcare.com, doyourownresearch.substack.com (G), longhaulwiki.com, Marinos, Marinos (B), stevekirsch.substack.com, trialsitenews.com, twitter.com (AD), , , ].
Protocols, approvals, and statistical analysis plans can be found here
[togethertrial.com (D)].
This study is excluded in the after exclusion results of meta
analysis:
multiple anomalies as per detailed analysis.