Delayed >6 months. The paper was delayed over 6 months with no explanation. The companion fluvoxamine arm, completed at the same time, was published Aug 23, 2021.
No response to data request.Authors have not responded to a request for the data (requests can be sent to
thetogethertrial@gmail.com, let us know the outcome).
Incorrect conclusion. The conclusion states that ivermectin "did not result in a lower incidence of [hospitalization] or of [ER observation >6hr]". This is incorrect, hospitalization was 17% lower (just not statistically significant).
Two different death counts. Table 3 shows 21 and 24 deaths, while Table S6 shows 20 and 25 [twitter.com (C)]. In Table 3, death and grade 5 events show the same 21/24 numbers, but different effect sizes, with 0.81 being closer to the 20/25 counts and the previously reported number. This is consistent with one death being moved between arms after manuscript generation, but not updated in Table S6 or the Table 3 AE RR. This cannot be explained by the safety population excluding patients with zero doses because the AE control deaths are higher.
Trial was not blind. Ivermectin/placebo blinding was done by assigning a letter to each group that was only known to the pharmacist. If a patient received a 3-dose treatment, investigators immediately know that the patient is more likely to be in the treatment group than the control group, because 3-dose placebo was relatively rare. If a patient received non-3-day treatment, investigators immediately know that the patient is not an ivermectin treatment patient. Moreover, by observing the frequency of allocations, investigators can easily determine which letter corresponds to active ivermectin 3-day treatment, thereby removing all blinding. Note that we only know about this blinding failure because the journal required the authors to restrict to the 3-day placebo group. Also note that it would have been trivial to avoid if desired.
Reportedly terminated for futility although futility threshold not reached. Authors report futility thresholds of 20%, 40% and 60%, the trial was reportedly terminated due to futility, however all published probabilities are >60% (ITT 79.4%) [twitter.com (D)]. Additionally, the fluvoxamine arm did not have the higher 60% threshold, only using 40%. Note the DSMC was not independent as below.
Conflicting target enrollment. There are conflicting target enrollment numbers. The protocol showed 800 patients per arm as of Mar 21, 2021 (after the trial started) [static1.squarespace.com, twitter.com (E)], the co-principal investigator reported 800 per arm in an interview published June 14, 2021 [halifaxexaminer.ca], and the protocol changed to 681 on June 22 [static1.squarespace.com (B)]. The fluvoxamine arm which started two months earlier was terminated at the same time, and was terminated due to superiority [Reis (B)] after 741/756 patients. For 800 patients per arm, the trial would have been terminated at 85% enrollment, however the protocol specifies interim analyses at 25%, 50%, and 75% [twitter.com (F)]. Note that Gamma was declining significantly around the termination point, which likely favors improved efficacy if the trial continued, given the late treatment and dosage used.
Funding conflict. The paper does not include the Bill and Melinda Gates Foundation or Unitaid as funders, however the Mar 25, 2021 protocol shows the Gates Foundation [static1.squarespace.com], and the web site shows Unitaid [togethertrial.com].
Unequal randomization, confounding by time. The trial reports 1:1:1:1 randomization, however independent analysis shows much higher enrollment in the ivermectin treatment arm towards the start of the trial [c19ivermectin.com, longhaulwiki.com], introducing potentially significant confounding by time due to changes in variants, SOC, hospital utilization, etc.
Missing time from onset patients show statistically significant efficacy. For the known time since onset subgroups, both groups show worse results than the overall results [twitter.com (G)], with the missing 317 patients showing significant efficacy RR 0.51, p = 0.02 (compared to 1.00 and 1.14 for known patients).
Unknown onset patients were enrolled, subgroup results opposite of previous trials. After imputation, the percentage of patients in the late treatment subgroup went from 46% to 56%. 87% of the unknown patients were predicted to be in the late group. This is reasonable and expected — patients that do not recall when the onset was are more likely to have had onset further in the past. What is not clear is how these patients could be enrolled in the trial, how many of these patients had onset >7 days, how this very late 317 patient subgroup could show much greater efficacy as above, and why authors did not report this result, analyze this in greater detail, or recommend further research.
Side effect profile consistent with many treatment patients not receiving authentic ivermectin and/or control patients receiving ivermectin. The side effects (e.g., gastrointestinal side effects were lower in the ivermectin arm) suggest that many ivermectin patients may not have received authentic ivermectin, or that placebo patients may have taken ivermectin. For comparison, there was a 3.6 times greater incidence of diarrhea in the treatment arm in [Lim].
A local Brazilian investigator reports that, at the time of the trial, there was only one likely placebo manufacturer, and they reportedly did not receive a request to produce identical placebo tablets. They also report that compounded ivermectin in Brazil is considered unreliable.
Ivermectin use widespread in the community. Recent ivermectin use was not in the exclusion criteria. Ivermectin was available OTC, was recommended by the government for COVID-19, and had nine times higher sales [twitter.com (H)]. Authors claim they ensured patients did not use ivermectin via "extensive screening", but do not explain why this was not an exclusion criterion, or how this unwritten exclusion was ensured even though there is extensive missing data related to written exclusion criteria. Similar unwritten exclusions were not mentioned for other arms [twitter.com (I)], a primary investigator previously stated such an exclusion should not be an issue [twitter.com (J)], and it is not mentioned in the interview sheets [osf.io]. After publication, a co-principal investigator reportedly wrote that "even if some patients did access IVM, the fact that it is blinded should still maintain balance", which is incorrect, placebo patients taking ivermectin are expected to improve, treatment patients that already having significant tissue distributions may have positive, neutral, or negative responses to additional treatment.
Patient counts do not match previously released enrollment graph. Authors claim the 679 ivermectin patients were all from on or after March 23, 2021, however independent analysis [c19ivermectin.com, longhaulwiki.com] of the previously released enrollment graph (contained in this presentation [dcricollab.dcri.duke.edu]) shows only 636 patients assigned to 3-dose ivermectin (78 were assigned to 1-dose, and there was a two-week gap). Similarly, the paper reports 679 placebo patients, however the analysis shows only 559 assigned during the same period. Note that the numbers in this analysis exactly match for fluvoxamine.
Per-protocol population different to the contemporary fluvoxamine arm. Table 2 per-protocol numbers show 92% per-protocol patients for ivermectin and only 42% for control. This appears to be a post-hoc change selecting only 3-day placebo patients, while similar selection does not appear to have been done for the companion fluvoxamine trial (showing 74% and 82% per-protocol patients for fluvoxamine and control) [Reis (B)].
Time of onset, required for inclusion, missing for 317 patients. For the companion fluvoxamine arm, 24% of patients had an unknown time from onset, including 179 of the control patients [Reis (B)]. In this trial, 0 patients have an unspecified time from onset in Table 1, due to imputation. However, Figure 2 reveals that the time from onset is unknown for 317 patients, similar to the fluvoxamine paper. However, time from onset is required for the inclusion criteria. According to Figure 2, age and BMI also show missing values.
Conflicting comorbidity counts. The companion fluvoxamine arm ran from Jan 20 to Aug 5, 2021, while this trial ran from March 23 to Aug 6, 2021 — most control patients should be shared, with an additional 10% for fluvoxamine from the earlier start. The fluvoxamine control arm shows 16/756 control patients with asthma. The ivermectin control arm has a subset of these patients (679), but shows a much higher prevalence of asthma (60 patients). It does not appear to be possible for both of these to be correct. There are similar issues with other comorbidities [Reis (B)]. Figure 2 shows 282 in the ivermectin arm with cardiovascular disease, which is greater than the sum of every comorbidity in Table 1, even if there was no overlap between comorbidities.
Screening to treatment delay. Table 2, schedule of study activities, shows treatment administration one day after screening, baseline, and randomization in the original protocol [drive.google.com], indicating an additional day delay in already late treatment for most patients. The protocol attached to this paper has changed, now stating that the treatment should be administered on the same day of randomization, however there is no explanation of when this change was made, how this change was implemented (there are many tasks in the screening and baseline visits), and no reporting for how many patients received treatment on the same day. The form for the first treatment visit asks if there were clinical events including >6hr ER visits since the baseline visit, which would not be possible if this visit was immediately after randomization. Time of first treatment was recorded [osf.io], but no information has been reported.
Mean delay. The reported mean number of days from symptoms to randomization likely only includes known onset patients and therefore is likely to significantly underestimate the actual average, in addition to not including the time between randomization and treatment.
Viral load not reported. The protocol has change in viral load as an outcome, however only viral clearance is reported, and without any details (for example, using a high Ct value would have limited relevance).
Incorrect dose reporting, many patients at higher risk due to BMI may have received lower per kg doses, and show lower efficacy. The paper reports 400μg/kg for 3 days, however the protocol indicates that this was only up to 90kg, meaning that the dose received for higher-risk high BMI patients was even further reduced from dosage which is already far below clinician recommendations for the dominant variant [twitter.com (K)]. 50% of patients had BMI ≥30. Much greater efficacy was seen in the low BMI subgroup (RR 0.77 vs 0.98).
Plasma concentration below known effective value. [Krolewiecki] show an antiviral effect only with plasma concentrations above 160ng/mL. Figure S5 shows that the authors expected the mean concentration to be well below this level [twitter.com (L)]. Dosage requirements are likely to vary significantly depending on many factors including the variant encountered, time of administration, mode of administration, patient genetics, concomitant medications, SOC, and the distribution of the infection in different tissues. However, the dose used is far below what is recommended by clinicians for post-infection treatment with the Gamma variant — about 2.5 - 6.5x lower, depending on the recommendation and which estimate of fasting/fed administration is used.
Primary outcome easy to game, selected after ivermectin one dose arm. The subjective "emergency room visit for >6 hours" criterion shows higher risk (RR 1.16), while hospitalization is lower (RR 0.84 in the appendix or RR 0.83 in the paper). The primary outcome results were set on March 21, 2021, after the single dose ivermectin arm. Given the known public biases of some investigators, this may have been specifically chosen to reduce efficacy. Authors claim that the 6hr threshold did not include waiting time, however the emergency visit form has no mention of waiting time, only recording presentation and discharge times [osf.io].
Including contraindicated chronic kidney disease patients. "Stage IV chronic kidney disease or on dialysis" was an inclusion criterion, however ivermectin is contraindicated with kidney disease [Arise, en.wikipedia.org, Nunes] (not always recognized, and may be less critical with very low dose use for other conditions). Only 7 CKD patients were enrolled.
Antigen test requirement. The protocol indicates that patients with a negative test may be included if they become positive a few days later, potentially resulting in a long unreported delay between randomization and treatment, depending on how investigators interpreted the protocol. The requirement for a positive antigen test excludes the possibility of early treatment in many cases - tests have very high false negative rates in the early stages of infection, and symptoms may appear before the test becomes positive.
Missing analysis. Authors do not provide time from onset analysis for either mortality or hospitalization, only the combined measure including the ER visits where anomalous results are seen.
Missing PP, mITT mortality, hospitalization results. Authors do not provide per-protocol or mITT results for mortality or hospitalization. Per-protocol mortality results were provided for the companion fluvoxamine trial.
Missing outcomes. Many outcomes specified in the protocol appear to be missing, including the co-primary outcome of COVID-19 mortality (only all-cause mortality is provided, specific AE details not provided), time to clinical failure, days with respiratory symptoms, mortality due to pulmonary complications, cardiovascular mortality, COVID-19 symptom scale assessment, WHO clinical worsening scale assessment, and 14 day mortality.
Missing age information. According to Figure 2, 98 patients are missing age information [twitter.com (M)].
Mid-trial protocol changes. There were several mid-trial protocol changes on July 5, 2021 [clinicaltrials.gov (B)]. The number of patients for viral load analysis was reduced, only for the ivermectin arm. All-cause, cardiovascular, and respiratory death outcomes were deleted. Exclusions were modified to allow enrolling patients vaccinated within the last 14 days. Inclusion criteria were modified to allow enrolling healthy young people — the criterion "fever >38C at baseline" was added, allowing enrollment independent of increased risk.
Vaccine status unclear. The trial appears to have switched to allowing vaccinated patients at some point, however details of any change, and results by vaccination status are not reported [twitter.com (N)].
Large change in results from previously released data. The published results are very different from the previously released results, for example 100/679 vs. 86/677 for the primary outcome ivermectin events. The mortality RR changed from 0.82 to [0.88/0.81/0.80] for [Table 2/Table 2 AE/Table S6].
Statistical analysis plan dated after trial start. The statistical analysis plan appears to be dated after the trial started [twitter.com (O)].
Per-protocol placebo results very different. The 3-dose placebo appears to have been much more effective [twitter.com (P)].
Imputation protocol violation. The protocol specifies multiple imputation with up to 20% of missing data, however imputation was done with time from symptom onset, which has >23% missing data [twitter.com (Q)].
Two different per-protocol counts. Figure 1 shows 228 per-protocol for the control arm, while Table 2 shows 288.
Possibly the largest financial conflict of interest of any trial to date. Disclosed conflicts of interest include: Pfizer, Merck, Bill & Melinda Gates Foundation, Australian Government, Rainwater Charitable Foundation, Fast Grants, Medicines Development for Global Health, Novaquest, Regeneron, Astrazeneca, Daichi Sankyo, Commonwealth Science and Research Organization, and Card Research. Many conflicts of interest appear unreported. For example, Unitaid is a sponsor [Harper, togethertrial.com].
Analysis done by a company that receives payment from and works closely with Pfizer. All analyses were done by Cytel. Cytel is a statistical modelling company that helps pharmaceutical companies get approval — they work very closely with Pfizer [cytel.com]. Cytel's software and services are used by the top 30 pharmaceutical companies [cytel.com (B)].
Re: I guess it's time to talk about the NIH...
Originally Posted by Chuck Naill
Posting Permissions
Reply With Quote
Bookmarks