In a recent letter, as part of an international group (Enrico Bucci, Johannes Berkhof, André Gillibert, Gowri Gopalakrishna, Raffaele A Calogero, Lex M Bouter, Konstantin Andreev, Florian Naudet, Vasiliy Vlassov) we expressed some concerns about reporting of interim data of Sputnik V phase 3 trial published in The Lancet.
Our priors for a potential problem were high as there was also a possible problem with Phase 1/Phase 2. Enrico Bucci and his colleagues already described these possible problems in the Lancet. See here for the detailed story. This study had data that appeared to be duplicated and offered data sharing. At least 3 independent groups asked for the data. No response was received; thus, these 3 groups have merged in the present letter.
The Lancet article reporting on the Phase 3 trial was problematic in many ways. For instance, the number of patients at risk increased between 2 visits in the Kaplan Meyer curve. This error appeared in the first published version of the paper and has been corrected in the latest version. However, the correction note did not detail the reasons for this correction: "...in figure 2, the number at risk at the day 20 timepoint has been corrected to 15 117."
Another example: the primary outcome was ill-defined in the trial registry. One can read it here. One cannot know if the primary outcome was assessed after the first or the second dose. A full protocol was not available with the publication. We describe other concerns with data reporting in this study. Importantly, we have some concerns about an excess of homogeneity in one analysis (i.e. the subgroup analysis by age group). In this analysis, efficacy across all age subgroups was very similar with points estimates around 90-92%. But the confidence intervals were large (from 50 to 99 %). We would therefore expect less homogeneous results, even under the null hypothesis, e.g. point estimates ranging from 70 % to 99 %. This is a central analysis as the market value of a Covid vaccine depends on its efficacy across age groups. Please note that we did not find an excess of homogeneity in other analyses presented in this paper. Therefore, this extreme homogeneity could have occurred by chance.
However, we found another excess of homogeneity across interim results reported by press releases. The Lancet refused to publish this information because press releases are not part of their publication repertoire. Even alluding to this was not possible in our Lancet letter. We, therefore, published this information as rapid response at The BMJ. In this letter, we explain that the ratio of the number of events between the placebo and vaccine groups is as homogeneous as can be permitted by the number of events in the different press releases and The Lancet publication.
In our opinion, there is an excess of homogeneity in two somewhat independent analyses. It is important to say that we cannot affirm that there is a problem. But we can say that it is suspect enough to warrant an inspection of the database.
However, the database is not available. The publication explains that data sharing will be considered after the publication of the completed study and after approval by a "security department". Importantly, please note that lack of data sharing is not only a problem with the sputnik V vaccine. The whole medical literature is obscure and clinical trial data sharing is the exception, even in COVID biomedical research.
3 of the authors responded to our letter. According to them, problems were just some typos. The excess of homogeneity was just proof of the vaccine efficacy: "the homogeneity of the values only confirms the fact that, as described in the article, the effectiveness of the vaccine does not differ between age groups." Again, our concerns do not mean that the vaccine does or doesn't work, but rather that any appraisal of this study deserves a careful look at the data. The authors' answer suggests that we are far from data sharing.