Ideally, medical practitioners and health policy analysts feel more confident when the results and conclusions of scientific experiments can be replicated. However, it has been found that many scientific studies are difficult or impossible to replicate or reproduce. In response, scientists have launched various replication initiatives to assess the robustness of published research. For example, in psychology, numerous labs have volunteered to re-run studies, with the methods used by the original researchers : in one examination of 100 experiments only 39 were successfully replicated. The same concerns have been expressed about biomedical research. But while attempts to replicate psychological studies are often easy to implement and feasible, attempts to replicate medical studies (pre-clinical or clinical) are often costly and difficult to run. This is especially true for randomized controlled trials (RCTs) although these studies are typically of major importance, and expense, in informing health practices and health policies.
Data sharing is increasing in medicine and will enable re-analyses of this sort to be run. Numerous pharmaceutical companies have created mechanisms for investigators to access IPD. Although more than 6000 trials are already available to investigators through open data platforms, only 15.5% had been requested by a small number of researchers. Most proposals have focused on non-pre-specified subgroups or predictors of response rather than on validation of study results.
To date, no independent, systematic and large scale re-analysis project has been published in the field of RCTs. We propose to conduct a project of this sort, which is needed 1/ to explore the existence and the extent of a ‘reproducibility crisis’ in biomedicine and 2/ to better inform editors, funders and other stakeholders as to whether their new policies really make science better, more transparent and reproducible.
Florian is a psychiatrist, meta-researcher and former post-doctoral fellow at METRICS (the Meta-research Innovation Center at Stanford). He's currently Professor of Therapeutics at Rennes 1 University, France. His research interests are evaluating and developing methodological solutions to assess treatments in patients, primarily but not exclusive in psychiatric research. He has a strong interest in studying research wastes and data-sharing practices.
FUNDER'S DATA SHARING POLICIES IN THERAPEUTIC RESEARCH