Skeptics in the Pub - COMPare: tracking switched outcomes in clinical trials, and trying to correct the record
COMPare project team
Wednesday, 13 July 2016, 7.30pm to 9pm
St Aldates Tavern, 108 St Aldate's, City Centre, Oxford. OX1 1BU
Hosted by Skeptics in the PubFind out more on Facebook
How and why the infrastructure of science is broken
For 6 weeks in late 2015, the COMPare team monitored every clinical trial published in the top 5 medical journals for “outcome switching”: when trialists report something different from what they originally said they would report. Of 67 trials assessed, 58 (87%) were found to contain discrepancies between prespecified and reported outcomes.
Outcome switching is already known to be extremely common, even in top medical journals. But COMPare went one step further: they wrote a letter to the journal for all 58 trials found to contain discrepancies; to correct the record on the individual trials, and to test the “self-correcting” properties of science.
The responses to these letters from journal editors and trial authors were unprecedented, and shed light on the reasons why this problem persists. The aim of COMPare was to fix outcome switching, through correction letters and open discussion. They never expected the levels of misunderstanding and bias at the heart of the issue.
Based at the Centre for Evidence-Based Medicine, COMPare is made up of three senior researchers, 5 graduate-entry medical students, and a programmer. The project was born when one medical student came to the department in search of a project. The idea of monitoring the outcomes in clinical trials was made possible by 4 more medical students, who were recruited to make the vast amount of analysis possible. All assessments are reviewed by senior colleagues, and decisions made at weekly team meetings. There is no specific funding for COMPare: all the students work for free, driven by the desire and opportunity to fix a broken system.