Sixth year medical student, Charlotte, shares evidence-based project as part of the special studies module (SSM)

My name is Charlotte and I’m a 6th year medical student. As part of my post-finals special study module (SSM) I chose to join the Centre for Evidence-Based Medicine within the Department of Primary Care Health Sciences for three weeks, as I had heard great things about it from previous students. I was keen to improve my knowledge of EBM beyond the classes I’ve had as a medical student. My goal for this module was to learn about evaluating literature from an EBM perspective and the process of leading a project from protocol to publication. The three-week long SSM provided a fabulous opportunity to do this, whilst also having some interesting debates about EBM along the way.

What Did We Do?

During my first meeting with Dr David Nunan (SSM departmental lead),  we reviewed the core principles of EBM, open science and discussed our project. I focussed on evaluating the reporting of medication adherence to pharmacological interventions in coronavirus (SARS-CoV-2) related randomized control trials (RCTs). I learnt the importance of developing and sharing a protocol before commencing a study and led on this for my project, publishing our protocol in the Open Science Framework.

To assess this we initially looked at applying the European Society for Patient Adherence, COMpliance, and Persistence (ESPACOMP) Medication Adherence Reporting Guidelines (EMERGE). However, we struggled with applying these guidelines to our data set. As such, the project evolved to evaluate reporting standards against a different guideline, the Template for Intervention Description and Replication Checklist (TIDieR). TIDieR is an extension of Consolidated Standards Of Reporting Trials, 2010 (CONSORT) statement 5 and Standard Protocol Items: Recommendations for Interventional Trials, 2013 (SPIRIT) item 11. Specifically, we focussed on TIDieR items eleven and twelve which state:

‘11. Planned: If intervention adherence or fidelity was assessed, describe how and by whom, and if any strategies were used to maintain or improve fidelity, describe them.

12. Actual: If intervention adherence or fidelity was assessed, describe the extent to which intervention was delivered as planned’

To identify relevant trials, we searched for RCTs focussing on SARS-CoV-2, published in five major medical journals: Annals of Internal Medicine, British Medical Journal (BMJ), Journal of the American Medical Association (JAMA), The Lancet,  and the New England Journal of Medicine (NEJM), between January 1 2021 and December 31, 2021. Each RCT was assessed for compliance with items 11 and 12 of the TIDieR checklist. Item 11 was assessed in two parts:

11a) If intervention adherence or fidelity was assessed, describe how and by whom

11b) If intervention adherence or fidelity was assessed… and if any strategies were used to maintain or improve fidelity, describe them

What Did We Find?

Our search returned 167 publications from which 75 were identified as fulfilling our inclusion criteria (Figure 1). Our search and included trials organized by journal is summarized in Figure 1. The pharmacological interventions studied  included vaccination (28 trials), monoclonal antibodies (15 trials), anticoagulant therapy (7 trials), convalescent plasma therapy (5 trials), steroid treatment (4 trials), macrolide treatment (3 trials), antimalarial therapy (3 trials), janus kinase inhibitors (2 trials), and oxygen therapy (2 trials). One trial each focused on antiplatelet therapy, antiparasitic therapy, ACE inhibitor therapy, and vitamin D supplementation respectively.

A                                                                                                             B

Figure 1. A Summary of search process. B Journal distribution of trials included

Overall, we found 9.9% of trials included in this study complied with TIDieR items 11a, 11b and 12 . This poor adherence was particularly demonstrated with item 11b (describing strategies used to increase and/or maintain intervention adherence) across all journals. 

All five journals endorse the use of CONSORT in their publication process, with BMJ and JAMA requiring a completed CONSORT checklist submitted with manuscripts. JAMA additionally endorses extensions of CONSORT, such as TIDieR. It is imperative to maintain rigorous reporting standards during public health crises. Despite commitment to rigorous reporting standards of intervention adherence, we found poor compliance with known reporting guidelines. Plausible reasons to explain these findings will be discussed in the final write up.

What I learned

I really enjoyed having dedicated time to focus on the principles of EBM and my project! Through this SSM, I have learned how to design a study, from protocol composition to results. Specifically, my project has highlighted to me the importance of critical appraisal; I was surprised by the number of non-compliant publications in my project, especially given reputable journals we appraised. 

Having now completed my SSM, I feel like I’ve gained a good foundation to begin my AFP and clinical career. I hope to write up and present these results more formally.