Assessing risk of bias judgments for blinding of outcome assessors in Cochrane reviews
Publication: Journal of Comparative Effectiveness Research
Abstract
Aim: Adequate judging of risk of bias (RoB) for blinding of outcome assessors (detection bias) is important for supporting highest level of evidence. Materials & methods: Judgments and supporting comments for detection bias were retrieved from RoB tables reported in Cochrane reviews. We categorized comments, and then compared judgment and supporting comment with instructions from the Cochrane Handbook. Results: We analyzed 8656 judgments for detection bias from 7626 trials included in 575 reviews. Overall, 1909 judgments (22%) were not in line with the Cochrane Handbook. In 9% of trials, the authors split the detection bias domain according to outcomes. Here, prevalence of inadequate judgments was 19%. Conclusion: Interventions to improve RoB assessments in systematic reviews should be explored.
Supplementary Material
References
1.
Higgins JPT, Green S (Eds). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. (2011). The Cochrane Collaboration, London, UK. http://handbook-5-1.cochrane.org/
2.
Hrobjartsson A, Thomsen AS, Emanuelsson F et al. Observer bias in randomised clinical trials with binary outcomes: systematic review of trials with both blinded and non-blinded outcome assessors. BMJ 344, e1119 (2012).
3.
Kahan BC, Rehal S, Cro S. Blinded outcome assessment was infrequently used and poorly reported in open trials. PLoS ONE 10(6), e0131926 (2015).
4.
Barcot O, Boric M, Poklepovic Pericic T et al. Risk of bias judgments for random sequence generation in Cochrane systematic reviews were frequently not in line with Cochrane Handbook. BMC Med. Res. Methodol. 19(1), 170 (2019).
5.
Propadalo I, Tranfic M, Vuka I, Barcot O, Poklepovic Pericic T, Puljak L. In Cochrane reviews, risk of bias assessments for allocation concealment were frequently not in line with Cochrane’s Handbook guidance. J. Clin. Epidemiol. 106, 10–17 (2019).
6.
Barcot O, Boric M, Dosenovic S, Poklepovic Pericic T, Cavar M, Puljak L. Risk of bias assessments for blinding of participants and personnel in Cochrane reviews were frequently inadequate. J. Clin. Epidemiol. 113, 104–113 (2019).
7.
Saric F, Barcot O, Puljak L. Risk of bias assessments for selective reporting were inadequate in the majority of Cochrane reviews. J. Clin. Epidemiol. 112, 53–58 (2019).
8.
Babic A, Tokalic R, Amilcar Silva Cunha J et al. Assessments of attrition bias in Cochrane systematic reviews are highly inconsistent and thus hindering trial comparability. BMC Med. Res. Methodol. 19(1), 76 (2019).
9.
Babic A, Pijuk A, Brazdilova L et al. The judgement of biases included in the category “other bias” in Cochrane systematic reviews of interventions: a systematic survey. BMC Med. Res. Methodol. 19(1), 77 (2019).
10.
Moustgaard H, Bello S, Miller FG, Hrobjartsson A. Subjective and objective outcomes in randomized clinical trials: definitions differed in methods publications and were often absent from trial reports. J. Clin. Epidemiol. 67(12), 1327–1334 (2014).
11.
Review Manager (RevMan) [Computer program]. Version 5.3. Copenhagen: The Nordic Cochrane Centre, The Cochrane Collaboration (2014). https://revman.cochrane.org/
12.
Cambridge Dictionary (2019). https://dictionary.cambridge.org/dictionary/english/objective
13.
De Jong JD, Westert GP, Lagoe R, Groenewegen PP. Variation in hospital length of stay: do physicians adapt their length of stay decisions to what is usual in the hospital where they work? Health Serv. Res. 41(2), 374–394 (2006).
14.
Sterne JaC, Savovic J, Page MJ et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ 366, l4898 (2019).
15.
Puljak L, Ramic I, Arriola Naharro C et al. Cochrane risk of bias tool was used inadequately in the majority of non-Cochrane systematic reviews. Journal of Clinical Epidemiology 123, 114-119 (2020).
16.
Berger VW, Exner DV. Detecting selection bias in randomized clinical trials. Control. Clin. Trials 20(4), 319–327 (1999).
17.
Mickenautsch S, Fu B, Gudehithlu S, Berger VW. Accuracy of the Berger-Exner test for detecting third-order selection bias in randomised controlled trials: a simulation-based investigation. BMC Med. Res. Methodol. 14, 114 (2014).
18.
Berger V. Selection Bias and Covariate Imbalances in Randomized Clinical Trials. John Wiley & Sons, Chichester, UK (2005).
Information & Authors
Information
Published In
Pages: 585 - 593
PubMed: 32459105
Copyright
© 2020 Future Medicine Ltd.
History
Received: 1 December 2019
Accepted: 23 March 2020
Published online: 27 May 2020
Keywords:
Topics
Authors
Metrics & Citations
Metrics
Article Usage
Article usage data only available from February 2023. Historical article usage data, showing the number of article downloads, is available upon request.
Citations
How to Cite
Assessing risk of bias judgments for blinding of outcome assessors in Cochrane reviews. (2020) Journal of Comparative Effectiveness Research. DOI: 10.2217/cer-2019-0181
Export citation
Select the citation format you wish to export for this article or chapter.
Citing Literature
- Abdullah Yonis, Sarah Gerard Dean, Fiona C. Warren, Rod S. Taylor, William Levack, Jean Hay-Smith, Navigating blinding challenges in complex intervention trials: insights from a UK researcher survey, Trials, 10.1186/s13063-025-09223-9, 26, 1, (2025).
- Nawras Fashafsheh, Ismail A Elhaty, The Effectiveness of Bladder Filling Technique for Preventing Intraoperative Bladder Injury in Pregnant Women Undergoing Placenta Accreta Surgery: A Systematic Review, Sage Open Nursing, 10.1177/23779608251342751, 11, (2025).
- Livia Puljak, Andrija Babić, Ognjen Barčot, Tina Poklepović Peričić, Evolving use of the Cochrane Risk of Bias 2 tool in biomedical systematic reviews , Research Synthesis Methods, 10.1002/jrsm.1756, 15, 6, (1246-1247), (2024).
- Daniel Rehlicki, Mia Plenkovic, Ljerka Delac, Dawid Pieper, Ana Marušić, Livia Puljak, Author instructions in biomedical journals infrequently address systematic review reporting and methodology: a cross-sectional study, Journal of Clinical Epidemiology, 10.1016/j.jclinepi.2023.11.008, 166, (111218), (2024).
- Andrija Babić, Ognjen Barcot, Tomislav Visković, Frano Šarić, Aleksandar Kirkovski, Ivana Barun, Zvonimir Križanac, Roshan Arjun Ananda, Yuli Viviana Fuentes Barreiro, Narges Malih, Daiana Anne‐Marie Dimcea, Josipa Ordulj, Ishanka Weerasekara, Matteo Spezia, Marija Franka Žuljević, Jelena Šuto, Luca Tancredi, Anđela Pijuk, Susanna Sammali, Veronica Iascone, Thilo von Groote, Tina Poklepović Peričić, Livia Puljak, Frequency of use and adequacy of Cochrane risk of bias tool 2 in non‐Cochrane systematic reviews published in 2020: Meta‐research study , Research Synthesis Methods, 10.1002/jrsm.1695, 15, 3, (430-440), (2024).
- Silvia Minozzi, Marien Gonzalez-Lorenzo, Michela Cinquini, Daniela Berardinelli, Celeste Cagnazzo, Stefano Ciardullo, Paola De Nardi, Mariarosaria Gammone, Paolo Iovino, Alex Lando, Marco Rissone, Giovanni Simeone, Marta Stracuzzi, Giovanna Venezia, Lorenzo Moja, Giorgio Costantino, Angelo Cianciulli, Andrea Cinnirella, Francesca Grosso, Francesco Luceri, Giuseppe Venuti, Stefania Vultaggio, Emiliano Zambarbieri, Adherence of systematic reviews to Cochrane RoB2 guidance was frequently poor: a meta epidemiological study, Journal of Clinical Epidemiology, 10.1016/j.jclinepi.2022.09.003, 152, (47-55), (2022).
- Bart Torensma, Mohamed Hisham, Abdelazeem A. Eldawlatly, Mohamed Hany, Differences Between the 2016 and 2022 Editions of the Enhanced Recovery After Bariatric Surgery (ERABS) Guidelines: Call to Action of FAIR Data and the Creation of a Global Consortium of Bariatric Care and Research, Obesity Surgery, 10.1007/s11695-022-06132-7, 32, 8, (2753-2763), (2022).
- Ognjen Barcot, Matija Boric, Svjetlana Dosenovic, Livia Puljak, Assessing the risk of performance and detection bias in Cochrane reviews as a joint domain is less accurate compared to two separate domains, BMC Medical Research Methodology, 10.1186/s12874-021-01339-1, 21, 1, (2021).
- Ognjen Barcot, Matej Ivanda, Ivan Buljan, Dawid Pieper, Livia Puljak, Enhanced access to recommendations from the Cochrane Handbook for improving authors' judgments about risk of bias: A randomized controlled trial, Research Synthesis Methods, 10.1002/jrsm.1499, 12, 5, (618-629), (2021).
- Ognjen Barcot, Matija Boric, Svjetlana Dosenovic, Marija Cavar, Antonia Jelicic Kadic, Tina Poklepovic Pericic, Ivana Vukicevic, Ivana Vuka, Livia Puljak, Adequacy of risk of bias assessment in surgical vs non-surgical trials in Cochrane reviews: a methodological study, BMC Medical Research Methodology, 10.1186/s12874-020-01123-7, 20, 1, (2020).