Systematic reviews are built on the principles of rigor, transparency, and replicability. However, many current AI solutions don’t meet these principles. This risks a surge in unreliable, biased and low-quality reviews.
At Cochrane, we are committed to addressing this challenge with an approach that is measured and responsible. Here, we set out some of the work we are doing to ensure AI is used responsibly in evidence synthesis.
Using AI to support our review authors
Within Cochrane, we have a history of implementing automation solutions in our review process. For example, we have been using machine learning to identify randomized controlled trials since 2016. And we are now investing in the integration of emerging technologies, like generative AI.
We are primed to use AI and automation, thanks to our wealth of high-quality, structured data from systematic reviews and included studies. Recent innovations include:
dynamic analysis reporting in RevMan, which allows authors to insert live results directly into their reviews while they’re writing and updating them
a new feature in CENTRAL, our clinical trials database, that flags retracted publications to authors
In the future, we are planning to increase the proportion of reviews supported by Cochrane’s Evidence Pipeline, a service that combines automation and crowd verification to help authors identify relevant studies. In addition, we are leveraging annotations on Patient/Population, Intervention, Comparison, Outcome (PICO) to inform decisions about new intervention review proposals. Alongside the technical advances, we are working on further guidance and training to improve AI literacy across our organization.