The impact of machine learning for systematic reviews
We are all too well aware of how time consuming systematic reviews are, as the flood of research continues to enter the world, the job only gets bigger. Fortunately, recent progress in automation and machine learning has made lots of these tasks quicker and more efficient for review teams.
Here at Covidence we continue to build in efficiencies to help you finish your review sooner. That’s why we’re delighted to bring you the Cochrane Randomized Controlled Trial (RCT) classifier, which helps you to quickly and accurately filter out the studies that are not RCTs.
Firstly, what is the Cochrane RCT Classifier?
Led by James Thomas, the Cochrane RCT Classifier is a machine learning tool that automatically retrieves randomized controlled trials for Cochrane reviews (Thomas et al, 2021). The classifier has now been integrated into Covidence and can be used for all medical and health science systematic reviews.
How does it work in Covidence?
If you are reviewing medical or health science literature, there is a good chance that one of your inclusion criteria is that the study must be an RCT. The classifier looks at all the studies you import into Covidence and automatically assigns one of two tags:
- Possible RCT
- Not RCT
You can filter the studies by the ‘Possible RCT’ tag’ to quickly see the studies that meet the inclusion criteria.
Could the classifier miss a genuine RCT?
This is very unlikely. The biggest concern about the use of machine learning in systematic reviews is the incorrect removal of relevant evidence. The power of the classifier is its recall (sensitivity): it can correctly identify a non-RCT with 99% accuracy. The classifier does not remove any studies from the review. Review teams can decide for themselves whether to check some, all, or none of the studies tagged with ‘Not RCT’ before voting to exclude them.
As we might expect from a tool with such high recall, its precision is correspondingly low. When testing the classifier, Thomas and colleagues found that only 8% of studies labelled ‘Possible RCT’ were, in fact, RCTs. While this means that the list of Possible RCTs will contain a high proportion of ineligible studies, review teams can make a start on this list with confidence that the non-RCTs have been tagged correctly by the classifier. This helps review teams to make the most efficient use of their time and resources.
How can I access this feature?
The classifier is trusted by Cochrane and has been expertly integrated into Covidence to help screen faster. To access it, just opt in when setting up a new review in the Medical and Health Sciences category as shown in the image below.
Conclusion
Technology and automation is reducing the manual workload and creating efficiencies in the production of systematic reviews. This not only benefits the review community, but also ensures research enters the world faster and provides clinicians with access to the latest evidence to provide the best care for their patients.
Try Covidence and this feature for yourself, with a free trial.
