Is there better teaching opportunity than assigning systematic reviews in a semester?
Carrie Price tweeted about the phenomenon of assigning students a systematic review within a semester. Her tweet got a lot of attention from medical educators and librarians alike. We reached out to Carrie to find out more on the topic and she shares her thoughts with us.
If you’re teaching faculty who has assigned systematic-style literature reviews to your students for a semester project, it might be time to revisit the exercise. Trying to fit a full systematic review into a semester is almost always problematic.
The goal, after all, is to have students leave with skills they can use in real-life research scenarios, and not frustration and a misunderstanding of the comprehensive literature review process. Consider that a true systematic review requires a full expert team that includes librarians, statisticians, and methodologists; follows a rigorous methodology that includes adopting a critical appraisal tool and a standard for synthesizing the evidence; and takes on average 67.3 weeks to complete (Borah et al., 2017). Expecting students to grasp this process in the scope of a typical semester is a big ask, and often leaves them defeated and discouraged, with more questions than answers they started with.
What are some options?
Here are some exercises that can incorporate that will equip students with a better understanding of the systematic review process and evidence syntheses overall. Think of this as a buffet – use one assignment or several.
Rapid Review Exercise
A rapid review provides a rapid synthesis of knowledge about a policy or clinical practice issue and attempts to inform an evidence-based decision quickly. It follows many of the stages of a systematic knowledge synthesis but may modify stages to shorten the timeline.
Read more:
- Garritty, C., Gartlehner, G., Nussbaumer-Streit, B., King, V. J., Hamel, C., Kamel, C., … & Stevens, A. (2020). Cochrane Rapid Reviews Methods Group offers evidence-informed guidance to conduct rapid reviews. Journal of Clinical Epidemiology. https://doi.org/10.1016/j.jclinepi.2020.10.007
- Khangura, S., Konnyu, K., Cushman, R., Grimshaw, J., & Moher, D. (2012). Evidence summaries: the evolution of a rapid review approach. Systematic Reviews, 1, 10. http://doi.org/10.1186/2046-4053-1-10
Scoping Review
A scoping review might be used to map evidence and key concepts in fields that are new or very broad in scope, or where there is not enough data to make an effective synthesis. They may sometimes be done as a preliminary step to a systematic review.
Read more:
- Arksey, H., & O’Malley, L. (2005). Scoping studies: towards a methodological framework. International Journal of Social Research Methodology, 8(1), 19–32. http://doi.org/10.1080/1364557032000119616
- Munn, Z., Peters, M., Stern, C., Tufanaru, C., McArthur, A., & Aromataris, E. (2018). Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Medical Research Methodology, 18(1), 143. https://doi.org/10.1186/s12874-018-0611-x
Learning Outcome for Rapid or Scoping Review Exercises
Students:
- compare the differences between a systematic and scoping review.
- identify the appropriate methodology for future research
- gain practice in topic development and an applicable methodology.
Protocol Exercise
Ask students to select an appropriate topic and develop a protocol based on the PRISMA-P protocol reporting guideline (Moher et al, 2015). Developing a protocol ahead of the review helps students understand all that is involved in the workflow and process. Instructors can use the PRISMA-P Checklist as a rubric.
Learning Outcome for Protocol Exercise
Students:
- become familiar with the work and effort involved in comprehensive literature review methods.
- gain experience in topic development.
accurately report a systematic review protocol. - are prepared for the systematic review development process.
Annotated Bibliography of Classic Articles
Review Typologies
There exists within the corpus of literature a handful of classic “typology” articles. These articles aim to describe different approaches to evidence syntheses through literature review methodologies.
Read more:
- Akl, E.A., Haddaway, N., Rada, G., & Lotfi, T. (2020). Evidence synthesis 2.0: when systematic, scoping, rapid, living, and overviews of reviews come together? Journal of Clinical Epidemiology. https://doi.org/10.1016/j.jclinepi.2020.01.025
- Grant, M. J., & Booth, A. (2009). A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information & Libraries Journal, 26(2), 91-108. https://doi.org/10.1111/j.1471-1842.2009.00848.x
- Munn, Z., Stern, C., Aromataris, E., Lockwood, C., & Jordan, Z. (2018). What kind of systematic review should I conduct? A proposed typology and guidance for systematic reviewers in the medical and health sciences. BMC Medical Research Methodology, 18(1), 5. https://doi.org/10.1186/s12874-017-0468-4
- Sutton, A., Clowes, M., Preston, L., & Booth, A. (2019). Meeting the review family: exploring review types and associated information retrieval requirements. Health Information and Libraries Journal, 36(3), 202–222. https://doi.org/10.1111/hir.12276
Systematic Reviews
In addition to the above, anyone who may ever complete a systematic review should be aware of the following guidance.
- Aromataris E, Munn Z (Eds.). (2020). JBI Manual for Evidence Synthesis. Available from https://synthesismanual.jbi.global and https://doi.org/10.46658/JBIMES-20-01
- Campbell, M., McKenzie, J. E., Sowden, A., Katikireddi, S. V., Brennan, S. E., Ellis, S., … & Thomson, H. (2020). Synthesis without meta-analysis (SWiM) in systematic reviews: reporting guideline. BMJ, 368.https://doi.org/10.1136/bmj.l6890
- Higgins J.P.T., Thomas J., Chandler J., Cumpston M., Li T., Page M.J., Welch V.A. (Eds.). (2020 update). Cochrane Handbook for Systematic Reviews of Interventions version 6.1 (updated September 2020). Cochrane. https://training.cochrane.org/handbook/current
- Institute of Medicine (US) Committee on Standards for Systematic Reviews of Comparative Effectiveness Research, Eden, J., Levit, L., Berg, A., & Morton, S. (Eds.). (2011). Finding what works in health care: Standards for systematic reviews. National Academies Press (US). https://pubmed.ncbi.nlm.nih.gov/24983062/
- Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., … & Moher, D. (2021). The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. https://doi.org/10.1136/bmj.n71
A student would benefit from being asked to evaluate and reflect on any or all of these seminal publications.
Learning Outcomes for Annotated Bibliography of Classic Articles
Students:
- will describe different types of evidence syntheses.
- able to distinguish methodologies and their appropriateness for different topics
- name the various guidance documents that exist for the conduct and reporting of systematic reviews.
Search Methods Exercise
This exercise may work best in tandem with the PRISMA-P protocol exercise. After students have developed a protocol, have them develop, document, and carry out a search and screen the results. Ask them to write the search methods for their future proposed manuscript, where the PRISMA-S (Rethlefsen et al., 2021) functions as a rubric.
Learning Outcomes for Search Methods Exercise
Students:
- acquire valuable expert searching skills.
- document a reproducible and transparent search.
- recognize the importance of search documentation reporting within the broader scope of a systematic review.
Conclusion
If you are assigning a rapid or scoping review, be prepared to support your students through the entire process by sharing your expertise, bringing in other experts, and allowing students time to work and discuss during class. Consider inviting a librarian, a statistician, and/or a methodologist to your class. They will each have unique perspectives worthy of consideration.
Evidence-based practice champions can work together to empower students to complete evidence syntheses in their future careers by assigning stimulating and thought-provoking assignments without leaving them demoralized.
More references
Borah, R., Brown, A. W., Capers, P. L., & Kaiser, K. A. (2017). Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry. BMJ Open, 7(2). http://dx.doi.org/10.1136/bmjopen-2016-012545
Moher, D., Shamseer, L., Clarke M., Ghers,i D., Liberati, A., Petticrew, M., Shekelle P., Stewart LA. (2015). Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Systematic Reviews, 4(1). https://doi.org/10.1186/2046-4053-4-1
Rethlefsen, M.L., Kirtley, S., Waffenschmidt, S., Ayala, A.P., Moher, D., Page M.J., Koffel J.B. (2021). PRISMA-S Group. PRISMA-S: an extension to the PRISMA Statement for reporting literature searches in systematic reviews. Systematic Reviews, 10(1). https://doi.org/10.1186/s13643-020-01542-z
