Skip to main content
×
×
Home

USING THE AMSTAR CHECKLIST FOR RAPID REVIEWS: IS IT FEASIBLE?

  • Jessica Tajana Mattivi (a1) and Barbara Buchberger (a1)
Abstract

Objectives: Rapid reviews can be conducted in a narrower time frame, as compared to systematic reviews, by featuring restrictions. To estimate the validity of the results, assessment of methodological quality is required. Our aim was to analyze the methodological restrictions of rapid reviews compared with systematic reviews using the AMSTAR checklist and assess its feasibility for rapid reviews.

Methods: A systematic search for literature on rapid reviews of surgical interventions was conducted in three databases: Medline, Embase, and the Cochrane library. Additionally, health technology assessment (HTA) databases were searched. We analyzed reviews using AMSTAR and additionally compared the results with those of an overview of reviews on the same topic.

Results: Items found more frequently in rapid reviews were search for gray literature (65 percent versus 33 percent), listing of excluded studies (59 percent versus 37 percent), and provision of study characteristics (77 percent versus 44 percent), whereas consideration of study quality in formulating conclusions, conduct of meta-analysis, and statement of conflicts of interest were less frequent. Median time between search and publication was 8 months, with a range between 1 and 27.

Conclusions: With some adjustments, AMSTAR can be used as a checklist for rapid reviews to describe methodological restrictions in comparison to systematic reviews and to roughly estimate the validity of the results. Strikingly, only 14.3 percent of rapid reviews were published within 3 months.

Copyright
References
Hide All
1. Guyatt, GH, Sackett, DL, Sinclair, JC, et al. Users’ guides to the medical literature: IX. A method for grading health care recommendations. JAMA. 1995;274:18001804.
2. Wallace, BC, Dahabreh, IJ, Schmid, CH, Lau, J, Trikalinos, TA. Modernizing the systematic review process to inform comparative effectiveness: Tools and methods. J Comp Effect Res. 2013;2:273282.
3. Dobbins, M, Jack, S, Thomas, H, Kothari, A. Public health decision-makers’ informational needs and preferences for receiving research evidence. Worldviews Evid Based Nurs. 2007;4:156163.
4. Perleth, M, Lühmann, D, Gibis, B, Droste, S. “Rapid Assessments”: Schnelle Bewertung medizinischer Technologien. ZaefQ. 2001;95:7680.
5. Oxman, AD, Schünemann, HJ, Fretheim, A. Improving the use of research evidence in guideline development: 8. Synthesis and presentation of evidence. Health Res Policy Syst. 2006;4:20.
6. Ganann, R, Ciliska, D, Thomas, H. Expediting systematic reviews: Methods and implications of rapid reviews. Implement Sci. 2010;5:56.
7. AMSTAR – Assessing the methodological quality of systematic reviews. AMSTAR Checklist. http://www.amstar.ca/Amstar_Checklist.php (accessed June 4, 2015).
8. Watt, A, Cameron, A, Sturm, L, et al. Rapid versus full systematic reviews: Validity in clinical practice? ANZ J Surg. 2008;78:10371040.
9. Martel, G, Duhaime, S, Barkun, JS, et al. The quality of research synthesis in surgery: The case of laparoscopic surgery for colorectal cancer. Syst Rev. 2012;1:14.
10. Shea, BJ, Grimshaw, JM, Wells, GA, et al. Development of AMSTAR: A measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol. 2007;7:10.
11. Shea, BJ, Bouter, LM, Peterson, J, et al. External validation of a measurement tool to assess systematic reviews (AMSTAR). PLoS One. 2007;2:e1350.
12. Shea, BJ, Hamel, C, Wells, GA, et al. AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. J Clin Epidemiol. 2009;62:10131020.
13. Bond, K, Dennett, L. Stingray™ catheter and guidewire for recanalization of coronary chronic total occlusions. A rapid evidence assessment. Edmonton: Institute of Health Economics; 2013. http://www.ihe.ca/documents/Stingray%20catheter%20and%20guidewire%20for%20recanalization%20of%20coronary%20chronic%20total%20occlusions.pdf (accessed June 4, 2015).
14. Fischer, S, Zechmeister, I. Sakralnervenstimulation bei fäkaler Inkontinenz. Rapid Assessment 004. Ludwig Boltzmann Institut, Wien, 2011. http://eprints.hta.lbg.ac.at/925/1/Rapid_Assessment_004.pdf (accessed June 4, 2015).
15. Hill, R, Bagust, A, Bakhai, A, et al. Coronary artery stents: A rapid systematic review and economic evaluation. Health Technol Assess. 2004;8:iii–iv, 1–242.
16. Humphreys, K. Upper airway surgery for the treatment of adult obstructive sleep apnoea. ASERNIP-S Report No. 67. Adelaide, South Australia: ASERNIP-S; 2008. http://www.surgeons.org/media/300727/Upper_airway_surgery_for_adult_OSA.pdf (accessed June 4, 2015).
17. Jobanputra, P, Parry, D, Fry-Smith, A, Burls, A. Effectiveness of autologous chondrocyte transplantation for hyaline cartilage defects in knees: A rapid and systematic review. Health Technol Assess. 2001;5:157.
18. Kim, DG, Choi, YY, An, JY, et al. Comparing the short-term outcomes of totally intracorporeal gastroduodenostomy with extracorporeal gastroduodenostomy after laparoscopic distal gastrectomy for gastric cancer: A single surgeon's experience and a rapid systematic review with meta-analysis. Surg Endosc. 2013;27:31533161.
19. Kvas, E. Medikamentfreisetzende Stents bei Koronarinterventionen im Vergleich zu nicht beschichteten Stents. Rapid Assessment 001. Ludwig Boltzmann Institut, Wien: 2006. http://eprints.hta.lbg.ac.at/34/1/Rapid_Assessment_001.pdf (accessed June 4, 2015).
20. Lee, I. Clinical treatments for wrist ganglia. ASERNIP-S Report No. 63. Adelaide, South Australia: ASERNIP-S; 2008. http://www.surgeons.org/media/310861/Clinical_treatment_for_wrist_ganglia.pdf (accessed June 4, 2015).
21. Leopardi, D, Hoggan, B. Treatments for varicose veins. ASERNIP-S Report No. 66. Adelaide, South Australia: ASERNIP-S; 2008. http://www.surgeons.org/media/300551/Treatments_for_varicose_veins.pdf (accessed June 4, 2015).
22. Meads, C, Cummins, C, Jolly, K, et al. Coronary artery stents in the treatment of ischaemic heart disease: A rapid and systematic review. Health Technol Assess. 2000;4:1153.
23. Migliore, A, Jefferson, T, Cerbo, M, Abraha, I, Montedori, A. Implantable devices for the closure of patent foramen ovale in adults: An Italian rapid health technology assessment. Expert Rev Med Devices. 2014;11:151161.
24. Murphy, G, Cunningham, J. Percutaneous heart valve replacement for valvular heart disease: A review of the clinical effectiveness, cost-effectiveness, and guidelines. CADTH (Health Technology Inquiry Service) 2010. http://www.cadth.ca/media/pdf/l0149_percutaneous_heart_valves_htis-2.pdf (accessed June 4, 2015).
25. Parkes, J, Bryant, J, Milne, R. Implantable cardioverter-defibrillators in arrhythmias: A rapid and systematic review of effectiveness. Heart. 2002;87:438442.
26. Perera, C. Male non-therapeutic circumcision. ASERNIP-S Report No. 65. Adelaide, South Australia: ASERNIP-S; 2008. http://www.surgeons.org/media/292210/Male_non-therapeutic_circumcision.pdf (accessed June 4, 2015).
27. Stordeur, S, Gerkens, S, Roberfroid, D. Interspinous implants and pedicle screws for dynamic stabilization of lumbar spine: Rapid assessment. KCE Reports 116, 2009. https://kce.fgov.be/sites/default/files/page_documents/d20091027346.pdf (accessed June 4, 2015).
28. Thavaneswaran, P. Robotic-assisted surgery for urological, cardiac and gynaecological procedures. ASERNIP-S Report No. 75. Adelaide, South Australia: ASERNIP-S; 2009. http://www.surgeons.org/media/299238/RPT_2009-12-09_Robotic-assisted_Surgery.pdf (accessed June 4, 2015).
29. Vlayen, J, Camberlin, C, Paulus, D, Ramaekers, D. Rapid assessment of emerging spine technologies: Intervertebral disc replacement and vertebro/balloon kyphoplasty. KCE Reports 39, 2006. https://kce.fgov.be/sites/default/files/page_documents/20061027339.pdf (accessed June 4, 2015).
30. Harker, J, Kleijnen, J. What is a rapid review? A methodological exploration of rapid reviews in Health Technology Assessments. Int J Evid Based Healthc. 2012;10:397410.
31. Da Costa, BR, Cevallos, M, Altman, DG, Rutjes, AW, Egger, M. Uses and misuses of the STROBE statement: Bibliographic study. BMJ Open. 2011;1:e000048.
32. Buchberger, B, von Elm, E, Gartlehner, G, et al. Assessment of risk of bias in controlled studies. Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz. 2014;57:14321438.
33. Dreier, M, Borutta, B, Stahmeyer, J, Krauth, C, Walter, U. Comparison of tools for assessing the methodological quality of primary and secondary studies in health technology assessment reports in Germany. GMS Health Technol Assess. 2010;6:Doc07.
34. Buscemi, N, Hartling, L, Vandermeer, B, Tjosvold, L, Klassen, TP. Single data extraction generated more errors than double data extraction in systematic reviews. J Clin Epidemiol. 2006;59:697703.
35. Royle, P, Waugh, N. Literature searching for clinical and cost-effectiveness studies used in health technology assessments reports carried out for the National Institute for Clinical Excellence appraisal system. Health Technol Assess. 2003;7:iii, ix51.
36. Oxman, AD, Schunemann, HJ, Fretheim, A. Improving the use of research evidence in guideline development: 14. Reporting guidelines. Health Res Policy Syst. 2006;4:26.
37. Palmer, S, Jansen, A, Leitmeyer, K, Murdoch, H, Forland, F. Evidence-based medicine applied to the control of communicable disease incidents when evidence is scarce and the time is limited. Euro Surveill. 2013;18:pii:20507.
38. Higgins, JPT, Altman, DG, Sterne, JAC. 8.3.3 Quality scales and Cochrane reviews. InHiggins JPT, Green S, eds. Cochrane handbook for systematic reviews of interventions version 5.1.0. The Cochrane Collaboration, 2011. www.cochrane-handbook.org (accessed June 4, 2015).
39. Watt, A, Cameron, A, Sturm, L, et al. Rapid reviews versus full systematic reviews: An inventory of current methods and practice in health technology assessment. Int J Technol Assess Health Care. 2008;24:133139.
40. Sampson, M, Shojania, KG, Garritty, C, et al. Systematic reviews can be produced and published faster. J Clin Epidemiol. 2008;61:531536.
41. Shojania, KG, Sampson, M, Ansari, MT, et al. How quickly do systematic reviews go out of date? A survival analysis. Ann Intern Med. 2007;147:224233.
42. Bastian, H, Glasziou, P, Chalmers, I. Seventy-five trials and eleven systematic reviews a day: How will we ever keep up? PLoS Med. 2010;7:e1000326.
43. EUnetHTA – European Network for Health Technology Assessment 2013. HTA Core Model® for rapid relative effectiveness assessment of pharmaceuticals. http://www.eunethta.eu/sites/5026.fedimbo.belgium.be/files/Model%20for%20Rapid%20REA%20of%20pharmaceuticals_final_20130311_reduced.pdf (accessed June 4, 2015).
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

International Journal of Technology Assessment in Health Care
  • ISSN: 0266-4623
  • EISSN: 1471-6348
  • URL: /core/journals/international-journal-of-technology-assessment-in-health-care
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×

Keywords

Type Description Title
WORD
Supplementary materials

Mattivi and Buchberger supplementary material
Supplementary Table 2

 Word (17 KB)
17 KB
WORD
Supplementary materials

Mattivi and Buchberger supplementary material
Supplementary Table 3

 Word (18 KB)
18 KB
WORD
Supplementary materials

Mattivi and Buchberger supplementary material
Supplementary Table 1

 Word (15 KB)
15 KB

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 18
Total number of PDF views: 159 *
Loading metrics...

Abstract views

Total abstract views: 984 *
Loading metrics...

* Views captured on Cambridge Core between 18th October 2016 - 19th August 2018. This data will be updated every 24 hours.