Skip to main content
    • Aa
    • Aa


  • Craig R. Ramsay (a1), Lloyd Matowe (a2), Roberto Grilli (a3), Jeremy M. Grimshaw (a4) and Ruth E. Thomas (a1)...

Objectives: In an interrupted time series (ITS) design, data are collected at multiple instances over time before and after an intervention to detect whether the intervention has an effect significantly greater than the underlying secular trend. We critically reviewed the methodological quality of ITS designs using studies included in two systematic reviews (a review of mass media interventions and a review of guideline dissemination and implementation strategies).

Methods: Quality criteria were developed, and data were abstracted from each study. If the primary study analyzed the ITS design inappropriately, we reanalyzed the results by using time series regression.

Results: Twenty mass media studies and thirty-eight guideline studies were included. A total of 66% of ITS studies did not rule out the threat that another event could have occurred at the point of intervention. Thirty-three studies were reanalyzed, of which eight had significant preintervention trends. All of the studies were considered “effective” in the original report, but approximately half of the reanalyzed studies showed no statistically significant differences.

Conclusions: We demonstrated that ITS designs are often analyzed inappropriately, underpowered, and poorly reported in implementation research. We have illustrated a framework for appraising ITS designs, and more widespread adoption of this framework would strengthen reviews that use ITS designs.

Linked references
Hide All

This list contains references from the content that can be linked to their source. For a full set of references and notes please see the PDF or HTML where available.

AvornJ, SoumeraiSB, TaylorWet al. 1988 Reduction of incorrect antibiotic dosing through a structured educational order form. Arch Intern Med. 148: 1720- 1724.

BarefordD, HaylingA. 1990 Inappropriate use of laboratory services: Long term combined approach to modify request patterns. BMJ. 301: 1305- 1307.

BarnettGO, WinickoffR, DorseyJL, MorganMM, LurieRS. 1978 Quality assurance through automated monitoring and concurrent feedback usinga computer-based medical information system. Med Care. 16: 962- 970.

BeroL, GrilliR, GrimshawJMet al. 1998 Closing the gap between research and practice: An overview of systematic reviews of interventions to promote implementation findings by health care professionals. BMJ. 317: 465- 468.

BlohmM, HerlitzJ, HartfordMet al. 1992 Consequences of a media campaign focusing on delay in acute myocardial infarction. Am J Cardiol. 69: 411- 413.

BrookRH, WilliamsKN. 1976 Effect of medical care review on the use of injections: A study of the New Mexico Experimental Medical Care Review Organization. Ann Intern Med. 85: 509- 515.

BrufskyJW, Ross-DegnanD, CalabreseD, GaoX, SoumeraiSB. 1998 Shifting physician prescribing to a preferred histamine-2-receptor antagonist. Effects of a multifactorial intervention in a mixed-model health maintenance organization. Med. Care 36: 321- 332.

ClarkeJA, AdamsJE. 1990 The application of clinical guidelines for skull radiography in the accident and emergency department: Theory and practice. Clin Radiol. 41: 152-5.

Crosbie 1993 J. Interrupted time-series analysis with brief single-subject data. J Consult Clin Psychol. 61: 966- 974.

ElamK, TaylorV, CiolMA, FranklinGM, DeyoRA. 1997 Impact of a worker's compensation practice guideline on lumbar spine fusion in Washington State. Med Care. 35: 417- 424.

EpplerE, EisenbergMS, SchaefferS, MeischkeH, LarsonMP. 1994 911 and emergency department use for chest pain: Results of a media campaign. Ann Emerg Med. 24: 202- 208.

EverittDE, SoumeraiSB, AvornJ, KlapholzH, WesselsM. 1990 Changing surgical antimicrobial prophylaxis practices through education targeted at senior department leaders. Infect Control Hosp Epidemiol. 11: 578- 583.

FowkesFG, HallR, JonesJHet al. 1986 Trial of strategy for reducing the use of laboratory tests. BMJ Clin Res Ed. 292: 883- 885.

FraserGL, WennbergDE, DickensJDJr, LambrewCT. 1996 Changing physician behavior in ordering digoxin assays. Ann Pharmacother 30: 449- 454.

GamaR, NightingalePG, RatcliffeJG. 1992 Effect of educational feedback on clinicians' requesting of cardiac enzymes. Ann Clin Biochem. 29: 224- 225.

GortmakerSL, BickfordAF, MathewsonHO, DumbaughK TirrellPC. 1988 A successful experiment to reduce unnecessary laboratory use in a community hospital. Med Care. 26: 631- 642.

GurwitzJH, NoonanJP, SoumeraiSB. 1992 Reducing the use of H2-receptor antagonists in the long-term-care setting. J Am Geriatr Soc. 40: 359- 364.

HammondKW, SnowdenM, RisseSC, AdkinsTG, O'BrienJJ. 1995 An effective computer-based tardive dyskinesia monitoring system. Am J Med Qual. 10: 133- 137.

HealsmithMF, Graham-BrownRA, OsborneJE, LondonSP, FletcherA. 1993 Further experience of public education for the early diagnosis of malignant melanoma in Leicestershire. Clin Exp Dermatol. 18: 396- 400.

HerdRM, CooperEJ, HunterJAet al. 1995 Cutaneous malignant melanoma. Publicity screening clinics and survival – the Edinburgh experience 1982-90. Br J Dermatol. 132: 563- 570.

JoshiUY, CameronSO, SommervilleJM, SommervilleRG. 1988 HIV testing in Glasgow Genito-Urinary Medicine Clinics 1985-1987. Scott Med J. 33: 294- 295.

LegorretaAP, HasanMM, PetersAL, PelletierKR, LeungKM. 1997 An intervention for enhancing compliance with screening recommendations for diabetic retinopathy. A bicoastal experience. Diabetes Care 20: 520- 523.

LomasJ, AndersonGM, Domnick-PierreKet al. 1989 Do practice guidelines guide practice? The effect of a consensus statement on the practice of physicians. N Engl J Med. 321: 1306- 1311.

MaclureM, DormuthC, NaumannTet al. 1998 Influences of educational interventions and adverse news about calcium-channel blockers on first-line prescribing of antihypertensive drugs to elderly people in British Columbia. Lancet. 352: 943- 948.

MatoweL, RamsayCR, GrimshawJM, GilbertFI, MacleodMJ, NeedhamG. 2002 Effects of mailed dissemination of the Royal College of Radiologists' guidelines on general practitioner referrals for radiography: A time series analysis. Clin Radiol. 57: 575- 578.

MorrisonJC, SumrallDD, ChevalierSPet al. 1993 The effect of provider education on blood utilization practices. Am J Obstet Gynecol 169: 1240- 1245.

NattingerAB, HoffmannRG, Howell-PelzA, GoodwinJS. 1998 Effect of Nancy Reagan's mastectomy on choice of surgery for breast cancer by US women. JAMA. 279: 762- 766.

NovichM, GillisL, TauberAI. 1985 The laboratory test justified. An effective means to reduce routine laboratory testing. Am J Clin Pathol. 84: 756- 759.

PaunioM, VirtanenM, PeltolaHet al. 1991 Increase of vaccination coverage by mass media and individual approach: Intensified measles, mumps, and rubella prevention program in Finland. Am J Epidemiol. 133: 1152- 1160.

PehambergerH, BinderM, KnollmayerS, WolffK. 1993 Immediate effects of a public education campaign on prognostic features of melanoma. J Am Acad Dermatol. 29: 106- 109.

PomaPA. 1998 Effect of departmental policies on cesarean delivery rates: A community hospital experience. Obstet Gynecol 91: 1013- 1018.

1993 Royal College of Radiologists Working Party. Influence of Royal College of Radiologists' guidelines on referral from general practice. BMJ. 306: 110- 111.

SanterreRE. 1996 The effect of the ACOG guideline on vaginal births after cesarean. Med Care Res Rev. 53: 315- 329.

ShelleyJM, IrwigLM, SimpsonJM, MakaskillP. 1991 Evaluation of a mass-media-led campaign to increase Pap smear screening. Health Educ Res. 6: 267- 277.

ShermanCR, PotoskyAL, WeisKA, FergusonJH. 1992 The Consensus Development Program. Detecting changes in medical practice following a consensus conference on the treatment of prostate cancer. Int J Technol Assess Health Care. 8: 683- 693.

ShorrRI, FoughtRL, RayWA. 1994 Changes in antipsychotic drug use in nursing homes during implementation of the OBRA-87 regulations. JAMA. 271: 358- 362.

SoumeraiSB, Ross-DegnanD, KahnJS. 1992 Effects of professional and media warnings about the association between aspirin use in children and Reye's syndrome. Milbank Q. 70: 155- 182.

SoumeraiSB, AvornJ, GortmakerS, HawleyS. 1987 Effect of government and commercial warnings on reducing prescription misuse: The case of propoxyphene. Am J Public Health. 77: 1518- 1523.

StudnickiJ, RemmelR, CampbellR, WernerDC. 1997 The impact of legislatively imposed practice guidelines on cesarean section rates: The Florida experience. Am J Med Qual. 12: 62- 68.

ThamerM, RayNF, HendersonSCet al. 1998 Influence of the NIH Consensus Conference on Helicobacter pylori on physician prescribing among a Medicaid population. Med Care. 36: 646- 660.

TurnerGC, MuttonKJ. 1987 HIV testing: Changing trends. BMJ. 295: 502.

van WalravenC, GoelV, ChanB. 1998 Effect of population-based interventions on laboratory utilization: A time-series analysis. JAMA. 280: 2028- 2033.

WongET, McCarronMM. ShawSTJr. 1983 Ordering of laboratory tests in a teaching hospital. Can it be improved? JAMA. 249: 3076- 3080.

ZehrKJ, DawsonPB, YangSC, HeitmillerRF. 1998 Standardized clinical care pathways for major thoracic cases reduce hospital costs. Ann Thorac Surg. 66: 914- 919.

Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

International Journal of Technology Assessment in Health Care
  • ISSN: 0266-4623
  • EISSN: 1471-6348
  • URL: /core/journals/international-journal-of-technology-assessment-in-health-care
Please enter your name
Please enter a valid email address
Who would you like to send this to? *



Full text views

Total number of HTML views: 16
Total number of PDF views: 197 *
Loading metrics...

Abstract views

Total abstract views: 590 *
Loading metrics...

* Views captured on Cambridge Core between September 2016 - 23rd September 2017. This data will be updated every 24 hours.