It was an honor to have known and worked with Seymour Perry, MD, MACP, and to havewitnessed his many accomplishments and contributions in several fields—cancerresearch, government administration, the development and promotion of procedures forevaluating health technologies, particularly consensus development, and leadership infostering national and international collaboration in health technology assessment—representing a sustained effort in which both quantitative and qualitative benefits have been demonstrated.

The profusion of clinical practice guidelines has been remarked on by many (3;6;8) and ascribed to a need to address the wide geographic differences in practice documented in the last quartercentury (9;10), as well as to the increasing concern with the cost of healthcare. The idea is thatguidelines, by outlining efficient care strategies, will enhance the quality of care and reduceunnecessary or unproductive expenditures. Others hold that guidelines are simply a means oftransferring the results of research from the literature to clinicians (2;5). A darker view ofguidelines sees them as instruments of control of medical practice by uncaring administratorsconcerned solely with cost reduction (4;7).

Through the use of three scenarios, this paper presents thechallenges for clinical practice guidelines in the 21st century.Such challenges relate to technological developments to improve theefficiency and pace of the development process, to ensure thatclinical practice guidelines are kept up to date, and to facilitateimplementation of guidelines in the clinical setting. To improve andensure the validity of the content of clinical practice guidelines,we need to address the important problem of publication bias, forwhich researchers, granting agencies, industry, and journal editorsshare responsibility. This means insisting on registration of trialsat their inception, and incentives backed up by rules for funding andpeer review publication that would promote behaviors to avoidpublication bias. The more difficult challenges for clinicalpractice guidelines relate to what are referred to as attitudinalfactors. To achieve optimal efficiencies in development andmaintenance of clinical practice guidelines, we need to promotecooperation among various information resource providersinternationally and to stress partnership over leadership. Finally,there needs to be reconciliation of the different stakeholderperspectives of the value and purpose of clinical practice guidelinesso that they are used appropriately as aids to decision making andare not abused as tools for controlling clinical practice.

In an effort to control ambulatory care costs, regulatory practiceguidelines (références médicales opposables or RMOs) wereintroduced by law in France in 1993. RMOs are short sentences,negatively formulated (“it is inappropriate to \ldots”), covering medical and surgical topics, diagnosis, and treatment. Since theirintroduction, physicians who do not comply with RMOs can be fined.The fine is determined by a weighted combination of indices ofharm, cost, and the number of violations.

The impact of the RMO policy on physician practice has been questioned, but so far few evaluations had been performed. At the end of 1997, only 121 physicians had been fined (0.1% of French private physicians). The difficulty of controlling physicians, the large number of RMOs, and the lack of a relevant information system limit the credibility of this policy.

The simultaneous development of a clinical guideline program to improve the quality of careand of a program to control medical practice can lead to a misunderstanding among clinicians and health policy makers. Financial incentives or disincentives could be used to change physician behavior, in addition to other measures such as education and organizational changes, if they are simple, well explained, and do not raise any ethical conflict. But these measures are dependent on the structure and financing of the healthcare system and on the socioeconomic and cultural context. More research is needed to assess the impact of interventions using financial incentives and disincentives on physician behavior.

Economic guidelines recommend methods that should be employed in conducting economic evaluations of healthcare programs. The nature of theefficiency or equity goal underpinning economic guidelines isunclear. What is also unclear is how the methods recommended in theguidelines are linked to the underlying efficiency or equity goalbeing targeted. If it is unclear what efficiency/equity objectivesare being pursued, then it is unlikely that even full implementationof economic guidelines will improve resource allocation.

Clinical practice guidelines are expanding their scope of authority from clinical decision making to collective policy making, and promise to gain ground as resource allocation tools in coming years. A close examination of how guidelines approach patient selection criteria offers insight into their ethical implications when used as resource allocation or rationing instruments. The purposes of this paper are: a) to examine the structure of allocative reasoning found in clinical guidelines; b) to identify the ethical principles implied and compare how guidelines enact these principles with how explicit systems-level rationing exercises and health policy analyses have approached them; and c) to offer some preliminary suggestions for how these ethical issues might be addressed in the process of guideline development. The resulting framework can be used by guideline developers and users to understand and address some of the ethical issues raised by guidelines for the use of scarce technologies.

Practice guidelines are rapidly becoming preferred decision-makingresources in medicine, as advances in technology and pharmaceuticscontinue to expand. An evidence-based approach to the development ofpractice guidelines serves to anchor healthcare policy to scientificdocumentation, and in conjunction with practitioner opinion canprovide a powerful and practical clinical tool. Three sources ofinformation are essential to an evidence-based approach: a) anexhaustive literature synthesis; b) meta-analysis; and c) consensusopinion. The systematic merging of evidencefrom these sources offers healthcare providers a scientificallysupportable document that is flexible enough to deal with clinicallycomplex problems. Evidence-based practice guidelines, inconjunction with practice standards and practice advisories, areinvaluable resources for clinical decision making. The judicious useof these documents by practitioners will serve to improve theefficiency and safety of health care well.

Objectives: To identify and examine the methodologic issuesrelated to evaluating the effectiveness of treatment adherence toclinical guidelines. The example of antiretroviral therapy guidelinesfor human immunodeficiency virus (HIV) disease is used to illustratethe points.

Methods: Regression analysis was applied to observational HIV clinic datafor patients with CD4+ cell counts less than 500 per μL and greaterthan 50 per μL at baseline (n = 704),using Cox proportional hazardstime-varying covariates models controlling for baseline risk. Theresults are compared with simpler models (Cox model [withouttime-varying covariates] and logistic regression). In addition, theeffect of including a measure of exposure to antiretroviralguidelines in the model is explored.

Results: This study has three implications for modelingclinical guideline effectiveness. To capture events that are time-sensitive, a duration model should be used, and covariates that aretime-varying should be modeled as time-varying. Thirdly,incorporating a threshold measure of exposure to reflect the minimumperiod of time for guideline adherence required for a measurableeffect on patient outcome should be considered.

Conclusions: The methods proposed in this paper are importantto consider if guidelines are to evolve from being a tool forsummarizing and transferring the results of research from theliterature to clinicians into a practical tool that influencesclinical practice patterns. However, the methodology tested in thisstudy needs to be validated using additional data on similar patientsand using data on patients with other diseases.

Objective: To identify and compare clinical practiceguideline appraisal instruments.

Methods: Appraisal instruments,defined as instruments intended to be used for guideline evaluation,were identified by searching MEDLINE (1966–99) using the MedicalSubject Heading (MeSH) practice guidelines, reviewing bibliographiesof the retrieved articles, and contacting authors of guidelineappraisal instruments. Two reviewers independently examined thequestions/statements from all the instruments and thematicallygrouped them. The 44 groupings were collapsed into 10 guidelineattributes. Using the items, two reviewers independently undertook acontent analysis of the instruments.

Results: Fifteen instruments were identified, and two were excludedbecause they were not focused on evaluation. All instruments weredeveloped after 1992 and contained 8 to 142questions/statements. Of the 44 items used for the content analysis,the number of items covered by each instrument ranged from 6 to 34.Only the instrument by Cluzeau and colleaguesincluded at least one item for each of the 10 attributes, andit addressed 28 of the 44 items. This instrument and that of Shaneyfelt et al. are the only instruments that have so far been validated.

Conclusions: A comprehensive, concise, and valid instrument could helpusers systematically judge the quality and utility of clinicalpractice guidelines. The current instruments vary widely inlength and comprehensiveness. There is insufficient evidenceto support the exclusive use of any one instrument, although theCluzeau instrument has received the greatest evaluation. Moreresearch is required on the reliability and validity of existingguideline appraisal instruments before any one instrument can becomewidely adopted.

Objectives: To identify major differences and similarities inthe development of clinical guidelines in different Europeancountries.

Methods: A collaboration of researchers is funded by theEuropean Commission to compare the approaches to guidelinedevelopment in collaborators' countries. The program encompasses aseries of tasks, the first being to identify and document currentguideline procedures in the collaborating countries. A surveygathered information on guideline production, dissemination, andimplementation in the 10 European countries involved in the projectconsortium: Denmark, England and Wales, Finland, France, Germany,Italy, the Netherlands, Scotland, Spain (both the Basque Country andCatalonia), and Switzerland.

Results: Seven countries have a national policy on guidelineproduction, dissemination, and implementation, and three countriesare discussing their policies. A majority of guidelines arecurrently produced at the national level in six of the countries andat the regional or local level in the other four. Central ornational funding supports guideline production in six countries.Additional sources of funding include medical societies,pharmaceutical companies, and health insurance companies. Several ofthe countries have published or are preparing evaluations of theirdissemination strategies.

Conclusions: The survey highlighted wide variation in themethods and policies of guideline development in Europe. TheAppraisal of Guidelines, Research, and Evaluation in Europe (AGREE)Collaboration research program will identify the characteristics ofthe “better” guideline programs and will provide the basis for moreresearch-generated policy initiatives in the future, helping toensure that guidelines play a major role in improving patient care inthe millennium.

Clinicians make judgments under conditions of uncertainty. Decisionresearch has shown thatin uncertain situations individuals do not always act rationally, coherently, or to maximize theirexpected utility. Advocates of clinical guidelines believe that these guidelineswill eliminate some of thecognitive biases that the practitioner may introduce into the medical decision-making process inan attempt to reduce its uncertainty. Other physicians have grave doubts about guidelines' application in practice. Guideline implementation lags well behind their development. Studiesof practicing physicians and a survey of clinicians in one specialty and setting indicate thatexperienced clinicians may be implementing guidelines selectively. Many clinicians areconcerned that guidelines are based on randomized trials and do not reflect the complexity of thereal world, in which a decision's context and framework are important. Their reluctance alsomay be due to the difficulty of applying general guidelines to specific clinical situations. Theproblem will only increase in the future. The patients of the 21st century will be older and havemore complex disease states. Physicians will have more patient-specific therapies and need toexercise more sophisticated clinical judgment. They may be more willing to use guidelines inmaking those judgments if research can demonstrate guidelines' effectiveness in improvingdecision making for individual patients.

Clinical practice guidelines have been available to clinicians foralmost two decades, but the consistency of their implementation inpractice remains highly variable. This paper describes the variousprocesses and mechanisms used by one managed care organization todevelop clinical guidelines and promote their adoption. Some of thesemechanisms include provision of individual physician report cards,financial incentives, and various documentation tools that serve asreminders of guideline recommendations and provide an easy format todocument recommended services.

There have been measurement challenges in evaluating theeffectiveness of selected interventions designed to enhance guidelinecompliance. Most of these challenges relate to reliability andvalidity concerns regarding the three primary data sources used inthe evaluation process: medical records, administrative claims data,and member survey data. Some of the interventions the health planhas implemented to address these measurement challenges include usinghybrid methods of data collection and developing collaborativepartnerships with outside organizations to enhance the accuracy andcompleteness of the available data. Outcomes of these efforts aredescribed, as are physician response and recommendations for futureenhancement of practice guidelines.

Healthcare policy, medical practice, and cost of care are no longer considered distinctentities. Each is an integral factor in determining not only what, but how, patient carewill be delivered. Clinical practice guidelines are the lynchpin that connectsthem. This paper addresses the various components of the clinical practice guideline—cost alliance.

Objective: To examine the bidirectional influence of choice of care on costs and of cost of care on decision making.

Methods: The literature was used to identify cost-relatedfactors that influence development of guidelines and change in physician practicebehavior. In a MEDLINE search with modifiers to the keywords “clinical practiceguidelines,” particular attention was paid to identifying surveys of practitioners. Ananalysis, prompted by a recently published guideline, of treating penetrating intraperitoneal coloninjuries by different surgical approaches (primary repair versus diverting colostomy) exemplified how implementation of a guideline can affect the cost of care. Inpatientcost estimates, adjusted for medical inflation and cost-to-charge ratios and reported in1999 U.S. dollars, were developed using data from 1996 and 1997 discharge databases fromCalifornia and Massachusetts.

Results: The results showed that a substantial savings in hospital costs wasachieved when a primary repair surgical technique, as advocated by the guideline, was used. The effect of cost influences on the development of clinical practice guidelines wasestablished by demonstrating the cyclical effect between usual and customarypractices, guideline implementation, changing practice patterns, and the economicconsiderations influencing the process.

Conclusions: A growing, albeit uneasy, alliance between costs and clinical practice guidelines is evident.

Objective: We reviewed and appraised the methodsby which the issue of the learning curve has been addressedduring health technology assessment in the past.

Method: We performed a systematic review of papers inclinical databases (BIOSIS, CINAHL, Cochrane Library, EMBASE,HealthSTAR, MEDLINE, Science Citation Index, and Social ScienceCitation Index) using the search term “learning curve.”

Results: The clinical search retrieved 4,571 abstracts forassessment, of which 559 (12%) published articles were eligiblefor review. Of these, 272 were judged to have formally assesseda learning curve. The procedures assessed were minimal access (51%), other surgical (41%), and diagnostic (8%). Themajority of the studies were case series (95%). Some 47% ofstudies addressed only individual operator performance and 52% addressed institutional performance. The data were collected prospectively in 40%, retrospectively in 26%, and the methodwas unclear for 31%. The statistical methods used were simplegraphs (44%), splitting the data chronologically and performinga t test or chi-squared test (60%), curve fitting (12%), and other model fitting (5%).

Conclusions: Learning curves are rarely considered formallyin health technology assessment. Where they are, the reportingof the studies and the statistical methods used are weak. As aminimum, reporting of learning should include the number and experience of the operators and a detailed description of datacollection. Improved statistical methods would enhance theassessment of health technologies that require learning.

Objective: To explore the impact of methodologic issues onthe results of meta-analyses. The following issues were examined:the type of literature search strategy used; inclusion or exclusionof non–peer-reviewed studies; the inclusion or exclusion of non-English language publications; the effect of trial quality; and theinclusion or exclusion of non–placebo-controlled studies.

Methods: The International Study of Perioperative Transfusion (ISPOT) meta-analyses were used to evaluate each of the methodologic issues. The 10 meta-analyses consisted of technologies to reduce the need for perioperative red blood cell transfusion. The number of trials for each of the meta-analyses varied from 2 to 45. Both EMBASE and MEDLINE searches were conducted, including the use of systematic search strategies.

Results: MEDLINE identified the vast majority of trials. Alone,MEDLINE would have missed 8 studies compared to 10 for EMBASE. Use ofthe systematic search strategies greatly reduced the number ofarticles to be reviewed compared to open searches. Type ofpublication, country of study origin, inclusion of non-Englishpublications, and trial quality had very little impact on theestimates of effect. The use of placebo versus open-label controlaffected the magnitude of the odds ratio for two of the meta-analyses. The results of the two meta-analyses were not statisticallysignificant if only placebo-controlled trials were included.

Conclusions: While methodologic issues had very little impacton the ISPOT meta-analyses, further studies are needed in a varietyof other clinical settings. Because MEDLINE, coupled with a reviewof the references in the identified trials, identified the vastmajority of trials, one needs to consider the costs and benefits ofsearching EMBASE and the pursuance of unpublished and unindexed trials.

Objective: Severe to profound hearing impairment affects one-halfto three-quarters of a million Americans. To function in a hearingsociety, hearing-impaired persons require specialized educational,social services, and other resources. The primary purpose of thisstudy is to provide a comprehensive, national, and recent estimate ofthe economic burden of hearing impairment.

Methods: We constructed a cohort-survival model to estimate thelifetime costs of hearing impairment. Data for the model werederived principally from the analyses of secondary data sources,including the National Health Interview Survey Hearing Loss andDisability Supplements (1990–91 and 1994–95), the Department ofEducation's National Longitudinal Transition Study (1987), andGallaudet University's Annual Survey of Deaf and Hard of HearingYouth (1997–98). These analyses were supplemented by a review of theliterature and consultation with a four-member expert panel. MonteCarlo analysis was used for sensitivity testing.

Results: Severe to profound hearing loss is expected to cost society $297,000 over the lifetime of an individual. Most of these losses(67%) are due to reduced work productivity, although the use of specialeducation resources among children contributes an additional 21%.Life time costs for those with prelingual onset exceed $1 million.

Conclusions: Results indicate that an additional $4.6 billion willbe spent over the lifetime of persons who acquired their impairment in1998. The particularly high costs associated with prelingual onsetof severe to profound hearing impairment suggest interventions aimedat children, such as early identification and/or aggressive medicalintervention, may have a substantial payback.

Objective: This study investigates the influence of different aspects ofmethodologic quality on the conclusions of a systematic reviewconcerning treatments of acute lateral ankle sprain.

Method: A data set of a systematic review of 44 trials was used, of which 22 trialscould be included in this study. Quality assessment of the individualstudies was performed using the Delphi list. We calculated effectsizes of the main outcome measure in each study in order to evaluatethe relationship between overall quality scores and outcome. Next, weinvestigated the impact of design attributes on pooled effectsizes by subgroup analysis.

Results: The quality of most studies (82%) waslow; only 4 of 22 trials were of high quality. Studies with proper randomization and blinding procedure produce a slightly higher (not statistically significant) effectestimate compared to the other studies.

Conclusion: Previous research has suggested that methodologicallypoorly designed studies tend to over-estimate the effectestimate. Our study does not confirm these conclusions.

Objectives: Inappropriate hospital admissions are commonly believed torepresent a potential for significant cost reductions. However, thispresumes that these patients can be identified before the hospitalstay. The present study aimed to investigate to what extent this ispossible.

Methods: Consecutive admissions to a department of internalmedicine were assessed by two expert panels. One panel predicted theappropriateness of the stays from the information available atadmission, while final judgments of appropriateness were made afterdischarge by the other.

Results: The panels correctly classified 88% of the appropriate and27% of the inappropriate admissions. If the elective admissionspredicted to be inappropriate had been excluded, 9% of the costswould have been saved, and 5% of the gain in quality-adjusted life-years lost. The corresponding results for emergency admissions were14% and 18%.

Conclusions: The savings obtained by excluding admissions predicted tobe inappropriate were small relative to the health losses. Programsfor reducing inappropriate health care should not be implementedwithout investigating their effects on both health outcomes and costs.

Objectives: Cost-effectiveness analystsoften present cost-effectiveness results by age to help informdecisions about the use of an intervention. Yet it is not known howwell studies model the risks and costs associated with age. Wereviewed published studies to examine their modeling of agedifferences.

Methods: MEDLINE searches identified all cost-effectiveness analysespublished between 1985 and 1997 that included adults 50 years of age and older,were based on data for developed countries, and comparedcost-effectiveness ratios for adults of different ages or forinitiation of an intervention at different ages; 36 articles metthese criteria. They were reviewed to determine the extent to whichthey incorporated age-specific data. Studies that justified usingthe same data for all ages were counted as having varied the dataelement by age.

Results: All studies varied life expectancy by age. Most also variedthe incidence/prevalence of the target condition and the casefatality rate. Only 36% varied the effectiveness rate of theintervention by age. Costs were usually assumed constant: 42% ofstudies varied the cost of treating adverse effects and 17% variedthe cost of treating the target condition. Whether a data elementwas varied did not appear to be related to the pattern ofcost-effectiveness ratios by age.

Conclusions: Many studies have not modeled age differences insufficient detail to ensure that differences in cost-effectivenessratios by age are accurate and a sound basis for decisions. Ascost-effectiveness analysis becomes more widespread, analysts shouldstrive to incorporate more complete age-specific data.

Objectives: To estimate the potential for cost reduction in the acute care setting and the required investment in the home care setting of implementing an outpatient/early discharge strategy for operable (stages I and II) breast cancer in Canada.

Methods: Data from a community hospital were augmented by expert knowledge and incorporated into the breast cancer submodel of Statistics Canada's Population Health Model. For the estimated 90% of patients for whom this approach was assumed to be appropriate, the resource utilization for outpatient breast-conserving surgery and 2 days of hospitalization for those women undergoing mastectomy was quantified and costed, as were the appropriate home care services. A 5% readmission rate for complications was assumed. Cost per case, total cost burden, investment in home care, savings in acute care, and net savings were calculated. Sensitivity analyses were performed around readmission rates and home care/surgical follow-up costs. All costs were determined in 1995 Canadian dollars.

Results: The cost of initial treatment for the 15,399 women diagnosed with stages I and II breast cancer in 1995 in Canada was estimated to be $127.6 million. Hospitalization made up 53% of these costs. Under the outpatient/early discharge strategy, the acute care cost of initial breast cancer management could be reduced by $47.2 million, with an investment in home care of $14.5 million ($453 per patient), resulting in an overall net saving of $33 million. Under this strategy, hospitalization would contribute only 21% to the total care cost.

Conclusions: If Canadian surgeons and healthcare administrators were to worktogether to put in place processes to support ambulatory breastcancer surgery and if resources were redirected to the provision ofhome-based post-operative care, there would be potential for a largenet healthcare saving and preservation of high-quality patient care.