Skip to main content Accessibility help
×
Home

Data Quality in an Information-Rich Environment: Canada as an Example

  • Leslie L. Roos (a1), Sumit Gupta (a2), Ruth-Ann Soodeen (a1) and Laurel Jebamani (a1)

Abstract

This review evaluates the quality of available administrative data in the Canadian provinces, emphasizing the information needed to create integrated systems. We explicitly compare approaches to quality measurement, indicating where record linkage can and cannot substitute for more expensive record re-abstraction. Forty-nine original studies evaluating Canadian administrative data (registries, hospital abstracts, physician claims, and prescription drugs) are summarized in a structured manner. Registries, hospital abstracts, and physician files appear to be generally of satisfactory quality, though much work remains to be done. Data quality did not vary systematically among provinces. Primary data collection to check place of residence and longitudinal follow-up in provincial registries is needed. Promising initial checks of pharmaceutical data should be expanded. Because record linkage studies were “conservative” in reporting reliability, the reduction of time-consuming record re-abstraction appears feasible in many cases. Finally, expanding the scope of administrative data to study health, as well as health care, seems possible for some chronic conditions. The research potential of the information-rich environments being created highlights the importance of data quality.

Cette étude vise à évaluer la qualité des données administratives disponibles dans les provinces canadiennes, tout en mettant l'accent sur les renseignements nécessaires pour créer des systèmes intégrés. Nous comparons explicitement diverses approches en matière de mesure de la qualité, en indiquant dans quel cas le couplage des dossiers peut ou non se substituer à la méthode plus onéreuse de la seconde saisie des dossiers. Quarante-neuf études originales visant à évaluer les données administratives canadiennes (registres, résumés d'hospitalisation, demandes des médecins et médicaments sur ordonnance) sont résumées de manière structurée. Les registres, les résumés d'hospitalisation et les dossiers des médecins semblent généralement de qualité satisfaisante, bien qu'il reste beaucoup de travail à accomplir. La qualité des données n'a pas fait l'objet de variations systématiques entre les provinces. Des données primaires doivent être recueillies afin de vérifier les lieux de résidence et effectuer un suivi longitudinal dans les registres provinciaux. Les vérifications initiales des données pharmaceutiques se sont révélées prometteuses et doivent être poursuivies. É tant donné que les études fondées sur le couplage des dossiers étaient «prudentes» dans leurs conclusions en matière de fiabilité, la réduction du nombre de secondes saisies qui prennent beaucoup de temps semblerait faisable dans bien des cas. Enfin, il pourrait être possible d'étendre la portée des données administratives de manière à étudier l'état de santé, ainsi que les soins de santé pour certaines conditions chroniques. Le potentiel de recherche des milieux riches en informations qui sont en train d'être créés permet de souligner l'importance de la qualité des données.

Copyright

Corresponding author

Requests for offprints should be sent to: / Les demandes de tirés-à-part doivent être addressées à : Leslie L. Roos, Ph.D., Manitoba Centre for Health Policy, Department of Community Health Sciences, Faculty of Medicine, University of Manitoba, 4th Floor Brodie Centre, Room 408, 727 McDermot Avenue, Winnipeg, MB R3E 3P5. (Leslie_Roos@cpe.umanitoba.ca)

References

Hide All
Acheson, E.D. (1967). Medical record linkage. London: Oxford University Press.
Anderson, G., & Kerluke, K. (1996). Distribution of prescription drug exposures in the elderly: Description and implications. Journal of Clinical Epidemiology, 49, 929935.
Anderson, G.M., Kerluke, K.J., Pulcins, I.R., Hertzman, C., & Barer, M.L. (1993). Trends and determinants of prescription drug expenditures in the elderly: Data from the British Columbia Pharmacare Program. Inquiry, 30, 199207.
Armstrong, B.K., & Kricker, A. (1999). Record linkage: A vision renewed. Australian New Zealand Journal of Public Health, 23, 451452.
Austin, P.C., Daly, P.A., & Tu, J.V. (2002). A multicenter study of the coding accuracy of hospital discharge administrative data for patients admitted to cardiac care units in Ontario. American Heart Journal, 144, 290296.
Berkman, L.F., & Kawachi, I. (Eds.). (2000). Social epidemiology. New York: Oxford University Press.
Bernstein, C.N., Blanchard, J.F., Rawsthorne, P., & Wajda, A. (1999). Epidemiology of Crohn's disease and ulcerative colitis in a central Canadian province: A population-based study. American Journal of Epidemiology, 149, 916924.
Blanchard, J.F., Ludwig, S., Wajda, A., Dean, H., Anderson, K., Kendall, O., & Depew, N. (1996). Incidence and prevalence of diabetes in Manitoba, 1986–1991. Diabetes Care, 19, 807811.
Blanchard, J.F., Dean, H., Anderson, K., Wajda, A., Ludwig, S., & Depew, N. (1997). Incidence and prevalence of diabetes mellitus in children 0–14 years in Manitoba, Canada 1985–1993. Diabetes Care, 20, 512515.
Bryan, H., & Brasher, P. (1995). Breast implants and breast cancer: Re-analysis of a linkage study. New England Journal of Medicine, 332, 15351539.
Brown, K. (2000). The human genome business today. Scientific American, 283, 5055.
Brownell, M., & Yogendran, M. (2001). Attention-deficit hyperactivity disorder in Manitoba children: Medical diagnosis and psychostimulant treatment rates. Canadian Journal of Psychiatry, 46, 264272.
Canadian Health Services Research Foundation. (2003). User fees would stop waste and ensure better use of the health care system. Journal of Health Services Research Policy, 8, 105107.
Canadian Institute for Health Information. (2000). Health care in Canada 2000: A first annual report. Ottawa: Author.
Chamberlayne, R., Green, B., Barer, M.L., Hertzman, C., Lawrence, W.J., & Sheps, S.B. (1998). Creating a population-based linked health database: A new resource for health services research. Canadian Journal of Public Health, 89, 270273.
Cohen, M.M. (1993). Using administrative data for case-control studies: The case of the Papanicolaou smear. Annals of Epidemiology, 3, 9398.
Cohen, M.M., Kaufert, P.A., MacWilliam, L., & Tate, R.B. (1996). Using an alternative data source to examine randomization in the Canadian National Breast Screening Study. Journal of Clinical Epidemiology, 49, 10391044.
Cox, J.L., Melady, M.P., Chen, E., & Naylor, C.D. (1997). Towards improved coding of acute myocardial infarction in hospital discharge abstracts: A pilot project. Canadian Journal of Cardiology, 13, 351358.
Davidoff, F. (2000). Suppose there were no printers. Annals of Internal Medicine, 133, 5758.
Davidson, W., Molloy, W., Somers, G., & Bedard, M. (1994). Relation between physician characteristics and prescribing for elderly people in New Brunswick. Canadian Medical Association Journal, 150, 917921.
Delfino, R J., Backlake, M.R., & Hanley, J.A. (1993). Reliability of hospital data for population-based studies of air pollution. Archives of Environmental Health, 48, 140146.
Dimich-Ward, H., Hertzman, C., Teschke, K., Hershler, R., Marion, S.A., Ostry, A., & Kelly, S. (1996). Reproductive effects of paternal exposure to chlorophenate wood preservatives in the sawmill industry. Scandinavian Journal of Work Environment and Health, 22, 267273.
Edouard, L., & Rawson, N.S.B. (1996). Reliability of the recording of hysterectomy in the Saskatchewan health care system. British Journal of Obstetrics and Gynaecology, 103, 891897.
Fellegi, I.P., & Sunter, A.B. (1969). A theory for record linkage. Journal of the American Statistical Association, (328), 11831210.
Ghali, W.A., Rothwell, D.M., Quan, H., Brant, R., Tu, J.V., & Steering Committee of the Cardiac Care Network of Ontario. (2000). A Canadian comparison of data sources for coronary artery bypass surgery outcome “report cards.” American Heart Journal, 140, 402408.
Ghali, W.A., Quan, H., & Brant, R. (2002). Risk adjustment using administrative data: Impact of a diagnosis-type indicator. Journal of General Internal Medicine, 16, 519524.
Hatcher, J., & Hervas, M. (2001). Emigration patterns of cancer cases in Alberta, Canada. Chronic Diseases in Canada, 22, 1217.
Hawaleshka, D. (2002). The Maclean's Health Report: Measuring health care: North and West Vancouver, Edmonton, Victoria, Kelowna, B.C.—Western centres top the list, Maclean's, June 17, 23–31.
Hawker, G.A., Coyte, P.C., Wright, J.G., Paul, J.E., & Bombardier, C. (1997). Accuracy of administrative data for assessing outcomes after knee replacement surgery. Journal of Clinical Epidemiology, 50, 265273.
Hertzman, C., Teschke, K., Ostry, A., Hershler, R., Dimich-Ward, H., Kelly, S., Spinelli, J.J., Gallagher, R.P., McBride, M., & Marian, S.A. (1997). Mortality and cancer incidence among sawmill workers exposed to chlorophenate wood preservatives. American Journal of Public Health, 87, 7179.
Holman, C.D.J., Bass, A.J., Rouse, I.L., & Hobbs, M.S.T. (1999). Population-based linkage of health records in Western Australia: Development of the health services research linked database. Australian New Zealand Journal of Public Health, 23, 453459.
Houle, C., Berthelot, J.-M., David, P., Wolfson, M.C., Mustard, C.A., & Roos, L.L. (1999). Matching census database and Manitoba health care files. In Record linkage techniques – 1997: Proceedings of an international workshop and exposition. Washington, D.C.: National Academy Press, 305318.
Howe, G.R. (1998). Use of computerized record linkage in cohort studies. Epidemiologic Reviews, 20, 112121.
Humphries, K.H., Rankin, J.M., Carere, R.G., Buller, C.E., Kiely, F.M., & Spinelli, J.J. (2000). Co-morbidity data in outcomes research: Are clinical data derived from administrative databases a reliable alternative to chart review? Journal of Clinical Epidemiology, 53, 343349.
Huzel, L., Roos, L.L., Anthonisen, N.R., & Manfreda, J. (2003). Diagnosing asthma: The fit between survey and administrative database. Canadian Respiratory Journal, 9, 407412.
Iron, K., Goel, V., & Williams, J.I. (1995). Concordance of hospital discharge abstracts and physician claims for surgical procedures in Ontario. (ICES working paper series #42.) North York: Institute for Clinical Evaluative Sciences in Ontario.
Jacobs, P., Blanchard, J.F., James, R., & Depew, N. (2001). Excess costs of diabetes in the aboriginal population of Manitoba, Canada. Canadian Journal of Public Health, 91, 298301.
Jacobs, P., & Roos, N.P. (1999). Standard cost lists for health care in Canada: Issues in validity and interprovincial consolidation. Pharmacoeconomics, 15, 551560.
Kohler, R.E. (1994). Lords of the fly. Chicago: University of Chicago Press.
Koran, L.M. (1975). The reliability of clinical methods, data and judgements (Part 1). New England Journal of Medicine, 293, 642646.
Kozyrskyj, A., & Hildes-Ripstein, G.E. (2002). Assessing health status in Manitoba children: Acute and chronic conditions. Canadian Journal of Public Health, 93, S63S69.
Kozyrskyj, A., & Mustard, C.A. (1998). Validation of an electronic, population-based prescription database. Annals of Pharmacotherapy, 32, 11521157.
Kozyrskyj, A., Mustard, C.A., Cheang, M.S., & Simons, F.E.R. (2001). Income-based drug benefit policy: Impact on receipt of inhaled corticosteroid prescriptions by Manitoba children with asthma. Canadian Medical Association Journal, 165, 897902.
Levy, A.R., Tamblyn, R., Fitchett, D., McLeod, P.J., & Hanley, J.A. (1999). Coding accuracy of hospital discharge data for elderly survivors of myocardial infarction. Canadian Journal of Cardiology, 15, 12771282.
Malenka, D.J., McLerran, D.F., Roos, N.P., Fisher, E.S., & Wennberg, J.E. (1994). Using administrative data to describe casemix: A comparison with the medical record. Journal of Clinical Epidemiology, 47, 10271032.
Manitoba Health. (2002). Manitoba's health indicators report. Winnipeg: Author.
Miller, E., Blatman, B., & Einarson, T.R. (1996). A survey of population-based drug databases in Canada. Canadian Medical Association Journal, 154, 18551864.
Muhajarine, N., Mustard, C.A., Roos, L.L., Young, T.K., & Gelskey, D.E. (1997). Comparison of survey data and physician claims data for detecting hypertension. Journal of Clinical Epidemiology, 50, 711718.
Mustard, C.A., Harman, C.R., Hall, P.F., & Derksen, S. (1995). Impact of a nurses' strike on the cesarean birth rate. American Journal of Obstetrics and Gynecology, 172, 631637.
Naylor, C.D. (1999). Health care in Canada: Incrementalism under fiscal duress. Health Affairs (Millwood), 18, 926.
Naylor, C.D., & Slaughter, P.M. (Eds.). (1999). Cardiovascular health & services in Ontario: An ICES atlas. Toronto: Institute for Clinical Evaluative Sciences.
Newcombe, H.B. (1988). Handbook of record linkage. New York: Oxford University Press.
Pinfold, S.P., Goel, V., & Sawka, C. (2000). Quality of hospital discharge and physician data for types of breast cancer surgery. Medical Care, 38, 99107.
Potvin, L., & Champagne, F. (1986). Utilization of administrative data files in health research. Social Indicators Research, 18, 409423.
Quan, H., Parsons, G.A., & Ghali, W.A. (2002). Validity of information on comorbidity derived from ICD-9-CCM administrative data. Medical Care, 40, 675685.
Rankin, J.M., Spinelli, J.J., Carere, R.G., Ricci, D.R., Penn, I.M., Hilton, J.D., Henderson, M.A., Hayden, R.I., & Buller, C.E. (1999). Improved clinical outcome after widespread use of coronary-artery stenting in Canada. New England Journal of Medicine, 341, 19571965.
Rawson, N.S.B., & D'Arcy, C. (1998). Assessing the validity of diagnostic information in administrative health care utilization data: Experience in Saskatchewan. Pharmacoepidemiology & Drug Safety, 7, 389398.
Rawson, N.S.B., & Malcolm, E. (1995a). Validity of the recording of cholecystectomy and hysterectomy in the Saskatchewan health care datafiles. (Technical report series report #3). Saskatoon: Pharmacoepidemiology Research Unit.
Rawson, N.S.B., & Malcolm, E. (1995b). Validity of the recording of ischaemic heart disease and chronic obstructive pulmonary disease in the Saskatchewan health care datafiles. Statistics in Medicine, 14, 26272643.
Rawson, N.S.B., Malcolm, E., & D'Arcy, C. (1997). Reliability of the recording of schizophrenia and depressive disorder in the Saskatchewan health care datafiles. Social Psychiatry and Psychiatric Epidemiology, 32, 191199.
Reid, R.J., MacWilliam, L., Verhulst, L., Roos, N.P., & Atkinson, M. (2001). Performance of the ACG Case-Mix System in two Canadian provinces. Medical Care, 39, 8699.
Reid, R.J., Roos, N.P., MacWilliam, L., Frohlich, N., & Black, C. (2002). Assessing population health care need using a claims-based ACG morbidity measure: A validation analysis in the province of Manitoba. Health Services Research, 37, 13451364.
Richards, J., Brown, A., & Homan, C. (2001). The data quality study of the Canadian discharge abstract database. Proceedings of the Statistics Canada's 2001 Symposium October 16–19, “Achieving Data Quality in a Statistical Agency: A Methodological Perspective.” Ottawa: Statistics Canada.
Risch, H.A., & Howe, G.R. (1994). Menopausal hormone usage and breast cancer in Saskatchewan: A record-linkage cohort study. American Journal of Epidemiology, 139, 670683.
Roberts, J.D., Poffenroth, L.A., Roos, L.L., Bebchuk, J.D., & Carter, A.O. (1994). Monitoring childhood immunizations: A Canadian approach. American Journal of Public Health, 84, 16661668.
Roberts, J.D., Fransoo, R., Black, C., Roos, L.L., & Martens, P. (2002). Research meets reality: Administrative data to guide planning for Canadian Regional Health Authorities. Healthcare Management Forum, 15, 1321.
Robinson, J.R., & Tataryn, D.J. (1997). Reliability of the Manitoba Mental Health Management Information System for Research. Canadian Journal of Psychiatry, 42, 744749.
Robinson, J.R., Young, T.K., Roos, L.L., & Gelskey, D.E. (1997). Estimating the burden of disease: Comparing administrative data and self-reports. Medical Care, 35, 932947.
Roos, L.L., Magoon, J., Gupta, S., Chateau, D., & Veugelers, P.J. (2004). Socioeconomic determinants of mortality in two Canadian provinces: Multilevel modelling and neighborhood context. Social Science and Medicine, (7), 14351447.
Roos, L.L., Menec, V., & Currie, R.J. (2004). Policy analysis in an information-rich environment. Social Science and Medicine, (11), 22312241.
Roos, L.L., Nicol, J.P. & Wajda, A. (1985). Improving the quality of data banks through linkage. Chronic Diseases in Canada, 5, 8182.
Roos, L.L., & Nicol, J.P. (1999). A research registry: Uses, development, and accuracy. Journal of Clinical Epidemiology, 52, 3947.
Roos, L.L., Nicol, J.P., Johnson, C., & Roos, N.P. (1979). Using administrative data banks for research and evaluation: A case study. Evaluation Quarterly, 3, 236255.
Roos, L.L., Roos, N.P., Cageorge, S.M., & Nicol, J.P. (1982). How good are the data? Reliability of one health care data bank. Medical Care, 20, 266276.
Roos, L.L., Sharp, S.M., & Cohen, M.M. (1991). Comparing clinical information with claims data: Some similarities and differences. Journal of Clinical Epidemiology, (9), 881888.
Roos, L.L., Sharp, S.M., & Wajda, A. (1989). Assessing data quality: A computerized approach. Social Science and Medicine, 28, 175182.
Roos, L.L., & Wajda, A. (1991). Record linkage strategies: Part I. Estimating information and evaluating approaches. Methods of Information in Medicine, 30, 117123.
Roos, L.L., Walld, R., Wajda, A., Bond, R., & Hartford, K. (1996). Record linkage strategies, outpatient procedures, and administrative data. Medical Care, 34, 570582.
Roos, N.P., Carrière, K.C., & Friesen, D. (1998). Factors influencing the frequency of visits by hypertensive patients to primary care physicians in Winnipeg. Canadian Medical Association Journal, 159, 777783.
Roos, N.P., & Shapiro, E. (Eds). (1999). Academics at the policy interface: Revisiting the Manitoba Centre for Health Policy and Evaluation and its population-based health information system. Medical Care, 37(Suppl. 6), JS1JS308.
Sackett, D.L., Haynes, R.B., Guyatt, G.H., & Tugwell, P. (Eds.). (1991). Clinical epidemiology: A basic science for clinical medicine (2nd ed.). Boston: Little, Brown and Company.
Sheps, S.B., Reid, R.J., Barer, M.L., Krueger, H., McGrail, K.M., Green, B., Evans, R.G., & Hertzman, C. (2000). Hospital downsizing and trends in health care use among elderly people in British Columbia. Canadian Medical Association Journal, 163, 411412.
Starr, P. (1997). Smart technology, stunted policy: Developing health information networks. Health Affairs (Millwood), 16, 91105.
Statistics Canada. (2001). Population by aboriginal group, 1996 Census. Retrieved December 14, 2004, from http://www.statcan.ca/english/Pgdb/People/Population/dem039a.htm
Stukenborg, G.J., Wagner, D.P., & Connors, A.F. Jr. (2001). Comparison of the performance of two comorbidity measures, with and without information from prior hospitalizations. Medical Care, 39, 727739.
Tamblyn, R. (2002). The new millennium model for health care and research. In Downey, J. & Claxton, L. (Eds), Innovation: Essays by leading Canadian researchers. 185–193. Toronto: Key Porter Books Ltd.
Tamblyn, R., Lavoie, G., Petrella, L., & Monette, J. (1995). The use of prescription claims databases in pharmacoepidemiological research: The accuracy and comprehensiveness of the prescription claims database in Quebec. Journal of Clinical Epidemiology, 48, 9991009.
Tamblyn, R., Reid, T., Mayo, N.E., McLeod, P.J., & Churchill-Smith, M. (2000). Using medical services claims to assess injuries in the elderly: Sensitivity of diagnostic and procedure codes for injury ascertainment. Journal of Clinical Epidemiology, 53, 183194.
Thiessen, B.Q., Wallace, S.M., Blackburn, J., Wilson, T.W., & Bergman, U. (1990). Increased prescribing of anti-depressants subsequent to B-blocker therapy. Archives of Internal Medicine, 150, 22862290.
Tu, J.V., Austin, P.C., Walld, R., Roos, L.L., Agras, J., & McDonald, K.M. (2001). Development and validation of the Ontario acute myocardial infarction mortality prediction rules. Journal of the American College of Cardiology, 37, 992997.
Veugelers, P.J., Yip, A.M., & Kephart, G. (2001). Proximate and contextual socioeconomic determinants of mortality: Multilevel approaches in a setting with universal health care coverage. American Journal of Epidemiology, 154, 725732.
Virnig, B.A., & McBean, M. (2001). Administrative data for public health surveillance and planning. Annual Review of Public Health, 22, 213230.
Wajda, A., & Roos, L.L. (1987). Simplifying record linkage: Software and strategy. Computers in Biology and Medicine, 17, 239248.
Wajda, A., Roos, L.L., Layefsky, M., & Singleton, J.A. (1991). Record linkage strategies, part 2: Portable softwarre and deterministic matching. Methods of Information in Medicine, 30, 210214.
Weiner, J. (1999). Time, love, memory: A great biologist and his quest for the origins of behavior. New York: Alfred A. Knopf.
West, S.L., Richter, A., Melfi, C.A., McNutt, M., Nennstiel, M.E., & Mauskopf, J.A. (2000). Assessing the Saskatchewan database for outcomes research studies of depression and its treatment. Journal of Clinical Epidemiology, 53, 823831.
Young, T.K., Roos, N.P., & Hammarstrand, K.M. (1991). Estimated burden of diabetes mellitus in Manitoba according to health insurance claims: A pilot study. Canadian Medical Association Journal, 144, 318324.

Keywords

Metrics

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed