Skip to main content Accessibility help

Can electronic routine data act as a surrogate for patient-assessed outcome measures?

  • Hayley A. Hutchings (a1), Wai-Yee Cheung (a1), John G. Williams (a1), David Cohen (a2), Mirella F. Longo (a2) and Ian Russell (a3)...


Objectives: There has been a rapid growth in the use of patient-assessed outcomes (PAOs) that are measured in the assessment of health technologies. The process of collection of such measures can be costly, and there may be problems associated with the ability of the patient to complete them. The use of electronically stored routine data may reduce costs and overcome the problems associated with patient completion. The feasibility of using routine data surrogates for the UK Inflammatory Bowel Disease Questionnaire (UKIBDQ) and the Short Form 36 (SF-36) was examined.

Methods: Clinical terms and codes for the UKIBDQ and SF-36 questions were identified, and data from electronic routine sources were sought on patients participating in a randomized controlled trial. The presence or absence of relevant symptoms was used to generate surrogate scores, which were compared with the original scores.

Results: Most questions in the UKIBDQ and SF-36 were codable but only one third of the terms were recorded routinely in electronic form. The surrogate total IBDQ score had reasonable reliability (Kuder–Richardson coefficient = 0.51), but this reliability could not be determined for the SF-36. Intraclass correlations between routine and designed data were poor to weak.

Conclusions: Although electronic routine data sources had the capacity to develop surrogate measures for patient assessed outcomes, there was evidence of wide underutilization of coding systems leading to an underreporting of symptoms. This finding is consistent with previous literature where only poor correlations were illustrated between patient assessed outcomes and surrogate scoring of symptoms.



Hide All
Addington-Hall J, Kalra L. 2001 Who should measure quality of life? BMJ. 322: 14171420.
Academy of Colleges Information Group. 2001. Specification of core requirements for clinical information systems in support of secondary care. Available at:; Accessed:
Bakken Henry S, Lush M, Costantino M, et al. 1997 Health status measurement in computer-based patient record systems. Nurs Admin Q. 21: 5060.
Bakken Henry S, Morris J, Holzemer W. 1997 Using structured text and templates to capture health status outcomes in the electronic health record. J Qual Improv. 23: 667677.
Brazier J, Harper R, Jones N, et al. 1992 Validating the SF-36 health survey questionnaire: A new outcome measure for primary care. BMJ. 305: 160164.
Carr A, Gibson B, Robinson P. 2001 Is quality of life determined by expectations or experience? BMJ. 322: 12401243.
Cheung W-Y, Garratt A, Russell I, Williams J. 2000 The UK IBDQ- A British version of the inflammatory bowel disease questionnaire: Development and validation. J Clin Epidemiol. 53: 297306.
Cole L, Houston S. 1997 Integrating information technology with an outcomes management program. Crit Care Nurs Q. 19: 7179.
Deyo R, Diehr P, Patrick D. 1991 Reproducibility and responsiveness of health status measures. Statistics and strategies for evaluation. Control Clin Trials. 12: 142S158S.
Department Of Health. The NHS Plan—a plan for investment, a plan for reform. London: Stationary Office; July 2000.
Department Of Health. 2001. Building the information core: Implementing the NHS plan. London: Department of Health;
Dolin R. 1997 Outcome analysis: Considerations for an electronic health record. M.D. Computing. 14: 5056.
Galland R, Magee T, Berridge D, et al. 1998 Accuracy of centrally recorded OPCS codes for vascular surgery in the United Kingdom. Eur J Vasc Endovasc Surg. 16: 415418.
Galland R, Whatling P, Crook T, Magee T. 2000 Regional variation in varicose vein operations in England 1989-1996. Ann R Coll Surg Engl. 82: 275279.
Garratt A, Schmidt L, Mackintosh A, Fitzpatrick. 2002 Quality of life measurements: Bibliographic study of patient assessed health outcome measures. BMJ. 234: 14171419.
Higginson I, Carr A. 2001 Using quality of life measures in the clinical setting. BMJ. 322: 12971300.
Johnson R, Hornbrook M, Nichols G. 1994 Replicating the chronic disease score (CDS) from automated pharmacy data. J Clin Epidemiol. 47: 11911199.
Johnston R. 2000 Implementation of a computer-based patient record and an outcomes data-collection system at the Department of Orthopaedic Surgery, University of Iowa. J Bone Joint Surg Am. 82: 15021506.
Mann R, Williams J. 2003 Standards in medical record keeping. Clin Med. 3: 329332.
Medical Outcomes Trust. 1994. SF-36 Health Survey Scoring Manual for English Language Adaptations. Australia/New Zealand, Canada, United Kingdom. Boston, MA: Medical Outcomes Trust;
Moher D, Schulz K, Altman D. 2001 The CONSORT statement: Revised recommendations for improving the quality of reports of parallel-group randomised trials. Lancet. 357: 11911194.
National Health Service Information Authority. Clinical terminology and classification services. Available at:
Office of Population Censuses and Surveys. 1990. Tabular list of the classification of surgical operations and procedures. 4th revision. London: HMSO;
Pollock AM, Vickers N. 2000 Reducing DCO registration through electronic matching of cancer registry data and routine hospital data. Br J Cancer. 82: 712717.
Rietman L, Hendricks H. 2000 Speaking the same language: Improving evaluation of patient-focused outcomes using standardized terminology. Top Health Inform Manage. 20: 3543.
Streiner G, Norman R. 1995. Health measurement scales. A practical guide to their development and use. Oxford: Oxford University Press;
Ware J, Sherbourne C. 1992 The MOS 36-item short form health survey (SF-36). I. Conceptual framework and item selection. Med Care. 30: 473483.
Welsh Assembly Government. 2002. Informing healthcare-towards a knowledge-based NHS. Cardiff: Welsh Assembly Government;
Whitman G. 2002 Outcomes research. Getting started, defining outcomes, a framework, and data sources. Crit Care Nurs Clin N Am. 14: 261268.
World Health Organization. 1992. International statistical classification of diseases and related health problems. 10th revision. Geneva: World Health Organization;
Williams J, Morgan J, Howlett P, Severs M. 1993 Let there be light. Br J Healthcare Comput. 10: 3032.
Williams J, Cheung W, Russell I, et al. 2000 Open access follow up for inflammatory bowel disease: Pragmatic randomised trial and cost effectiveness study. BMJ. 320: 544548.
Williams J, Cheung W, Cohen D, et al. 2003 Can randomized trials rely on existing electronic data? A feasibility study to explore the value of routine data in health technology assessment. Health Technol Assess. 7 (26).
Wilson I, Cleary P. 1995 Linking clinical variables with health-related quality of life. A conceptual model of patient outcomes. JAMA. 273: 5965.
Zielstorff R. 1995 Capturing and using clinical outcome data: Implications for information systems design. J Am Med Informatics Assoc. 2: 191196.


Can electronic routine data act as a surrogate for patient-assessed outcome measures?

  • Hayley A. Hutchings (a1), Wai-Yee Cheung (a1), John G. Williams (a1), David Cohen (a2), Mirella F. Longo (a2) and Ian Russell (a3)...


Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed