Introduction
Over recent years, there has been an increased emphasis on the development and application of patient-reported outcome measures.Reference Nelson, Eftimovska, Lind, Hager, Wasson and Lindblad 1 This drive to assess the impact of illness or interventions, from the patient's perspective, has resulted in a large expansion of available questionnaires.Reference Dawson, Doll, Fitzpatrick, Jenkinson and Carr 2 Patient-reported outcome measures are usually designed to measure one of two broad themes, either patients’ perceptions of their general health, or their perceptions of their health in relation to specific diseases or conditions.
The selection of a patient-reported outcome measure requires careful consideration regarding the content of the questionnaire and its relevance to the intended patient group. An appropriate measure is one that is supported by published evidence demonstrating that it is: acceptable to patients, reliable, valid and responsive (sensitive to change).Reference Fitzpatrick, Davey, Buxton and Jones 3 In addition, evidence for these properties needs to have been obtained in a similar context, and on similar types of patients (in terms of age range, gender, and diagnostic or surgical category) to those whom the patient-reported outcome measure is to be applied.
The importance of selecting an appropriate patient-reported outcome measure is specifically emphasised in the paediatric population. Many adult-designed patient-reported outcome measures may contain items that are irrelevant to children (e.g. driving, financial outcomes), or use language and response categories that are not age-appropriate.
We therefore reviewed the current literature on patient-reported outcome measures used in paediatric otolaryngology. This included general paediatric patient-reported outcome measures applied to otolaryngology patients, and otolaryngology disease-specific patient-reported outcome measures.
Materials and methods
A comprehensive literature search was conducted on 14 December 2016 using the databases Medline, Embase, Cumulative Index to Nursing and Allied Health Literature, and PsycInfo. The search terms used were: ‘health assessment questionnaire’, ‘structured questionnaire’, ‘questionnaire’, ‘patient reported outcome measures’, ‘PROM’, ‘quality of life’ or ‘survey’, and ‘children’ or ‘otolaryngology’.
Results
The search yielded 656 articles. The results were limited to English-language articles published from 1996 to 2016; this yielded 562 articles. Removal of duplicates returned 395 articles. Of these, 82 article abstracts were screened; this yielded a total of 63 relevant articles. Searching the bibliographies of these articles identified further articles that were reviewed.
Disease-specific patient-reported outcome measures
A large number of questionnaires were identified related to the conditions of otitis media,Reference Brouwer, Maille, Rovers, Grobbee, Sanders and Schilder 4 –Reference Habesoglu, Habesoglu, Deveci, Kulekci, Kalaycik and Gokceer 14 hearing loss,Reference Umansky, Jeffe and Lieu 15 –Reference Lim, Xiang, Wong, Yuen and Li 18 obstructive sleep apnoea,Reference Franco, Rosenfeld and Rao 19 –Reference de Serres, Derkay, Astley, Deyo, Rosenfeld and Gates 30 voice disorders,Reference Walz, Hubbell and Elmaraghy 31 , Reference Boseley, Cunningham, Volk and Hartnick 32 and sore throat and tonsillitis.Reference Thong, Davies, Murphy and Keogh 33 –Reference Goldstein, Stewart, Witsell, Hannley, Weaver and Yueh 38 A number of other questionnaires were infrequently described, related to other specific otolaryngology conditions (Table I).Reference Hartnick, Giambra, Bissell, Fitton, Cotton and Parsons 39 , Reference Fraser, Montgomery, James, Wynne, MacGregor and Clement 40 Each tool was of a varying length and underwent variable degrees of validation (Table I). We also identified multiple institutional designed questionnaires, which underwent little or no validation.Reference McCormick, Ward, Roberson, Shah, Stachler and Brenner 41 –Reference Boss and Thompson 43
Table I Disease-specific patient-reported outcome measures used in paediatric otolaryngology*
* Identified from the literature review.
General patient-reported outcome measures
The paediatric otolaryngology literature describes a large number of general patient-reported outcome measures that have been utilised.Reference Kubba, Swan and Gatehouse 8 , Reference Goldstein, Fatima, Campbell and Rosenfeld 20 , Reference Tran, Nguyen, Weedon and Goldstein 21 , Reference Goldstein, Stewart, Witsell, Hannley, Weaver and Yueh 38 , Reference Kubba, Swan and Gatehouse 44 –Reference Boruk, Lee, Faynzilbert and Rosenfeld 64 Overall, these are extensively validated tools from a non-otolaryngology origin that have been applied to otolaryngology conditions or procedures (Table II).
Table II General patient-reported outcome measures used in paediatric otolaryngology*
* Identified from the literature review.
Discussion
We identified a large number of both disease-specific and general patient-reported outcome measures that have been used in the paediatric otolaryngology literature. Many publications used a mixture of a disease-specific patient-reported outcome measures and general patient-reported outcome measures. Disease-specific patient-reported outcome measures are usually more sensitive to differences in one organ system, compared with generic instruments. For otolaryngology, this may be particularly important to detect improvements with treatment. Nevertheless, general tools are important to patients and healthcare commissioners, as they measure more global changes in health status and allow comparison with other conditions.Reference Nelson, Eftimovska, Lind, Hager, Wasson and Lindblad 1 –Reference Fitzpatrick, Davey, Buxton and Jones 3
A large number of disease-specific patient-reported outcome measures were identified for the five conditions of otitis media, hearing loss, obstructive sleep apnoea, voice disorders, and sore throat and tonsillitis, which not unsurprisingly are the more common paediatric otolaryngology presentations.
In cases of otitis media, the Otitis Media-6 (‘OM-6’) questionnaire was by far the most extensively validated tool.Reference Brouwer, Maille, Rovers, Grobbee, Sanders and Schilder 4 –Reference Rosenfeld, Goldsmith, Tetlus and Balzano 11 The Otitis Media Outcome-22 (‘OMO-22’) questionnaire was another prominent tool.Reference Richards and Giannoni 10 , Reference Alsarraf, Jung, Perkins, Crowley and Gates 12 These tools were principally discriminated by their number of questions, providing a trade-off between response rate and sensitivity to change (lower in longer tools), versus the collection of what may be important information (easier in longer tools). A specific chronic suppurative otitis media tool was also identified, the Chronic Otitis Media-5 (‘COM-5’) survey.Reference Vlastos, Kandiloros, Manolopoulos, Ferekidis and Yiotakis 13 , Reference Habesoglu, Habesoglu, Deveci, Kulekci, Kalaycik and Gokceer 14
For hearing loss, more frequently generic quality of life (QoL) measures were used. However, disease-specific tools were also described; these included the adolescent-completed Hearing Environments and Reflection on Quality of Life (‘HEAR-QL’) questionnaire,Reference Umansky, Jeffe and Lieu 15 the Youth Quality of Life Instrument – Deaf and Hard of Hearing (‘YQOL-DHH’) module,Reference Patrick, Edwards, Skalicky, Schick, Topolski and Kushalnagar 16 and the caregiver-reported Paediatric Hearing Impairment Caregiver Experience (‘PHICE’) questionnaire.Reference Meinzen-Derr, Lim, Choo, Buyniski and Wiley 17 , Reference Lim, Xiang, Wong, Yuen and Li 18 All of these are fairly lengthy questionnaires related to hearing and general QoL.
A number of obstructive sleep apnoea specific health-related QoL measure were identified. Of these, the Obstructive Sleep Apnea-18 (‘OSA-18’) questionnaire, which was first described by Franco et al.,Reference Franco, Rosenfeld and Rao 19 was the most widely used and validated QoL survey for the assessment of paediatric obstructive sleep apnoea.Reference Goldstein, Fatima, Campbell and Rosenfeld 20 –Reference Kang, Weng, Yeh, Lee and Hsu 25 A number of other validated tools have also been described.Reference Goldstein, Stefanov, Graw-Panzer, Fahmy, Fishkin and Jackson 26 –Reference de Serres, Derkay, Astley, Deyo, Rosenfeld and Gates 30 Once again, the greatest discriminator was the length and form of the tool. The Clinical Assessment Score-15 (‘CAS-15’) stood out as the only clinic-completed tool identified, which included clinical and examination findings.
The literature concerning paediatric voice specific questionnaires was dominated by three frequently used tools, adapted from previously validated adult forms: the Pediatric Voice-Related Quality-of-Life (‘PVRQOL’) survey,Reference Walz, Hubbell and Elmaraghy 31 , Reference Boseley, Cunningham, Volk and Hartnick 32 the Paediatric Vocal Handicap Index (‘pVHI’)Reference Zur, Cotton, Kelchner, Baker, Weinrich and Lee 65 and the Pediatric Voice Outcomes Survey (‘PVOS’).Reference Walz, Hubbell and Elmaraghy 31 , Reference Hartnick 66 , Reference Merati, Keppel, Braun, Blumin and Kerschner 67
Paediatric throat health related QoL was almost entirely reported on using the Paediatric Throat Disorders Outcome Test (‘T-14’).Reference Thong, Davies, Murphy and Keogh 33 –Reference Soni-Jaiswal, Anderco and Kumar 37 Designed by Hopkins et al.,Reference Hopkins, Fairley, Yung, Hore, Balasubramaniam and Haggard 34 this 14-item disease-specific questionnaire has been extensively validated and reported upon in children. The only other identified measure was the infrequently used Tonsil and Adenoid Health Status Instrument (‘TAHSI’), from which the Paediatric Throat Disorders Outcome Test derives many question items.Reference Goldstein, Stewart, Witsell, Hannley, Weaver and Yueh 38 A number of other questionnaires were infrequently described, including the Pediatric Tracheotomy Health Status Instrument (‘PTHSI’)Reference Hartnick, Giambra, Bissell, Fitton, Cotton and Parsons 39 and the Post-Operative Pinnaplasty Questionnaire (‘POPQ’).Reference Fraser, Montgomery, James, Wynne, MacGregor and Clement 40
A number of general patient-reported outcome measures are described in the paediatric otolaryngology literature. The Glasgow Children's Benefit Inventory (‘GCBI’) was developed by Kubba et al.Reference Kubba, Swan and Gatehouse 44 to assess health-related QoL pre- and post-intervention, and has been extensively validated and used in a number of otolaryngology interventions.Reference Langille and El-Hakim 45 –Reference Schwentner, Schwentner, Schmutzhard, Radmayr, Grabher and Sprinzl 48 The less widespread use of this tool outside otolaryngology limits the global comparability of its findings with other non-otolaryngology interventions, which may be of importance to certain groups such as commissioners.
Tools such as the Pediatric Quality of Life Inventory (‘PedsQL’) have also been extensively used in otolaryngology, and in other specialties. It gives a global health overview, and usefully has two forms: the parent proxy report form and the age-specific child self-report form.Reference Naiboglu, Kulekci, Kalaycik, Sheidaei, Toros and Egeli 49 –Reference Lindman, Lewis, Accortt and Wiatrak 51
Similarly, the KINDL-R questionnaire for measuring health-related QoL in children and adolescents,Reference Loy, Warner-Czyz, Tong, Tobey and Roland 60 , Reference Morettin, Santos, Stefanini, Antonio, Bevilacqua and Cardoso 61 the Child Health Questionnaire (‘CHQ’)Reference Goldstein, Stewart, Witsell, Hannley, Weaver and Yueh 38 , Reference Georgalas, Tolley and Kanagalingam 54 –Reference Cunningham, Chiu, Landgraf and Gliklich 56 and the Child Behavior Checklist (‘CBCL’)Reference Goldstein, Fatima, Campbell and Rosenfeld 20 , Reference Tran, Nguyen, Weedon and Goldstein 21 , Reference Mazefsky, Anderson, Conner and Minshew 62 are general patient-reported outcome measures that have different forms for various age groups. These tools have the benefit of age-appropriate questions, but their use would potentially limit comparison of the findings outside of that age group.
Other generic tools have been used in otolaryngology and are completed by the caregiver, such as the TNO-AZL (Netherlands Organisation for Applied Scientific Research Academic Medical Centre) Preschool Quality of Life Questionnaire (‘TAPQOL’), specifically designed for pre-school children.Reference Kubba, Swan and Gatehouse 8 , Reference Raat, Mohangoo and Grootenhuis 58 , Reference Raat, Botterweck, Landgraf, Hoogeveen and Essink-Bot 59 Several generic tools have been described to specifically assess the impact of a disease on the family or caregiver, such as the Pediatric Quality of Life Inventory Family Impact survey,Reference Varni, Sherman, Burwinkle, Dickinson and Dixon 52 , Reference Blank, Grindler, Schulz, Witsell and Lieu 53 the Parenting Stress Index Short Form (‘PSI-SF’)Reference Davies, Batra, Mehanna and Keogh 63 and the Caregiver Impact Questionnaire (‘CIQ’).Reference Boruk, Lee, Faynzilbert and Rosenfeld 64
There were several incidences of patient-reported outcome measures being used inappropriately in the paediatric otolaryngology literature. We found instances where the wording of questionnaires was changed. It is important to note that the wording of a validated patient-reported outcome measure should not be changed because even relatively small alterations can make a considerable difference to the meaning of the questions and consequently to the measurement properties of a questionnaire.Reference Dawson, Doll, Fitzpatrick, Jenkinson and Carr 2 , Reference Fitzpatrick, Davey, Buxton and Jones 3
We also identified numerous cases where patient-reported outcome measures were applied to very different groups or situations to those on which they were validated; for example, the use of adult patient-reported outcome measures, completed by caregivers on the child's behalf, or the use of adult patient-reported outcome measures in adolescents. The term ‘paediatric’ covers a broad range, and while many of the general patient-reported outcome measures had multiple age-appropriate questions, many of the disease-specific questionnaires were only validated for specific age ranges. Furthermore, patient-reported outcome measures data need to be obtained from relevant patients at the same point in time relative to the date of an intervention or event of interest.Reference Dawson, Doll, Fitzpatrick, Jenkinson and Carr 2 , Reference Fitzpatrick, Davey, Buxton and Jones 3
It should also be noted that a number of the patient-reported outcome measures, particularly the general patient-reported outcome measures, had different versions available, and this needs to be considered when comparing to previously published studies using older versions.
We also identified a distinct lack of consistency regarding the methods for the development of patient-reported outcome measures in paediatric otolaryngology. The methods of validation contained many similarities, but were not universal; statistical methods and validation samples sizes varied radically.
The availability of multiple disease-specific and general patient-reported outcome measures is useful to the paediatric otolaryngologist. However, more consistent use of a smaller number of tools would allow for greater standardisation, and would assist in the pooling of data from multiple institutions and studies.
Selecting the appropriate patient-reported outcome measure can be challenging. This is emphasised in the otolaryngology literature from the extensive number of tools used for certain conditions; for example, otitis media.Reference Brouwer, Maille, Rovers, Grobbee, Sanders and Schilder 4 –Reference Habesoglu, Habesoglu, Deveci, Kulekci, Kalaycik and Gokceer 14
When assessing a patient-reported outcome measure, it is crucial to review the six key areas of: validity, test–retest reliability, precision, responsiveness, acceptance and response rate, and feasibility.Reference Nelson, Eftimovska, Lind, Hager, Wasson and Lindblad 1 –Reference Fitzpatrick, Davey, Buxton and Jones 3 Regarding validity, one should consider whether the patient-reported outcome measure assesses what it is supposed to. Changes in patient-reported outcome measure scores can be caused by a multitude of factors, not just the intervention that is being measured. With regard to test–retest reliability, do respondents score similarly on different occasions? Precision is epitomised by the disease-specific or general patient-reported outcome measure debate; can the patient-reported outcome measure discriminate a disease or intervention from a control group? Linking in with precision, responsiveness refers to the ability to measure change after an intervention or change in disease state. Acceptance and response rate are particularly important in paediatric questionnaires. Are the questions appropriate for children (e.g. do they contain questions about employment)? Linking in with acceptance and response rate, a long questionnaire may not be feasible in many clinic settings.
It is also crucial to review the literature on the previous use of any patient-reported outcome measure, considering particularly the age groups that it is appropriate for and whether there are reference data for the comparison group.
It is impossible to have one patient-reported outcome measure that covers all potential research or clinical questions posed in paediatric otolaryngology. For example, the 22-item Otitis Media Outcome-22 questionnaire provides more information than the 6-item Otitis Media-6 questionnaire, but may have a reduced response rate given its length and reduced sensitivity to change.Reference Brouwer, Maille, Rovers, Grobbee, Sanders and Schilder 4
Researchers and clinicians should aim to use the most appropriate patient-reported outcome measure for their question, but also consider the comparability with previous studies on that condition. We would advocate, whenever possible, using a disease-specific and/or general patient-reported outcome measure that is frequently cited for the condition of interest, and ensure it is suitably validated, with relevant reference data.
