Sick children are at high risk of malnutrition and identification of such children is important, since malnutrition is associated with increased morbidity, mortality(Reference Klein, Forbes and Nader1, Reference Schneider, Veyres and Pivot2), length of hospital stay(Reference Kac, Camacho-Dias and Silva-Coutinho3) and health-care costs(Reference Elia4). Recent national guidelines state that all patients should be screened for risk of malnutrition on admission and periodically during their hospital stay using a validated screening tool(5).
A number of malnutrition screening tools have been developed for adults(Reference Green and Watson6) but peer-reviewed validated screening tools suitable for generic use in paediatrics are scarce. A paediatric malnutrition screening tool (Paediatric Yorkhill Malnutrition Score; PYMS) was developed for use at the Royal Hospital for Sick Children (RHSC), Yorkhill, Glasgow, a large tertiary children's hospital. This was based on the guidelines of the European Society of Clinical Nutrition and Metabolism for nutritional screening(Reference Kondrup, Allison and Elia7). The development and performance of the tool at the RHSC, as well as in a district general hospital (DGH), are described elsewhere(Reference Gerasimidis, Macleod and McGrogan8). The PYMS assesses four steps, all recognised predictors or symptoms of malnutrition: BMI, history of recent weight loss, changes in nutritional intake, and the predicted effect of the current medical condition on the nutritional status of the patient. Each step bears a score of up to 2 and the total score reflects the degree of the nutrition risk of the patient. A score of 1 indicates medium risk and 2 or above, high risk.
Pilot work suggests that this tool has a high yield of identifying children at true risk of malnutrition, without generating large numbers of false-positive cases(Reference Gerasimidis, Macleod and McGrogan8). What was not known was how many true cases were being missed, how it compared with other available assessment and screening tools and the extent to which screened children showed objective evidence of malnutrition. The present study therefore aimed to test the accuracy of the PYMS used by ward nursing staff and compare it with full dietetic assessment, and other validated malnutrition tools, in a large tertiary hospital and a DGH. It further explored discrepancies in body composition between different levels of malnutrition risk.
Material and methods
Introduction of the Paediatric Yorkhill Malnutrition Score
Following the development of the tool, the PYMS was introduced in four paediatric wards (three medical, one surgical) at the RHSC and in the general paediatric ward of a DGH with high patient turnover. All patients (aged 1–16 years) admitted over a 4-month pilot phase were eligible for screening by about 160 nursing staff. Patients from cardiology, renal and orthopaedic specialties and critical care were not included. Before the launch of the project, the nursing staff attended 1 h awareness sessions and received training on anthropometric measurements.
Validation study
During the pilot, two research dietitians visited the participating wards daily and recruited children who had already undergone PYMS screening by the nursing staff. This was a convenience sample, with children recruited if available to be seen on the days and times when research staff were able to attend the ward. The original aim was to over-recruit patients rated high and medium risk by the PYMS. In the event, due to clinical logistics, it proved easier to recruit medium- and low-risk subjects, so that the proportions studied were very similar to the total screened population. The diagnostic accuracy of the PYMS was assessed with a four-stage research process.
Criterion validity
Patients' malnutrition risk (PYMS scoring by the ward nursing staff) was compared with the respective nutrition outcome (low, medium or high malnutrition risk) from a full dietetic assessment. Two research dietitians assessed the nutritional status of the patients, applying a detailed dietary history, anthropometric measurements, nutrition-associated physical examination, ability of the patients to maintain age-appropriate energy levels and review of the medical notes.
Inter-rater reliability
The reliability of the PYMS to yield the same nutrition outcome when used on the same patients by different health professionals was tested between the two research dietitians and the ward nursing staff. An additional PYMS form was completed by the two research dietitians, for those patients who had been screened on admission, and the malnutrition outcome was compared with that of the ward nursing staff.
Comparison with other tools
The concordant validity and agreement of the PYMS was assessed by means of comparison with other similar paediatric tools and by comparison of all tools against the same research dietetic assessment. The research dietitians applied the PYMS, the Screening Tool for the Assessment of Malnutrition in Paediatrics (STAMP)(Reference McCarthy, McNulty and Dixon9), and the Paediatric Subjective Global Nutritional Assessment (SGNA)(Reference Secker and Jeejeebhoy10) on the same patients and compared their concordance and how these compared with the full research dietetic assessment.
Discriminant validity
The ability of PYMS malnutrition classification to discriminate between patients with high and low fat or lean mass was tested using body composition assessment. We assumed that patients who are currently malnourished are likely to have lower levels of fat mass and possibly lean mass than children at low risk. For children older than 5 years, impedance was measured with a foot-to-foot bioelectrical impedance analyser (Tanita TBF-300; Tanita, Tokyo, Japan). For children under 5 years of age, the mid-upper arm circumference was measured with a tape, and the triceps and subscapular skinfold thicknesses with a Harpenden calliper using standard protocols.
In all stages of the validation process one research dietitian identified those patients who had their nutrition screening completed on admission by reviewing the nursing notes and the second carried out the dietetic assessment, whilst blinded to the nursing PYMS outcome. However, as the same dietitian carried out the dietetic assessment and compared the individual nutritional screening tools, blinding each outcome was not feasible.
Statistical analysis
As the PYMS is intended for clinical use by the nursing staff, its diagnostic validity was evaluated on the nurses' screening. In contrast, for the comparison of PYMS validity with that of other nutritional screening tools, the dietetic screening ratings were used so that all tools were used by the same assessors. In all cases the nutritional outcome of the PYMS (completed by nursing staff or research dietitian) was cross-tabulated with the respective outcome of the research dietetic assessment or other screening tools. As only patients at high risk of malnutrition were to be referred for dietetic review, we combined the low- and medium-risk categories in order to calculate the diagnostic values (sensitivity, specificity, negative and positive predictive values) of the PYMS on the contingency tables. The agreement between the methods was assessed using Cohen's κ statistics and interpreted according to the tables by Landis & Koch(Reference Landis and Koch11).
Anthropometric measurements of height, weight and BMI were converted to z-scores using 1990 British reference data(Reference Cole, Freeman and Preece12). Measurements of skinfolds and mid-upper arm circumference were converted to z-scores according to the WHO reference range. For children between 5 and 14 years bioelectrical impedance values (Ω) were converted into z-scores for lean and fat mass adjusted for height, sex and age according to recent UK references values(Reference Wright, Sherriff and Ward13). Data analysis was conducted with Minitab 15 (Minitab Ltd, Coventry, UK). Differences in anthropometry and body composition characteristics between PYMS malnutrition risk categories were assessed with one-way ANOVA.
The present study was conducted according to the guidelines laid down in the Declaration of Helsinki and all procedures involving human subjects/patients were approved by the West Glasgow of Scotland Research Ethics Committee. Both the patient and their carer received written information on the study and gave written consent.
Results
Patient characteristics
Between 23 June and 28 October 2008, 2174 children were admitted for care to the five pilot sites (RHSC, n 1640; DGH, n 534). Of these patients, 1571 (72·3 %) were successfully screened (RHSC, 1208 (73·7 %) v. DGH, 363 (68 %)) by the ward nurses. Of these children, 247 (96 % of all approached; 103 female; 8·1 (sd 4) years) participated in this part of the study, 134 (54 %) being under various medical specialties at the RHSC, ninety-one (37 %) surgical or burns patients and twenty-two (9 %) being DGH patients.
Prevalence of malnutrition by nutrition tool
Prevalence of malnutrition risk in the cohort of patients recruited for the validation study varied considerably between the malnutrition tools and the research dietetic assessment (Fig. 1). The ward nursing staff rated proportionally fewer patients at high risk of malnutrition than the PYMS or the STAMP completed by the research dietitian, but more than the SGNA and the research dietetic assessment. The STAMP rated more patients at high and medium risk of malnutrition (P < 0·0001) and the SGNA fewer (P < 0·0001) compared with the PYMS (Fig. 1).
Prevalence of malnutrition risk according to malnutrition assessment and screening tool. (□), Low risk; (
), medium risk; (■), high risk; PYMS, Paediatric Yorkhill Malnutrition Score; STAMP, Screening Tool for the Assessment of Malnutrition in Paediatrics; SGNA, Paediatric Subjective Global Nutritional Assessment.
Criterion validity
Overall, 218 (88 %) patients were classified at the same nutrition risk between the research dietetic assessment and PYMS screening completed by the nursing staff (Table 1). The agreement between the research dietetic assessment and the PYMS completed by the nursing staff was moderate (κ = 0·46; 95 % CI 0·27, 0·64). Eleven patients (41 %) who were judged at ‘true’ high risk of malnutrition by the research dietetic assessment had not been rated at high risk by the nursing staff (Table 1). This gave a sensitivity for the nurse-rated PYMS of 59 %. Likewise, eighteen patients who were rated at high risk by the nursing PYMS were assessed as at ‘true’ low risk by the dietitians and thus were false positives, yielding a positive predictive value for the nursing PYMS of 47 % (Table 1). Among the patients who were judged to be false positives, 87 % were scored high on the basis of decreased intake and 75 % on the predicted effect of the current medical condition on future nutritional status. Manipulation of the rating system within the individual steps of the PYMS and of the threshold for referral for dietetic review did not improve the diagnostic accuracy.
Cross-classification of patient malnutrition risk based on the dietetic assessment and the Paediatric Yorkhill Malnutrition Score (PYMS) completed by the nursing staff
PPV, positive predictive value; NPV, negative predictive value.
* Low- and medium-risk categories grouped.
Inter-rater reliability
The inter-rater agreement for the PYMS completed by the two dietitians compared with the nursing staff was moderate (κ = 0·53; 95 % CI 0·38, 0·67) and concurred for 86 % of patients (213 out of 247) when low- and medium-risk categories were grouped together (Table 2).
Inter-rater reliability based on the Paediatric Yorkhill Malnutrition Score (PYMS) completed by the nursing staff and the two dietitians
* Low- and medium-risk categories grouped.
Comparison with other tools
When used by the research dietitians, 80 % (198 out of 247) of the patients were classified at the same risk of malnutrition between the STAMP and the PYMS and 81 % (199 out of 247) between the SGNA and the PYMS. The agreement between the STAMP and the PYMS was moderate (κ = 0·47; 95 % CI 0·34, 0·61) and between the SGNA and the PYMS slight (κ = 0·12; 95 % CI − 0·11, − 0·34). The PYMS identified all the children who screened at high risk by the SGNA, but only 52 % of those screened at high risk by the STAMP. Likewise, 20 % of the patients screened at low risk by the SGNA and 9 % by the STAMP were classified at high risk by the PYMS.
The STAMP and the PYMS completed by the research dietitians both achieved high specificity and sensitivity compared with the research dietetic assessment, but the positive predictive value was higher for the PYMS and showed higher agreement with the research dietetic assessment (Table 3). The SGNA showed extremely high specificity, and negative and positive predictive values, but had very low sensitivity compared with the research dietetic assessment (Table 3).
Cross-classification of patient malnutrition risk based on the dietetic assessment, the Screening Tool for the Assessment of Malnutrition in Paediatrics (STAMP), the Paediatric Subjective Global Nutritional Assessment (SGNA) and the Paediatric Yorkhill Malnutrition Score (PYMS) completed by the research dietitians
PPV, positive predictive value; NPV, negative predictive value.
* Low- and medium-risk categories grouped.
Discriminant validity
Anthropometry and body composition were measured in the majority, but not all of the children, as some were unable to stand or were uncooperative (Table 4). Weight, BMI and lean index z-scores (for children aged 5–14 years) varied significantly by PYMS risk group but fat did not (Table 4). When patients who scored at high risk due to a low BMI (i.e. BMI z-score ≤ − 2) or were underweight (weight z-score ≤ − 2) were excluded, the difference in BMI z-scores attenuated but remained significant, but the differences in weight and lean z-scores became non-significant (Table 4). For children younger than 5 years no significant differences were found for subscapular skinfold, triceps skinfold, and mid-upper arm circumference z-scores between the different malnutrition risk categories.
(Mean values and standard deviations)
MUAC, mid-upper arm circumference; TSF, triceps skinfold; SUB, subscapular skinfold.
* Mean value was significantly different from that of the high-risk group (P < 0·05).
† Excluding children who would only score at high risk due to a low BMI (BMI z-score ≤ − 2) or were underweight (weight z-score ≤ − 2).
‡ n ≤ 5.
Discussion
These data suggest that when used by mainstream ward nursing staff in a general paediatric population, this new tool presents acceptable levels of sensitivity and specificity compared with the proposed ‘gold standard’ of a full dietetic assessment and that it has a more useful profile than other screening tools already in use. The ideal malnutrition screening method should identify all children at high risk of malnutrition, while not misclassifying those at low risk. As this is generally not achievable, in previous studies the need to correctly identify the majority of patients who are malnourished has tended to take precedence over the risk of misclassifying well-nourished patients(Reference Ferguson, Capra and Bauer14, Reference Gerasimidis, Drongitis and Murray15). However, such tools are likely to increase staff workload inappropriately and waste available resources, making their routine use in clinical practice not feasible. The PYMS was developed for routine clinical use and was designed to ensure a balance between the need to identify more patients at high risk of malnutrition, without significantly increasing the staff workload. In a related study we have found that use of the tool did not produce large numbers of false positives(Reference Gerasimidis, Macleod and McGrogan8).
In contrast to previous studies which used health professionals (dietitians or specialist nurses) as both the user and assessor of the validity of a screening tool(Reference Ferguson, Capra and Bauer14, Reference Gerasimidis, Drongitis and Murray15), the present study used ward nursing staff as raters and then used dietitians who were blinded to the nurses' rating to assess their accuracy. Although a lower diagnostic accuracy of the PYMS is expected when completed by the nursing staff(Reference Bavelaar, Otter and van Bodegraven16) compared with when it is carried out by the dietitians, the present study aimed to test how the PYMS would perform in actual clinical practice, used by a large number of non-specialist nursing staff. This evaluation revealed that nurses using the PYMS identified over half the children deemed by full assessment to actually be at high risk. If most of these children would not have been identified at all without this formal screening process, this is an acceptable result for a screening tool; earlier studies have shown that health staff are poor at recognising undernutrition(Reference Bavelaar, Otter and van Bodegraven16). However, it is also important to ensure that staff using the tool are aware that cases may be missed and therefore a low score does not negate referral for dietetic assessment for other valid reasons. From the present data we are not able to determine how many children would have been missed without use of the PYMS, but in a related study we found that more than 50 % of valid referrals from the PYMS were not known to the dietetic service(Reference Gerasimidis, Macleod and McGrogan8). The fact that use of the PYMS by a dietitian identified more true cases also suggests that the diagnostic accuracy of the PYMS could possibly be improved by further training and continuous use.
The specificity and positive predictive value shown by the PYMS was good by screening standards, but it still has to be borne in mind that a positive predictive value of 47 % could potentially result in a doubling of dietetic referral rates. However, one cannot know how many referrals are already made to dietitians for inappropriate reasons, or how many of these will have been prevented by reassurance when the PYMS score was low. Most of the false positives were triggered due to decreased intake or the likely effect of the current condition on future nutritional status. These false positives occurred most frequently in patients with acute medical (for example, gastroenteritis, asthma) and surgical conditions (for example, appendicitis, tonsillectomy), which usually resolve quickly once medical or surgical treatment is in place. As reduction in nutritional intake over a longer period is likely to be a strong predictor of future nutritional status, the stated threshold for reduced intake will be increased in the final version of the tool to 1 week rather than 5 d as in the original version of the tool.
The comparisons with other paediatric malnutrition tools again demonstrated that the prevalence of malnutrition in a population depends largely on the screening methods used(Reference Gerasimidis, Drongitis and Murray15, Reference Stratton, Hackston and Longmore17). In contrast to previous studies(Reference Gerasimidis, Drongitis and Murray15, Reference Stratton, Hackston and Longmore17), which assessed only the agreement and concordance between screening tools, the present study explored how all the different tools performed against the same reference method. The PYMS identified fewer patients and hence a more manageable sample of patients at risk of malnutrition than the STAMP, as well as demonstrating slightly better sensitivity.
Although the SGNA achieved very high specificity and positive predictive value, its sensitivity was very low. This is not surprising, as the SGNA is an assessment method, rather than a screening tool, which aims to identify children with established malnutrition, while the PYMS and STAMP were developed also to identify children whose nutritional status is likely to deteriorate as the result of an acute medical condition, despite normal anthropometry at the time of the screening.
The body composition data themselves raise new questions. While the patients at high risk of malnutrition had significantly lower BMI and lean mass, they showed no evidence of lower fat stores. A low BMI ( ≤ 2nd centile) was used in the PYMS to screen for malnutrition risk, but even when those high-risk children identified solely on the basis of a low BMI were excluded, the remainders (who were the great majority) still had significantly lower BMI and relatively low lean mass. A low BMI was also strongly associated with being judged at high malnutrition risk on full assessment. This therefore suggests that the children judged high risk using either method were not, on average, acutely malnourished. The significance of their low lean mass is not clear. Other studies have found that children with chronic illness and disability may have very low lean mass(Reference Wright, Sherriff and Ward13, Reference Burnham, Shults and Semeao18) and such children are more likely to be at long-term nutritional risk and comprised the majority who scored high risk in the present study.
In conclusion, the PYMS appears to be effective at identifying children at risk of malnutrition and should produce fewer false-positives case than the STAMP screening tool. Further work is needed to explore its utility in more specialist paediatric areas and when applied in new centres.
Acknowledgements
We would like to acknowledge the contribution of the members of the ‘Malnutrition Screening Tool Steering Group’: Mr Toby Mohammed, Ms Christina McGuckin, Dr Paraic McGrogan, Dr Graeme Stewart, Mrs Isabel Swinbank, Ms Anne Maclean, Ms Elaine Buchanan and Ms Mary McAuley. We are particularly grateful to the nursing and dietetic departments at the Royal Hospital for Sick Children (Yorkhill, Glasgow, UK) and the Royal Alexandra Hospital (Paisley, UK), and the patients who participated in the study. We would like also to acknowledge the Department of Human Nutrition, University of Glasgow for equipment provision.
The project was funded by the National Health Service Greater Glasgow & Clyde, Food Fluid & Nutritional Care Planning Implementation Group, Glasgow, UK. A small grant was generously provided by the Yorkhill Children Foundation to cover consumable expenses.
K. G. designed the study, undertook the main research activities, conducted data analysis and drafted the manuscript; O. K. undertook the main research activities; I. M. contributed to the research activities and revised the manuscript; D. M. F. coordinated the project and revised the manuscript; C. M. W. designed and coordinated the study and revised the manuscript.
There are no conflicts of interest to declare.
