The prevalence of acute malnutrition is widely used to assess the nutritional status of populations of children aged 6–59 months in developing countries and to plan nutrition programmes. Moderate acute malnutrition (MAM) is defined as a weight-for-height Z-score (WHZ) <−2 and ≥−3 without bilateral pitting oedema and severe acute malnutrition (SAM) as WHZ < −3 or the presence of bilateral pitting oedema(1, 2). In these definitions, WHZ is most frequently calculated using the WHO growth standards(3).
The most commonly used method for estimating the prevalence of global acute malnutrition (GAM; grouping MAM and SAM), MAM and SAM is by conducting two-stage cluster sampled surveys requiring the measurement of several hundreds of children, typically 900 children (thirty clusters of thirty children) for achieving a sufficient precision for decision making(4, 5). In 2006, the SMART (Standardized Monitoring and Assessment of Relief and Transitions) method was introduced(6), addressing the problems of lack of standardization and lack of methodological rigour in the way nutritional surveys were undertaken(Reference Spiegal, Salama and Maloney7). The method allowed for standardization of surveys for the assessment of nutrition emergencies and provided a generic tool that can be used by various organizations working in the field(6). Despite the consistency in methodology and analysis that this method has provided for the nutrition community, there remains concern about the difficulty in obtaining usefully precise estimates of SAM prevalence, the large sample size required within the constraints of security and accessibility to villages, and the cost in applications such as surveillance by repeated cross-sectional surveys.
The classical method of estimating prevalence is to calculate the number of children meeting a case definition in the sample divided by the total number of children in the sample. In 1995, the WHO proposed the PROBIT method as an alternative method for prevalence estimation. The PROBIT method estimates prevalence indirectly using the inverse cumulative normal distribution function, which converts parameters of a normally distributed variable (i.e. the mean and standard deviation) to cumulative probability below any cut-off, which is equivalent to the proportion of individuals below the cut-off(8). The 1995 WHO document states that the main advantage of the PROBIT method is that it requires a smaller sample size than the classical method(8); however, no evidence of this is given. Golden and Grellety stated in 2002 that ‘calculation from the mean and standard deviation could give a more rapid, efficient and precise estimate of the extent of malnutrition than counting affected individuals’(9). There was no specific evidence provided either and no further work on this has been done, although this did lead to the inclusion in the SMART software output of PROBIT estimated prevalence.
The aim of the present study was to compare the PROBIT method with the classical method for estimating prevalence of GAM, MAM and SAM using a computer-based simulation approach to generate populations from real-world survey data sets and then simulate surveys sampled from these populations. Bias in the estimation of prevalence using the classical and PROBIT methods was investigated. The precision obtained for a given sample size when using the classical and PROBIT methods for estimating prevalence were compared.
In addition to weight-for-height, WHO and UNICEF also recommend a SAM case definition based on mid-upper arm circumference (MUAC) of <115 mm or the presence of bilateral pitting oedema(10). Several agencies also use MUAC ≥ 115 mm and <125 mm as a MAM case definition for programmatic purposes(11). We also tested the PROBIT and classical methods using these case definitions.
Methods
The classical method and PROBIT method for calculating prevalence of acute malnutrition were compared using computer-based simulations. First, populations were created from a database of existing surveys. Then surveys describing these populations were simulated by sampling from populations created from these original survey data sets.
Database
The database used in the analysis consisted of 560 nutritional surveys involving children aged between 6 and 59 months from thirty-one different countries, totalling 459 036 children. The surveys were carried out by eleven different organizations involved in nutrition programmes throughout the world. The surveys included measurements of weight, height, MUAC and assessment of oedema. WHZ was calculated using the WHO growth standards(3). A summary of the prevalence results for GAM, MAM and SAM from the surveys is shown in Table 1.
Summary of prevalence results from the 560-survey data set
MUAC, mid-upper arm circumference; WHZ, weight-for-height Z-score; GAM, global acute malnutrition; MAM, moderate acute malnutrition; SAM, severe acute malnutrition.
Creation of simulated populations
Each of the 560 surveys in the database was used to create a simulated population of 17 000 children by sampling with replacement from the survey data set. This size of population was chosen as being typical of the populations in which nutritional anthropometry surveys are commonly performed. Highly improbable values of the selected indicator were censored before the population was created (i.e. records in which weight-for-height was <−5 sd or >5 sd from the WHO growth standard median or MUAC was <80 mm or >240 mm) and appropriate case definitions (Table 2) were applied to the remaining records. Sampling with replacement from the survey data sets was done to create simulated populations of the desired size.
Case definitions of acute malnutrition used in the present study
GAM, global acute malnutrition; WHZ, weight-for-height Z-score; MUAC, mid-upper arm circumference; MAM, moderate acute malnutrition; SAM, severe acute malnutrition.
*WHZ using the WHO growth standards(3).
Simulating surveys with different sample sizes
Each of the 560 simulated populations was sampled using simple random sampling without replacement. Fifteen different sample sizes (fifty, seventy-five, 100, 125, 150, 175, 200, 225, 250, 275, 300, 350, 400, 450 and 500) were used for these simulated surveys. One hundred and fifty surveys were simulated for each sample size from each population. This process led to a total of 1·26 million simulated surveys: 560 populations × 15 sample sizes × 150 simulations = 1·26 million simulated surveys.
Calculation of true prevalence in the simulated populations
We calculated the true prevalence in each simulated population by counting the number of children meeting the case definition of SAM or MAM and calculating the ratio of this number to the total population.
Estimation of prevalence with classical and PROBIT methods in the simulated surveys
First, we calculated the prevalence using the classical method, by counting the number of children with the case definition of SAM or MAM in the simulated survey and calculating the ratio of this number to the total sample. Second, we estimated the prevalence with a PROBIT method. We used three different PROBIT approaches. The first approach was based on the approach recommended by WHO that involved the sample median WHZ and assuming sd = 1(8). The second approach used the sample mean and sd. The last approach involved the sample mean and sd calculated from data transformed towards normal, as explained below. Prevalence estimates for all three approaches using the PROBIT function were looked at as the cumulative probability of WHZ<−2 (GAM), WHZ < −3 (SAM) with and without oedema, and −3 ≤ WHZ < −2 (MAM). The approach using median and sd = 1 was not applicable to MUAC because the assumption that sd = 1 is only applicable to WHZ which is assumed to follow the standard normal distribution. The two other approaches of the PROBIT function were used to calculate prevalence estimates as the cumulative probability of MUAC < 125 mm (GAM), MUAC < 115 mm (SAM) with and without oedema and 115 mm ≤ MUAC < 125 mm (MAM).
Investigating normality of distributions
We investigated the normality of distributions of anthropometric indices in the simulated surveys using the Shapiro–Wilk test(Reference Armitage and Berry12). If there was evidence of non-normality (i.e. P < 0·05 for the Shapiro–Wilk test) then data were transformed towards normality using a power transformation with the transforming power found using the Box–Cox method(Reference Box and Cox13).
Investigation of bias and precision
Bias was investigated for the PROBIT method by the estimation of mean error (true prevalence minus estimated prevalence). Figure 1 shows an example of the plot of true prevalence against the difference between true and estimated prevalence for the PROBIT method. Figure 1 involves GAM prevalence by MUAC using PROBIT with transformed data with a sample size of 250 created using twenty replicates (a small number of replicates was used for illustrative purposes). Precision was investigated by the 95 % limits of agreement (mean (error) ± 1·96 × sd (error))(Reference Bland and Altman14). For all methods, the half width of the 95 % limits of agreement was calculated for different sample sizes based on the 150 simulated surveys. The analyses were also performed excluding children with oedema as we suspected that oedema might bias WHZ upwards, leading to downwardly biased estimates of prevalence.
Example of the plot of true prevalence v. the difference between true and estimated prevalence for the PROBIT method (global acute malnutrition prevalence by mid-upper arm circumference)
Results
Bias
Biases are shown for the three PROBIT methods for GAM, MAM and SAM defined by WHZ in Table 3. They indicate that the methods using mean and sd of non-transformed and transformed data are similar, with the method using median and sd = 1 inferior for both GAM and SAM but slightly better for MAM. Biases for the two PROBIT methods for GAM, MAM and SAM defined by MUAC again showed similarity between the methods using mean and sd of transformed and non-transformed data (Table 4). The biases for the classical approach are not shown since the classical method is known to be generally unbiased and the results of the simulations showed that any ‘bias’ was very close to zero and evenly distributed around zero. Results without oedema are not shown since excluding oedema in the analysis did not substantially change the results.
Bias (true prevalence minus calculated prevalence, in percentage points) for acute malnutrition according to weight-for-height in simulated surveys
GAM, global acute malnutrition; MAM, moderate acute malnutrition; SAM, severe acute malnutrition; WHZ, weight-for-height Z-score.
*Sample median WHZ and assuming sd = 1.
†Sample mean WHZ and sd.
‡Sample mean WHZ and sd with data transformed towards normal.
Bias (true prevalence minus calculated prevalence, in percentage points) for acute malnutrition according to mid-upper arm circumference in simulated surveys
GAM, global acute malnutrition; MAM, moderate acute malnutrition; SAM, severe acute malnutrition; MUAC, mid-upper arm circumference.
*Sample mean MUAC and sd.
†Sample mean MUAC and sd with data transformed towards normal.
Precision
Figure 2 shows precision for the classical method and three PROBIT methods for GAM, MAM and SAM using WHZ and the classical and two PROBIT methods for GAM, MAM and SAM using MUAC. The precision of the PROBIT methods (using the mean and sd of the survey with transformed and non-transformed data) is slightly better than for the classical method for sample sizes n < 150 for GAM and SAM for both MUAC and WHZ. However, the precision of the PROBIT method (using the mean and sd of the survey with transformed and non-transformed data) is better for MAM for sample sizes n < 300 for both MUAC and WHZ. The method using median and sd = 1 is generally inferior to the classical method except for small sample sizes for MAM. Results without oedema are not shown since excluding oedema in the analysis did not substantially change the results.
Observed precision for global acute malnutrition (GAM), moderate acute malnutrition (MAM) and severe acute malnutrition (SAM) calculated by the classical method (——) or the PROBIT approaches (- - -, median and sd = 1; · · ·, observed mean and sd; – · – · –, transformed data) using mid-upper arm circumference (MUAC) or weight-for-height Z-score (WHZ), according to sample size in simulated surveys: (a) GAM by MUAC; (b) GAM by WHZ; (c) MAM by MUAC; (d) MAM by WHZ; (e) SAM by MUAC; (f) SAM by WHZ
Discussion
The main limitation of the present study was that it was impractical to know the true prevalence of a large number of populations and to perform repeated surveys to estimate bias and precision of different estimators. The only feasible approach to testing the validity of the PROBIT approach was through simulation of surveys.
The study confirms that the PROBIT method can estimate prevalence of GAM and MAM using WHZ or MUAC. The PROBIT method provides an estimate of prevalence that is proportional to the true prevalence with a small bias that can be corrected for by simple subtraction of a small value found of bias in Table 2. The study shows, however, that the PROBIT method is inferior to the classical method for estimating the prevalence for SAM by WHZ or MUAC at sample sizes n > 150, although it does seem suitable for small sample sizes which may be useful for applications such as surveillance. These results do not seem to be influenced by a bias resulting from the inclusion of cases with bilateral pitting oedema since the results are similar with or without oedema in the analysis.
For WHZ, the PROBIT method of mean with observed sd of the data shows an improvement compared with using sd = 1. This suggests that when choosing to use the PROBIT method, it would be useful to use the observed sd to calculate prevalence. Checking for normality and, if necessary, transforming data towards normality may further improve the estimation.
An explanation for the PROBIT method not estimating the prevalence of SAM as well as the classical method may be that perhaps the tail of the distribution of WHZ or MUAC does not follow the normal distribution and relates to children who may have other health problems in addition to primary malnutrition. One could argue that SAM children do not predictably follow the general pattern due to fact that they are often infected or suffer from a family crisis which makes them shift in an unpredictable way.
In conclusion, the PROBIT method could be useful in sentinel-site surveillance systems using repeated small sample surveys or small spatially stratified samples so as to allow the course mapping of prevalence. The classical method should be preferred when estimating prevalence with larger samples.
Acknowledgements
This work was supported by the Health and Nutrition Tracking Service. All co-authors have seen and agree with the contents of the manuscript and there is no conflict of interest to report. N.M.D. contributed to the conception and design, analysis and interpretation of data and drafting of the manuscript. M.M. participated in the design of the study, performed the data analysis and revised the manuscript critically for important intellectual content. C.P. and A.B. participated in the design of the study, analysis and interpretation of the data and revised the manuscript critically for important intellectual content.
