To assess and monitor the eating habits of large populations of adolescents, FFQ provide a reproducible and relatively inexpensive tool for repeated measurements(Reference Willett1). Validation of new FFQ is important because poor nutrient intake data may lead to false associations between diet and disease. Validation studies assess whether: the FFQ is measuring what it should measure; the degree to which the FFQ agrees with other established dietary intake measurement methods; the level of measurement error associated with use of the FFQ; and to estimate the necessary sample size for studies examining the role of diet in disease(Reference Cade, Thompson, Burley and Warm2).
A limited number of validation studies of FFQ for adolescents have been conducted internationally(Reference Frank, Nicklas, Webber, Major, Miller and Berenson3–Reference Andersen, Nes, Lillegaard, Sandstad, Bjorneboe and Drevon8), with a minority in developing countries(Reference Slater, Philippi, Fisberg and Latorre9). Results generally show a strong agreement in test–retest for foods(Reference Frank, Nicklas, Webber, Major, Miller and Berenson3) and nutrients(Reference Rockett, Wolf and Colditz4–Reference Field, Peterson, Gortmaker, Cheung, Rockett, Fox and Colditz6, Reference Andersen, Nes, Lillegaard, Sandstad, Bjorneboe and Drevon8), but often a high variability in the dietary intake of adolescents(Reference Rockett, Breitenbach, Frazier, Witschi, Wolf, Field and Colditz5, Reference Slater, Philippi, Fisberg and Latorre9).
Only one study to date has assessed the validity and reliability of an FFQ for an adult population in Ho Chi Minh City, Vietnam(Reference Kusama, Le, Hanh, Takahashi, Hung, Yoshiike and Yamamoto10) and none have been reported for adolescents in Vietnam.
We developed a new FFQ to assess dietary intake in an urban adolescent population in Vietnam for use in a cohort study examining risk factors related to excess weight gain. Although the initial use of this FFQ will focus on macronutrient intake, it was developed to allow measurement of a wide range of nutrients for use in future studies of diet and health in adolescents in urban Vietnam. The aim of the present study was to assess reliability and validity of nutrient intake measured with this new FFQ, comparing results to those obtained from 24 h recalls.
Materials and methods
The FFQ was administered three times in Ho Chi Minh City, Vietnam: FFQ1 in early December 2003, FFQ2 in January 2004 (4 weeks after FFQ1) and FFQ3 in late June 2004 (6 months after FFQ1). The validity of the instrument was evaluated by comparing mean nutrient intake data obtained from FFQ1 and FFQ3 with mean nutrient intake data obtained from four 24 h recalls (collected every 5 weeks from December 2003 to June 2004). The mean of FFQ1 and FFQ3 was selected as the main validation approach, because the reference period for FFQ1 did not correspond to the time period covered by the 24 h recall comparison method, and using FFQ1 alone might have produced apparently lower validity results. Although the reference period for FFQ3 corresponded to the time period covered by the 24 h recall comparison method, the repeated questioning about diet prior to administering FFQ3 was expected to have enhanced the skills of the respondents to answer questions about diet. Thus using FFQ3 alone in the validation analysis might have produced apparently higher validity results. For this reason, we assessed the performance of the FFQ by taking the average nutrient intakes from FFQ1 and FFQ3. However, we did also compare the nutrient intake data obtained from FFQ1 alone, and FFQ3 alone, with the mean nutrient intake data obtained from the four 24 h recalls.
The sample size for the study was calculated using a standard formula for correlation coefficients(Reference Willett and Lenart11). The number of participants required was about 110. However, a sample size of 150 to 200 subjects has been reported to provide sufficient precision for FFQ validation studies(Reference Cade, Thompson, Burley and Warm2). Thus, 180 subjects aged 11–15 years were randomly chosen for the validation study, with the sample divided between the wealthy and less wealthy districts. Subjects were recruited from three schools that were randomly selected from the list of junior high schools in urban areas of Ho Chi Minh City: two from wealthy districts and one from a poorer district. In each school, classes were randomly selected from the lists consisting of grades six, seven and eight. In each grade, one class was chosen, and in the nine selected classes, twenty students were randomly selected.
Study instruments
FFQ
The FFQ with 6 months recall was prepared by obtaining open-ended data via the method described by Buzzard(Reference Buzzard, Stanton, Figueiredo, Fries, Nicholson, Hogan and Danish7) in fifty junior high-school students in urban districts, who were interviewed about food and drink consumed during the 24 h time period prior to interview(Reference Willett1). Energy, protein, carbohydrate and fat content of most foods were computed using EIYOKUN version 1(Reference Hanh, Yoshimura, Takahashi, Kusama, Hanh, Chuyen and Yamamoto12), a nutrient database developed from Vietnamese food consumption tables(13). The remaining nutrient data were computed using the composition tables for common foods found in southern Vietnam(14). A forward stepwise multiple linear regression model was used to rank nutrients in terms of contribution to between-person variation(Reference Willett1), with increased percentage variance expressed as a cumulative R 2 value derived from the addition of a food to a progressively larger list of food items. Foods in this model that explained at least 80 % of between-person variability were considered for the FFQ(Reference Shai, Shahar, Vardi and Fraser15, Reference Shahar, Shai, Vardi, Brener-Azrad and Fraser16), with thirty-nine foods being selected via stepwise regression and accounting for 49 % of energy intake. Additional food items derived from another validated FFQ (developed for use in adults)(Reference Kusama, Le, Hanh, Takahashi, Hung, Yoshiike and Yamamoto10) were also included in the FFQ, resulting in a preliminary list of 170 food items.
Based on the ‘Composition of 400 common foods’ prepared by the Nutrition Centre of Ho Chi Minh City(14), these food items were divided into eight groups: (i) processed foods; (ii) rice, breads and cereals; (iii) meat, fish and seafood; (iv) fruits and vegetables; (v) sweets and snacks; (vi) milk and dairy; (vii) drinks; and (viii) miscellaneous. The study time frame was set at 6 months to account for seasonal variation of foods.
A picture book of full-sized photographs illustrating common portion sizes for each food was used in conjunction with the FFQ (this photo album is available on request to the corresponding author). As the participants in our study were junior high-school students having eating habits very similar to adults, we used the portion sizes that have been described and evaluated for use in Vietnamese adults(Reference Kusama, Le, Hanh, Takahashi, Hung, Yoshiike and Yamamoto10). We used multiple portion sizes in this FFQ, as various foods that have different ways of cooking were included. Determination of portion sizes was mainly derived from the results of 24 h recalls (prior to the validation study in order to develop the FFQ), and partly from the composition of commonly prepared foods(14).
The weight of a cooked food was calculated by dividing its cooked, processed weight by its uncooked, processed weight (processed meaning cleaned, peeled and cut), to obtain a ‘transferred index’, which was used to calculate nutrient data using uncooked processed weight figures from the EIYOKUN database(Reference Hanh, Yoshimura, Takahashi, Kusama, Hanh, Chuyen and Yamamoto12). Average daily nutrient intake contributed by each food was based on: frequency of consumption, mean amount eaten and nutrients per gram; and total daily nutrient intake was calculated by summing individual food values.
This preliminary FFQ was trialled in twenty junior high-school students. Following three group (ten males and ten females per group) discussions, some foods were deleted, two snacks were added and one frequency option was omitted (greater than four times per day), resulting in a final FFQ containing 160 food items.
The FFQ was administered by trained interviewers from the Pham Ngoc Thach University of Medicine (PNTUM), during class time over approximately 30 min. Participants were asked to include any foods regularly consumed which were not listed in the questionnaire.
24 h recall
Each 24 h recall was also conducted in-person by trained staff from PNTUM: three recalls were on weekdays and one on a weekend day. Participants were asked to recall what they ate and drank for the 24 h prior to interview, stating frequency and amount. Accuracy of the latter was increased by showing pictures of utensils such as glasses, bowls and spoons to illustrate standard measurements, as well as using a ruler to help students estimate portion size.
Statistical methods
Data were analysed using the STATA statistical software package version 9 (Stata Corporation, College Station, TX, USA). Nutrient intakes were initially loge transformed to improve the normality of data because most nutrient distributions were right-skewed towards higher values. Three food records were excluded due to mean energy intake being <2093 kJ (500 kcal)/d or >20 930 kJ (5000 kcal)/d(Reference Rockett, Breitenbach, Frazier, Witschi, Wolf, Field and Colditz5).
Pearson product–moment correlation coefficients, i.e. observed coefficients, were calculated(Reference Willett and Lenart11) to assess short-term reliability (FFQ1 v. FFQ2), long-term reliability (FFQ1 v. FFQ3) and validity.
Energy-adjusted nutrient intakes were computed using the residuals method described by Willett(Reference Willett1), with coefficients being adjusted for residual within-person variability in the four 24 h recalls such that correlations between nutrient intakes derived from this comparison method v. the FFQ method would be attenuated(Reference Rockett, Breitenbach, Frazier, Witschi, Wolf, Field and Colditz5).
Bland–Altman analysis was applied using graphs to evaluate the level of agreement between the two methods of assessing nutrient intake(Reference Cade, Thompson, Burley and Warm2), with differences plotted against means of paired intake values. If the difference between paired values was approximately equal to 1sd of the mean of the paired values, the width of the limits of agreement (LOA) was considered ‘good’; 2sd, ‘fair’; and 3sd, ‘poor’.
As recommended(Reference Masson, McNeill, Tomany, Simpson, Peace, Wei, Grubb and Bolton-Smith17, Reference Flood, Smith, Webb and Mitchell18), misclassification error was assessed by dividing mean intake values derived from the three FFQ and four 24 h recalls into quintiles. Gross misclassification was defined as when the mean nutrient intake was categorised into opposite lowest/highest quintiles by the two methods (i.e. four quintiles apart). Quadratic-weighted kappa statistics compared quintiles of nutrient intake: FFQ1 v. FFQ2; FFQ1 v. FFQ3; mean of the three FFQ v. mean of the four 24 h recalls. The weighted kappa was assessed according to the guide provided by Landis and Koch(Reference Landis and Koch19).
Results
All students (46 % male and 54 % female, aged 11–14 years) completed the three FFQ and the four 24 h recalls. The mean weight, height and BMI of the participants were 43·0 kg, 151·4 cm and 18·6 kg/m2, respectively. The difference between boys and girls were not significant.
FFQ reliability for nutrients
Short-term reliability (FFQ1 v. FFQ2)
Coefficients after energy adjustment ranged from 0·22 for retinol to 0·78 for fibre. Percentage of subjects correctly classified within one quintile was highest for zinc (91 %) and calcium (88 %) and lowest for retinol (70 %) and fat (73 %). Percentage of subjects grossly misclassified was highest for retinol (6 %) and protein (5·5 %). Weighted kappa values showed mostly ‘fair’ to ‘good’ agreement, with the lowest for retinol (0·24) and highest for zinc (0·72; see Table 1).
Energy and nutrient intakes were assessed for reliability using Pearson correlation coefficients and joint classification of subjects by quintiles of nutrient intakes between baseline FFQ1 and 4-week FFQ2, as well as baseline FFQ1 and 6-month FFQ3, respectivelyFootnote *
* Three cases were excluded because the energy intakes from the FFQ1 exceeded 20 930 kJ (5000 kcal).
† Energy-adjusted correlation coefficients.
Long-term reliability (FFQ1 v. FFQ3)
Mean reported nutrient intakes from FFQ3 were not significantly lower than FFQ1, with the exception of energy, protein, calcium, carotene, palmitic acid, stearic acid, linolenic acid and cholesterol (data not shown). Coefficients after energy adjustment ranged from 0·30 for retinol to 0·81 for zinc. Percentage of subjects correctly classified within one quintile was highest for zinc (88 %) and calcium (88 %). Percentage of subjects grossly misclassified was lowest for carbohydrate (0 %) and highest for retinol (6 %). Weighted kappa values showed mostly ‘fair’ to ‘good’ agreement, again with the lowest for retinol (0·34) and highest for zinc (0·73).
FFQ v. 24 h recall
Table 2 illustrates the differences in nutrient intakes estimated by FFQ1 and 3 and the four 24 h recalls (P < 0·05). The mean values from the FFQ were similar to the 24 h recalls except for protein, carbohydrate, retinol, thiamin, riboflavin, sodium, palmitic acid, stearic acid, linolenic acid and cholesterol, which were all significantly higher for FFQ.
Energy and nutrient intakes of the average of FFQ and the average of four 24 h recalls and validity was assessed by Pearson correlation coefficients, joint classification of subjects by quintiles of nutrient intakes between the FFQ and the four 24 h recalls administered over 6 months in a sample of 177 junior high-school studentsFootnote *
* Three cases were excluded because the energy intakes from the FFQ1 exceeded 20 930 kJ (5000 kcal).
† Pearson correlation coefficients.
Energy-adjusted coefficients between the FFQ1 and 3 and the four 24 h recalls were slightly higher than the unadjusted coefficients, except for fibre, linoleic acid, vitamin C, thiamin, riboflavin, sodium and linolenic acid. For unadjusted nutrients, coefficients ranged from 0·32 for stearic acid to 0·64 for fibre, with a mean of 0·42. After correction for within-person variation, the de-attenuated coefficients ranged from 0·33 for stearic acid to 0·63 for fibre, with a mean of 0·43. All coefficients were statistically significant.
Table 2 also shows that the proportion of subjects correctly classified within one quintile food category was highest for carbohydrate (80 %) and lowest for copper (60 %). Weighted kappa values showed mostly ‘fair’ to ‘good’ agreement for nutrient categories, with the lowest for lipid (0·28) and highest for cholesterol (0·66).
As seen in Table 3, except for fibre, vitamin C and linoleic acid, a significant linear relationship was found between the difference and the average of the two methods for all nutrients. The mean agreement between average of FFQ and four 24 h recalls ranged from 56 % for sodium to 130 % for zinc. Except for protein, lipid, carbohydrate, iron, thiamin, riboflavin, sodium, cholesterol, the FFQ could under- or overestimate 24 h recalls. The narrowest LOA was found for carbohydrate and widest for retinol.
Limits of agreementFootnote * between average of FFQ and the four 24 h recalls and the slopeFootnote † with 95 % CI for a linear regression of the difference against the means of the two method administered over 6 months in a sample of 177 junior high-school studentsFootnote ‡
* All nutrients were log-transformed before being examined the agreement.
† Slope of linear regression line, where difference in methods = intercept + coefficient (average of methods).
‡ Three cases were excluded because the energy intakes from the FFQ1 exceeded 20 390 kJ (5000 kcal).
Bland–Altman plots comparing mean nutrient intakes estimated by the three FFQ v. the four 24 h recalls were examined, although only plots for protein are presented. Figure 1 shows a mean difference of −2·76 for protein, indicating that the FFQ recorded lower levels of protein intake than the 24 h recalls. The regression line revealed no significant linear trend and indicated no need to use log-transformed nutrients to interpret the LOA. The LOA for protein were approximately equal to 2sd, suggesting ‘fair’ agreement.
Bland–Altman plot to assess validity of protein intake estimated by the means of FFQ1–3 and the means of the four 24 h recalls. LOA, limits of agreement
Coefficients for different genders showed a mean correlation before energy adjustment of 0·38 for boys and 0·43 for girls. After adjustment for energy and de-attenuation, mean correlations were 0·39 and 0·43, respectively, for boys; and 0·45 and 0·51, respectively, for girls. Coefficients between the two methods of assessing nutrient intake were usually higher for girls than for boys, but there were no consistent patterns between the two genders (data not shown).
Similar findings to those reported above for the average of FFQ1 and FFQ3 were found when FFQ1 alone and FFQ3 alone were compared with the 24 h recalls (data not shown). The correlations between the post-recall FFQ (FFQ3) and the 24 h recalls were slightly higher than those between the baseline FFQ (FFQ1) and the 24 h recalls, for all three types of correlation (before adjustment, after adjustment and de-attenuation).
Discussion
The present study showed a mostly fair to good correlation between repeated administrations of a novel FFQ, suggesting that this instrument is a reliable method of assessing nutrient intake in adolescents. Results also indicate that this FFQ is a valid tool, based on the Bland–Altman plots and coefficients comparing the FFQ method with the 24 h recall method. Coefficients were mostly around 0·40, with higher correlations for energy-adjusted nutrients. After correcting for within-person variation in the 24 h recalls, coefficients were higher, with a mean of 0·49 for macronutrients and 0·41 for micronutrients. The relatively high proportion of nutrient intakes classified within one quintile, and the small number grossly misclassified, suggests that the FFQ is capable of adequately ranking the nutrient intake of different subjects.
The short-term reliability of this FFQ was high for almost all nutrients, except retinol due to high within-person variability. It is not possible to compare short-term reliability of the present study with other studies of FFQ in adolescents due to the absence of reporting it. As expected, the long-term reliability of the FFQ was lower than the short-term reliability for most nutrients, and was similar to that reported in other studies(Reference Rockett, Wolf and Colditz4, Reference Andersen, Nes, Lillegaard, Sandstad, Bjorneboe and Drevon8).
Coefficients for crude nutrients ranged from 0·32 for stearic acid to 0·64 for fibre, suggesting reasonably valid estimates of crude energy and nutrient intakes. Similar correlations were found when comparing FFQ1 and FFQ3 with the mean values derived from the four 24 h recalls. Such correlations were generally higher than those in other validation studies conducted in adolescents(Reference Frank, Nicklas, Webber, Major, Miller and Berenson3, Reference Rockett, Breitenbach, Frazier, Witschi, Wolf, Field and Colditz5, Reference Field, Peterson, Gortmaker, Cheung, Rockett, Fox and Colditz6, Reference Slater, Philippi, Fisberg and Latorre9), possibly because our FFQ was administered in-person, rather than self-administered, and subjects were asked to include any foods regularly consumed which were not listed in the FFQ.
Energy intake was positively correlated with individual nutrient consumption. In these circumstances, if energy intake is associated (although not causally associated) with disease, the effects of specific nutrients may be confounded by the individual’s total energy intake(Reference Willett1) and hence nutrient intakes should be adjusted for energy. In the present study, after energy adjustment we found higher correlations for most nutrients, except for total, saturated and unsaturated fats, with the nutrients measured by the comparison method. These higher correlations may be explained by a decrease in correlated measurement error for total energy and for macronutrients that exceeded the reduction in between-person variation for nutrient intake as a consequence of controlling for total energy intake(Reference Rockett, Breitenbach, Frazier, Witschi, Wolf, Field and Colditz5). Some validation studies have reported similar increased correlations(Reference Rockett, Breitenbach, Frazier, Witschi, Wolf, Field and Colditz5) as we found, but others have reported decreased correlation coefficients(Reference Slater, Philippi, Fisberg and Latorre9) after energy adjustment. This could be because subjects may not have reported foods rich in nutrients such as fat in the same way during assessment for both dietary intake methods.
Within-person variability in the recalls would reduce the correlations between nutrients in the two methods, because some of the comparison measurements would be much higher or lower than usual long-term intake. Similar to other studies, we observed that adjustment for within-person variability in the reference recall method increased all coefficients(Reference Rockett, Breitenbach, Frazier, Witschi, Wolf, Field and Colditz5, Reference Field, Peterson, Gortmaker, Cheung, Rockett, Fox and Colditz6, Reference Slater, Philippi, Fisberg and Latorre9).
As suggested in a consensus statement on methods for assessing FFQ(Reference Cade, Thompson, Burley and Warm2), we used Bland–Altman plots to assess the level of agreement between the FFQ and recalls. We compared the widths of the LOA from the FFQ with nutrient intake distribution derived from recalls, assuming if the LOA was too wide, FFQ may misclassify individuals when ranked by quintiles in comparison to recalls. We found differences ranging between 1sd and 2sd compared to recalls, indicating that the FFQ is accurate in assessing nutrient intake at a group level. Other studies have found similar variation between methods of assessing nutrient intake(Reference Andersen, Nes, Lillegaard, Sandstad, Bjorneboe and Drevon8, Reference Flood, Smith, Webb and Mitchell18, Reference Lietz, Barton, Longbottom and Anderson20, Reference Montomoli, Gonnelli, Giacchi, Mattei, Cuda, Rossi and Gennari21). The widest LOA found for retinol in the present study can be explained by the difficulty in estimation of such a nutrient because it may be highly concentrated in some foods.
We also examined agreement between methodologies via quintiles: the highest percentage grossly misclassified was for total, saturated and unsaturated fats, findings which corroborate correlation figures and may be explained by the occurrence of Tet (New Year) holidays within the study duration (a festive period where many foods high in fat are ingested). While the recalls only investigated short periods of dietary intake and did not ask about foods common to Tet holidays, the FFQ covered longer periods and asked about foods common to Tet holidays.
There were a number of potential limitations and biases in the present study. Firstly, the 24 h recall reference method used in the present study, like the FFQ, relies on memory for identification of foods and portion sizes(Reference Buzzard22), which may result in errors such as under- or overestimation of food quantities(Reference Slater, Philippi, Fisberg and Latorre9). However, repeated 24 h recalls are less demanding for subjects and can minimise respondent burden, and as such have been commonly used as the reference method in other validation studies with adolescents. Secondly, the study was conducted over 6 months rather than across a full year. Eating patterns may be different within the 6-month period from the rest of the year and the coverage of Tet holidays described above is an example. Third, research was only carried out in urban areas and it is yet to be determined whether the FFQ will be as accurate in rural areas. Many kinds of fast food are not available in rural areas, so the tool should be re-validated before being used in a rural setting.
In conclusion, the validity of our novel FFQ in estimating intake of many nutrients, including crude energy and macronutrients, was relatively high. After being adjusted for energy, as well as corrected for within-person variability in the 24 h recall reference method, reliability and validity were increased. Findings suggest that this FFQ could be applied in epidemiological studies of diet and lifestyle in adolescents living in Ho Chi Minh City and other urban areas to adequately rank nutrient intakes at a group level.
Acknowledgements
Source of funding: Data collection was conducted with a grant from the Nestlé Foundation (Switzerland). Conflict of interest declaration: The authors declare that they have no conflicts of interest. No author has any financial or personal relationships with the organisation sponsoring this research. Author contributions: T.K.H. conducted the data analysis and prepared the manuscript; M.J.D. contributed to the analysis of data and preparation of the manuscript; D.S. provided advice about the analysis of data and preparation of the manuscript. T.K.H. conducted the analyses and prepared this report while studying for a PhD at the University of Newcastle with a scholarship provided by the Ministry of Education and Training, Government of Vietnam. Acknowledgements: The authors would like to thank Dr Ho Thi Kim Lien and Dr Pham Lan Tran, Department of Community Health, PNTUM, Ho Chi Minh City for their helpful support in data collection and entry.
