The association between habitual dietary intake and chronic diseases has been measured by various dietary methods such as food records(Reference Hong, Song and Lee1, Reference Villani, Clifton and Keogh2), multiple 24 h dietary recalls(Reference Flegal, Carroll and Kit3, Reference Maillot and Drewnowski4) and FFQ(Reference Mekary, Rimm and Giovannucci5). Food records and multiple dietary recalls are accurate methods of assessing individuals’ nutritional status; but the cost, time, participants’ motivation and literacy are important factors limiting their use in large epidemiological studies. FFQ are time efficient, less expensive and are most commonly used to rank individuals by nutrient intake categories for assessing diet–disease relationships in large epidemiological studies. As the conceptual exposure in studies of chronic disease is long-term diet, the FFQ is suitable for measuring exposure as it assesses habitual dietary intake over 1 year. One of the main limitations is that an FFQ does not accurately measure absolute intake. However, the FFQ is able to assess relative intake, a property that is useful in diet–disease studies. Also, an FFQ developed for one population cannot be readily used in another population because different groups of people eat different foods. As such, FFQ are developed specifically for populations of interest. There is no single gold standard method for developing an FFQ or assessing its validity, but multiple dietary recalls are used as the reference method by 75 % of FFQ validation studies(Reference Cade, Thompson and Burley6).
We developed and validated an FFQ to be used for assessing the dietary intake of Chilean adults participating in an ongoing cohort called the Prospective Urban and Rural Epidemiological (PURE) study. As far as we are aware, PURE is the first study to construct and validate an FFQ for measuring the habitual dietary intake of adults in Chile. The main purpose of the present paper is to report the relative validity of this FFQ for ranking individuals based on dietary intake.
Materials and methods
Overall study design
The PURE study is a large ongoing prospective cohort being conducted in seventeen low-, middle- and high-income countries and has recruited approximately 153 996 men and women aged 35–70 years, of whom 3451 (2808 urban 643 rural) are from Chile. Within urban and rural settings, we recruited participants from low-, middle- and high-income areas. The main objective of the PURE study is to examine the association between societal influences on human lifestyle and risk factors of non-communicable diseases. The design and main findings of the PURE study have been reported previously(Reference Teo, Chow and Vaz7, Reference Yusuf, Islam and Chow8).
FFQ development
Overall, 100 volunteers residing in Temuco were enrolled in the FFQ development study. They were trained and asked to record their food intake for 24 h (construct validity). Then, the most commonly reported foods were compiled and the initial food list was created. The face validity and content validity of the food list were checked by two experienced nutritionists (M.D. and S.M.). Nutrient-rich foods that were not captured by the dietary recalls were added to the food list. The resulting FFQ consisted of a food list, portion size and frequency of consumption. The frequency of consumption was assessed using nine categories ranging from never to more than 6 times/d. Frequencies were formatted to recall food consumption during the previous year.
FFQ validation
Initially, seventy-six individuals voluntarily participated in the FFQ validation but only sixty-six of them completed the study. The average food and nutrient intakes from four 24 h dietary recalls were collected over 1 year (approximately 3 months apart) and used as the reference dietary intake. To ensure the quality of the data and completeness of reports, the four dietary recalls were administered by a nutritionist through structured interviews. On average each dietary recall interview took 1 h. During administration of the dietary recalls, recipes of mixed dishes were collected from women more involved in cooking at home. To reduce bias related to portion sizes, a photograph food atlas was used. The food atlas consisted of two parts: (i) photographs of single food items (such as milk, an apple or a piece of cheese); and (ii) photographs of mixed dishes (e.g. empanada, earthen chicken and corn humita). For single food items, we used Nelson et al.'s photographic atlas of food portion size(Reference Nelson, Atkinson and Meyer9). We followed Nelson's method for constructing the second section of the food atlas, which contained different portion sizes of each mixed dish.
Food composition database
To estimate daily energy, macro- and micronutrient intakes, a food composition database was required. As the tool is to be used for an international study, a food composition database containing nutrient estimates was developed that allows comparison between PURE countries. The nutrient database was based primarily on the US Department of Agriculture's food composition database and was modified appropriately with reference to Chilean food composition tables(Reference Merchant and Dehghan10). Based on the food's nutrient profile, the daily nutrient intake for each individual was calculated.
Food groups
All food items reported in the FFQ were grouped based on nutrient profile similarities; for example, milk, cheese and yoghurt were grouped as dairy products, while the vegetable group included all types of vegetables consumed (raw, cooked) and legumes. We did not include potatoes in the vegetables group. The frequency of consumption for some food items such as liquor was reported as never by the majority of participants; hence, we did not include such items in the related food group. In total, thirteen food groups were constructed.
Sociodemographic variables
Sociodemographic information was obtained at the first visit and trained research assistants measured the weight and height of participants. Body weight was measured with a digital scale to the nearest 100 g while participants wore no shoes and only light clothing; height was measured to the nearest 1 cm.
Ethics statement
The study received approval from Hamilton Health Sciences/McMaster Health Sciences Research Ethics Board and the Comité de Etica, Servicio de Salud Araucanía Sur, Temuco, Chile. Written informed consent was obtained from all participants in the study.
Statistical methods
We computed the mean and standard deviation of intake for each nutrient and food group as obtained from the FFQ and the dietary recalls separately. To improve the normality of the distribution, the data were log transformed. The validity of the FFQ was assessed by comparing the intake of each nutrient/food group estimated from the FFQ with that estimated from the average of the four 24 h dietary recalls using the Pearson correlation coefficient. Variation due to daily energy intake was removed by adjusting for total energy using the residual model(Reference Willett and Lenart11). Also, energy-adjusted de-attenuated correlations were calculated to remove the within-person variability(Reference McKeown, Day and Welch12). Relative agreement between the two dietary assessment methods was tested by cross-classification of the nutrient score and estimation of the proportion of participants classified by the two methods into the same and extreme quartiles. To remove the proportion of agreement that may have happened by chance we used the weighted kappa statistic (κ w) using the Fleiss–Cohen method(Reference Huybrechts, De Backer and De Bacquer13), calculated as:
$$ {{\kappa }_w}\: = \:\frac{{P{{r}_{\rm o}} (w){\rm{ - }}P{{r}_{\rm{e}}}(w)}}{{1{\rm{ - }}P{{r}_{\rm{e}}}(w)}},\eqno\rm$$where Pr o is the relative observed agreement between two raters, Pr e is the probability of agreement by chance and w is the computed weight for each quintile.
To assess the level of agreement between the FFQ and dietary recalls, Bland–Altman(Reference Bland and Altman14) analysis was performed for energy, macro- and some micronutrients. The difference between mean intakes estimated by the FFQ and dietary recalls was plotted against the average of mean intakes by the two methods for each participant and nutritional measure. It is expected that the mean difference (bias) lies between ±3 sd and a non-linear association indicates that the magnitude of error does not vary with the range of measurement. All statistical analyses were performed using the statistical software packages SAS version 9·1 and STATA version 10·0.
Results
Overall, seventy-six women and men participated in the study; however, ten people were excluded as they had only three dietary recalls. Table 1 shows the sociodemographic characteristics of the participants. The participants were aged 52·1 (9·2) years on average and had a mean BMI of 29·1 (5·1) kg/m2; the majority of them were married (65·0 %) and had secondary/high school education (52·0 %).
Sociodemographic characteristics of the participants in the FFQ validation study
Energy, macro- and micronutrients
The mean daily energy and nutrient intakes estimated by the average of the four 24 h dietary recalls and the FFQ, and comparisons of the two methods, are shown in Table 2. Compared with the average of four dietary recalls, the FFQ overestimated daily intakes of energy and all macro- and micronutrients. The crude correlation coefficients between the dietary recalls and FFQ varied from 0·14 (protein) to 0·43 (fat and MUFA). Energy adjustment improved the correlation coefficients for almost all nutrients except protein (0·07), thiamin (0·14) and folate (0·04). To correct day-to-day variation between dietary recalls, we computed the de-attenuated and energy-adjusted correlation coefficients. The de-attenuated and energy-adjusted correlation coefficient for most nutrients varied from 0·40 to 0·99 except for protein (0·10), thiamin (0·22) and folate (0·07). Cross-classification of nutrient intake categories obtained by the FFQ and dietary recalls into the same quartiles was highest for MUFA (82 %) and lowest for Na (64 %) and thiamin (64 %). The mean misclassification into extreme quartiles was 6·0 % and varied from 3·0 % (energy) to 12·0 % (PUFA). The κ w values showed a moderate agreement between the two dietary assessment methods except for protein (0·08), Na (0·10), thiamin (0·14), folate (0·15) and PUFA (0·15). The absolute agreement between macronutrient intakes estimated by the FFQ and dietary recalls was assessed using Bland–Altman plots (shown in Fig. 1). For all nutrients, the mean difference (bias) lay between ±3 sd and we observed a non-linear association between measurement error and average of mean. We found that the magnitude of measurement error did not vary with the range of measurements and the FFQ over/underestimation did not vary in a systematic way.
Mean daily nutrient intakes estimated by the average of four 24 h dietary recalls (DR) and the FFQ, and correlations between the two methods: men and women (n 66) participating in the FFQ validation study
*Indicates correct classification.
†Indicates misclassification.
Bland–Altman plots assessing the level of agreement between the FFQ and four 24 h dietary recalls among men and women (n 66) participating in the FFQ validation study. The difference between mean intakes estimated by the FFQ and dietary recalls (DR) was plotted against the average of mean intakes by the two methods for each participant and for: (a) energy; (b) protein; (c) carbohydrate; (d) total fat; (e) SFA; (f) PUFA; (g) fibre; (h) folate; (i) vitamin C; (j) sodium; (k) potassium; (l) calcium. ———, Mean difference; – – – –, 95 % limits of agreements
Food groups
Table 3 shows the correlation coefficients between daily food intakes estimated by the FFQ and the average of four 24 h dietary recalls. The crude correlations between the FFQ and dietary recalls varied from 0·04 (potatoes) to 0·49 (whole grains) and de-attenuation improved the correlation for most food groups; the highest correlation was found for margarine and butter (0·80) and the lowest for potatoes (0·11). For some food groups, energy adjustment improved the estimated correlation slightly, but energy-adjusted de-attenuated correlations were lower for vegetables, refined grains and non-alcoholic beverages than were energy-unadjusted correlations. The exact agreement in quartile categorization varied between 59·0 % (potatoes and non-alcoholic beverages) and 85·0 % (margarine and butter).
Mean daily food intakes estimated by the average of four 24 h dietary recalls (DR) and the FFQ, and correlations between the two methods: men and women (n 66) participating in the FFQ validation study
*Indicates correct classification.
†Indicates misclassification.
Discussion
We developed and evaluated the relative validity of a 109-food-item FFQ among women and men in Temuco, Chile. Compared with the mean of four 24 h dietary recalls, the de-attenuated and energy-adjusted correlation coefficients for the majority of nutrients exceeded 0·40. The average of classification into the same quartiles (correct classification) was approximately 70 % for macro- and micronutrients. For food groups, the highest energy-adjusted de-attenuated correlation was 0·75 for butter and margarine, and most correlation coefficients were greater than 0·40.
Overall, 166 individuals participated in the study (development n 100 and validation n 66). The results of the validation study are based sixty-six individuals who completed four 24 h dietary recalls. For FFQ validation, wide ranges of sample size have been reported in the literature (five to 3750)(Reference Cade, Thompson and Burley6), but the number of participants for most studies has been less than 200 individuals. To assess the absolute agreement using the Bland–Altman method, at least fifty, and preferably 100, participants are needed(Reference Cade, Thompson and Burley6). Although fewer numbers of people are motivated and agree to a 1-year commitment in less privileged populations, we enrolled sixty-six individuals for the validation study. Our sample size is similar to that of previous studies(Reference Pakseresht and Sharma15) and met the requirements of the Bland–Altman method.
Based on the main objectives of the study, the number of food items in an FFQ varies from five to more than 350 with a median of seventy-nine(Reference Cade, Thompson and Burley6) and our FFQ has 109 food items. Considering low food variation and availability in developing countries, the number of food items in our FFQ seems to be reasonable for assessing overall diet. Elongating the food list increases the time and cost of FFQ administration and provides little benefit in the accuracy of collected data(Reference Willett16).
Compared with energy and nutrient intakes estimated from the four dietary recalls, our FFQ overestimated energy and nutrients. Similar overestimations have been reported before(Reference Bohlscheid-Thomas, Hoting and Boeing17–Reference Paalanen, Mannisto and Virtanen19). Xia et al. validated an eighty-one-item FFQ and found that nutrient intakes estimated by FFQ were higher than those by dietary recalls except for protein and fat(Reference Xia, Sun and Zhang20). The overestimation may have occurred due to a long food list, difficulty in conceptualizing the assigned portion sizes and difficulties in reporting the frequencies of usual intake(Reference Jackson, Walker and Cade21).
The association between an FFQ and a comparison method is usually assessed by correlation coefficients and due to the various measurement errors for each dietary assessment method, the observed correlation coefficients are measures of relative validity. In our study, the de-attenuated and energy-adjusted correlations varied from 0·10 (protein) to 0·99 (MUFA), and were comparable to those in other studies that assessed FFQ validity in middle-income countries(Reference Jackson, Walker and Younger22, Reference Loy, Marhazlina and Nor23). Since we found a low correlation for protein, we computed the cross-classification for folate and protein into the same quartiles and found 89·4 % agreement for the FFQ and 77·3 % for the dietary recalls, highlighting the value of the FFQ in categorizing study participants into nutrient quartiles.
Adjusting for energy removes partial error related to energy intake and de-attenuation corrects for day-to-day within-person variation. Improvement of the correlation coefficients after de-attenuation supports the presence of high day-to-day variation of some nutrients. We observed low correlation coefficients for certain nutrients such as protein, which may be due to high day-to-day variation in intake and not necessarily a limitation of the FFQ.
For food groups, we observed high correlation for almost all foods, especially for margarine and butter, fruits, vegetables, whole grains and milk and dairy products. We observed poor correlation for potatoes, red meat and processed meats, white meat and sea foods. Similar to our findings, Hong et al.(Reference Hong, Choi and Lee24) reported poor correlations for four out of sixteen food groups and Hu et al.(Reference Hu, Rimm and Smith-Warner25) found poor correlations for organ meats, other vegetables, garlic and pizza. However, the FFQ is designed to rank individuals and based on the classification results, we noted that more than 65 % of individuals were correctly classified into the same quartiles.
Our study has some limitations. Biomarkers are the gold standard for some nutrients but due to feasibility and financial limitations, similar to most validation studies(Reference Cade, Thompson and Burley6), we chose the dietary recall as the reference method. Although we trained interviewers extensively, we acknowledge that both dietary recalls and FFQ have similar sources of error that may inflate the observed correlations. In order to reduce the number of attempts for contacting participants, we informed them in advance about the dates of dietary recall administrations. Hence, participants may have altered their food intake for those days. We collected only four dietary recalls and when habitual dietary intake is measured by a large number of dietary recalls, the estimated daily intake is a more accurate measure of true intake. Because the study was part of an ongoing cohort and the burden on participants was a limiting factor, we were unable to increase the number of dietary recalls.
Conclusions
The 109-item FFQ has moderate relative validity, similar to other FFQ validation studies, and can be used to rank individuals based on energy, nutrient and food intakes. The validation study was conducted in a unique setting and will be used for assessment of diet–disease associations by the PURE study.
Acknowledgements
Source of funding: This research received no specific grant from any funding agency in the public, commercial or not-for-profit sector. Conflict of interest: The authors declare that they have no conflicts of interest. Authors’ contributions: M.D. contributed to the design of the study, analysis plan, data interpretation and wrote the initial draft of the manuscript. X.Z. performed all statistical analyses. S.M., P.S. and F.L. performed the study in Chile and critically reviewed the manuscript. A.T.M. and S.I. contributed to the analysis plan, data interpretation and gave detailed comments on several versions of the manuscript. All authors approved the final version submitted for publication. Acknowledgements: The authors are grateful to the participants of the study. They also thank Ms Roxanna Solano for her assistance in data entry and data quality control; and Dr Salim Yusuf, Director of the Population Health Research Institute, for his support and guidance.
