To investigate how diet impacts health in epidemiological studies requires assessment tools that are capable of estimating long-term dietary intake, yet inexpensive and simple enough to administer to a large population( Reference Willett 1 ). Although the FFQ has these advantages, the cognitive challenges in filling out the FFQ, the potential omission of important food items and the reliance on generic memory may introduce error, limiting its ability to assess dietary intake( Reference Kristal, Peters and Potter 2 , Reference Smith 3 ). Therefore, the validity and reproducibility of an FFQ need to be confirmed in the population of its intended use, as the quantitative understanding of the tool's performance and limitations will be important in the interpretation of the future diet–health analysis( Reference Pisani, Faggiano and Krogh 4 ).
A vegetarian diet has been shown to protect against heart diseases( Reference Crowe, Appleby and Travis 5 ), diabetes( Reference Tonstad, Stewart and Oda 6 ) and other chronic degenerative diseases( Reference Huang, Yang and Zheng 7 ) in the West. However, relatively little is known about vegetarians in Asia who may consume a different diet from vegetarians in the West and from the non-vegetarian general population in Asia. To study how vegetarian diet affects health, we need a tool to assess the nutritional intake of such a dietary pattern. While several Taiwanese FFQ have been developed and validated( Reference Huang, Lee and Pan 8 – Reference Lyu, Lin and Chang 10 ), none has been designed with the vegetarian diet in mind. The current FFQ was modified from the FFQ used in the Nutrition and Health Survey in Taiwan (NAHSIT)( Reference Huang, Lee and Pan 8 ) to include more commonly consumed vegetarian foods. The present study aimed to examine the relative validity and reproducibility of the modified FFQ in a new cohort study with a high proportion of true vegetarians and part-time vegetarians in Taiwan.
Methods
The Tzu Chi Health Study
The Tzu Chi Health Study (TCHS) is the first cohort to look into diet and health outcomes in vegetarians in Taiwan and in Asia. It enrolled 6002 adults that came to the Buddhist Dalin Tzu Chi Hospital for a health examination from 2007 to 2009. The recruitment focused on Tzu Chi commissioners, a devoted group of volunteers to the Buddhist Tzu Chi Foundation who receive a free health examination at Tzu Chi hospitals every 2–3 years. Approximately one-third of participants were vegetarians, and 53% of omnivores may be considered part-time vegetarians as they reported regular consumption of a vegetarian diet without completely avoiding meat and fish. The participants were interviewed on lifestyle, medical history and FFQ. The institutional review board at the Buddhist Dalin Tzu Chi Hospital approved the cohort study and the validation sub-studies.
The FFQ
The TCHS FFQ was modified from the one used in NAHSIT( Reference Huang, Lee and Pan 8 ). The original FFQ contains only twenty-eight items on top of an open-ended section on staple foods (such as rice, noodles and bread). We modified the FFQ in several ways:
-
1. Multiple foods grouped into single items in the original FFQ were expanded into separate items.
-
2. More commonly consumed vegetarian products (such as meat analogues, gluten products and fermented soya products) were added in consultation with dietitians experienced with vegetarian dietary habits.
-
3. The open-ended staple food section was modified into individual items in the FFQ.
-
4. A portion size section was added for each item in the FFQ.
The modified FFQ contained sixty-four food items or food groups, and separate sections on cooking methods, dietary habits and the use of condiments and dietary supplements. The FFQ presented items by major food groups in the following order: fish, seafood and meat; eggs and dairy; soya and other vegetarian protein products; vegetables; fruits and nuts; staples (i.e. rice, noodles, bread, oats); beverages; and cooking oils. Each item in the FFQ had on average five accompanying photographs (the photographs were published by the Department of Health in Taiwan for portion-size education( 11 )). The research dietitians asked the participants to quantify the frequency (in terms of per day, per week, per month, or never) they consumed a food or food group, and then to quantify the amount of the food consumed compared with the photographs (i.e. similar portion, half or twice the amount). Prior to administering the FFQ, the dietitians asked questions to determine participants’ vegetarian status (vegetarian defined by not eating any meat, fish and animal flesh) and then skipped non-relevant sections to reduce participant burden.
Reproducibility study
The reproducibility of the FFQ was evaluated by comparing the baseline FFQ (FFQ1) with a second administration of the FFQ (FFQ2). Between March 2009 and February 2010, a total of 103 previously enrolled TCHS participants returned to Dalin Tzu Chi Hospital for a health examination and all of them agreed to be interviewed on the FFQ the second time. Nutrient intakes were calculated for both FFQ and compared to evaluate reproducibility. The mean time interval from FFQ1 to FFQ2 was 23·2 (sd 3·6) months.
Diet record-validity study
The relative validity of the FFQ was assessed by comparison with 3 d of diet records (DR). From February to June of 2011, 101 TCHS participants (not overlapping with the reproducibility study participants) returning to Dalin Tzu Chi Hospital for the second health examination were invited to participate in the DR-validity study. Of these, seventy-eight (77%) agreed to participate and completed the instructed dietary records. The research dietitian instructed the participants on keeping 3 d of DR, including two weekdays and one weekend day over a 2-week period. The participants were taught to record in a structured booklet the details of everything they consumed, including the time and location, the brands for commercial items, the names of the food or the dishes, the ingredients, the food preparation method and the portion size, with reference to standard household measurements such as bowls, cups, spoons and reference objects. A few participants who did not have the ability to record diet by themselves received assistance from their daughters-in-law who lived with the participants and prepared their meals. The research dietitian made follow-up telephone calls to clarify any recording that appeared ambiguous or lacking the required detail description. Nutrient intakes as assessed by the FFQ were compared with those obtained by the DR to evaluate for relative validity. The mean time interval from FFQ1 to DR was 30 (sd 9; range: 21–54) months.
Biomarker-validity study
The FFQ's relative validity for folate and vitamin B12 was further assessed by comparing nutrient intakes calculated from the FFQ with the levels of serum folate and vitamin B12 in a subgroup of 1528 participants. These participants were selected at a ratio of two omnivores to one vegetarian among those enrolled during the early phase of the cohort (October 2007 to June 2008). All participants completed the FFQ at enrolment and had fasting blood drawn the next morning. Concentrations of serum folate and vitamin B12 were measured using a commercial RIA kit (Siemens, Malvern, PA, USA) at the Radioimmunoassay Laboratory, Department of Nuclear Medicine of the Chiu Clinic in Taiwan. As the blood for these biomarkers was drawn one day after the baseline FFQ, there is no time lag between these two measurements.
Computation of nutrients
Nutrient analysis programs were developed to calculate intakes from the FFQ and the DR based on values in Taiwan's Food Composition Database( 12 ), with the addition of folate compiled by experts in Taiwan( Reference Lee, Lee and Wong 13 ). When food items could not be found within the database, nutrient values from other sources( Reference Gebhardt, Lemar and Pehrsson 14 – Reference Yang, He and Pan 16 ) were adopted. For FFQ items that represent food groups, we computed the average food composition of the relevant foods, weighted by the frequency of consumption based on the 2005–2008 NAHSIT.
Statistical analysis
Demographics of participants in the reproducibility/validity studies were compared with those of the entire TCHS cohort using the independent-samples t test (continuous variables) and the χ
2 test (categorical variables). Reproducibility and validity were assessed by the following methods. First, median nutrient intakes were compared by the Wilcoxon signed-rank test. Second, Spearman correlations were calculated for both crude and energy-adjusted nutrient intakes; we adjusted for energy by (i) the residual model(
Reference Willett and Stampfer
17
) for normally distributed nutrients (loge transformation applied when necessary to improve normality) and (ii) the nutrient density model (divided by energy intake) for nutrients that violated normality (defined as P value <0·05 on a one-sample Kolmogorov–Smirnov test) and could not be normalized with loge transformation. For the DR-validity study, as the 3 d of DR were not sufficient to capture the day-to-day variation in diet, we deattenuated for the within-person variation by multiplying the correlation coefficient by the factor
$$--><$>\sqrt {1\, + \,(\sigma _{\rm w}^{2} \,/\,\sigma _{\rm b}^{2} )\,/\,n} $$$
, where n is the average number of repeated DR,
$$--><$>\sigma _{\rm w}^{2} $$$
is the within-person variance and
$$--><$>\sigma _{\rm b}^{2} $$$
is the between-person variance calculated from the DR(
Reference Willett
18
). The deattenuation was carried out only for nutrients with normal distribution after energy adjustment. Third, cross-classification was used to evaluate the ability of the FFQ to rank order participants by quartiles of dietary intake. For the biomarker-validity study, we computed Spearman correlation (with and without energy adjustment by the residual model) and ‘actual values for surrogate categories’(
Reference Willett and Lenart
19
), in which the average of actual values of biomarkers for each quartile of FFQ intakes were computed. All statistical analyses were performed using the statistical software package IBM SPSS Statistics Version 17.
Results
The 103 participants in the reproducibility study represented the total TCHS cohort study well in terms of demographics, dietary patterns and BMI. The seventy-eight DR-validity study participants, in comparison, were slightly younger, had a higher proportion of females and of those with a middle/high-school education, while the dietary pattern and BMI did not differ significantly (Table 1). The 1528 biomarker-validity study participants were 0·1 years younger, with 3·8% more vegetarian compared with the cohort, and no significant difference seen for other variables.
Comparison of baseline characteristics of participants in the reproducibility, diet record (DR)-validity and biomarker-validity studies with those of the cohort study
*P value < 0·05 for comparison with Tzu Chi Health Study participants.
For reproducibility (Table 2), FFQ1 tended to show slightly higher intake than FFQ2 for most nutrients but the differences were small (<20% for most nutrients) and statistically insignificant, except for folate and vitamin B12. The energy-adjusted correlations for macronutrients ranged from 0·46 (protein) to 0·65 (fibre), and for micronutrients from 0·35 (riboflavin) to 0·67 (Fe). Inclusion of supplements decreased correlations for most vitamins and minerals but increased correlations for thiamin and riboflavin. The cross-classification showed that (on average) 40% and 81% of participants were classified into the same quartile and the same-and-adjacent quartiles, respectively. Energy adjustment improved the agreement slightly (data not shown). Misclassification into extreme quartiles was about 3–4% for both crude and energy-adjusted nutrients.
Daily intake estimates from, correlation between and cross-classification of the two FFQ in the reproducibility study
P25, 25th percentile; P75, 75th pecentile; RE, retinol equivalents; α-TE, α-tocopherol equivalents.
*P value < 0·05 in Wilcoxon signed-rank test.
†(FFQ1 – FFQ2)/FFQ2.
‡Energy adjustment by the density model due to non-normal distribution.
For the DR-validity study (Table 3), DR and FFQ showed comparable median intakes for most nutrients, except cholesterol, niacin, vitamin B6, vitamin B12 and vitamin E. Correlations were low for polyunsaturated fat, vitamin E and B vitamins, but reasonable for most macronutrients, minerals, vitamin C and vitamin A. Inclusion of supplemental sources of vitamins and minerals resulted in an increased correlation for Fe, decreased correlations for Zn, folate, vitamin B12 and vitamin E, and similar correlations for other nutrients. The overall mean correlation was similar whether supplements were included or not. Since only two participants recorded use of vitamin/mineral supplements in the DR, the median intakes with and without supplements were almost identical. In cross-classification, on average, 36% and 76% of participants were placed in the same quartile and same-and-adjacent quartiles, respectively. Adjustment for energy resulted in similar agreement rates: 35% and 76% for the same quartile and same-and-adjacent quartiles, respectively (data not shown). Misclassification into extreme quartiles was about 5% for both crude and energy-adjusted intakes.
Daily intake estimates from, correlation between and cross-classification of the FFQ and diet records (DR) in the DR-validity study
P25, 25th percentile; P75, 75th pecentile; RE, retinol equivalents; α-TE, α-tocopherol equivalents.
*P value < 0·05 in Wilcoxon signed-rank test.
†(FFQ – DR)/DR.
‡Deattenuation factor was calculated from energy-adjusted nutrients from DR.
§Energy adjustment by the density model due to non-normal distribution.
Table 4 shows the FFQ's relative validity to assess folate and vitamin B12 with biomarkers as the reference method. The correlation for FFQ folate and serum folate improved substantially from 0·29 to 0·41 after energy adjustment. Serum folate levels increased by 51·5% and 70·7% across the quartiles for crude and energy-adjusted FFQ folate intake, respectively. For vitamin B12, energy adjustment changed the correlation slightly from 0·40 to 0·41. Serum vitamin B12 levels increased by 71·6% and 86·6% across the quartiles for crude and energy-adjusted FFQ vitamin B12 intake, respectively.
Spearman correlation between FFQ intake and biomarkers, and mean values of biomarkers by quartile of FFQ intake, in the biomarker-validity study
*P < 0·001 for all correlations.
Discussion
The reproducibility of the FFQ is reasonable despite being administered two years apart. Our results are comparable to those observed in the Shanghai Women's Health Study (SWHS)( Reference Shu, Yang and Jin 20 ), which had crude correlations ranging from 0·48 to 0·51 for macronutrients and from 0·33 to 0·39 for vitamins; and the Shanghai Men's Health Study (SMHS)( Reference Villegas, Yang and Liu 21 ), in which crude correlations were in the range of 0·39–0·53 for macronutrients and 0·38–0·52 for vitamins. Our correlations for most nutrients are lower than those reported in a Taiwanese meal-based FFQ by Lyu et al.( Reference Lyu, Lin and Chang 10 ), which had the two FFQ administered at a much shorter interval (7·5 months). Due to the longer time interval between our two FFQ administrations (2 years), our reported data may be affected by both the actual reproducibility of the questionnaire and the potential change in diet over time. The Dutch arm of the European Prospective Investigation into Cancer and Nutrition (EPIC) also observed a trend of decreasing reproducibility with increasing time lag between the two FFQ for all nutrients and food groups( Reference Ocke, Bueno-de-Mesquita and Goddijn 22 , Reference Ocke, Bueno-de-Mesquita and Pols 23 ).
The present FFQ has reasonable validity for energy and most macronutrients, except polyunsaturated fat. Our validity for energy, protein, carbohydrates and fibre (correlations: 0·47–0·66, same quartile agreement: 38–45%) are slightly lower than those in the SWHS (correlations: 0·52–0·66, same quartile agreement: 38–46%) but comparable to the SMHS (correlations: 0·48–0·58, same quartile agreement: 35–47%) and Taiwanese FFQ by Lee (correlations: 0·20–0·46)( Reference Lee, Pan and Liu 9 ) and Lyu (correlations: 0·36–0·67)( Reference Lyu, Lin and Chang 10 ). The low correlation of polyunsaturated fat likely results from challenges in assessing this nutrient. The main predictors for unsaturated fat in the Taiwanese population include several cooking oils such as soyabean oil and peanut oil( Reference Lee, Pan and Yu 24 ), which could vary considerably in the amount used in typical stir-fried dishes and would be difficult to estimate for those not involved in cooking. In addition, the surface area of the food (cutting into smaller pieces would increase the surface area and result in a coating of more oil) and the amount of juice in a dish eaten may also affect the amount of oil consumed( Reference Lee, Pan and Yu 24 ). These details are difficult to collect in self-reported data such as FFQ and DR, and thus require considerable assumptions from researchers. Other researchers have used fatty acid composition in adipose tissue as the reference method( Reference Knutsen, Fraser and Beeson 25 ), which may be useful when validating more specific types of fatty acids.
Among micronutrients, our FFQ is expected to provide good relative ranking for most minerals, vitamin C and vitamin A (similar or better correlation and agreement compared with other Chinese FFQ( Reference Lee, Pan and Liu 9 , Reference Lyu, Lin and Chang 10 , Reference Shu, Yang and Jin 20 , Reference Villegas, Yang and Liu 21 )), but we had low correlations for B vitamins and vitamin E. The low correlations for B vitamins may have resulted from higher within-person to between-person variation, as found in another Taiwanese study( Reference Lee, Pan and Liu 9 ). This suggests that more repeats of DR throughout a longer period may be needed to minimize random errors in the reference method. Moreover, the predicting foods for B vitamins may be different for vegetarians. For example, pork is a strong predicting food for thiamin in the Taiwanese population( Reference Pan, Lee and Yu 26 ), but is not eaten by vegetarians. Further studies to find better predicting foods for vegetarians are needed to improve the FFQ's ability to assess B vitamins. The low correlation for vitamin E may have been caused by missing values in the nutrient database.
Biomarker validation usually shows lower correlation and is considered to be the lower bound of questionnaire validity due to independence in sources of errors( Reference Jacques, Sulsky and Sadowski 27 ). Our correlations for folate (crude: 0·29, energy-adjusted: 0·41) are moderate, lower than in an American( Reference Jacques, Sulsky and Sadowski 27 ) and a Canadian( Reference Green, Allen and O'Connor 28 ) study, and higher than in two Japanese( Reference Iso, Moriyama and Yoshino 29 , Reference Yoshino, Nishide and Sankai 30 ), two Dutch( Reference van de Rest, Durga and Verhoef 31 , Reference Verkleij-Hagoort, de Vries and Stegers 32 ) and one Jamaican( Reference Jackson, Walker and Younger 33 ) studies. This may be related to bioavailability, as the North American populations may have consumed more bioavailable folic acid from fortified food and supplements that is more readily reflected in serum levels. Our correlations for vitamin B12 (crude: 0·40, energy-adjusted: 0·41) are substantially higher than those reported by others( Reference Jacques, Sulsky and Sadowski 27 , Reference Green, Allen and O'Connor 28 , Reference Yoshino, Nishide and Sankai 30 , Reference Verkleij-Hagoort, de Vries and Stegers 32 ). This may have resulted from both good validity and the wide range of nutrient intake and nutrient status due to the high proportion of vegetarians. Similar to other biomarker validation studies( Reference Jacques, Sulsky and Sadowski 27 , Reference Bingham, Gill and Welch 34 ), we found energy-adjusted nutrients correlated better with biomarkers, suggesting that energy-adjusted nutrient intake may have greater biological relevance( Reference Willett 35 ).
FFQ validation is usually challenged by the lack of a gold standard reference method. The next best are those with different sources of errors, as non-independent measurement errors would tend to falsely elevate correlation( Reference Willett, Sampson and Stampfer 36 ). DR may have less correlated errors with FFQ than does 24 h recall, as it allows direct measurement of portion size and does not rely on memory( Reference Willett, Sampson and Stampfer 36 ). However, it may be difficult for those with lower education, adding to selection bias. The assistance from family members enabled us to gather some data from the less educated, although not to the extent to be representative of the TCHS population. Although not perfect, our result is more generalizable to the present cohort than if we were to use the other Taiwanese FFQ( Reference Lee, Pan and Liu 9 , Reference Lyu, Lin and Chang 10 ), as our FFQ was validated among our cohort participants, while the other FFQ were validated among nutrition students or university staff/students.
Our DR validation is hampered by the inaccurate reference method used. The DR were obtained for only 3 d within a short period of time, which could hardly capture foods and nutritional supplements consumed episodically and nutrients with a high ratio of within-person to between-person variation, leading to potential underestimation for validity. In addition, comparing baseline FFQ (which assessed previous diet) with DR obtained 30 months post FFQ is of incorrect time frame. Potential dietary changes throughout this long duration would likely decrease the correlations. Although these limitations would both have led to underestimation of the FFQ's performance, our results are still comparable to those from other Chinese FFQ that had more replicates of dietary records or 24 h recalls completed across the span of 6–12 months prior to the compared FFQ(( Reference Lee, Pan and Liu 9 , Reference Lyu, Lin and Chang 10 , Reference Shu, Yang and Jin 20 , Reference Villegas, Yang and Liu 21 ).
The additional biomarker validation has enabled us to overcome the limitations of DR and further ascertain the relative validity of folate and vitamin B12, both otherwise shown to be low–moderate in the DR-validity study. The higher correlations for biomarker validation (the supposed lower bound for validity) than in DR validation confirm that our estimation from DR validation is a severe underestimation and a conservative estimation of the real validity of the present FFQ.
Conclusion
Our study suggests that the FFQ has acceptable reproducibility and relative validity for ranking intakes of energy and major nutrients among TCHS participants. For the first time, an FFQ for assessing nutrient intakes of Taiwanese and Asian vegetarians is available.
Acknowledgements
Sources of funding: The TCHS received grants (TCRD-I9605-01 and TCRD-I9605-02) from the Buddhist Tzu Chi Hospital which covered the expenses for baseline data collection of the cohort study and the biomarkers, while the DR-validity study did not receive any extra funding. Conflicts of interest: None of the authors have conflicts of interests. Authors’ contributions: study design by T.H.T.C., M.-N.L., K.-J.C. and C.-L.L.; coordination and data collection by M.-N.L., H.-Y.H. and Y.-R.W.; data computation, analysis and interpretation by T.H.T.C., J.P.C.C., Y.-H.L. and B.C.-H.C.; writing of the manuscript by T.H.T.C. All authors contributed to the final version of this report. Acknowledgements: The authors would like to thank Dr Wen Harn Pan of Academia Sinica in Taiwan for providing the database of food intake in NAHSIT, the consumption frequency of which was used as the basis to assign weight distribution for our FFQ items; Dr Rwei-Fen S. Huang from the Department of Nutrition in Catholic Fu-Jen University, for sharing the folate database; and Dr Meei Shyuan Lee from the School of Public Health, National Defense Medical Center in Taiwan, for consultation on computation of FFQ.
