Botswana has undergone a rapid epidemiological transition characterized by increased incidences of non-communicable diseases, namely CVD, diabetes and cancers(Reference Livingston1). Dietary intake is associated with morbidity and mortality from non-communicable diseases(Reference Beaglehole, Bonita and Horton2). Assessment of the usual food intake of individuals and groups is central to the investigation of eating patterns, nutrient intakes and their relationship to health outcomes, but the measurement of food and nutrient intakes can be challenging. The FFQ is used to obtain estimates of habitual diet and in contrast to other methods, such as single 24 h recalls and food records which do not reflect past diet or usual intake, has been used in a variety of settings to measure intakes in epidemiological studies. The FFQ ranks individuals according to levels of consumption rather than providing estimates of absolute quantities of energy and nutrient intakes. In addition, the technique has the advantage of being relatively easy to administer and analyse, thereby incurring lower costs compared with other methods of dietary assessment(Reference Willett3). The use and limitations of the FFQ have been extensively reviewed and the method can be prone to measurement errors; thus validation studies are an essential step in the development of new FFQ.
Whereas FFQ have been developed and validated for many developed and developing economies, there are few published reports of validated instruments designed to measure habitual diet in African populations, except for subgroups in South Africa(Reference MacIntyre, Venter and Vorster4–Reference Senekal, Steyn and Nel6), Mali(Reference Parr, Barikmo and Torheim7, Reference Torheim, Barikmo and Hatloy8) and Cameroon(Reference Sharma, Cade and Jackson9). Investigations of diet–disease relationships require that the dietary assessment method adequately estimates intake; hence it is essential that the FFQ is developed and validated in the population being studied in order to be culturally appropriate for that population(Reference Cassidy10). In the current paper we present the validity and reproducibility of a 122-item FFQ designed to evaluate habitual intakes of the adult population in Botswana for use in epidemiological studies of non-communicable chronic diseases.
Materials and methods
Development of the FFQ – Botswana
The quantitative FFQ was designed to assess habitual diet of adults in urban and rural areas over the previous year and categorize participants by intakes of food, energy and nutrients hypothesized to influence the development of chronic diseases.
Data collection
Two sources of data were used to identify food items to be included in the FFQ: food intake data obtained from focus group discussions and a large survey of dietary intakes of adults aged 18–75 years.
Focus group discussions. Nineteen focus group discussions were held in urban and rural areas in five geographical regions in Botswana (Gaborone, Gantsi, Maun, Kasane and Francistown). Data from the Central Statistics Office indicated that the regions consisted of populations that were representative of Botswana in its sociodemographic characteristics. Focus groups were organized by home economists together with agricultural demonstrators in each region. Each group consisted of ten persons (women and men) who were interviewed about foods commonly eaten in the area. Discussions were tailored to identify food items that were typically consumed, including ingredients used and methods of preparation. Information obtained also included the availability of foods during the wet and dry seasons, traditional and ceremonial dishes consumed and foods that were infrequently consumed.
Dietary intakes survey. Information on food intakes were obtained from an unpublished cross-sectional survey of adults aged 18–75 years (n 1426) that sought to provide data on procurement patterns, hunger scale and food consumption patterns.
Participants in the cross-sectional survey were enrolled by a multistage random cluster sampling technique using 2001 census data obtained from the Central Statistics Office. Data were collected from June to August 2006. A single 24 h recall provided information on dietary intakes. The FFQ also included other commonly used foods that were uniquely consumed in the geographic regions.
Relative validity
The relative validity of the FFQ was evaluated against repeated 24 h recalls (n 4).
A single 24 h recall was administered to each participant at 3-month intervals, yielding a total of four recalls for each participant over 1 year.
Sampling
The validation study was conducted in Kanye, a large village with mixed characteristics of urban and rural settings in southern Botswana, from September 2006 to August 2007. Participants were selected from those enrolled in the epidemiological survey of diet and weight status. The Central Statistics Office maps of urban and rural villages indicated the starting point and boundaries of the community. From the starting point, one adult from every second household included in the epidemiological study was systematically selected to provide information on dietary intakes. It was envisaged that the validation study would recruit a sample of ninety-six persons equally divided into eight age/sex categories; however, few men were available. Seventeen participants (six males; eleven females) who had only partially completed the study were excluded from the analyses; the final sample consisted of sixteen males and sixty-three females.
Eligibility criteria
Individuals were recruited to the study if they were 18–75 years old and not pregnant.
Data collection
One trained interviewer collected data on 24 h recalls and participants were unaware of the days the interview would be conducted. The interviewer requested participants to recall all foods and drinks consumed over the previous 24 h. Portions were carefully estimated by use of food models, household measures and utensils in conjunction with a detailed description of the food and method of preparation. The FFQ was administered at the beginning and at the end of the study period.
The FFQ was pretested and in its final form consisted of 122 food and drink items (see Appendix). Foods were grouped in ten categories on the basis of either physical composition (e.g. cereals, milk and milk products) or cultural use (e.g. desserts). Infant feeding foods, tsabana (sorghum–soya weaning food) and infant formula were included in the FFQ as they reflected eating habits of low-income households. Infant formula is used in hot beverages and tsabana is shared by the whole family. Frequency of usual food consumption was estimated using one of eight pre-coded categories of response that ranged from ‘almost never’ to ‘2 or more times per day’. For each food item, participants were asked to supply information on portion size by using food models, commonly used household utensils, measuring cups and a measuring tape to indicate, on average, the portion size usually consumed. All data were collected by trained personnel in English and Setswana (the common language of the country).
To determine the weight of the food items used for the estimation of standard weights in grams, dishes were prepared at the National Food Technology Research Centre and adults in nearby communities, using bowls or plates, were asked to serve themselves portions that reflected their usual intake. In Botswana, food is generally eaten from individual plates with a spoon; however, eating with hands is still done. The majority of households purchase foods from supermarkets and most foods (>80 %) are imported from South Africa. The weights of foods in varying household measures were obtained and used to estimate the weight of food in grams. Participants were asked to recall on average how much food and drink they consumed and the frequency with which food/drink was consumed.
Nutrient intakes
Daily nutrient intakes were calculated from the questionnaire by multiplying the frequency of consumption by the nutrient composition specified for each food item and its portion weight, using a computer program written for SPSS. In analysis, coefficients of 0·0, 0·03, 0·08, 0·14, 0·40, 0·80, 1·00 and 2·5 were used to indicate frequencies of ‘almost never’, ‘once per month’, ‘2–3 times per month’, ‘once per week’, ‘2–4 times per week’, ‘5–6 times per week’, ‘once per day’ and ‘2 or more times per day’, respectively. Nutrients from all foods were summed to obtain a total nutrient intake for each individual. Nutrient content of food items was calculated using the FoodFinder, a South African food composition database(11).
Reproducibility
The reproducibility of the instrument was determined by the test–retest method. Seventy-nine participants enrolled in the validity study repeated the FFQ one year later to assess the reproducibility of the instrument.
The study was approved by the Ethics Committee of the University of Botswana and the Ministry of Health, Botswana.
Statistical analyses
Non-parametric methods were used to describe nutrient and food intakes as most data were not normally distributed. The median and 25th and 75th percentiles of food and nutrient intakes were computed and differences between methods were tested with Wilcoxon's signed-rank test.
Nutrient intakes were adjusted for total energy by computing residuals from regression analyses with energy intake as the independent variable and nutrient intake as the dependent variable(Reference Willett3). Residuals were added to the expected nutrient value for the mean energy intake of the sample to obtain a score adjusted to the average energy intake. Spearman correlation coefficients were computed to determine the associations between the FFQ and 24 h recalls before and after adjustment for total energy intake. Method agreement and misclassification were expressed as the proportion of individuals classified respectively into the same and extreme quartiles of the distribution for a given nutrient intake. Weighted kappa statistics (κ w)(Reference Horn-Ross, Lee and Collins12) were calculated comparing quartiles of intake for each nutrient from the FFQ and 24 h recalls. The following values were used to evaluate agreement between the dietary methods: κ w ≥ 0·80 indicates very good agreement, κ w = 0·61–0·80 good agreement, κ w = 0·41–0·60 moderate agreement, κ w = 0·21–0·40 fair agreement and κ w ≤ 0·20 indicates poor agreement(Reference Jackson, Walker and Cade13).
To evaluate the reproducibility between the first and second estimation of the FFQ, Spearman correlation coefficients were used to assess the correspondence between the two measurements. Intra-class correlation coefficients were calculated as a measure of the extent of within- to between-subject agreement in nutrient intake between the two administrations of the questionnaire.
All analyses were performed using the statistical software package SPSS version 15·0. Statistical significance was achieved when P < 0·05.
Results
The characteristics of participants enrolled in the validity and reproducibility study are displayed in Table 1. More women (n 63) than men (n 16) were included in the study. Women had higher mean BMI on average and had a higher prevalence of obesity.
Characteristics of study participants enrolled in the Botswana FFQ reproducibility and validation study
Mean values were significantly different from those of men: *P < 0·05, **P < 0·0001.
Validity
The median daily intakes of energy and nutrients, estimated from the mean of the four 24 h recalls and from the two FFQ that were administered at the beginning (FFQ1) and end (FFQ2) of the 1-year validation study, are presented in Table 2. Intakes of energy, macronutrients and some micronutrients (Fe, retinol and β-carotene) were significantly higher on both FFQ than the 24 h recalls; dietary estimates of Ca, vitamin E and fibre were lower on FFQ1.
Daily energy and nutrient intake estimates (median and 25th, 75th percentile (P25, P75)) from the average of the four 24 h recalls and the first (FFQ1) and second (FFQ2) administrations of the Botswana FFQ, and Spearman correlation (coefficient r and 95 % confidence interval) between the two dietary methods
Median values were significantly different from those estimated from repeat recalls: *P ≤ 0·05, **P ≤ 0·001.
†Energy-adjusted correlations between dietary methods as suggested by Willett(Reference Willett3).
Spearman correlation coefficients for unadjusted and energy-adjusted nutrient intakes estimated from the first administration of the FFQ and the 24 h recalls are also presented in Table 2. Correlation coefficients between the recalls and FFQ for unadjusted energy and nutrients ranged from 0·19 for β-carotene to 0·74 for alcohol. Energy adjustment increased the correlations between the recalls and FFQ for all nutrients. Partial correlations between the recalls and FFQ, controlling for age and BMI, revealed coefficients similar to those presented in Table 2 (data not shown).
Agreement in cross-classification by the two methods was assessed as the proportion of participants similarly classified into the highest or lowest quartiles, and misclassification as the proportion classified into the opposite extreme quartile, for unadjusted and energy-adjusted nutrients (Table 3). Cross-classification of the nutrients into quartiles showed that more than half of participants were correctly classified into the same quartile for MUFA, folate and alcohol intake. Gross misclassification into the opposite quartile was evident for Fe, retinol and β-carotene. The κ w values are also shown in Table 3 and ranged from 0·13 for retinol to 0·66 for alcohol. Moderate to good agreement (κ w ≥ 0·4) was observed for energy, protein, carbohydrate, fat, MUFA, folate, fibre and alcohol.
Percentages of participants classified into the same and opposite quartiles of intake according to the average of the four 24 h recalls and the first administration (FFQ1) of the Botswana FFQ, and weighted kappa statistics (κ w)
†Quartile by 24 h recalls.
Comparisons of the median intakes of selected food groups measured by the recalls and the first administration of the FFQ are shown in Table 4. The two instruments gave similar median intake estimates for poultry, fish, eggs, other vegetables and legumes. For cereals, red meat, fruits and dark green leafy and yellow vegetables, the FFQ gave higher estimates of intake. Correlation coefficients between the 24 h recalls and the FFQ were highest for poultry (0·58), fish (0·52) and eggs (0·49) and were lowest for fruits (0·23) and dark green leafy and yellow vegetables (0·18; Table 4).
Daily intakes of food groups (median and 25th, 75th percentile (P25, P75)) from the average of the four 24 h recalls and the first (FFQ1) administration of the Botswana FFQ, and Spearman correlation (coefficient r and 95 % confidence interval) between the two dietary methods
Median values were significantly different from those estimated from repeat recalls: *P ≤ 0·05, **P ≤ 0·001.
Reproducibility
Reproducibility of the FFQ was determined by intra-class and Spearman correlation coefficients for unadjusted and energy-adjusted nutrient intakes comparing intakes obtained one year apart (Table 5). Intra-class correlation coefficients ranged from 0·50 to 0·68 except for retinol (0·37). Spearman's correlation coefficients for the macronutrients ranged from 0·54 (protein) to 0·68 (carbohydrate). Among micronutrients, coefficients were highest for alcohol (0·68), Fe (0·68) and vitamin E (0·66) and lowest for retinol (0·41). Energy adjustment tended to reduce correlation coefficients between FFQ1 and FFQ2 for macronutrients (carbohydrate and fat) but improved coefficients for most micronutrients.
Intra-class and Spearman correlation coefficients (ICC and r, respectively) for daily energy and nutrient intake estimates from the Botswana FFQ completed at the beginning (FFQ1) and end (FFQ2) of the validity study
Discussion
We evaluated the 122-item quantitative FFQ against repeated 24 h recalls (4 d) for the assessment of habitual diet of adults in Botswana. The FFQ performed consistently well in comparing energy and nutrient intakes estimated from 24 h recalls as assessed by correlations, percentage correct classification and kappa statistics. Correlations between the two methods were moderate to good for macronutrients and most micronutrients, ranging from 0·33 to 0·74. However, the FFQ appeared not able to adequately assess intakes of Fe, retinol and β-carotene as indicated by weak correlation coefficients (0·19–0·23) and evaluation of agreement between the FFQ and 24 h recalls. Evaluation of intakes of food groups suggested moderate relative validity.
Validity
Studies of the accuracy of an FFQ in measuring habitual intake are typically based on repeated 24 h recalls(Reference Horn-Ross, Lee and Collins12–Reference Johansson, Hallmans and Wikman15) or daily weighed food intakes(Reference MacIntyre, Venter and Vorster4, Reference MacIntyre, Venter and Vorster5, Reference Parr, Barikmo and Torheim7, Reference Marks, Hughes and van der Pols16) as the reference measurement. In the present study, repeated 24 h recalls were used as the reference method to determine the relative validity of the FFQ. Twenty-four hour recalls were selected to ensure a high degree of compliance. Other methods of dietary assessment require high literacy and motivation and are not suitable for a developing country such as Botswana. Four recalls were conducted over a 1-year period to allow for seasonal variations and included all days of the week.
Correlations of estimates of micronutrient intakes between recalls and the FFQ were low for Fe, retinol and β-carotene. Low correlation coefficients associated with vitamin A intakes may be related to seasonal availability of fruits and vegetables resulting in substantial fluctuations in intake. Seasonal fruits and vegetables include wild fruits (morula, mmilo and moretlwa), melons (maraka and lerotse) and vegetables such as morogo wa dinawa (cowpea leaves). Of note, correlation coefficients were low for fruits and dark green leafy and yellow vegetables. It is possible also that green leafy and yellow vegetables consumed in mixed dishes were under-reported and may have contributed to the low correlation. Low correlations for vitamin A are not unusual and may reflect not only limitations of the FFQ but also the difficulty in estimating retinol and β-carotene intake by the reference method. Weak correlations for Fe have been observed in other studies(Reference MacIntyre, Venter and Vorster4, Reference Larkin, Metzner and Thompson17, Reference Rodriguez, Mendez and Torun18) and in the present study may be related to poor measurement of cereal intake. Prolonged recording periods in excess of the 4 d used in the present study may be necessary to reflect day-to-day variation in intake of these micronutrients. Food intake estimated by the FFQ showed moderate levels of agreement compared with recalls. Correlation coefficients comparing 24 h recalls and the FFQ ranged from 0·18 to 0·58. Correlation coefficients were comparable to those observed in other validation studies(Reference Jackson, Walker and Younger14–Reference Marks, Hughes and van der Pols16, Reference Bohlscheid-Thomas, Hoting and Boeing19, Reference Feskanich, Rimm and Giovannucci20).
Nutrient data are often adjusted for energy to eliminate confounding due to energy intake and further reduces bias due to low energy reporting(Reference Willett3). In the present study, energy adjustment increased correlation coefficients for macronutrients and most micronutrients. However, adjustment for energy intake did not improve the coefficients for Fe and it is suggested that this occurs when variability is related more to systematic errors of under- or overestimation of specific foods than to energy intake(Reference Malekshah, Kimiagar and Saadatian-Elahi21, Reference Rimm, Giovannucci and Stampfer22). Furthermore, energy adjustment may also reduce the between-subject variability, leading to a reduction in correlation coefficients. Excluding Fe, retinol and β-carotene, correlation coefficients between the first FFQ and recalls ranged from 0·33 to 0·74 and compared well with other studies(Reference Horn-Ross, Lee and Collins12, Reference Jackson, Walker and Cade13, Reference Rodriguez, Mendez and Torun18, Reference Malekshah, Kimiagar and Saadatian-Elahi21, Reference Rimm, Giovannucci and Stampfer22).
The ability to rank individuals according to food or nutrient intakes is important in epidemiological studies(Reference Cade, Thompson and Burley23). Cross-classification of nutrients into quartiles showed that 40·0–72·2 % of participants in the lowest and highest quartiles according to the 24 h recalls were classified in the same quartiles by the FFQ, excluding SFA, PUFA, Fe, retinol and β-carotene. Misclassification into extreme quartiles tended to be low for most nutrients, with few participants grossly misclassified into extreme quartiles. Misclassification was higher for vitamin E, Ca, Fe, retinol and β-carotene. In general, agreement between the FFQ and the reference method was good and was comparable to that in other studies(Reference Jackson, Walker and Younger14, Reference Johansson, Hallmans and Wikman15, Reference Rodriguez, Mendez and Torun18).
The κ w statistic was used to evaluate the level of agreement and κ w values above 0·4 are valid for conclusions(Reference Masson, McNeill and Tomany24). The FFQ performed reasonably well (κ w = 0·40–0·66) for energy, carbohydrate, total fat, MUFA, cholesterol, folate, fibre and alcohol. Similar to correlational analyses and the percentages classified/misclassified, the κ w results indicate that the FFQ did not adequately classify individuals with respect to β-carotene, retinol and Fe intake levels, suggesting that intakes of these nutrients were not well assessed by the FFQ.
Reproducibility
Reproducibility of the FFQ expressed as correlations between the measurements was generally good, with Spearman and intra-class correlation coefficients for most of the nutrients varying between 0·54 and 0·68. When compared with other reproducibility studies of dietary habits over extended periods, such as the previous year, the level of agreement for repeated intakes of most of the nutrients was similar to that reported as unadjusted correlation coefficients for frequency questionnaires used in adults(Reference Johansson, Hallmans and Wikman15, Reference Malekshah, Kimiagar and Saadatian-Elahi21, Reference Rimm, Giovannucci and Stampfer22, Reference Villegas, Yang and Liu25). Correlation coefficients in the present study were higher than those reported in South Africa (0·14–0·75) in which the FFQ was repeated 6 to 12 weeks after first administration(Reference MacIntyre, Venter and Vorster5).
Agreement between repeated measurements of energy and macronutrient intakes was good (carbohydrate, Spearman's r = 0·68, highest; protein, r = 0·59, lowest) probably reflecting the fact that they are derived from several sources and mainly from staple foods which are eaten regularly and so may be reliably recalled. Among the micronutrients (highest r = 0·64; lowest r = 0·37), reproducibility for retinol was the lowest with a correlation coefficient of 0·37. Real changes in diet over time and differences in the participants’ ability to describe food intakes may have influenced reproducibility measures. Other factors such as the manner in which the interviewer asks questions, the interviewer effect(Reference Willett3), may also influence the reproducibility of the FFQ. In the present study however, all interviewers were trained and hence the administration of the FFQ should not vary widely. Nutrient intakes were calculated from the same food composition tables for both administrations of the FFQ and all computations were made using the same software.
The group enrolled in the validation study was recruited from a study that investigated the association of dietary intakes with weight status. Participants in the validation study were recruited after enrolment in the country-wide study. Participants were eligible for recruitment if they were 18–75 years old and not pregnant; consequently normal-weight, underweight and overweight individuals were included in the study. Subar et al.'s assessment of biomarkers of energy and protein against reported intakes highlighted that errors in reporting are likely to occur among both obese and non-obese individuals(Reference Subar, Kipnis and Troiano26). It is possible therefore participants may have under- or over-reported their intakes in this validation study.
Strengths/limitations
For validation of the FFQ we used the 24 h recall as the reference method. Both methods rely on memory and hence their sources of error may not be independent(Reference Willett3). Four days of recalls (three weekdays and one weekend day), administered over 1 year, were used to measure usual intakes. Dietary data collected on non-consecutive days increases the within-person variation; however, it is suggested that to adequately assess diet, a minimum of 7 d of data are needed for some nutrients and food groups(Reference Willett3). A self-reported method of dietary assessment was used as the reference method and may be biased, underestimating true intake(Reference Black, Goldberg and Jebb27). Our assessment of the validity of the FFQ included energy adjustment of nutrients in order to eliminate confounding due to total energy intake(Reference Willett3). A limitation of this validation study is the small sample size. The data set consisted of sixty-three females and sixteen males; hence the validity in estimates of intakes among men may be less certain and should be interpreted with caution. The FFQ did not include composite dishes and hence participants may have had difficulty estimating portions consumed, resulting in over- or under-reporting of intakes.
Conclusions
The FFQ developed for Botswana showed good relative validity for estimating most food and nutrient intakes. For Fe, retinol and β-carotene and related food groups (fruits and dark green leafy and yellow vegetables), estimates must be interpreted with caution because of the poor agreement. The reproducibility of the FFQ was comparable to that reported elsewhere. The inclusion of composite dishes should be given consideration for further refinement of the FFQ. These results are encouraging and acceptable for the use of the FFQ in the investigation of associations of diet with risk of disease in Botswana.
Acknowledgements
Sources of funding: This study was supported by the US Agency for International Development (USAID) Partnership for Food Industry Development (PFID-MSU), the National Food Technology Research Centre, the University of Botswana and the Ministry of Health, Botswana. Conflicts of interest: None of the authors had a conflict of interest to disclose. Authors’ contributions: The authors’ responsibilities were as follows. M.D.J., B.S.M., L.D.K., T.B.R. and J.J-M. contributed to concept development; B.S.M., L.D.K., R.I.K.-L., T.M. and M.M. were responsible for execution of the study and processing of the data; M.D.J. conducted the analysis and drafted the manuscript. All authors critically reviewed the manuscript and have responsibility for the final content. Acknowledgements: The authors wish to thank the interviewers and study participants for their support in the investigation.
Food items (local name in Setswana) listed in the FFQ
