The Strategic Action Plan to Tackle Non-Communicable Chronic Diseases in Brazil, 2011–2022( Reference Malta, Moraes Neto and Silva Junior 1 ), launched by the Ministry of Health in 2010, defined and prioritized actions and investments necessary to address non-communicable diseases in the next 10 years based on three principles: (i) surveillance, information, evaluation and monitoring; (ii) health promotion; and (iii) integral care. The aim is to diminish obesity rates in 5–9-year-old children and adolescents and to halt the rise of obesity in adults (≥18 years old)( Reference Malta, Moraes Neto and Silva Junior 1 , Reference Malta and Silva Junior 2 ). In Brazil, there are surveillance systems for adolescents( 3 ) and adults( Reference Malta, Andrade and Claro 4 ) but none for 5–9-year-old-children whose overweight (including obesity) prevalence tripled between 1974 and 2009, from 11 % to over 33 %( 5 ). Therefore it is necessary to develop surveillance and monitoring systems to determine compliance with the obesity prevalence targets for 5–9-year-old children, set at 8 % for boys and 5 % for girls( Reference Malta and Silva Junior 2 ).
Periodic population-based surveys are the principal source of time trends in nutritional status and associated lifestyle factors such as diet and physical activity. Monitoring these parameters over the past decades has shown a dramatic increase in excess weight worldwide, especially in younger generations and children( Reference Wang and Lobstein 6 – Reference Ochola and Masibo 8 ). In many countries, the number and complexity of survey items covering these issues have also increased, incurring higher costs( Reference Levine, Feltbower and Connor 9 ). To strike the right balance between the rising costs and a growing need to update our knowledge on population diet and physical activity, it is of utmost importance to clarify what sort of decision could be made on the basis of such surveys and what is an acceptable margin of error.
Information technology has reduced the cost of surveying lifestyle factors to a cost lower than traditional paper-and-pencil methods( Reference Illner, Freisling and Boeing 10 ). Several countries have developed computer-based questionnaires to assess dietary intake and/or physical activity in schoolchildren: the UK( Reference Moore, Ells and McLure 11 , Reference Foster, Hawkins and Delve 12 ), Denmark( Reference Biltoft-Jensen, Bysted and Trolle 13 ), Norway( Reference Medin, Astrup and Kåsin 14 ), Belgium( Reference Vereecken, Covents and Maes 15 ), Portugal( Reference Carvalho, Baranowski and Foster 16 ), the USA( Reference Baranowski, Islam and Baranowski 17 , Reference Diep, Hingle and Chen 18 ), Canada( Reference Storey and McCargar 19 ) and Brazil( Reference Ruggeri, Voci and Borges 20 , Reference da Costa, Schmoelz and Davies 21 ).
The Schoolchildren Food Consumption and Physical Activity (WebCAAFE) is an online previous-day questionnaire (recall of frequency of thirty-two food items and thirty-two types of activities) aimed at Brazilian children attending 2nd to 5th grades of elementary schools( Reference da Costa, Schmoelz and Davies 21 , Reference Davies, Kupek and de Assis 22 ). It originated from paper-and-pencil pictorial structured questionnaires on schoolchildren’s lifestyle that have been validated over the past 10 years( Reference Assis, Guimarães and Calvo 23 – Reference Assis, Benedet and Kerpel 25 ). Usability and validity tests have shown very good acceptability and child capacity to understand and respond to WebCAAFE( Reference da Costa, Schmoelz and Davies 21 , Reference Davies, Kupek and de Assis 22 ). However, the bias and precision of the questionnaire have not yet been reported.
There are challenges in such evaluation which include the question of a suitable gold/reference standard, the influence of the questionnaire on eating behaviour and its reporting (e.g. social desirability of the report), cognitive skills such as memory and attention necessary to provide an accurate report (especially for younger children), retention interval (the time elapsed between meal and recall), child motivation and optimal instructions to answer the questionnaire( Reference Illner, Freisling and Boeing 10 , Reference Assis, Benedet and Kerpel 25 – Reference Lu, Baranowski and Islam 31 ). Web-based applications may be rather limited in lower socio-economic areas in developing countries. In addition, the errors in self-reported food consumption are usually systematic, heteroscedastic, not normally distributed, and correlated within person and with reference se, as well as between dietary items( Reference Kipnis, Freedman and Brown 32 ).
Although the cost-effectiveness of health interventions has become a backbone of decision making for resource allocation in various areas of medical care, its application in nutritional epidemiology has been rarely debated, with few exceptions( Reference Carroll, Pee and Freedman 33 – Reference Stram, Longnecker and Shames 35 ). In order to extend this methodology to lifestyle interventions such as reinforcing healthy diet, quantification of bias and precision of the monitoring questionnaire (e.g. food questionnaires) is necessary. For example, a health or school authority may wish to know whether a particular food questionnaire provides sufficient precision to inform whether at least 90 % of children follow the recommendation to avoid fried food.
The objectives of the present paper are twofold: (i) to estimate the bias of WebCAAFE in relation to direct observation of food intake in school as the reference standard; and (ii) to estimate the sample size necessary to detect 75 % compliance with recommended frequency of daily consumption of major food groups in the Brazilian Food Guidelines.
Methods
WebCAAFE questionnaire
The WebCAAFE questionnaire is part of a web-based monitoring system to gather data on behavioural and environmental variables related to obesity in Brazilian schools. After usability( Reference da Costa, Schmoelz and Davies 21 ) and validity( Reference Davies, Kupek and de Assis 22 ) studies, the questionnaire was applied for three consecutive years (2013–2015) in thirty-four elementary public schools in the city of Florianopolis, with the participation of about 6000 schoolchildren. Currently, the questionnaire is being tested in a north-eastern Brazilian city. The system (www.caafe.ufsc.br) allows access to four types of user: (i) researchers can set up the survey parameters such as participating schools, dietary items and physical activities to be surveyed; (ii) school principals can access a questionnaire designed to evaluate the school environment in terms of physical activity and healthy eating opportunities, as well as the charts summarizing main research findings; (iii) children can access the WebCAAFE questionnaire using the passwords valid only during the time of answering the questionnaire; and (iv) health and education authorities can access the charts summarizing main research findings for each school. The WebCAAFE questionnaire was designed to operate in the school setting to help public health and education professionals to evaluate schoolchildren regarding: (i) nutritional status, food consumption, physical activity and sedentary behaviours; (ii) assessment of school compliance and children’s acceptability of the National School Meals Program; (iii) participation in and satisfaction with physical education classes at school; and (iv) health and education authorities can access the charts summarizing main research findings for each school.
The questionnaire is structured in three sections: registration, diet and physical activity( Reference da Costa, Schmoelz and Davies 21 ). The registration section collects information about respondents: their name, mother’s name, sex, weight, height, age, date of birth and study period. The child’s weight and height measures should always be taken by a trained researcher at most 15 d before responding to WebCAAFE and must be recorded in the school diary of each child. The researcher should also note the child’s name, sex, date of birth (provided by school administration) and anthropometric measures in a spreadsheet. The latter are used to calculate BMI and determine the weight status according to age and sex( Reference de Onis, Onyango and Borghi 36 ).
The diet section is a previous-day recall of frequency of intake of thirty-two dietary items offered for each of the six structured eating events (breakfast, morning snack, lunch, afternoon snack, dinner and supper) presented sequentially on the screen. For each event, thirty-two images (icons) of foods and beverages are presented on the computer screen, so that a child can select these by clicking in order to report the ingested meal composition (Fig. 1). The foods and beverages include both healthy and unhealthy items. The item selection took into account their frequency of intake in this age group as reported previously by 180 schoolchildren in 7 d food diaries, as well as the composition of school meals and the recommendations of the Brazilian Food Guidelines( 37 ). After responding on the six eating events, children are requested to answer four questions about school meals.
WebCAAFE food choices presented on computer screen
The physical activity section is divided into three parts of the day (morning, afternoon and evening)( Reference da Costa, Schmoelz and Davies 21 ). For each part, children can choose sedentary and physical activity icons among thirty-two options. Children are also asked about physical activity classes at school (frequency of weekly participation and degree of satisfaction) and the means of transport they use to go to and from school (car, school bus, regular bus, motorbike, walking, bike, skateboard and boat).
An avatar in the form of a robot (Fig. 1) guides the child while completing the questionnaire. For the diet section, the avatar explains the concept of each meal, the time of the day and the importance of reporting food consumption from the previous day. The avatar also instructs the child not to click on any food not consumed during the preceding day.
Participants
An intentional sample of five public schools with children aged 7–11 years, attending 2nd to 5th grades, was selected in the city of Florianopolis, southern Brazil. Having an information technology room for child use with personal computers and Internet access was the inclusion criterion for school selection. School selection was based on the need to include low, intermediate and high level of information technology resources within the school, such as the number and the quality of personal computers and Internet connection at child disposal within the school, as well as to cover the geographical regions of the municipality (North, South, East, West and Central). Six classes were randomly chosen from the list of eligible classes within each school by a lottery-like drawing without replacement.
The target sample included 778 children invited to answer WebCAAFE. Of these, 129 children were excluded because they were absent from school either during the observation of the meals and/or when WebCAAFE was applied on the following day. The final sample was composed of 629 children who answered the WebCAAFE and had their food intake in school directly observed (273, 104 and 252 children observed during the morning snack, lunch and the afternoon snack, respectively). As the school schedule in these public schools runs over one-half day, the majority of the children were observed during one meal only but thirty-one children reported taking two meals, thus producing the total of 660 WebCAAFE reports for the purpose of the current analysis.
Parents completed a questionnaire with sociodemographic information, as well as the information regarding the child’s access to a computer at home. They also gave written consent whereas the children gave their oral consent to participate in the study. The study protocol was approved by the Ethics Comittee of the Federal University of Santa Catarina (number 2250/11).
Procedures
Anthropometric measurements of weight and height were carried out by trained researchers in the schools using standard techniques( Reference Lohman, Roche and Martorell 38 ). Weight was measured using portable digital scales with a capacity of 180 kg (Marte LTDA, Belo Horizonte, Brazil). Height was measured using a metal stadiometer (Seca, Hamburg, Germany). The children were weighed and measured barefoot, wearing light clothing. BMI was computed as weight divided by height squared (kg/m2). The 2007 WHO-recommended growth reference for school-age children was used to convert BMI quintiles of the children in the present study to corresponding BMI-for-age Z-scores( Reference de Onis, Onyango and Borghi 36 ).
The children’s food intake was observed directly during school hours on the day before WebCAAFE was applied, keeping in mind that the questionnaire asked about the consumption of thirty-two food items on the previous day. Direct observation of food intake was conducted in the school environment, including meals that were prepared on school premises, those bought in the school canteen and food brought from home. A list of food items available from the school, as such, was known to the observers, so they could distinguish the origin of the items consumed by children. Although the children were told that the observers were in school to visit the premises, they were not aware of the exact purpose of this visit (i.e. to observe food intake and activities during school break time).
The observers of school meals were sixteen graduate students of nutrition sciences who received at least 20 h of training before the research, including a pilot observation of the school environment. The training consisted of six practice sessions, during which each observer had to observe at least thirty children in total. Agreement on ingested foods was 96 % between the more experienced observer and the other observers during training( Reference Davies, Kupek and de Assis 22 ). During data collection, each observer was assigned to observe a maximum of five children identified with name tags and coloured ribbons on their arms( Reference Davies, Kupek and de Assis 22 ).
The day after food consumption was observed, a trained researcher gave a demonstration of how to complete the computer questionnaire in each classroom. With the aid of a banner (140 cm×105 cm), the WebCAAFE food images were shown, as well as examples of how to answer the questionnaire. Subsequently, the children were escorted to the school computer room to complete the WebCAAFE which took approximately 15 min. The data were collected from March to October 2012, covering all days of the week except Friday. After completing the questionnaire, the children were individually asked about the presence of an Internet connection at home.
Statistical analysis
For each food/beverage group, the frequency of consumption (in times per day) was calculated by summing reported consumption over six daily eating events (e.g. the fruits and vegetables food group was calculated by adding reported consumption of fruits, starchy vegetables, vegetable soup and leafy vegetables over six eating events on the previous day). Food items were aggregated to food groups according to the dietary recommendations in the 2006 version of the Brazilian Food Guidelines( 37 ) based on frequency of consumption, without specifying corresponding portion sizes and nutrient quantities. Although a new version of the guidelines( 39 ) was launched in 2014, it did not specify the daily recommended frequencies or food portion sizes, so the 2006 version was used in the present study.
WebCAAFE food items were divided according to their nutritional properties or according to their roles as positive or negative markers of a healthy diet into the following groups: cereals (bread, biscuits, manioc flour, pasta, breakfast cereal, cheese bread, porridge); dairy products (milk, cheese, yoghurt); sweets (candies, chocolate bars, ice cream, cookies, cakes with icing); fish/seafood (includes shellfish); meats (red meats, poultry, pork, chicken); eggs (fried, boiled or omelette); fruits (all kinds of traditional Brazilian fruits such as bananas and oranges); green leaves/vegetables (green leaves and others such as carrots, pumpkin, broccoli, vegetable soup); fried food (French fries, hot dogs, fried snacks, pizzas, hamburgers); sugar-added beverages (sodas, fruit juices, chocolate drinks); cooked beans; and some combinations thereof, such as meat/fish/seafood and fruits/green leaves/vegetables. Some of these groups are partially overlapping to mimic the variations in healthy diet recommendations which also use overlapping components (e.g. eating six fruits/vegetables/green leaves or at least three fruits daily).
Bias was defined as the difference between WebCAAFE reports and directly observed food consumption within school, weighted by the inverse probability of having individual eating behaviour observed during each of the three school meals analysed. The method assumes a representative sample of the target population and applies the weighting scheme routinely used in multiple stages of a survey to account for unequal probability of recruiting individuals with certain demographic or behavioural profile( Reference Xue, Kim and Castle 40 ). For example, instead of oversampling hard-to-reach groups such as street children, with extremely low probability of recruiting them in a school-based survey, statistical adjustment can be made to account for this probability in the survey weights. Selection probabilities regarding demographic characteristics (child sex and school grade), child BMI-for-age Z-score, child access to Internet at home, as well as parental income and educational level, were all calculated via logistic regression and compared by a χ 2 test in order to verify the assumption that the observed dietary intake adequately represented school food intake of the target sample. The above sample characteristics were chosen as already established factors associated with reporting food consumption, except for access to Internet at home. BMI-for-age Z-score quintiles were used to equally space its range in order to verify a dose–response relationship with compliance.
Poisson regression used count for each food group as the dependent variable and the counting method (WebCAAFE v. direct observation) as the independent variable, together with the above-mentioned weights and adjustment for the covariance between reports on the same child. In addition, the total of food items consumed in school was added as offset to arrive at incidence rates for each of the groups analysed. The bias was expressed as a percentage.
The next step was to calculate the sample size necessary to detect a statistically significant difference between WebCAAFE reports and the recommendations for a healthy diet according to the Brazilian Food Guidelines( 37 ). The error types I and II were fixed at 5 % and 10 %, respectively, thus providing 90 % power of statistical tests. The difference between bias-corrected WebCAAFE reports and the frequency stated in the above recommendations, as well as the variances of estimated bias and of WebCAAFE-reported food consumption, were all used to determine the sample size with one-sided hypotheses of satisfying a dietary recommendation. For example, to determine the sample size needed to detect a statistically significant difference between at least 75 % compliance with the recommendation to avoid beverages with added sugar and their consumption rate reported by WebCAAFE, the formula:
$$\eqalignno {n{\equals}\{ Z_{{1{\minus}\alpha \,/\,2}} [\,p_{0} (1{\minus}p_{0} )]^{{1\!/2}} {\plus}Z_{{1{\minus}\beta }} [\,p_{{\mathop{\rm A}\nolimits} } (1{\minus}p_{{\mathop{\rm A}\nolimits} } )]^{{1\!/2}} \} ^{2} \! /\,[\,p_{{\mathop{\rm A}\nolimits} } (1{\minus}p_{0} )]$$
was used, where n is the required sample size, p 0 is the population proportion under the null hypothesis (e.g. 0 for the foods that are not recommended), p A is the alternative hypothesis that p 0 is different from the hypothesized value, and Z 1–α/2 and Z 1–β are the Z-values from a normal distribution corresponding to the type I and II errors, respectively( Reference Pagano and Gauvreau 41 ). The design effect was assumed to be 1 as for simple random sampling. In addition, it was assumed that the reporting bias observed for the school meals holds for other meals taken outside school, so that the daily intake on an individual level can likely be estimated by multiplying WebCAAFE daily totals with the incidence rate ratio from the Poisson regression used for bias estimation. Seventy-five per cent compliance was chosen as a useful population parameter for screening purposes. The null hypotheses compared this value with those from bias-corrected WebCAAFE reports. The upper limit of the estimated sample size was increased by 20 % and rounded to the nearest integer to account for eventual selection bias and/or confounding likely to be present in many field studies. The latter was denominated the target sample size.
For some food groups such as meat, fish and seafood, separate analyses of these components were added in order to identify specific sources of the bias.
The statistical software package Stata version 12.1 was used for all calculations. The command sampsi was used for the sample size calculation.
Results
Mean age of the children sampled was 9.50 (sd 1.25) years, with distribution of 7.86 (sd 0.33), 8.85 (sd 0.45), 9.93 (sd 0.56) and 10.98 (sd 0.71) years over school grades 2 to 5, respectively. The proportion of girls was about 5 % higher than that of boys (Table 1). Most of the families fell into the lowest income quintiles and had no college education, but had a personal computer at home.
Selection probabilities for observing food intake in school, according to some demographic characteristics, BMI and access to Internet at home, among schoolchildren (n 629), aged 7–11 years, attending 2nd to 5th grades of five elementary public schools in Florianopolis, Brazil, March–October 2012
* Number of children observed for a given school meal.
† Probability of observing a child with a given characteristic (percentage).
‡ Hypothesis of equal percentages rejected by χ 2 test (P<0.05).
Overall, there were no statistically significant differences in the probability of observing children with particular demographic characteristics, BMI-for-age Z-scores and Internet access during school meals, except for the preference in selecting children in 2nd and 3rd grades compared with those in 4th and 5th grades (last column in Table 1). There was also a higher probability of observing children without information on BMI and, to a lesser extent, those whose parents did not report their income and educational level, although these differences did not reach statistical significance. Most of the significant differences were identified for school lunch (sex, income, education), whereas morning and afternoon snacks showed more balanced selection probabilities.
Percentage bias v. the reference standard (observed school meals) showed moderate bias magnitude for most food groups (Fig. 2) except for almost fourfold overestimation of fish/seafood frequency and about threefold underestimation of the fruit intake during school meals (Table 2). Frequency of consumption did not seem to be related to the bias except for fish/seafood, which was very rarely consumed (0.6 %). The imprecision of bias estimates (95 % CI) was the highest for eggs and beans but did not reveal any discernible pattern in relation to the bias magnitude.
WebCAAFE bias as percentage difference relative to observed food consumption among schoolchildren (n 629), aged 7–11 years, attending 2nd to 5th grades of five elementary public schools in Florianopolis, Brazil, March–October 2012. Values are means with their 95 % confidence interval represented by vertical bars
Mean frequency of food group intake based on direct observation and estimated WebCAAFE bias (N 660 eating events) among schoolchildren (n 629), aged 7–11 years, attending 2nd to 5th grades of five elementary public schools in Florianopolis, Brazil, March–October 2012
* Multiplied by 100.
The compliance with dietary recommendations varied widely, from <2 % for fruits and vegetables to about half the schoolchildren who did not report avoiding sugary beverages, fried food and sweets (Table 3). The least significant differences between the 75 % compliance criterion and rarely v. frequently met recommendations (e.g. fruits and vegetables v. sugary beverages) were large, so was the variation in the sample sizes necessary to detect these differences. A large, least significant difference implies a large effect size and therefore requires smaller sample size to detect it by test statistics, so only a handful of children need to be sampled for the foods for which daily intake very rarely meets the daily recommendations. After increasing the estimated sample size by 20 % to account for eventual selection bias, the target sample size varied between only four and seventy-four schoolchildren.
Observed compliance with dietary recommendations and the sample size needed to detect 25 % or higher level of non-compliance using WebCAAFE among schoolchildren (n 629), aged 7–11 years, attending 2nd to 5th grades, of five elementary public schools in Florianopolis, Brazil, March–October 2012
* Recommended daily frequency of consumption.
† Estimated sample size with 90 % power to detect 75 % compliance or higher.
‡ Targeted sample size with 20 % added to the upper bound of the 95 % CI.
Discussion
To the best of our knowledge, this is the first validation study that used observed food intake at school as a reference method to estimate previous-day recall bias and the sample size necessary to detect 75 % compliance with recommended frequency of daily consumption based on food guidelines, thus allowing evaluation of the viability of a food questionnaire for decision making at school level. The findings of the present study add to previous WebCAAFE validation study results( Reference Davies, Kupek and de Assis 22 ) and to other web-based 24 h recall/record validation studies in children using observation during school meals as a reference method( Reference Biltoft-Jensen, Bysted and Trolle 13 , Reference Medin, Astrup and Kåsin 14 , Reference Carvalho, Baranowski and Foster 16 – Reference Diep, Hingle and Chen 18 ).
According to the sample sizes needed to detect a significant difference between self-reported and observed food/beverage consumption, WebCAAFE is a valid questionnaire for screening compliance with dietary recommendations analysed in the present study for medium and sometimes even small groups of children (e.g. in a classroom). If the difference between the recommendations and estimated daily consumption is large, a handful of WebCAAFE reports may be sufficient to detect the stipulated level of non-compliance (≥25 %) in the group analysed. However, as the difference gets smaller, the sample size rises to the size of several classrooms (seventy-four children) to test the hypothesis that at least a quarter of schoolchildren do not comply with the recommendation to avoid sugar-added beverages. Although these sample sizes may appear surprisingly small, the following factors that may explain these results need to be considered: (i) as recommended dietary proportions are stipulated population constants with zero variance, the variance of the difference between WebCAAFE proportion and the stipulated proportion was considerably reduced; (ii) the differences between observed and recommended dietary proportions were large for most foods due to low level of compliance; and (iii) the comparisons were one-sided as the dietary recommendations were directed, in the sense of either avoiding or consuming the foods by at least certain number of times per day.
The 25 % criterion for non-compliance is both plausible and relevant for intervention decision making such as school-based promotion of healthy diet. Different compliance criteria (e.g. 90 % compliance equivalent to 10 % non-compliance) are easy to substitute in the sample size formula.
Most publications on sample size calculations for diet validation studies focused on continuous outcomes such as the quantity of nutrient intake( Reference Kipnis, Freedman and Brown 32 , Reference Carroll, Pee and Freedman 33 , Reference Freedman, Schatzkin and Wax 42 , Reference Freedman, Carroll and Wax 43 ) as opposed to binary outcomes such as dietary compliance analysed in the present study. A computer simulation showed that large within-subject variation in reporting food frequency required a calibration study design with more repeated records over fewer individuals as the most efficient study design when a bias was set to 17 % under-reporting of the true value( Reference Carroll, Pee and Freedman 33 ). This figure is close to the bias magnitude estimated for most foods in the present study, thus suggesting that several repetitions of WebCAAFE for a cohort of schoolchildren may be the best way forward in future validation studies with this questionnaire.
Regression calibration has been the prevailing method in diet validation of nutrient intake components and the indicators derived thereof. In most cases, it assumes no correlation between measurement errors in the reference standard and alternative questionnaire subject to calibration and applies normalizing transformations to highly skewed distributions to improve se estimation of parameter estimates( Reference Freedman, Midthune and Carroll 44 ). The present study used Poisson regression regularly applied to count outcomes such as the number of food items consumed per day, with logarithmic transformation of the counts and population variance assumed to equal the mean consumption, both of which take into account the peculiarities of the measurement error structure in diet validation( Reference Kipnis, Freedman and Brown 32 ).
The WebCAAFE presents a different framework from other web-based questionnaires( Reference Moore, Ells and McLure 11 – Reference Ruggeri, Voci and Borges 20 ) mainly because it was formulated as a previous-day recall of frequency of intake of thirty-two food items in each of six daily eating events, so the dietary assessment does not involve a fully quantified 24 h recall of foods and beverages. The cognitive task required for estimating portion size, frequency and averaging may not be compatible with the perceptual and conceptual capacities of children who have not reached the stage of abstract reasoning at approximately 10–11 years of age( Reference Baranowski and Domel 26 , Reference Baxter 27 , Reference Livingstone, Robson and Wallace 29 ). Non-quantified recall of the dietary items considered most relevant in the national food guidelines may be a feasible method for surveillance of dietary intake in children, including those just starting school, in order to reduce cognitive demands related to food quantities. This is also important in order to reach out to disadvantaged children. In the present study, about a quarter of the children analysed were 7–8 years old, about 20 % had parents with at most elementary school education and a further 20 % had parents with high school education (Table 1). Compared with aforementioned validation studies regarding other web-based food questionnaires( Reference Biltoft-Jensen, Bysted and Trolle 13 , Reference Medin, Astrup and Kåsin 14 , Reference Diep, Hingle and Chen 18 ), the social and educational background of the WebCAAFE respondents was considerably lower.
The high percentage bias for fish/sea products, meats, fruits and beans may have occurred because the food images in the WebCAAFE screen for these items did not match the way children often eat (Fig. 1). Fish/sea products, meats and beans are generally prepared in school canteens in mixed dishes, like casseroles and stews. This is in line with attributing lower accuracy of pulses intake to the discrepancy between their screen representation in PAC 24( Reference Carvalho, Baranowski and Foster 16 ), a web-based questionnaire for Portuguese children, and their appearance when served to the children.
The strengths of the present study include a reasonably large sample size, geographical coverage of the city regions and a reference standard for school meals based on direct comparison between observed behaviour and reported food choices, thus providing a solid assessment of the bias. Furthermore, the study tackled important methodological issues of imprecision of the reference standard and only partial verification of the food consumption, which was limited to the school premises. The former was quantified by 95 % CI whereas the latter was adjusted for by the inverse probability weights. No indication of significant selection bias was found regarding key control variables (child sex, BMI-for-age Z-scores and Internet access; parental income and education level) except for the higher chance of selecting younger children from the second and third school grades (Table 1). Older children might have more autonomy to skip school meals and use this time for other activities, such as talking or playing with friends. In addition, they were more accurate in reporting their dietary intake than younger (7–8 years) children( Reference Davies, Kupek and de Assis 22 ). On the other hand, 43 % of the matches between WebCAAFE and observed dietary intake in school, as well as 29 % of intrusions and 28 % of omissions, put this questionnaire’s accuracy close to that of other similar instruments( Reference Baranowski, Islam and Baranowski 17 , Reference Diep, Hingle and Chen 18 ).
Finally, screening and decision making with data from WebCAAFE is highly flexible with respect to local food menus. Some of the Brazilian Food Guidelines’ dietary recommendations( 37 ) were chosen to exemplify the sample size calculations because of a widespread agreement on their importance for child growth and development. However, achievement of other targets can be tested in the same way. This makes the above approach flexible to assess adaptive cross-cultural differences in eating behaviour.
Among the limitations of the present study, it is worth noticing that the sample size calculation did not take into account within-subject variation of repeated WebCAAFE reports. Another study found this variation considerably higher for episodic compared with regular food intake( Reference Dodd, Guenther and Freedman 45 ) and unstructured meal components. However, dietary recommendations usually deal with more frequently consumed foods such as those analysed in the present study. Furthermore, statistical adjustments applied in the present study would not be effective for eventual systematic differences in child reporting on the food consumed in school v. outside school. A child has more autonomy to choose what to eat outside school compared with a more restricted school environment. Therefore, the number of food choices is typically higher outside school, especially for less structured meals such as snacks. Even within the school environment, under-reporting with WebCAAFE was higher when more food choices were available( Reference Davies, Kupek and de Assis 22 ). Also, there is often less adult control over the food consumed at home, such as while watching television or playing with friends. In addition, the sampling was restricted to the schools with computer laboratories where a child has access to the Internet, so the findings cannot be generalized to the schools without such conditions. Furthermore, the cluster sampling design underestimates the variance of simple random sampling without adjusting for design effect. Finally, increasing estimated sample size by 20 % may be insufficient to provide adequate target samples in the case of a strong selection bias (e.g. if compliers with food guidelines were much more likely to participate in the study) or large systematic differences between observed and unobserved eating behaviour (e.g. in school v. out of school).
In the city where the present study was conducted, all public and private schools had Internet access and a designated room for its use by students, although the quality of the Internet connection was pretty variable. However, in a north-eastern Brazilian city where WebCAAFE was also validated, only half of the public schools had the quality of Internet connection necessary for its application. For areas with no Internet access, the paper-and-pencil versions of the questionnaire( Reference de Assis, Kupek and Guimarães 24 , Reference Assis, Benedet and Kerpel 25 ) are current alternatives to WebCAAFE. Future development of the questionnaire will add interfaces for tablets and smart phones, which may be provided by researchers to the schools without Internet access, as well as outside the school. In addition, improving screen representation of the highly biased foods such as fish/sea products, fruits, beans and meats, as well as including an open field to write down a food item not found in the current menu, are important lines of future research to increase the accuracy of WebCAAFE reports.
Despite the limitations, WebCAAFE appears to be a simple, low-cost and highly adaptable food questionnaire for evaluation of schoolchildren’s dietary compliance on a group level, even with relatively low sample size. As such, it may be used to aid decision making for school or health authorities.
Acknowledgements
Acknowledgements: The authors gratefully thank Sanlina Hulse Barreto from the Education Department of the Municipality of Florianopolis, the children, their parents and the school authorities for their participation in the study. Financial support: This study was supported by the Brazilian Ministry of Health (Departamento de Ciência, Tecnologia e Insumos Estratégicos; DECIT) and the Scholarship Program from the Fundação de Amparo à Pesquisa e Inovação do Estado de Santa Catarina (FAPESC). DECIT and FAPESC had no role in the design, analysis or writing of this article. Conflict of interest: The authors have no competing interests. Authorship: E.K. participated in the concept, data analysis, interpretation and preparation of the paper. M.A.A.A. participated in study design, sought the ethics committee approval, supervised the study implementation (including recruitment and anthropometric measurements) and helped interpreting its results. F.B. and A.S.L. assisted in the manuscript interpretation and revision. All authors approved the submitted manuscript. Ethics of human subject participation: This study was conducted according to the guidelines laid down in the Declaration of Helsinki and all procedures involving human subjects were approved by the Federal University of Santa Catarina Human Research Ethics Committee (Protocol 2250/11). Oral consent was obtained from participating children and written informed consent was given by their parents and educators.
