Worldwide, many children and adolescents do not meet the recommendations for fruit, vegetable, fish, dairy or whole grains consumption and are over-consuming energy-dense, sugary and salty foods(Reference Holman and White1–Reference Vereecken and Maes4).
One of the many factors influencing dietary intake is nutritional knowledge(Reference Story, Neumark-Sztainer and French5). Evidence of the association between nutritional knowledge and dietary intake has been found in several studies(Reference Kristjansdottir, De Bourdeaudhuij and Klepp6–Reference Gibson, Wardle and Watts10). Moreover studies examining the effects of programmes designed to increase nutrition knowledge have found positive results(Reference Rasanen, Niinikoski and Keskinen11, Reference Rasanen, Niinikoski and Keskinen12). However, other studies found no significant relationship(Reference Perez-Lizaur, Kaufer-Horwitz and Plazas13, Reference Calfas, Sallis and Nader14). Explanations for the inconsistent results are poor measurement of knowledge (lack of relevance, poor conceptualization, different levels of specificity of knowledge and dietary habits), different measurements of knowledge, poor measurement of dietary intake, lack of statistical power and the many factors that influence dietary behaviours of which nutritional knowledge is just one(Reference Worsley15).
To improve children's and adolescents’ food habits, local, national and international(Reference Vereecken, Huybrechts and Van Houte16–Reference Brug, Velde and Chinapaw21) interventions are being developed. One of these local interventions in Belgium-Flanders is an Internet intervention targeting primary-school children. The aim of the intervention will be to improve children's dietary habits by increasing children's nutritional knowledge and awareness. To evaluate the effectiveness of the intervention, changes in nutritional knowledge will be evaluated in addition to changes in dietary intake, as traditional dietary assessment instruments are not always appropriate to assess the impact of nutrition education programmes(Reference Turconi, Celsa and Rezzani22). Hence, a web-based nutritional knowledge questionnaire was developed.
Limited surveys are available to assess the nutrition knowledge of primary-school children and even fewer have investigated the psychometric properties of nutritional knowledge questionnaires in this age group. Item and scale analyses are however important as they can help to improve the questionnaire. Additionally, poor reliability degrades the precision of a measurement and reduces the ability to track changes or link changes to an intervention. To our knowledge, only one other study has developed and evaluated a computer-based nutritional knowledge questionnaire for school-aged children(Reference Gower, Moyer-Mileur and Wilkinson23). Based on moderate test–retest correlations, appropriate content validity and the questionnaire's ability to measure improvement in the control group of an intervention study, the authors of that study concluded that the computer-based survey is a promising medium for assessing nutrition knowledge.
Methods
Participants and procedure
Ethical approval for the current study was obtained from the ethical board of the Ghent University Hospital.
A convenience sample of fourteen primary schools in Flanders (the northern part of Belgium) participated in the study. Children in the second, fourth and sixth grades (n 1029; 7–12 years of age) were invited to participate in the study. Parents of children participating in the study signed informed consent. Children completed a nutrition knowledge questionnaire and a food preferences questionnaire(Reference Vereecken, Covents and Parmentier24) online during school hours. To assess test–retest reliability, half of the second and fourth graders completed the nutritional knowledge questionnaire a second time, the other half completed the preference questionnaire a second time; sixth graders completed both instruments twice. The test–retest interval was one to two weeks. The children completed the programme autonomously, although a teacher and a researcher stayed with the children during the entire administration to address any problems. To assess the children's appreciation of the instrument a short evaluation tool was completed by a sub-sample at the second measurement occasion. Data collection took place in February–March 2011.
Material
The nutritional knowledge questionnaire was developed by two student dietitians and a health psychologist. In Belgium-Flanders, it is not defined which aspects of nutritional knowledge should be covered in the curricula of primary schools. Therefore the questionnaire was developed based on the literature(Reference Gibson, Wardle and Watts10, Reference Anderson, Bell and Adamson25–Reference Parmenter and Wardle27) and the Flemish Food Behaviour Dietary Guidelines model (‘The Active Food Triangle’)(28), which is used in most schools.
Five main areas of interest were identified: (i) healthy food choices; (ii) nutrient content; (iii) main function of food items/nutrient; (iv) food group categorization; and (v) estimation of adequate portions.
Healthy food choices were assessed with three question formats. For five questions respondents were asked to rank-order a set of two to four items from the healthiest to the least healthiest. For fourteen questions respondents were asked to select the healthiest alternative from two to five multiple-choice items. Finally children were asked to select from a list of eight food items the healthy snacks. Five items asked about the nutrient content of common foods with two to three response options and another five items asked questions related to main health functions (four response options). The next set of questions (food group categorization) contained eight sets of four food images, of which the food item not belonging to the same food group as the other items had to be identified. To assess children's knowledge about adequate portions as recommended by health experts (doctors and dietitians), for eight items respondents had to select the recommended amount from four to five options. The correct responses were based on the Belgian-Flemish Food Behaviour Dietary Guidelines(29). Items were selected to represent each of the main food groups.
A primary-school teacher, a dietitian and six student dietitians in their last year provided comments on the clarity and content of the items (content validity). A pilot test was done among children of the researchers’ family and acquaintances (n 10) and members of a volleyball team (n 6) to check for clarity, resulting in a number of small modifications.
All items, except the ranking questions, were scored taking into account correction for guessing, so +1 for a correct answer and −1, −0·5, −0·33 or −0·25 for a wrong answer depending on the number of response options (respectively 2, 3, 4 or 5). The ranking questions received one point if the first item was ranked correctly and another point if the last item was ranked correctly. Finally the children could also indicate for each item that they did not know the answer, scored as zero. After the pilot test the ‘don't know’ option was dropped for the ranking question as this was confusing for the second graders. However, respondents could skip these questions (scored as 0) while all other questions were obligatory. A sum score was created for each domain and a sum score was computed over all domains to represent a general nutritional knowledge score; however, the healthy choices score was divided by three to have a more equal weight over the different domains.
We chose to develop the questionnaire in computer format as the use of computers can help to make questionnaires look simple and attractive(Reference Borgers, DE Leeuw and Hox30). Moreover, this way, many of the food items and recommended servings/portions could be visualized by food images which improves clarity and recognition(Reference Hamilton-Ekeke and Thomas31). Access to the online questionnaire can be obtained from the authors on request.
Children's appreciation of the tool was assessed with a short online questionnaire asking if the questionnaire was clear, interesting, nice, difficult, suitable for children, too long, if the pictures were clear and if there was enough explanation.
The questionnaires were developed in Limesurvey 1·85 (Open Source Software).
Statistical analyses
Percentages are shown, with the percentage of respondents giving the correct answer representing the difficulty index(Reference Parmenter and Wardle32). As knowledge questions should be not too easy and not too difficult, an appropriate range falls between 20 % and 80 % of correct responses(Reference Kline33). The item discrimination index reflects an item's ability to discriminate between individuals who scored high and those who scored low on the entire test and was computed as percentage correct in the highest scoring tertile minus percentage correct in the lowest scoring tertile. The discrimination index of each item was assessed within each subscale. Discrimination indices above 20 % are acceptable and above 30 % good(Reference Sabbe, Van de Poele and De Cock34).
Because of the known differences in children's general and nutrition-related cognitive capacities by age(Reference Zeinstra, Koelen and Kok35), percentages are presented for the total sample and by grade. In addition, a new data set was created including all difficulty and discrimination indices by item and grade to investigate significant differences by grade. Repeated-measures ANOVA was used to investigate these differences.
Kappa statistics were used to assess agreement between test and retest for each item separately. For this, items were first dichotomized into correct v. wrong responses. Values κ<0 are considered as poor, κ = 0–0·20 as slight, κ = 0·21–0·40 as fair, κ = 0·41–0·60 as moderate, κ = 0·61–0·80 as substantial and κ = 0·81–1·00 as almost perfect(Reference Kramer and Feinstein36). A low κ value might indicate that the question is not clear and/or that the respondents are guessing. The intra-class correlation (ICC)(Reference Kianifard37) was used to assess agreement on a scale level for each of the domains and for the total knowledge score. An ICC > 0·8 is usually regarded as indicating good to excellent reliability, whereas an ICC between 0·6 and 0·8 may be taken to represent substantial reliability(Reference Pinna, Maestri and Torunski38). Systematic differences (higher or lower scores on the retest) were investigated by paired-sample t tests.
Because of the ordinal level and skewed distribution of the appreciation items, the non-parametric Kruskal–Wallis test was used to investigate grade differences in appreciation. Data were analysed using the SPSS statistical software package version 15·0·1·1 (2007; SPSS Inc.). The significance level was set at P < 0·05.
Results
Of the 1029 children approached for participation, 596 returned informed consent and 576 children (grade 2: 33 %, grade 4: 34 %, grade 6: 32 %, boys: 44 %, mean age: 9·7 (sd 1·7) years) filled in the nutritional knowledge questionnaire at the first measurement occasion (T1). Three hundred and ninety-six children completed the nutritional knowledge questionnaire a second time (T2), of whom 386 (grade 2: 25 %, grade 4: 28 %, grade 6: 47 %, boys: 42 %) could be matched with T1 measurements.
Descriptive statistics, including the difficulty and discrimination indices of the healthy choices, nutrient content, main food function and categorization of food items are summarized in Table 1; descriptive statistics of the estimation of the recommended portions/servings are described in Table 2. On average 73 % of the healthy choices were correctly responded. The percentage of correct responses was, however, lower for the remaining scales (nutrients: 59 %, food categorization: 49 %, main function: 38 %, portions: 36 %).
Overview of items in the healthy choices, nutrient, food function and food categorization scales of the children's nutritional knowledge questionnaire, difficulty and discrimination index by grade (G) and in the entire sample (Tot) for T1, and test–retest agreement (κ) between T1 and T2 for all grades: Belgian children (aged 7–12 years) from fourteen primary schools, February–March 2011
All items, except the ranking items, had ‘don't know’ as last the response option.
F, fruit; V, vegetables; G, grains and potatoes; M, meat and meat substitutes; D, dairy; O, oils and fats; T, top of the active food triangle; W, item belongs to the water group.
*Boiled potatoes and mashed potatoes were considered as correct as this might depend on the preparation. In future studies only one of both items should be kept.
†Response options were visualized with food images.
‡Correct response.
Responses, difficulty index (percentage correct) and discrimination index by grade (G) and in the total sample (Tot) at T1, and test–retest agreement (κ) between T1 and T2 for all grades, of the recommended portions scale of the children's nutritional knowledge questionnaire: Belgian children (aged 7–12 years) from fourteen primary schools, February–March 2011
κ = that between test and retest of dichotomized (correct v. wrong) responses.
*Correct response, the percentage of this option = the difficulty index.
†Response options visualized by food images.
A high difficulty index (easy items), in combination with a low discrimination index, was found for the identification of water, milk, fresh fruit salad, jam, oranges and grapes as the most healthy items; for the identification of an apple as a healthy snack; and for the identification of the three unhealthy snacks. A low difficulty index in combination with a low discrimination index was found for two items in the food group categorization, namely nuts in a series of grain products and cheese in a series of meats and meat substitutes. Finally, a low difficulty index was found for children's estimation of recommended portions of bread. The agreement on an item level of correct v. wrong responses on T1 v. T2 was fair to moderate for most items, with an average of κ = 0·39 (sd 0·11).
Repeated-measures ANOVA indicated a significant increase of the difficulty index by grade (grade 2: 48·9 (se 3·3), grade 4: 61·5 (se 3·6), grade 6: 66·0 (se 3·6); P < 0·001), but no significant difference was found for the discrimination index (grade 2: 31·9 (se 2·0), grade 4: 29·6 (se 2·2), grade 6: 28·5 (se 2·4); P = 0·466).
A good agreement between test and retest was found for the overall knowledge scale (ICC = 0·76) and the healthy food choices subscale (ICC = 0·75), a moderate agreement was found for the subscales on food categorization, estimated recommended portions and main functions of food items (ICC = 0·44 to 0·61), but a low agreement was found for the nutrient content scale (ICC = 0·33; Table 3). In general the ICC was lowest for grade 2, with some very low values for the nutrient content, main function of food items and recommended portions scales. The mean of the healthy choices scale decreased from T1 to T2. Results of an abbreviated nutrition knowledge score (excluding the ten easy items with low discrimination index in the healthy eating score) resulted in no major changes in test–retest statistics.
Test–retest statistics of the nutritional knowledge scales: mean and standard deviation of T1 and T2, significance of the difference (paired t test), and intra-class correlation (ICC) by grade (G) and in the total sample (n 386); Belgian children (aged 7–12 years) from fourteen primary schools, February–March 2011
In general the questionnaire was well received by the respondents (Fig. 1). No significant difference was found by grade except that second grade children found it more difficult (P = 0·04) and too long (P = 0·001) than fourth and sixth graders.
Appreciation of the test (
, completely agree; 
, rather agree; 
, rather disagree; 
, completely disagree) among Belgian children aged 7–12 years (n 150) from fourteen primary schools, February–March 2011
Discussion
In the present paper the development of a nutritional knowledge questionnaire is described. The overall nutritional knowledge questionnaire proved to be a reliable tool (test–retest ICC = 0·76).
The test–retest correlations of the subscales varied between 0·33 and 0·75 and are comparable with what has been found in the literature. In a study of Anderson et al.(Reference Anderson, Bell and Adamson25), test–retest correlations of different domains of knowledge in a sample of 11-year-olds (n 37) were 0·458 (applied nutrition knowledge), 0·577 (knowledge of food preparation) and 0·380 (confidence in cooking skills), although their values may have been attenuated due to some changes in the questions between the first and second measurement. In a study of Calfas et al.(Reference Calfas, Sallis and Nader14) in which children aged 4–8 years were presented with pairs of food images and asked to point to the food that would make their baby (a doll) healthy, big and strong, test–retest reliability was 0·72, with the highest reliability in 5–6-year-olds and an unexpectedly low value (r = 0·3) in the 7–8-year-olds. In a study of Gower et al.(Reference Gower, Moyer-Mileur and Wilkinson23) among children 6–10 years of age, the test–retest correlation of a fifteen-item scale was 0·54, with a correlation of r = 0·51 for the subscale food groups, r = 0·65 for healthful foods and r = 0·49 for food functions. In a group of children in grades 3 to 5, test–retest correlation of a ten-item knowledge of high fat foods was 0·52(Reference Stevens, Cornell and Story39). Very low ICC were found in a study of Wilson et al.(Reference Wilson, Magarey and Mastersson40) (fruit knowledge: ICC = 0·16, vegetables: ICC = 0·36); however, they used single items in stead of scales. Also in our questionnaire several single items and even some of the sum scales (e.g. ICC = 0·33 for the nutrient content scale) showed low reliability and this even more in the lowest grade (nutrient content, main function and recommended portions: ICC ≤ 0·18).
Unexpectedly, a significant decrease was found between T1 and T2 for children's healthy choices score and children's total knowledge score. We suspect that some might have been less motivated the second time due to questionnaire fatigue, leading to more superficial and less accurate responses(Reference Borgers41).
Comparison of the children from the different grades showed that the questions were more difficult for the second graders. Considering their lower cognitive abilities this is not surprising. Low cognitive ability and difficult questions may also lead respondents to provide more superficial responses instead of optimal ones(Reference Borgers41). This in turn may lead to more randomness and lower reliability(Reference Borgers41), explaining the lower reliability found in the second graders. The higher ICC in the total sample in comparison with the ICC of each of the grades can be explained by the higher heterogeneity in the total sample(Reference Pinna, Maestri and Torunski38).
Based on the analyses some changes are suggested. In general, most children identified water, milk, fresh fruit salad, jam, oranges and grapes as the healthiest choice, most knew that an apple was a healthy snack and identified the three unhealthy snacks. If all or most respondents answer an item in the same way then this item is not capable of discriminating between respondents(Reference Parmenter and Wardle32). Moreover, deleting these items from the healthy choices score did not change test–retest statistics substantially; as a consequence these items are not likely to contribute and may be removed in future studies.
In the nutrient content section, we suggest to replace the item ‘Frozen vegetables always contain more vitamins than fresh vegetables? yes v. no’ by ‘Fruits contain more vitamins and minerals than vegetables/Vegetables contain more … than fruit/Some vitamins and minerals are more available in fruits, others more in vegetables’ based on the difficulty index and the low κ value of this item. In the section on the main function of food items, a low difficulty index was found for the item on the main function of rice; we suggest replacing this item with a question on the main function of potatoes, as this is more common in Flemish food culture. In the categorization task a low difficulty index was found for the series of ‘muesli, nuts, rice and pasta’ and the series ‘ham, cooked egg, cheese and salami’. We suggest replacing the ‘rice toast’ with a more familiar food such as ‘rice’ or ‘cooked potatoes’, and the ‘cooked egg’ with an ‘omelette’, so that the shape is more equal to the other food items.
Children were in general positive about the questionnaire, even after a second measurement. This is not unimportant, especially when in the framework of an intervention children have to complete the same instrument more than once.
Finally some limitations should be noted. Representativeness cannot be assumed as the sample was a convenience sample and the response rate was rather low (56 % of parents gave consent). A second limitation is that only a small number of items could be included in each domain; however, in each domain the different main food groups were represented as much as possible. Finally, despite that a pilot test was done, some questions/remarks turned up during the data collection, which can help to further refine the instrument (e.g. rice crispies would better be replaced by a more familiar breakfast cereal). A strength of the study is the large sample size for a study of test–retest reliability and the multiple age groups included.
Conclusions
The instrument is a promising, practical, inexpensive, pleasing and easy-to-administer tool with an acceptable reliability for fourth and sixth graders. For ranking second graders according to nutritional knowledge, the healthy food choices and food categorization might be useful. Further research with the instrument to evaluate the effect of a web-based tailored intervention is warranted. But the instrument could also be useful outside an intervention study, for example as part or a starting point of a school-based nutrition education programme to highlight gaps in nutritional knowledge.
Acknowledgements
C.V. is a postdoctoral researcher supported by the Research Foundation – Flanders (FWO-Vlaanderen). There are no conflicts of interest. The data were collected by A.D.P., S.V.C. and Julie Parmentier as part of their Masters theses. A.D.P. and S.V.C. did the initial analyses; C.V. did the final analyses and wrote the manuscript. All other authors read and approved the manuscript. The authors would like to thank parents and children for participating in the study.
