Dietary assessment in large prospective epidemiological studies investigating diet–disease associations is particularly challenging. Large numbers of study participants are required to provide statistical power to produce reliable results and, in order to account for changes in diet as well as measurement error, it is desirable that dietary information be captured through repeated measures over a period of time(Reference Schatzkin, Subar and Moore1).
All the commonly administered dietary assessment tools used in prospective studies, the FFQ, multiday diet diary and 24 h dietary recall, have their limitations(Reference Bingham, Luben and Welch2, Reference Willett3). An FFQ is designed to capture ‘usual’ dietary intake, generally over a 1-year period, and is easy and cost-effective to administer in large populations; however, the dietary information captured by an FFQ is less detailed and possibly less accurate than that from other dietary assessment methods, as it requires the individual to estimate average intake over 1 year(Reference Schatzkin, Subar and Moore1, Reference Bingham, Luben and Welch2, Reference Thompson and Byers4). Multiday diet records capture precise dietary information over a number of days, as individuals usually record (and sometimes weigh) all foods and beverages consumed as they go through their day; however, they are time-consuming to complete, costly to code and usually record intakes for only a fixed number of consecutive days at one point in time. Moreover, they tend to be ‘reactive’ in that some participants alter their dietary intakes on the days that they are recording information(Reference Thompson and Byers4). Interviewer-administered 24 h dietary recalls also capture precise information, as interviewers ask individuals to recall everything they ate over the last 24 h; however, they are time-consuming, costly, represent intake from 1 d only and rely on the participant's memory. However, if administered repeatedly over time 24 h dietary recalls are considered to be representative of usual intake(Reference Schatzkin, Kipnis and Carroll5). Hence, a 24 h dietary assessment tool that can be efficiently and cost-effectively administered repeatedly over time is an attractive method of assessment to capture dietary information for large epidemiological studies(Reference Schatzkin, Subar and Moore1).
To address this need, we designed a low-cost, 24 h dietary assessment tool that can be completed over the Internet, the ‘Oxford WebQ’. We aimed to have an Internet-based dietary questionnaire that would be self-administered, would take no more than 15–20 min to complete, would calculate nutrient intakes automatically and would provide information comparable to a traditional interviewer-administered 24 h dietary recall on the types and quantities of foods and beverages consumed and the daily nutrient intakes. In the present report, we describe the development and initial evaluation of the Oxford WebQ.
Methods
The Oxford WebQ is a dietary questionnaire that is administered over the Internet. It has been designed for use in several large-scale prospective studies in the UK, including the European Prospective Investigation into Cancer and Nutrition (EPIC)–Oxford study(Reference Davey, Spencer and Appleby6) (65 000 men and women), the Million Women Study(7) (1·3 million women) and the UK Biobank(8) (500 000 men and women). Similar to a standard interviewer-administered 24 h dietary recall, the aim of the Oxford WebQ was to obtain information on the quantities of all foods and beverages consumed over the previous day.
Description of the Oxford WebQ
The current version of the Oxford WebQ can be viewed at https://questionnaires.ceu.ox.ac.uk/diet/show/index.html. The questionnaire was structured to record information about foods consumed from each of twenty-one food groups. For the majority of these groups, individuals were presented with a single question on the screen. The online questionnaire was developed to take advantage of computer technology in such a way that a positive answer to a question (e.g. Did you eat any bread or crackers yesterday?; Fig. 1a) would result in the screen expanding (Fig. 1b) to reveal a list of commonly consumed breads and crackers.
Screenshots of the compressed (a) and expanded (b) questions on bread and crackers in the Oxford WebQ
Participants are then asked to select the amount of each food consumed during the previous day using standard categories to indicate the amount consumed (e.g. four slices of bread during the day). For foods without a natural size (e.g. cheese, rice), a portion size is specified as a ‘serving’ with a description of that particular serving size in the help section of the WebQ. If participants’ serving of the food item is twice the specified amount, they are required to double it. Where appropriate, a positive response to certain food items will open up new screens. For example, selecting ‘bread’ will open up an additional screen asking what type of flour the bread contained (e.g. white, granary, wholemeal; Fig. 2). To overcome the problem of unanswered questions, participants cannot proceed to the next page until they have answered the question. So that the replies can be coded automatically to provide nutrient information, open-ended questions are not used, although some free text boxes are available for use when the options listed do not fit with what the participants have consumed. At the end of the WebQ, a summary page of all food and beverage items and the quantities reported appears and participants are asked to check the list and make amendments where necessary.
Screenshot of the expanded question on sliced bread in the Oxford WebQ with added detail on flour type
The quantity of each food or drink consumed during 24 h is calculated by multiplying the assigned portion size(9) of each food or beverage by the amount consumed. The nutrient intakes for each participant are calculated by multiplying the quantity consumed by the nutrient composition of the food or beverage, as taken from McCance and Widdowson's The Composition of Foods and its supplements(Reference Holland, Welch and Unwin10–Reference Holland, Unwin and Buss20), and stored in a secure database.
Further development of the Oxford WebQ
The format of the WebQ and the structure of the questions asked were based on feedback from extensive piloting of the questionnaire by staff working at the Cancer Epidemiology Unit and the Clinical Trial Service Unit at the University of Oxford. In the pilot studies, information typed in the free text sections was used to refine the WebQ until very few items were entered as free text. The food groups and foods in the questionnaire were chosen to encompass the major foods consumed in the UK using the information from population dietary surveys(Reference Hoare, Henderson and Bates21) and earlier pilot studies, and to address current hypotheses about certain foods and diseases.
We also used information collected in a prospective cohort – the EPIC–Oxford study(Reference Davey, Spencer and Appleby6) – to further develop the Oxford WebQ. We compared the nutrient intakes obtained from data recorded on a single day of a 7 d diet diary and entered into a nutrient calculation software program with the same data entered into the Oxford WebQ. We randomly selected 101 food diaries from 1400 study participants who had nutrient intakes calculated for each day of their diary using DINER (Data Into Nutrients for Epidemiological Research) and DINERMO(Reference Welch, McTaggart and Mulligan22) and then randomly selected a single day from each of the 101 diaries. The food items and corresponding quantities that were written in the diary on this day were entered into the Oxford WebQ by data entry personnel who were blinded to the nutrient intakes calculated previously. Overall, there was good agreement between the estimated energy and nutrient values (Spearman's ρ = 0·6–0·8). Where agreement was not good and where dietary items had been entered as free text we made modifications to the Oxford WebQ. This included revising portion sizes and the nutrient codes assigned to some of the food items, in particular to the composite food items. We also added more food items to the choices available.
Comparison of the WebQ with an interviewer-administered 24 h dietary recall
For the study reported here we recruited 116 adult volunteers from June to August 2008. Volunteers were invited to attend the Cancer Epidemiology Unit at the University of Oxford through an email invitation using the local university and staff mailing lists. The present study was approved by the Central University Research Ethics Committee of the University of Oxford and participants provided written informed consent. Participants were asked to complete the Oxford WebQ, unassisted, recording their dietary intake of the previous day. In this version of the Oxford WebQ was also a question asking participants, ‘Compared to other people, how would you describe your serving size?’ with the options of ‘smaller’, ‘average’ and ‘larger’ available. Immediately after completing the Oxford WebQ, the participants completed a standardised multiple-pass 24 h dietary recall administered by two trained interviewers regarding their consumption of food from the previous day. The interviewers were blinded to what the volunteers had entered in the Oxford WebQ. Briefly, participants were asked to provide a list of all foods and beverages that they had consumed the previous day. A checklist was then used to prompt for the consumption of snack foods and drinks between meal times. The interviewers captured more details of brands, accompaniments and portion sizes of all foods and beverages consumed using a food atlas to assist participants where possible(Reference Nelson, Atkinson and Meyer23). A final pass summarised what had been recorded and confirmed any extra items that may have been recalled. Dietary information collected from the interviewer-administered 24 h dietary recall was entered into the DINER program by the interviewers, followed by a series of checks regarding available foods and portion sizes and correction of data-entry mistakes in order to calculate total energy and nutrient intakes.
Statistical methods
The intakes of total energy, macronutrients and selected micronutrients from the Oxford WebQ were compared with those estimated from the interviewer-administered 24 h dietary recall. The nutrients compared were total energy, protein, total fat, saturated fat, polyunsaturated fat, carbohydrate, starch, total sugars, Englyst fibre, alcohol, Ca, Fe, Mg, K, carotene, folate and vitamins B6, B12, C, D and E. The percentage differences between energy and nutrient intakes were calculated by subtracting the comparator estimate (interviewer-administered 24 h dietary recall) from the Oxford WebQ estimate and dividing by the comparator estimate. Agreement between estimates for each nutrient was assessed using Spearman's ρ for rank correlation and calculating the percentage of participants who were categorised into the same sex-specific tertile. We also examined the effect of adjusting for portion size by subtracting or adding a percentage to the portion size of all foods in cases where participants indicated that their portion sizes were, respectively, smaller or larger than average. The tested portion size adjustments were 10 %, 20 % and 25 %.
As most prospective studies on diet and disease risk examine associations by comparing disease incidence in categories of the dietary factor of interest, we divided participants into tertiles of intake for each nutrient, as estimated from the Oxford WebQ. The mean intake among participants in each tertile of the WebQ was plotted against the mean intake among the same participants as estimated from the interviewer-administered 24 h dietary recall. The plots were visually inspected for evidence of consistent under- or overestimation of nutrient values between dietary assessment methods, and also for regression dilution(Reference MacMahon, Peto and Cutler24). All analyses were conducted using the STATA statistical software package version 9·2 (StataCorp., College Station, TX, USA).
Results
The mean age of the 116 participants included in this comparison was 42 (range: 19–82) years and 72 % were women. It took participants an average of 14 (median:12·5, interquartile range: 10·8–16·3) min to complete the online WebQ. On average, it took approximately 30 min to complete the interviewer-administered 24 h dietary recall and another 30 min for this information to be entered into a nutrient calculation software program. The results in Table 1 show the mean energy and nutrient intakes, differences in means, Spearman's correlation and percentages in the same sex-specific tertiles based on the WebQ and interviewer-administered 24 h dietary recall. Compared with the 24 h dietary recall, most nutrients estimated by the WebQ differed by <10 %. Only for carotene and vitamins B12 and D were the differences greater. The rank correlations between the 24 h dietary recall and the WebQ were mostly about 0·6 (mean = 0·6) and generally ranged between 0·5 and 0·9, but were 0·41 for carotene and 0·37 for vitamin E. The percentage of participants who were categorised into the same sex-specific tertile for the two dietary methods was about 50–70 % for most nutrients and ranged from 43 % for vitamin E to 92 % for alcohol.
Comparison of mean total energy and nutrient intakes between 116 men and women who completed a 24 h dietary recall that was entered into DINER and the same individuals who completed the Oxford WebQ
DINER, Data Into Nutrients for Epidemiological Research.
*All values are statistically significant at P < 0·001.
†Calculated as the difference of the mean (WebQ − 24 h dietary recall) divided by the mean 24 h dietary recall.
Adjusting the portion sizes used in the modified WebQ by adding 10 %, 20 % or 25 % for those participants who reported consuming a larger portion size (n 13) and subtracting 10 %, 20 % or 25 % for those who reported consuming a smaller portion size than average (n 9) had little effect on either the mean nutrient intakes from the WebQ or the measures of agreement between the 24 h dietary recall and the WebQ (results not shown).
Figure 3 shows the mean and se from the 24 h dietary recall plotted against the mean intake among tertiles in the WebQ for total energy and macro- and micronutrients. There were no systematic differences between the WebQ and the 24 h dietary recall because the plotted points did not sit consistently above or below the dotted line of equality. As expected, the relationship between the means generally followed a pattern consistent with regression to the mean on re-measurement; that is, for the lower tertile the mean value from the 24 h dietary recall was greater than that estimated from the WebQ, and for the upper tertile the mean value from the 24 h dietary recall was less than that estimated from the WebQ.
Mean and se of the interviewer-administered 24 h dietary recall plotted against the mean of the tertiles from the Oxford WebQ for total energy and nutrient intakes. The dotted line represents a line of equality between the two dietary assessment methods
When we compared detailed responses to the WebQ with those recorded in the interview we found several disagreements. The estimated amounts or portion sizes for food items occasionally differed slightly between the two dietary methods. For instance, participants recorded discordant numbers of cups of tea or coffee, or the estimated portion sizes of food items were not equivalent between methods; for example, one participant entered one portion of carrots that was assigned a portion size of 60 g in the Oxford WebQ, and in the interviewer-administered 24 h dietary recall this was recorded as 90 g. Differences were also apparent when recalling composite food items: one participant recorded one serving of ‘mixed fruit’ on the Oxford WebQ, but recalled consuming larger quantities of a number of different types of fruit in the interview. In another case, one participant recorded consuming a handful of nuts on the Oxford WebQ but did not report this item to the interviewer. This participant also reported consuming a jam tart to the interviewer, having failed to record an equivalent item in the Oxford WebQ.
Discussion
The present report describes the development and initial evaluation of the Oxford WebQ, a new low-cost instrument for assessing diet in large-scale prospective studies. We found that, compared with an interviewer-administered 24 h dietary recall, the Oxford WebQ provided similar mean estimates of energy and nutrient intakes and had reasonable ability to discriminate and rank participants according to the intakes of energy and most nutrients. These findings suggest that the low-cost Oxford WebQ may be a suitable method for assessing dietary intake, although results from a validation study are needed to confirm this.
Most published studies evaluating dietary assessment tools usually compare methods such as an FFQ (based on consumption over the previous year) with diet records(Reference Willett and Lenart25). For most nutrients, the correlations between the WebQ and the interviewer-administered 24 h dietary recall were higher than or equivalent to those reported in other studies comparing a new dietary assessment method such as a modified FFQ or food group questionnaire with multiple 24 h dietary recalls(Reference Boucher, Cotterchio and Kreiger26, Reference Kaaks, Slimani and Riboli27). The present study differed in that it aimed to assess how a single administration of the Oxford WebQ compared to a single administration of an interviewer-administered 24 h dietary recall for the estimation of energy and nutrient intakes. Nevertheless, the results from the present study were at least equivalent to those from other studies comparing newer dietary assessment tools used to assess dietary intake (e.g. interactive multimedia dietary recall, personal digital assistants or computerised dietary assessment tools) with the dietary intake assessed by a single 24 h dietary recall(Reference Ngo, Engelen and Molag28).
For some micronutrients such as carotene and vitamins B12, D and E the absolute values and rank correlations were only moderate to poor, although similar findings have been reported in other studies(Reference Bingham, Gill and Welch29–Reference Sevak, Mangtani and McCormack35). These discrepancies may have arisen because of differences in the codes assigned to foods, which are used to calculate the nutrient intakes for the two dietary assessment methods, or because of differences in the actual food items and the portion sizes recorded. In the UK, some but not all brands of margarine and breakfast cereals are fortified with nutrients such as vitamins B12, D and E; therefore, small differences between the two dietary assessment methods in the coding or quantity of these particular food items could lead to substantial differences in estimated nutrient intakes(Reference Gibson36).
Theoretically, the main limitations of the WebQ compared with a standard interviewer-administered 24 h dietary recall include less detailed dietary information and inability to probe for in-depth information on food preparation methods, food brands, and unusual food items that may have been consumed. For this reason the study was designed so that the WebQ was always completed before the interviewer-administered 24 h dietary recall, as we might expect the latter to capture more food items; however, on comparison of actual food items, this was not the case. Sometimes a food item was reported in only one type of assessment method but there was no consistency as to whether it was reported in the WebQ or in the interviewer-administered 24 h dietary recall. It is possible that discrepancies may have arisen instead from differences in the cues used for each dietary method; that is, photos of foods and lists of food items were prompted for in the Oxford WebQ, whereas verbal prompts were used for the interviewer-administered 24 h dietary recall.
For the purposes of estimating usual diet to investigate diet–disease associations in large studies, a low-cost questionnaire that can be administered repeatedly over time has the advantage of minimising random errors in the capture of dietary intake by obtaining repeat measures. Besides the low cost and ease of administration, other advantages of the Oxford WebQ are the immediate availability of nutrient intakes for analysis. The advantages of Internet dietary surveys are well recognised and other self-administered, web-based 24 h dietary recall questionnaires already exist(Reference Arab, Wesseling-Perry and Jardack37, 38). The Oxford WebQ was designed to be easy to use and takes approximately 15 min to complete, which has the theoretical advantage of being more accessible to groups of individuals who have lower rates of computer literacy. It also reduces the burden placed on study volunteers who would be asked to complete the questionnaire on repeated occasions.
The current version of the Oxford WebQ is being administered to participants in the UK Biobank and the Million Women Study, and the response rates of repeated administration of the WebQ will be assessed within these studies. A validation study in which participants will complete 4 d diet diaries and an online version of the EPIC–Oxford FFQ(Reference Bingham, Gill and Welch29), and then the Oxford WebQ on four occasions over 1 year is also underway. The nutrient intakes from the WebQ will be compared with intakes from the 4 d diet diaries and the FFQ, from recovery biomarkers – doubly labelled water, N and K in 24 h urine samples – and from plasma concentrations of carotenoids, folate and vitamins B12 and C. On the basis of the results from these studies, the allocation of food codes, portion sizes and food items in the Oxford WebQ could be amended if required.
Acknowledgments
The present study was funded by Cancer Research UK, the Medical Research Council, UK, and by the UK Biobank. The authors have no conflict of interest to declare. H.Y., V.S.B. and P.N.A. performed the statistical analyses and all authors were involved in conceiving and conducting the studies described, in developing the Oxford WebQ and in writing and reviewing drafts of the paper. The authors thank Maria MacGregor and Pam Dyson for their assistance in conducting the studies and Marleen Lentjes and the EPIC–Norfolk team for their help with DINER and DINERMO. The authors also acknowledge the Institute of Public Health at the University of Cambridge for the use of the DINER and DINERMO programs. Readers interested in using the Oxford WebQ are encouraged to contact Heather Young at dietweb@ceu.ox.ac.uk.
