Agreement between an online dietary assessment tool (myfood24) and an interviewer-administered 24-h dietary recall in British adolescents aged 11–18 years

Abstract myfood24 Is an online 24-h dietary assessment tool developed for use among British adolescents and adults. Limited information is available regarding the validity of using new technology in assessing nutritional intake among adolescents. Thus, a relative validation of myfood24 against a face-to-face interviewer-administered 24-h multiple-pass recall (MPR) was conducted among seventy-five British adolescents aged 11–18 years. Participants were asked to complete myfood24 and an interviewer-administered MPR on the same day for 2 non-consecutive days at school. Total energy intake (EI) and nutrients recorded by the two methods were compared using intraclass correlation coefficients (ICC), Bland–Altman plots (using between and within-individual information) and weighted κ to assess the agreement. Energy, macronutrients and other reported nutrients from myfood24 demonstrated strong agreement with the interview MPR data, and ICC ranged from 0·46 for Na to 0·88 for EI. There was no significant bias between the two methods for EI, macronutrients and most reported nutrients. The mean difference between myfood24 and the interviewer-administered MPR for EI was −230 kJ (−55 kcal) (95 % CI −490, 30 kJ (−117, 7 kcal); P=0·4) with limits of agreement ranging between 39 % (3336 kJ (−797 kcal)) lower and 34 % (2874 kJ (687 kcal)) higher than the interviewer-administered MPR. There was good agreement in terms of classifying adolescents into tertiles of EI (κ w =0·64). The agreement between day 1 and day 2 was as good for myfood24 as for the interviewer-administered MPR, reflecting the reliability of myfood24. myfood24 Has the potential to collect dietary data of comparable quality with that of an interviewer-administered MPR.

Using new technology for dietary assessment offers several possible advantages. For instance, it has the potential to improve data quality, it can help standardise the questions and questioning sequence, it makes the processing of data easy, it produces immediate results and it increases privacy and confidentiality (5,6,7) . Recently, most technology-based selfadministered dietary assessment methods have been developed specifically for adults (8)(9)(10)(11)(12)(13) , whereas a limited number have been designed for adolescents. For instance, Food Intake Recording Software System version 4 (14) , which is available now as the Automated Self-Administered 24-hour dietary recall (ASA24)-kids, was developed in the USA for self-completion by children aged 10 years and older. In Belgium, the Young Adolescents' Nutrition Assessment on computer (YANA-C) (15,16) was developed for use among 11-14-year-olds, and a web-based version has been improved and adapted for use among young adolescentsnamely, the Children's and Adolescents' Nutrition Assessment and Advice on the Web (17) . In the UK, the Synchronized Nutrition and Activity Program for self-completion by 7-15-year-old children (18) provides similar information to the FFQ, and the interactive portion size assessment system has been adapted and extended for use among 11-24-year-olds (INTAKE24). INTAKE24 is an online multiple-pass 24-h dietary recall tool developed for use in the Scottish food and nutrition survey (19) .
Recently, Measure Your Food On One Day (myfood24) has been developed (20) . It is a UK online 24 h dietary assessment tool designed to address the need for a valid, reliable, lowburden and user-friendly dietary assessment method suitable for use among different age groups (adolescents and adults), with the aim of standardising the method used for the whole population. There is still limited knowledge regarding the accuracy of fully automated 24-h dietary recalls, and more research is therefore required to investigate their validity, particularly among different age groups (6) . Thus, this study aimed to assess the agreement between myfood24 and an interviewer-administered 24-h multiple-pass dietary recall (interview (MPR)) for use among British adolescents aged 11-18 years.

Sample size
Sample size calculation was based on mean and standard deviations of energy intakes (EI), of 7511 kJ (1795 kcal) (SD 2100 kJ (502 kcal)), as reported by adolescents (11-18 years old) in the National Diet and Nutritional Survey 2008/2011 (21) . In all, seventy participants were required to obtain 90 % power to detect a 10 % (837 kJ (200 kcal)) difference in mean EI reported by the two methods at a significance level of 0·05. This number would also provide an adequate precision for the Bland-Altman limits-of-agreement test (22) . Allowing for 9 % attrition, the aim was to recruit seventy-seven participants. To ensure a representative sample of adolescents from each age group, an effort was made to balance the sample in terms of age and sex.

Recruitment criteria
All adolescents were recruited from two high schools; thirty participants were recruited from a high school in the centre of Leeds and forty were recruited from a high school in the northwest of Leeds. The inclusion criteria were as follows: aged 11-18 years and being able to speak, read and write English; exclusion criteria were having any limitation that could inhibit the adolescent's ability to recall their diet or use a computer. Having experience of using the Internet or computers was not required. Pupils who were interested in taking part were given an information sheet and consent form. Written parental consent was also obtained if the participants were younger than 16 years of age. Ethics approval for the study was granted by the University of Leeds (Ethics reference: MEEC 11-046 (Phase 2)).

Measure Your Food On One Day (myfood24)
myfood24 Is an online self-administered 24-h dietary recall/ record tool, which has been developed to help researchers collect multiple automated dietary data in large-scale epidemiological studies. To reduce completion time of the food intake report, the myfood24 consortium chose not to pursue the detailed Automated Multiple-Pass Method (AMPM). In myfood24, users are requested to go through as few webpages as possible to complete the food recall; pop-ups and prompts were limited. myfood24 Has retained some aspects of the AMPM by including an optional make-list function as the first pass, a detailed food search, prompts for commonly forgotten foods and a final review before submission. myfood24 Has the advantage of being linked to an extensive (approximately 50 000 items) branded electronic food composition database (23,24) . myfood24 Has food portion images that were obtained from the Young Person's Food Atlas (25) . myfood24 Was developed with the flexibility to be used as either a food diary or to be self-or interviewer-administered as required. More detailed information about myfood24's design and features has been provided in published reports (26) .
In this study, myfood24 was used as a self-administered 24-h dietary recall method, with free use of myfood24's features and functions (no instructions were given to adolescents about how to use myfood24, only brief polite points were presented on the first webpage of myfood24 (project instructions)). We wanted to reflect how the tool could be used in the future, where individuals will receive an email to use the system by themselves, with no additional guidance available.

Reference method: interviewer-administered 24-h multiple-pass recall
Face-to-face interviews were administered for the 24-h dietary recall with an identical time frame to myfood24 to ensure that participants completed myfood24 on the same day as the reference method. All interviewers were postgraduate nutritionist students and they received training, a standardised protocol and material to administer the multiple-pass method (27) . All the participants were interviewed at school in Online dietary assessment tool a separate area. The Young Person's Food Atlas (25) was used to estimate food portion size in the interview MPR. After collecting the 24-h dietary recalls, the interviewer reviewed the report once it was completed. Data generated from the interview MPR were coded by one of the three trained coders using myfood24 in order to have the same database available. Each coder received training and a standardised protocol for coding, which was developed by the researchers to deal with missing or ambiguous data. This was to increase consistency in coding and to minimise measurement errors related to coding and food composition databases. After coding all dietary recall interviews, a detailed review and verification for quality control were carried out by two members of the research team where any coding errors were corrected to match the reported interview. No data have been excluded and no changes were made to the student recalls from myfood24.

Data collection
After identifying adolescents who were willing to take part in the study, participants were asked to attend one of the fourteen groups at their school. They were asked to attend one session on 2 non-consecutive days over the course of 2 weeks for each session. The 2 d were selected based on the availability of students in school. As the interview was not conducted on Saturday or Sunday, the recall only covered Monday to Thursday and Sunday (week and weekend days). In each session, participants were asked to report all foods and drinks consumed in the previous day in myfood24 using the school's computer and then attend an interview with one of the two trained nutritionist interviewers in order to complete the interviewer-administered multiple-pass 24-h recall for the same day. The interviewers were blind to what the participants had entered into myfood24 and each participant was interviewed separately. All participants were given a unique username and password to use myfood24.
For logistical reasons, in order to manage the research in schools, each group of students was randomly assigned to attend a 1-h session, three of them were asked to start with myfood24 (in order to avoid the learning effect with myfood24) and two with the interviewer-administered MPR first. Next, they were swapped over when they had completed their first tool. On the 2nd day, participants who started with the interviews on the 1st day were asked to start with myfood24 and vice versa. When students asked for help, the researcher advised them to check the instructions on the website, and encouraged them to select the most appropriate food they consumed (no help was given in selecting food or portion size). The target reference period was from midnight to midnight the previous day, with an identical time frame for both methods.
On the 2nd day, after conducting the two tests, the participants completed an anonymous questionnaire to identify their backgrounds, attitudes towards technology, evaluation of myfood24 and tendencies to respond in a socially desirable way, using the Children's Social Desirability (CSD) scale, which contains fourteen items (online Supplementary Table S1). At the end of the study, each participant received a £5 voucher as a token of appreciation for their time and effort.

Statistical analyses
Analyses were performed using Stata statistical software release 11 (StataCorp LP) (28) , and for all analyses the significance level was two-sided and set at 0·05. Descriptive statistics (mean values and standard deviations) were used to define sample characteristics. The method of Bland and Altman was used to measure the extent of agreement between the two dietary assessment methods (29) , of which there are two components: bias and precision. These were assessed by using the limit of agreement between methods. Both aspects of agreement were assessed using information from two observations of each method per individual, assuming that the within-subject variance is constant and the two measures for the same subject on different days are independent (30) . The intraclass correlation coefficient (ICC) between myfood24 and interview (MPR) was also calculated using a two-way mixed-effects ANOVA model, with a subject-by-method interaction (31) . In order to assess whether myfood24 agrees with interviewer MPR in ranking individuals in the same category, dietary intake was classified into tertiles for both methods. A linear weighted kappa (κ w ) (32) was used to evaluate the level of agreement over and above that which would be expected by chance and to take into account the amount of disagreement between the methods. This analysis was carried out for the average of the 2 d.
Secondary analyses were carried out to investigate whether there was a significant difference between the differences in EI (myfood24-interview (MPR)) for the average of the 2 d and sex or age group (younger adolescents to older adolescents) using regression models; the difference in EI was the dependent variable, and sex (model-1) and age group (model-2) were the independent variables. Furthermore, a CSD score was calculated and possible scores ranged from 0 to 14, with higher scores indicating a higher tendency towards socially desirable responses (33) . A Spearman rank correlation was used to measure the association between the CSD score and the differences in EI. All secondary analyses were carried out with the average of EI for the 2 d. Analyses were conducted on key nutrients presented by the myfood24 summary file.

Characteristics of the study sample
In total, seventy-five adolescents took part in this study: thirtyeight girls (51 %) and thirty-seven (49 %) boys, aged between 11 and 18 years. In all, sixty-six (88 %) of them were of white ethnicity. An effort was made to balance the sample in terms of age and sex in each age group. Only five adolescents (7 %), three girls and two boys were unable to complete the 2nd round. The recall covered 2 d including weekdays (Monday to Thursday) and a weekend day (Sunday). From the 2 d, 26 (18 %) weekend days were included in the study ( Table 1). The mean social desirability score was 5·3 (95 % CI 4·8, 5·6), and there were no significant differences in adolescents' CSD scores between older adolescents and younger adolescents (mean difference 0·02; 95 % CI −1·2, 1·1, P = 0·1) or between girls and boys (mean difference 0·3; 95 % CI −1·4, 0·9, P = 0·6). Reported nutrient intakes by myfood24 and the interviewer-administered 24-h dietary recall multiple-pass method for each day Table 2 illustrates the daily intake of energy (kJ (kcal)), macronutrients (g) and some nutrients (g), as reported by myfood24 and the interview (MPR) for day 1 and day 2 separately. Daily EI, macronutrients and most nutrients were similar on day 1 and day 2, with SFA and sugars being slightly lower on the 2nd day. In general, the interview (MPR) appeared to record slightly higher EI and macronutrient values than myfood24.
Agreement between myfood24 and the interviewer-administered 24-h dietary recall multiple-pass method for each day Table 3 shows the limits of agreement and ICC for EI (kJ (kcal)), macronutrients (g), fibre (g), SFA (g), Na (g), sugars (g), total vegetables (g) and total fruits (g). myfood24 Underestimated EI compared with the interview (MPR); the mean difference was −230 kJ (−55 kcal) (95 % CI −490, 30 kJ (−117, 7 kcal), P < 0·398), with a limit of agreement of −3336 to 2874 kJ (−797 to 687 kcal). This difference was equivalent to 2·8 % of the average EI, and the limit of agreement ranged from an underestimation of 39 % to an overestimation of 34 % for an average EI (Fig. 1). Although there are significant differences between the two methods in the reported fibre and sugars intakes, the ICC was high at 0·76 and 0·75, respectively. The ICC for EI and other reported nutrients were high between the two methods, and ranged from 0·46 for Na to 0·88 for EI. The ICC between the time points (day 1 and day 2) for the two methods were similar (Table 4). It was 0·5 (95 % CI 0·37, 0·63) for myfood24 and 0·49 (95 % CI 0·36, 0·62) for the interview (MPR) for the reported EI, and similar ICC were also found for all other nutrients.
Agreement on ranking of energy and macronutrients Table 5 presents the strength of agreement between myfood24 and the interview (MPR) on ranking of EI and macronutrients into the same tertiles for the average of the 2 d. The percentage agreement between the two methods was good, with the percentage classified into the same or adjacent tertiles ranging from 80 % for proteins (κ w = 0·55) to 86 % for carbohydrates (κ w = 0·71).

Secondary analyses
Although girls had a greater difference in EI between the two methods than boys, no significant difference was found between the differences (−414·6 kJ (−99·1 kcal); 95 % CI −207, 9·3; P = 0·07). Moreover, no significant differences were found in younger adolescents compared with older adolescents in terms of the difference in EI between the two methods (41 kJ (9·8 kcal); 95 % CI −101, 120; P = 0·86). Furthermore, the correlation between CSD score and the difference in reporting EI between the two methods was tested to investigate whether the differences in reporting EI were related to the adolescents' social desirability. Adolescents' CSD was not associated with the difference between the two methods (Spearman rank correlation r −0·07; 95 % CI −0·29, 0·17; P = 0·58).

Discussion
This study demonstrates that myfood24 is an appropriate, reliable and easy-to-use (20,34) tool among British adolescents aged 11-18 years. myfood24 Has the potential to collect dietary data of comparable quality to that of an interview (MPR), which is considered to be the gold standard in the USA and is the most widely used method. There were strong ICC between myfood24 and the interview (MPR) for EI and most of the reported nutrients. The relative bias in EI between the two methods was small and not important.
There are a limited number of studies that have assessed the validity or relative validity of 'interactive computer-based' and 'web-based' 24-h recalls (10,11,(35)(36)(37) and specific limitations exist when adolescents are the target age group. Studies comparing self-administered with interviewer-administered computerised 24-h recalls have observed a small but significant underestimation of energy and fat intake with the self-administered tool in general (6) .
Most of the studies comparing a computerised approach to an interview or food diary have found larger differences and considerably wider variation between the methods for key nutrients compared with self-reported recalls in our study. For example, the EI reported by the computerised 24-h recall YANA-C was higher by 13 % (1096 kJ (262 kcal)) when it was compared against a 1-d food record (n 136), with a limit of agreement ranging from 86 % (7033 kJ (1681 kcal)) to 60 % (−4841 kJ (−1157 kcal)). However, the EI reported in YANA-C was higher by 5·5 % (427 kJ (102 kcal)) with a limit of agreement ranging from 68 % (5276 kJ (1261 kcal)) to -57 % (−4427 kJ (−1058 kcal)) when compared with the interview (n 101), and the researcher guided the pupils when completing YANA-C (15) . When YANA-C was compared with the intervieweradministered YANA-C data for (n 236) adolescents from eight European cities, the EI was higher compared with the interview by 3 % (255 kJ (61 kcal)), with a limit of agreement ranging from 41 % (3778 kJ (903 kcal)) to −47 % (-4289 kJ (-1025 kcal)) (16) .
myfood24, Measure Your Food On One Day. * Daily intake of energy (kJ and kcal) and macronutrients such as proteins, carbohydrates and fats (g) as recorded by myfood24 and the interview (MPR) for the equivalent day.
* Daily intake of energy (kJ and kcal) and macronutrients such as proteins, carbohydrates and fats (g) as recorded by myfood24 and the interview (MPR) for all days. † Intraclass correlation coefficients (ρ) between myfood24 and interview (MPR).

S. A. Albar et al.
A recent study compared INTAKE24 (un-aided) with an interviewer-administered 24-h recall on the same day for 4 non-consecutive days conducted among 11-16-year-old adolescents. The findings showed that the average EI reported by INTAKE24 was lower than the interviewer-administered 24-h recall by 3 %, with a mean ratio of 0·97; the limit of agreement ranged from 82 % (upper mean ratio 1·82) to −48 % (lower mean ratio 0·52) (38) . Findings from the YANA-C and INTAKE24 studies were consistent in terms of absolute differences with the results for myfood24. However, the limits of agreement were narrower in myfood24 compared with other tools, possibly reflecting the additional information provided by using brand-specific nutrients. ASA24-Kids-2012 was less accurate than the interviewer-administered 24-h dietary recall when compared with observed intakes; nonetheless, both methods performed poorly among children aged 9-11 years (39) . However, in a feeding study in which the true intake for three meals was known, eighty-one adults (20-70 years old) completing the ASA24 reported 80 % of the foods and drinks actually consumed compared with 83 % in AMPM (40) .
Adolescents in this study tended to underestimate fibre and sugar intakes using myfood24 compared with the interview MPR. Similarly, with INTAKE24, adolescents underestimated non-milk intrinsic sugar by 11 %, and it was found that high-sugar drinks were one of the most commonly omitted food items in the tool (38) . Furthermore, it was noticed that adolescents in this study, particularly young adolescents, found it difficult to estimate both the amount of corn flakes consumed and the amount of milk consumed with the corn flakes, as the food photographs available in myfood24 only included bowls of cereal without milk. Therefore, adding some information in 'project instruction' (first screen in myfood24) about how to estimate corn flakes portion size would be useful (e.g. give an example for the standard portion size of corn flakes with and without milk), as well as emphasise the importance of reporting all sugary drinks and added sugar (extrinsic sugars).
In this study, Na and total vegetables had the lowest ICC and this is likely to be related to the choice of different food codes between the two methods that did not match well (far matches) or omissions/intrusions. In INTAKE24, for example, the vegetable group had the largest percentage of food omissions (17 %) among adolescents aged 11-16 years (38) . Alternatively, it may relate to food items with a greater number of options, with respondents selecting items closer to the top of the list of search results rather than scrolling through the whole list to find the best match. This was suggested as being the case in an assessment of adults' responses to the ASA24 (40) . Similar mechanisms may explain the lower ICC for some nutrients and food groups.' In this study, no significant differences were found between males and females in the differences in terms of reported EI between the two methods. This may highlight an advantage of using new technology in a dietary assessment method, as most adolescents nowadays share the same social characteristics, particularly regarding their use of the internet (41) . Moreover, boys and girls have become similar in terms of their computer activities (42) . In addition, no significant differences were found between younger adolescents and older adolescents in the differences between the two methods when reporting EI.
Despite there being no statistically significant differences between younger and older adolescents, some difficulties were noticed in our study among year 7 adolescents (11 years old) when they were reporting their food intake, particularly when identifying certain types of food, such as whole milk or semi-skimmed milk, or the cooking method used. This corresponds with other studies (17,43) that have found that adolescents aged 11-12 years should complete the tools with assistance from an adult. Owing to adolescents' limited knowledge of food names and types, their ability to self-report their food intake without support is limited (3) . Therefore, we recommend that adolescents <12 years of age would require assistance from the researcher or parents to obtain more accurate data, regardless of the dietary assessment method used (traditional methods or an online dietary assessment method, especially if the new method contains a large database). However, the assistance could be provided at a group level (e.g. at school classes) when using an online dietary assessment tool, as there is no difference between 11-12-year-old adolescents and adults in technical understanding of websites (44) .
Self-reporting of dietary intake is susceptible to social desirability bias, which is rarely evaluated in the development of new dietary assessment tools (45) . The association between CSD score and the differences in EI between the two methods has been investigated in this study. The CSD score was developed based on the Marlowe-Crowne Social Desirability (MCSD) scale for adults (46) . Higher scores on the MCSD scale were related to lower accuracy of reporting EI, with under reporting of fats, sweets and total EI (47) . In this study, the mean tendency towards social desirability among adolescents was 5·3 out of 14 and it was not associated with the difference EI between the two methods.
Regarding study strengths, this study has followed the general principles of an appropriate relative validity study design, as the test method and the reference method have measured the same underlying concept over the same time period (48) . Moreover, 2 d were used to ensure the accuracy and reproducibility of myfood24. There are considerable advantages in collecting replicate observations, so that the repeatability of the methods can be observed. The present study has also examined whether social desirability response bias is a source of measurement error in validation.
In terms of the limitations of this study, 'relative' instead of 'absolute' validity was applied by comparing myfood24 with a face-to-face interview MPR; thus, measurement error may have occurred as the two methods are not totally independent from each other, and the subject is not totally independent on the 2 different days. However, measurement error cannot be independent in all dietary assessment methods and it is difficult to measure the absolute validity of dietary intake (49) . The main validation study for myfood24 is being conducted in a large sample of adults against reference measures (an intervieweradministered MPR) and biomarkers of nutritional exposure including urinary N (for protein), K, sucrose and fructose and total energy expenditure (TEE).
The process of collecting dietary information using one method may affect the response to the other method, as participants may become more conscious about their diet and improve their recall in the second method (1) . In order to limit this effect on the use of myfood24 and to test the tool in a way that will be used in practice, we had intended that all students would complete myfood24 before the MPR. However, owing to the availability of the interviewers, computers and students, a pragmatic approach was taken. An unbalanced design was used in the order of administration of the two methods, with more participants completing myfood24 first than the interviewer-administered MPR. However, no statistical differences were found in the order of the administration of the two methods, which was explored in a subsample analysis (online Supplementary Table S2). An alternative approach would have been to randomise the order of the two instruments, as it is possible that students would be more conscientious about completing the first instrument and perhaps would have lost interest for the second method. However, the mode of delivery of the two methods was different, and thus potentially minimising any such effect.
The aim of this study was to evaluate adolescents aged 11-18 years and their ability to use myfood24, rather than their ability to use any underlying database. Therefore, the same food database has been used in the two methods, which may enhance the agreement and correlation between them, but provides a clearer comparison of the tool itself.
Limited numbers of dietary assessment methods have been found to be reproducible and valid for use among adolescents (50) . A review by Forrestal (51) of twenty-eight studies found that retrospective methods, especially 24-h dietary recalls, are preferred for use among adolescents. In general, there is a tendency for a higher estimate of EI by two non-consecutive 24-h recalls compared with a 5-d estimated food record. The 24-h recall is a valid method for assessing dietary intake at the group level, and even young children could estimate their EI with 78 % accuracy when compared with food records (50,52,53) . At the group level, the multiple-pass 24-h recall accurately reflected mean EI with no difference between the mean of 3 d EI and TEE estimated by doubly labelled water (54) .
to rectify the issue of potential under-reporting of EI among adolescents or adults (3,6) . This may be due to the cognitive processes involved in dietary recall. It has been suggested that what one eats is stored in the generic memory and rarely encoded into long-term memory (3) . In addition, adolescents underestimate EI by 18-42 % when using a food record (55) . Findings from a focus group study found that adolescents preferred to report their food intake at the end of the day as they still remembered what they had consumed, rather than via a 24-h recall or food record (56) . However, that was not possible for the validation study; although myfood24 has the option to be used as a recall or food record, it would have been impractical to perform the interview at the end of the day. Therefore, further studies are required to investigate the accuracy of reported adolescents' food intakes using myfood24 at the end of the day, comparing the findings with estimated energy expenditure.

Conclusion
myfood24 Is an online 24-h dietary assessment method developed to meet the need for an accurate national online dietary assessment tool. The findings of this study confirm that myfood24 has the potential to collect accurate dietary data that are comparable in quality with an intervieweradministered 24-h recall (MPR) among adolescents aged 11-18 years, while being less laborious in terms of the data collection method. However, using an online dietary assessment tool may not be feasible for adolescents <12 years of age without assistance. myfood24 Is currently being validated in adults against nutritional biomarkers. Further research is required to test the feasibility of using myfood24 in a large epidemiological study with different age groups so as to standardise and automate dietary measurement within the UK population.