Introduction
Accurate measurements of dietary intake are fundamental to health and nutrition research; however, diet is inherently complicated to measure as it changes over time and varies across life stages amongst other factors( Reference Kirkpatrick and Collins 1 ). In recent years there have been many advances in the application of technology in the area of dietary assessment( Reference Arens-Volland, Spassova and Bohn 2 ). Innovative dietary assessment technologies using devices such as mobile telephones( Reference Six, Schap and Zhu 3 – Reference Harray, Boushey and Pollard 8 ), sensors, wearable cameras( Reference Sun, Burke and Mao 9 – Reference Barker, Blain and Russell 11 ) and web-based platforms are common ways of collecting dietary intake data and have become increasingly popular alternatives to traditional pen-and-paper versions of dietary assessment( Reference Illner, Freisling and Boeing 12 ). Web-based methodologies facilitate the collection of dietary intake across many geographic locations( Reference Fallaize, Forster and Macready 13 – Reference Kristal, Kolar and Fisher 16 ) and in some cases are preferred by participants compared with the traditional pen-and-paper alternatives( Reference Six, Schap and Zhu 3 , Reference Touvier, Kesse-Guyot and Méjean 17 ). Overall the application of technology in dietary intake assessment has been shown to reduce issues associated with traditional collection of dietary data, such as cost, participation rates (by reducing the burden associated with dietary assessment) and accuracy of data collected( Reference Foster, Hawkins and Simpson 18 ). The success of these methods has been attributed to a number of factors including the ability to collect data in a remote and neutral environment, the standardised sequence of questioning, the use of digital portion size assessment aids, automated analysis of data collected and the provision of dietary feedback( Reference Illner, Freisling and Boeing 12 ).
The reduced participant burden and time of data collection, associated with web- and computer-based 24-h dietary recall (24HDR) tools, support the use of this method in many different populations. Web- and computer-based 24HDR tools have been used to collect data from various population groups including young children( Reference Diep, Hingle and Chen 19 – Reference Carvalho, Baranowski and Foster 22 ), adolescents( Reference Foster 5 , Reference Vereecken, Covents and Maes 6 , Reference Storey and McCargar 23 ) and adults( Reference Subar, Thompson and Potischman 24 – Reference Comrie, Masson and McNeill 27 ). Two of the comparison and validation studies included in this review recruited elderly participants in their study population( Reference Shin, Park and Sun 28 , Reference Liu, Young and Crowe 29 ); however, the validity of these tools have not been tested explicitly in an elderly population. In general, the applications are based on the multi-pass approach described by Moshfegh et al. ( Reference Moshfegh, Raper and Ingwersen 30 ). The applications are either self-administered, whereby a participant completes the recall in the absence of a researcher( Reference Foster 5 , Reference Carter, Albar and Morris 31 – Reference Arab, Tseng and Ang 33 ), or interviewer administered, where the interviewer uses the application to collect/analyse dietary recall data in the presence of a participant( Reference Slimani, Casagrande and Nicolas 26 , Reference Shin, Park and Sun 28 , Reference Daniel, Kapur and McAdams 34 , Reference Park, Shin and Hwang 35 ). The software guides and prompts the participant/interviewer to recall and record food and drink consumed in the previous day (24 h). The design features, food and beverage lists, nutritional composition data, prompts and portion size assessment aids incorporated in these applications differ from model to model and are generally for target populations.
The ability of many of these tools to accurately assess dietary intakes has been investigated using different study designs, the most common of which are comparison and validation studies. A comparison study investigates the performance of a ‘test measure’ of dietary assessment against an alternative dietary recall (such as interviewer-led 24-h recall) to ascertain if the data collected by the test measure are comparable with those using the existing method. A validation study investigates the accuracy of a test measure compared with an objective measure of intake, such as biological markers or direct observation of intake. Assessing the validity of dietary assessment tools is difficult due to the risk of correlated error between the test method and a reference method( Reference Lombard, Steyn and Charlton 36 ). The cost and practicality associated with the direct observation of intake or the collection of biological samples for the analysis of biomarkers of intake can also be limiting factors. In the absence of biomarkers or direct observation, dietary records (preferably weighed food records) are often considered the ‘gold standard’ for the measurement of food and nutrient intake and are typically used as a reference comparison method( Reference Raatz, Scheett and Johnson 37 ).
De Keyzer et al. ( Reference De Keyzer, Bracke and McNaughton 38 ) notes that the 24HDR method is currently the most commonly used tool to collect dietary intake data in national surveys, usually taking multiple recalls to assess habitual intake and, in some instances, used in conjunction with an FFQ (for example, Canada and USA)( 39 , 40 ). In Europe, efforts are being made to standardise the collection of dietary intake data and the potential of computer-assisted 24HDR tools has been recognised( Reference Slimani, Casagrande and Nicolas 26 , 41 ). Both self-administered and interviewer-administered applications have demonstrated the ability to reach large population groups using these applications( Reference Subar, Kirkpatrick and Thompson 42 , Reference Ferrari, Roddam and Fahey 43 ). At present there are numerous 24HDR tools that have been developed, tested and validated worldwide. The aim of this review was to examine common design features amongst current 24HDR tools and investigate the methods used to assess the comparability and validity of these tools.
Methods
A literature search was conducted using the online databases PubMed, EMBASE and Web of Science to collect information on all 24HDR tools. The inclusion criteria were as follows: English-language publications from 2000 until 2016 reporting the development, comparison or validation of a web- or computer-based 24-h recall tool for any population group. The following search terms were used alone and in combination: ‘24h recall’, ‘24-HR’, ‘nutrition’, ‘food’, ‘intake’, ‘dietary assessment’, ‘validity’, ‘validation’, ‘comparison’ and ‘reliability’. In a similar approach described by Gemming et al. ( Reference Gemming, Utter and Ni Mhurchu 44 ), abstracts were initially screened by one author (C. M. T.) for relevance and then collected and distributed to three other authors. The literature search yielded a total of forty-four relevant papers. All papers were reviewed by three investigators (C. M. T., R. v. d. B. and R. J. B.). Sixteen papers were omitted for the following reasons: the papers did not describe a 24HDR tool (n 4) or the papers described aspects of the study, other than the development, comparison or validation of web-based dietary recall tools (n 12). A total of twenty-eight papers reporting twenty-one individual web- or computer-based 24HDR tools were included in the present review.
The design characteristics of the twenty-one 24HDR tools are described in Tables 1 and 2 and the findings of those tools whereby validity/comparability was investigated are described in Tables 3 and 4. To obtain an overview of the different tools, general characteristics were examined, such as participant age, number of food and drink items incorporated in the application, the use of prompt techniques to ensure the complete capture of data and the use of portion size assessment aids. For the assessment of techniques used to investigate validity and comparability, details such as number of recalls used, reference method, type of statistical analysis and markers of nutrient intake were compared to identify common features. Where the required information was not available in the included publications, the corresponding author was contacted via email to obtain/verify information when possible.
Design characteristics of the web- and computer-based 24-h dietary recall tools developed for children
CANAA-W, Children and Adolescents’ Nutrition Assessment and Advice on the Web (a web-based 24-h dietary recall program for children and adolescents); YANA-C, Young Adolescents’ Nutrition Assessment on Computer (a web-based 24-h dietary recall program for children and adolescents); 24HR, 24-h recall; MR, multiple-pass recall; SA, self-administered; SNAP™, Synchronised Nutrition and Activity Program (a web-based 24-h recall program which assesses energy balance-related behaviours in children and adolescents); FIRSSt, Food Intake Recording Software System (a computer-based 24-h dietary recall program for children); PAC24, Portuguese self-administered, computerised, 24-h dietary recall (a web-based 24-h dietary recall program for children); Web-SPAN, Web-Survey of Physical Activity and Nutrition (a web-based 24-h dietary recall program for children and adolescents); Intake24, a web-based 24-h dietary recall program for children and young adults; ASA24-Kids-2012, Automated Self-Administered 24-Hour Dietary Assessment Tool, Kids-2012 version; myfood24, Measure your food on One day (a web-based 24-h dietary recall program for children and adults).
Results
Design characteristics
Of the twenty-one 24HDR tools identified from the literature search, six tools were developed for use specifically with children (three developed in Europe, two in the USA and one in Canada) and thirteen tools were developed (seven in Europe, three in USA, two in Korea and one in India) specifically for adult populations and two tools (both in the UK) were developed for use both with children and adults( Reference Foster 5 , Reference Carter, Albar and Morris 31 ). All of the tools developed for use with children were self-administered, whereas four of the tools developed for adults were interviewer administered. The youngest age to test a 24HDR tool was 7 years of age( Reference Carvalho, Baranowski and Foster 22 , Reference Moore, Hillier and Batterham 45 ) and the oldest age was 80+ years( Reference Shin, Park and Sun 28 , Reference Liu, Young and Crowe 29 ) (Tables 1 and 2).
Only three 24HDR tools reported collecting intake data at a food-group level( Reference Comrie, Masson and McNeill 27 , Reference Liu, Young and Crowe 29 , Reference Freese, Feller and Harttig 46 ), including additional choices/questions within each food group to obtain more specific information about intakes. All other tools recorded intake by presenting lists of food and beverage items. The number of food and drink items included varied for each tool, ranging from forty( Reference Moore, Hillier and Batterham 45 ) to 45 000 food and beverage options( Reference Carter, Albar and Morris 31 ). The use of portion size images was the most popular aid to facilitate portion size estimation. The number of portion size images varied from tool to tool, with some investigators reporting as many as 17 000 images( Reference Subar, Kirkpatrick and Mittl 32 ).
The use of food prompts was prevalent across 24HDR tools developed for both children and adults. ‘Frequently forgotten foods’ was the most common prompt function reported in 24HDR tools developed for children. This prompt involves the presentation of a list of foods that are known to be frequently omitted from dietary recalls (for example, sugar in hot beverages, etc.)( Reference Carter, Albar and Morris 31 , Reference Kirkpatrick, Subar and Douglass 47 ). The Synchronised Nutrition and Activity Program (SNAPTM) tool( Reference Moore, Ells and McLure 21 ) used images of foods to prompt for frequently forgotten food items rather than a list of foods. In the tools developed for adults, ‘frequently forgotten foods’ was also a common prompt, as was ‘linked foods’ (Table 2). Linked food prompts offer the participant food or drink options that are often eaten in combination with the primary food or drink item selected( Reference Foster 5 , Reference Subar, Thompson and Potischman 24 ). Automated data entry checks, such as those implemented by Vereecken et al. ( Reference Vereecken, Covents and Haynie 48 ), for portion size information and meal gap reviews whereby any period greater than 3 h between meals was queried( Reference Subar, Kirkpatrick and Mittl 32 ) were also useful prompt functions incorporated into adult 24HDR tools. Of the 24HDR tools developed for children, two recorded supplement intake( Reference Diep, Hingle and Chen 19 ) compared with six of the identified 24HDR tools developed for adults (Table 2).
Design characteristics of the web- and computer-based 24-h dietary recall tools developed for adults
EPIC-Soft, European Prospective Investigation into Cancer and Nutrition Software (a standardised computerised 24-h dietary recall method); 24HR, 24-h recall; MR, multiple-pass recall; IA, interviewer administered; NutriNet-Santé, a web-based 24-h dietary recall program for adults; SA, self-administered; NINA-DISH, New Interactive Nutrition Assistant – Diet in India Study of Health (a computer-based 24-h dietary recall program for adults); CAPIS, Computer-Assisted Personal Interview System (a computer-based 24-h dietary recall program for adults); Intake24, a web-based 24-h dietary recall program for children and young adults; SNAPA™, Synchronised Nutrition and Activity Program for Adults (a web-based 24-h dietary recall program for adults); FoRC, Food Recall Checklist (121-item food recall list which has been used to measure diet in undergraduate students); Oxford WebQ, a web-based 24-h dietary recall program for adults; IMM, a bilingual interactive multimedia dietary assessment tool based on the 24-h dietary recall method (a web-based 24-h dietary recall program for adults); ASA24, Automated Self-Administered 24 Hour Dietary Assessment Tool (web-based 24-h dietary recall tool for adults, developed by the National Cancer Institute, USA); DietDay, a web-based 24-h dietary recall program for adults; GloboDiet, international computerised 24 hour dietary recall method; myfood24, Measure your food on One day (a web-based 24-h dietary recall program for children and adults); Compl-eat™, a computer-based 24-h dietary recall program for adults.
Main findings of the validation/comparison studies of the web- and computer-based 24-h dietary recall tools developed for children
SNAP™, Synchronised Nutrition and Activity Program (a web-based 24-h recall program which assesses energy balance-related behaviours in children and adolescents); 24HR, 24-h recall; CHO, carbohydrate; PAC24, Portuguese self-administered, computerised, 24-h dietary recall (a web-based 24-h dietary recall program for children); Web-SPAN, Web-Survey of Physical Activity and Nutrition (a web-based 24-h dietary recall program for children and adolescents); ICC, intraclass correlation coefficient; Intake24, a web-based 24-h dietary recall program for children and young adults; EI, energy intake; LOA, limits of agreement; FIRSSt, Food Intake Recording Software System (a computer-based 24-h dietary recall program for children); ASA24-Kids-2012, Automated Self-Administered 24-Hour Dietary Assessment Tool, Kids-2012 version; myfood24, Measure your food on One day (a web-based 24-h dietary recall program for children and adults).
Web- and computer-based 24-h dietary recall tools comparability/validation study design
Tables 3 and 4 present the comparison/validation study design features, and key findings for the identified 24HDR tools. For the purposes of this review, the authors considered a comparison study to be where the test measure was compared against a traditional 24-h recall or estimated/weighed food diary. A validation study was when the test measure was compared against direct observation (of eating occasions) or biological markers of nutrient intake. According to the authors’ interpretation of a comparison and validation study, of the 24HDR tools developed for both children and adults (Foster considered one study across two age groups), nine investigators conducted comparison studies and nine investigators conducted validation studies (EPIC-Soft and NutriNet-Santé were investigated in both comparison and validation studies). The remaining three tools, of the twenty-one identified in the present review, were not evaluated in either a comparison or validation study. Of the studies included in the present review the most common reference method in comparison studies was an interviewer-led multiple-pass recall and for validation studies was direct observation of eating occasions, closely followed by the use of biological markers of intake (Table 4). The number of participants ranged from forty-one( Reference Carvalho, Baranowski and Foster 22 ) to 459 for comparison and validation studies conducted with children( Reference Storey and McCargar 23 ). There was similar variation with the numbers that participated in the adult studies with fifty-three( Reference Comrie, Masson and McNeill 27 ) to 1072( Reference Ferrari, Roddam and Fahey 43 ). It is important to consider the reference method used when comparing participant numbers across studies; for example Touvier et al. ( Reference Touvier, Kesse-Guyot and Méjean 17 ) recruited 147 participants and used one web-based recall compared with one interviewer-led recall, whereas Comrie et al. ( Reference Comrie, Masson and McNeill 27 ) asked fifty-three participants to complete a 4-d estimated food diary as a reference. The lesser burden associated with the reference used by Touvier et al. ( Reference Touvier, Kesse-Guyot and Méjean 17 ) may allow for a greater number of participants to be involved compared with the burden (participant training, interview on diary and analysis of data) associated with a 4-d estimated food diary( Reference Comrie, Masson and McNeill 27 ).
Main findings of the validation/comparison studies of the web- and computer-based 24-h dietary recall tools developed for adults
EPIC-Soft, European Prospective Investigation into Cancer and Nutrition Software (a standardised computerised 24-h dietary recall method); CHO, carbohydrate; 24HR, 24-h recall; NutriNet-Santé, a web-based 24-h dietary recall program for adults; ICC, intraclass correlation coefficient; Intake24, a web-based 24-h dietary recall program for children and young adults; LOA, limits of agreement; SNAPA™, Synchronised Nutrition and Activity Program for Adults (a web-based 24-h dietary recall program for adults); FoRC, Food Recall Checklist (121-item food recall list which has been used to measure diet in undergraduate students); Oxford WebQ, a web-based 24-h dietary recall program for adults; IMM, a bilingual interactive multimedia dietary assessment tool based on the 24-h dietary recall method (a web-based 24-h dietary recall program for adults); ASA24, Automated Self-Administered 24 Hour Dietary Assessment Tool (web-based 24-h dietary recall tool for adults, developed by the National Cancer Institute, USA); DietDay, a web-based 24-h dietary recall program for adults; TEE, total energy expenditure; Compl-eat™: a computer-based 24-h dietary recall program for adults.
A variety of statistical measures were used to investigate comparability/validity of the 24HDR tools against reference methods. Some investigators used descriptive statistics such as the number of ‘matches’, ‘intrusions’ and ‘omissions’ between the food and drink items recorded by the web-based tool compared with the reference and in some cases linear and Poisson regression analysis was used to investigate the association of matches, intrusions and omissions between the test and reference( Reference Foster 5 , Reference Diep, Hingle and Chen 19 , Reference Baranowski, Islam and Baranowski 20 , Reference Kirkpatrick, Subar and Douglass 47 ). Correlation analysis (Pearson, Spearman, κ and intraclass coefficients) was the most popular statistical method used to investigate the validity of nutrient intake and in some instances food group intake( Reference Hillier, Batterham and Crooks 49 ) reported by 24HDR tools compared with the output of reference methods. A wide range of correlation coefficients was reported by investigators. De Keyzer et al. ( Reference De Keyzer, Huybrechts and De Vriendt 50 ) reported low correlation coefficients (r 0·16; P<0·001) for intakes of thiamine recorded by EPIC-Soft when compared with intakes from an estimated food diary. In contrast, Touvier et al. ( Reference Touvier, Kesse-Guyot and Méjean 17 ) reported a high correlation coefficient (r 0·92; P<0·001) for niacin intakes recorded by women using NutriNet-Santé and an interviewer-led 24HDR. Liu et al. ( Reference Liu, Young and Crowe 29 ) also reported a mean correlation coefficient (r 0·6; P<0·001) for twenty-one nutrients examined to demonstrate the relationship between the test and reference measure across a range of nutrients. In addition to correlation analysis, Storey et al. ( Reference Storey and McCargar 23 ) and De Keyzer et al. ( Reference De Keyzer, Huybrechts and De Vriendt 50 ) used paired t tests and Student’s t tests to investigate significant differences in the reporting of nutrient intakes between the test and reference methods.
Bland & Altman( Reference Bland and Altman 51 ) analysis was only used by six investigators, four of which investigated the agreement and bias in nutrient intakes recorded by test and reference methods( Reference Foster 5 , Reference Comrie, Masson and McNeill 27 , Reference Wardenaar, Steennis and Ceelen 52 , Reference Albar, Alwan and Evans 53 ). Moore et al. ( Reference Moore, Ells and McLure 21 ) investigated agreements between methods in the recording of food and drink items and Carvalho et al. ( Reference Carvalho, Baranowski and Foster 22 ) investigated estimated v. observed amounts consumed using Bland–Altman analysis.
The investigators of the 24HDR tools developed for children, included in this review, did not use biomarkers of nutrient intake to investigate validity. Table 4 shows the 24HDR tools developed for adults including four 24HDR tools that were validated using biological markers for nutrient intake. DietDay( Reference Arab, Tseng and Ang 33 ) is the only 24HDR tool that used doubly labelled water (DLW) in validation as a biomarker of usual energy intake( Reference Trabulsi, Troiano and Subar 54 ). Results indicated that the validity of DietDay was more superior than a paper-based FFQ or diet history questionnaire. Three investigators (Wardenaar et al. ( Reference Wardenaar, Steennis and Ceelen 52 ), Ferrari et al. ( Reference Ferrari, Roddam and Fahey 43 ) and Lassale et al. ( Reference Lassale, Castetbon and Laporte 55 )) used urinary biomarkers (urinary N, urinary N and K and urinary N, K and Na, respectively) to investigate the relationship between reported and true intakes. The investigators reported moderate to strong correlations between reported nutrient intakes and their respective urinary biomarkers: r 0·37 (95 % CI 0·03, 0·70) for Na intakes reported by women( Reference Lassale, Castetbon and Laporte 55 ), r 0·65 (95 % CI 0·45, 0·79) for protein intakes reported by athletes( Reference Wardenaar, Steennis and Ceelen 52 ) and r 0·83 (95 % CI 0·637, 0·932) for reported N intakes( Reference Ferrari, Roddam and Fahey 43 ).
Discussion
Tool design characteristics
The range of ages involved in the testing of the 24HDR tools demonstrates the applicability of web- and computer-based dietary assessment methods to the majority of the population. Investigators did note age-specific adaptations to some of the 24HDR tools including the use of cartoon avatars( Reference Biltoft-Jensen, Trolle and Christensen 56 , Reference Baranowski, Islam and Douglass 57 ) and portion size pictures specifically developed for children( Reference Biltoft-Jensen, Trolle and Christensen 56 , Reference Baranowski, Islam and Douglass 57 ). Baranowski et al. ( Reference Baranowski, Islam and Douglass 57 ) and subequently Diep et al. ( Reference Diep, Hingle and Chen 19 ) used adapted versions of the National Cancer Institute’s Automated Self-Administered 24-Hour Dietary Assessment Tool (ASA24®) which had been modified for children and excluded food preparation questions deemed too difficult for children to answer( Reference Douglass, Islam and Baranowski 58 ). However, none of the investigators noted changes to the 24HDR tools that would potentially enhance the user experience for older adults (for example, larger format or font). At present, there is no single 24HDR tool suitable for all age groups; however, some investigators such as Baranowski et al. ( Reference Baranowski, Islam and Douglass 57 ), Subar et al. ( Reference Subar, Kirkpatrick and Mittl 32 ), Hillier et al. ( Reference Hillier, Batterham and Crooks 49 ) and Moore et al. ( Reference Moore, Hillier and Batterham 45 ) have worked to developed separate age-appropriate versions of the same tool. Foster( Reference Foster 5 ) also tested the same tool with children and with young adults and Albar et al. ( Reference Albar, Alwan and Evans 53 ) proposes to test myfood24 with adults aged 18 years and over. Intuitive design features such as ‘same as before’, ‘meal gap review’ and spell check function were unique to some 24HDR tools but could potentially be useful in all web- and computer-based dietary assessment designs (whether it be diary or FFQ based).
Performance
For the majority of 24HDR tools included in this review, the investigators concluded that the performance of the respective tool was acceptable when compared with reference methods (comparison and/or validation studies). There was a variety of concluding remarks from the investigators: ‘The test measure was in good agreement with the reference method’( Reference Foster 5 , Reference Touvier, Kesse-Guyot and Méjean 17 , Reference Storey and McCargar 23 , Reference Comrie, Masson and McNeill 27 , Reference Liu, Young and Crowe 29 ); ‘The test method performed well but in some instances the reference method was more accurate’( Reference Diep, Hingle and Chen 19 , Reference Baranowski, Islam and Baranowski 20 , Reference Kirkpatrick, Subar and Douglass 47 ); ‘The test measure was more superior to the reference measure’( Reference Arab, Tseng and Ang 33 ); and ‘Although the findings were promising the test method required further modifications to improve accuracy’( Reference Carvalho, Baranowski and Foster 22 , Reference Moore, Hillier and Batterham 45 , Reference Hillier, Batterham and Crooks 49 ).
Overall, the 24HDR tools whose performance was investigated, either in a comparison and/or validation study, demonstrated promising potential. Investigators that reported the reference method as being more accurate used direct observation as the reference method( Reference Diep, Hingle and Chen 19 , Reference Kirkpatrick, Subar and Douglass 47 , Reference Baranowski, Islam and Baranowski 59 ). Whilst this objectively investigates the accuracy of the test measure, for example in the case of Kirkpatrick et al. ( Reference Kirkpatrick, Subar and Douglass 47 ), it may lead to the underestimation of the performance of ASA24 if compared with how other 24HDR tools were assessed for accuracy, for example, Zoellner et al. ( Reference Zoellner, Anderson and Gould 60 ). The performance of the ASA24 may be underestimated as the bilingual interactive multimedia (IMM) dietary assessment tool was compared against an interviewer-led 24-h recall, the findings of which may agree more favourably compared with direct observation. As eating is an observable behaviour( Reference Sherwood 61 ), direct observation of eating occasions is an expensive and invasive measure and is usually only feasible for short periods of time (for example, lunch or dinner times) which may not accurately represent the performance of the tool for recording longer periods of intake (for example, 24-h period)( Reference Diep, Hingle and Chen 19 ). Recovery biomarkers measure dietary intake with little error and have been successfully used to validate some of the studies included in this review; however, these biomarkers are limited to certain nutrients and cannot validate all nutrient estimates from a test dietary assessment measure. Based on the findings of this review, the most appropriate use of reference measures for investigating the validity of a 24HDR tool is the combination of an estimated dietary record in conjunction with biological markers of intake or direct observation of eating occasions as undertaken by Arab et al. ( Reference Arab, Tseng and Ang 33 ) and Baranowski et al. ( Reference Baranowski, Islam and Baranowski 20 ), respectively.
In the assessment of the performance of 24HDR tools, acceptability of the tool is an aspect that is not always considered. In the comparison of ASA24 with an interviewer-led 24-h recall, Thompson et al. ( Reference Thompson, Dixit-Joshi and Potischman 62 ) noted that a substantial proportion of study participants preferred the self-administered ASA24 approach compared with the interviewer-led recall. However, Vereecken et al. ( Reference Vereecken, Covents and Haynie 48 ) highlighted the opposite, when parents preferred a traditional approach to recording diet (i.e. a written record) compared with a web- or computer-based tool. Therefore, to ensure prolonged usage of web- and computer-based dietary assessment methods, research into the acceptability of these methods should be considered in the comparison/validation study design so that improvements can be made if warranted.
Accuracy/precision
Although the accuracy of the 24HDR tools is difficult to compare across the board, one can consider the results reported within the individual studies. It is known that one statistical measure in isolation cannot determine the validity of any method of dietary assessment( Reference Lombard, Steyn and Charlton 36 ). However, due to the variation and combination of statistical measures used across studies, for the purposes of this review statistical measures are compared in isolation across studies. Correlation coefficients are commonly reported across studies investigating the accuracy of 24HDR tools. Comparing the ranges of correlation coefficients for macronutrients and micronutrients reported across all 24HDR tools, NutriNet-Santé( Reference Touvier, Kesse-Guyot and Méjean 17 ) had the strongest r values for intakes derived from the web-based tool compared with a reference measure, with r values ranging for macronutrient intake as high as 0·88 (protein intakes recorded by women) and for micronutrient intake an r value of 0·92 (niacin recorded by women). As with all studies investigating the comparability/validity of a test measure of dietary assessment compared with a reference, it is important to consider the study design. Touvier et al. ( Reference Touvier, Kesse-Guyot and Méjean 17 ) compared intakes of NutriNet-Santé against intakes derived from an interviewer-led dietary recall at a single time point, whereas De Keyzer et al. ( Reference De Keyzer, Huybrechts and De Vriendt 50 ) compared intakes across two separate time points and Comrie et al. ( Reference Comrie, Masson and McNeill 27 ) compared intake across four separate time points. It is, therefore, difficult to make conclusions about accuracy of one tool compared with another, particularly using just one type of statistical analysis. The number of recalls administered across the comparison/validation studies varied from a single dietary recall( Reference Moore, Ells and McLure 21 ) to six recalls( Reference Arab, Tseng and Ang 33 ). The difference in the number of recalls administered compared with the number of days of dietary recording by the reference method is important to consider when investigating the agreement between methods in their assessment of nutrient intake.
However, when comparing correlation coefficients from studies that have similar study designs (for example, Liu et al. ( Reference Liu, Young and Crowe 29 ) (r 0·37 to 0·72); Touvier et al. ( Reference Touvier, Kesse-Guyot and Méjean 17 ) (r 0·54 to 0·92); Zoellner et al. ( Reference Zoellner, Anderson and Gould 60 ) (r 0·29 to 0·78)), all of whom used a single time point of data collection in their investigation and used an interviewer-led recall as a reference, Touvier et al. ( Reference Touvier, Kesse-Guyot and Méjean 17 ) performed the best based on this type of analysis. Another type of measure that was used to investigate validity of 24HDR tools was the incidence of ‘Matches, Intrusions and Omissions’ whereby food and drink items recorded using the test measure are compared with the reference measure. Again, there was variation across studies depending on the reference measure used; for example comparing the output of the web-based tool against direct observation, the number of percentage matches was not as high (ranging from 46 to 67 %)( Reference Baranowski, Islam and Baranowski 20 , Reference Carvalho, Baranowski and Foster 22 ) when compared with percentage matches identified when comparing the web-based tool with interviewer-led recalls (79·2 and 82·6 % both)( Reference Foster 5 ).
Precision (the degree to which the same method produces the same value on repeated measures) was less frequently reported. Based on the limited number of investigators that calculated the limits of agreement in their analysis, it is not possible to compare the precision of the methods included in this review. This type of analysis was included in a report on the performance of INTAKE24 (a web-based 24-h dietary recall program for children and young adults)( Reference Foster 5 ). Foster( Reference Foster 5 ) assessed the precision of INTAKE24 compared with other web- and computer-based methods of dietary assessment, which had included these findings in their analysis, by calculating the width of the limits of agreement. The lack of this statistical interpretation in dietary assessment validation studies was also noted by Lombard et al. ( Reference Lombard, Steyn and Charlton 36 ).
Another factor affecting the design of the comparison/validation study was the proximity and sequence of the test and reference measure. Margetts & Nelson( Reference Margetts and Nelson 63 ) suggested that the sequence (i.e. that the test measure be administered before the reference method) and the proximity of the test and reference measure need to be carefully considered to avoid raising the apparent level of agreement between the measures. The cross-over design is a popular design for testing the performance of 24HDR tools( Reference Foster 5 , Reference Subar, Thompson and Potischman 24 , Reference Zoellner, Anderson and Gould 60 ). The design involves the use of the test and reference method in the same day. Although this limits the impact of variation of diet on the results, completing both methods in the same day may introduce a ‘learning effect’. Having acknowledged this in the study design, Foster( Reference Foster 5 ) attempted to limit this effect by weighting the order in which the test measure was administered to participants so that the impact of a ‘learning effect’ could be investigated retrospectively. Interestingly, although the weighed food record is considered the ‘gold standard’ (in the absence of unbiased measures) of dietary assessment( Reference Zoellner, Anderson and Gould 60 ), none of the investigators included this as a reference method. Alternatives may have been used instead due to the high respondent burden and cost associated with weighed food diaries( Reference Wrieden, Peace and Armstrong 64 ).
Biological markers of nutrient intake can serve as an objective validation of dietary assessment methods as they reflect nutritional status, but are independent of dietary intake assessment( Reference McKeown, Day and Welch 65 ). Of all the tools included in this review, Arab et al. ( Reference Arab, Tseng and Ang 33 ) was the only investigator to use DLW to assess the validity. Whilst DLW is considered the ‘gold standard’ recovery biomarker of nutrient intake, urinary N( Reference Bingham, Gill and Welch 66 ) and urinary K( Reference Tasevska, Runswick and Bingham 67 ) are also useful biomarkers in the validation of dietary assessment tools. Biomarkers of nutrient intake from blood were not used by any of the investigators included in this review. Biomarkers of nutrient intake from plasma and serum samples (for example, plasma vitamin C and plasma carotenoids) have been used in other validation studies concerning novel dietary assessment methodologies( Reference Fallaize, Forster and Macready 13 , Reference Patterson, Hogg and Kishi 68 , Reference Timon, Astell and Hwang 69 ). In the studies investigating the validity of 24HDR tools in children (included in the present review), none of the investigators used biological markers of nutrient intake. A reason for this may be that biological samples are difficult to obtain from children( Reference Rockett, Lynch and Buck 70 ). Baranowski et al. ( Reference Baranowski, Islam and Baranowski 20 ) used other strategies such as obtaining a bogus pipeline hair sample from children using the FIRSSt tool to encourage more accurate reporting of dietary intake. Using this method, the researchers informed participants that the hair sample provided would reveal what they had eaten through chemical analysis. Collecting this sample appeared to reduce the level of omissions when intakes from FIRSSt were compared with an interviewer-administered recall( Reference Baranowski, Islam and Baranowski 20 ).
Conclusion
Web- and computer-based 24HDR tools are cost-effective, useful methods for assessing dietary intake from different populations. The overall accuracy of these methods is difficult to determine because, in many cases, direct comparisons cannot be made. Across-the-board agreement between these 24HDR tools for macro- and micronutrient intakes is generally strong when compared against references. Few studies used biomarkers in the validation of 24HDR tools; however, the investigators that did use biomarkers showed that these methods provide reliable estimates of protein intake, moderate estimates of energy intakes and reliable estimates for micronutrient intake such as K and Na. This review highlights some findings which may be applied when designing and investigating the performance of 24HDR tools in the future. Age-specific adaptations have been shown to be of benefit in younger populations, and may prove beneficial for older adult populations also. In the assessment of performance and validity, direct observation is a useful reference method; however, it is often limited to short periods of intake, whereas estimated records in conjunction with biological markers of intake may be more feasible methods of investigating validity. It is important to be mindful of the type of statistical analysis used to investigate validity. Lombard et al. ( Reference Lombard, Steyn and Charlton 36 ) noted that in some cases more than three statistical measures are required to truly assess the validity of dietary assessment measures. Lastly, this review demonstrates that although a standardised dietary assessment methodology is preferential for nutrition surveillance across countries, valuable technology-based 24HDR tools exist and could be used to collect intermittent data for continual health and nutrition research purposes.
Acknowledgements
The present review was completed as part of a Food Institutional Research Measure (FIRM) project, DietIreland, funded by the Department of Agriculture, Food and Marine (13/F/424).
The present review was completed as part of a project conceived by E. R. G. and A. F.; C. M. T. and E. R. G. devised the rationale and concept of the review; R. v. d. B., R. J. B., L. K., K. E. and J. W. were involved in the data collection for the review; C. M. T. drafted the manuscript. All authors approved the final content of the manuscript.
The authors declare no conflict of interest.
