National nutrition surveys collect population-based diet-related data that are essential in assessing food and nutrient intakes, monitoring nutritional status and informing and evaluating public health nutrition programmes and policies. Given the significance of the data obtained through these national surveys, it is important that the methodologies used are robust and valid(Reference Moshfegh, Rhodes and Martin1).
The most common dietary assessment method used in national nutrition surveys is the 24-h dietary recall(Reference Gazan, Vieux and Mora2,Reference Alkhaldy, Aljaadi and Jalil3) , due to its standardised data collection process and its ability to provide reasonably accurate nutrient intake data and collect data from large population groups while minimising participant burden. Many traditional 24-h dietary recall tools have undergone technological advancements in recent years to enhance their cost-efficiency, data quality, user experience and scalability(Reference Ngo, Engelen and Molag4,Reference Murai, Tajima and Matsumoto5) . Today, a wide range of online 24-h dietary recall tools are available, each varying in their functionalities, ease of use, validity and adaptability(Reference Gazan, Vieux and Mora2,Reference Timon, van den Barg and Blain6) . Selecting a 24-h dietary recall tool to collect population dietary intake data can therefore be challenging.
The methods for a future national nutrition survey in New Zealand were developed recently. Since the 24-h dietary recall tool used in the previous national nutrition surveys was no longer available, a new tool had to be selected. This short communication describes the process and results of a review and evaluation of online 24-h dietary recall tools for adaptation and use in a New Zealand national nutrition survey.
Methods
Updated review of existing tools
To identify existing online 24-h dietary recall tools that could potentially be adapted for use in a New Zealand nutrition survey, a systematic review published in 2021 by Gazan and colleagues was updated(Reference Gazan, Vieux and Mora2). The original review aimed to provide an overview of online 24-h dietary recall tools that had the potential to collect dietary intake data for national nutrition surveys and therefore provided an ideal starting point. The authors identified eighteen online 24-h dietary recall tools in publications between 2000 and 2019. We decided to update Gazan’s review for the 24-h dietary recall tools already identified and therefore limited the updated searches to the same eighteen tools. Any tools developed after the initial search was performed would have been in an early stage of development, and it was therefore less likely that validation and usability studies would have been conducted. The selection of a well-established tool was preferred over a new tool that required more testing. Furthermore, the contractor (New Zealand Ministry of Health) advised that they would prefer a 24-h dietary recall tool that had already been used in nutrition surveys in other countries and tool selection needed to be completed within a short timeframe to meet contract deliverables.
Search strategy, study selection and data extraction
The search for relevant online 24-h dietary recall tools was updated (from 2019 to January 2022) using the same methods (i.e. search strategy and eligibility criteria) described in Gazan et al. (Reference Gazan, Vieux and Mora2). The original search terms were combined with the names of the eighteen 24-h dietary recall tools to identify relevant publications in PubMed and Google (Scholar).
Study selection was based on publications or (scientific) literature that described the functionality, validity, user usability or flexibility of relevant online self-administered 24-h dietary recall tools(Reference Gazan, Vieux and Mora2). Relevant studies were only included if published in English. A standard screening process for rapid scoping reviews was performed(Reference Garritty, Gartlehner and Nussbaumer-Streit7) to determine eligibility. Search results were initially screened for inclusion based on title and abstract by one researcher (BF) followed by assessment by a second independent reviewer (SM) of a random sample (∼20 %) of excluded records only. Full-text articles were obtained and screened following the same process. Records marked as ‘unsure’ or where decisions were in conflict were resolved through discussion by both researchers or with the wider research team.
Data extraction was undertaken in line with the previous review and included updated information on tools’ characteristics, collection methods and functionalities (e.g. dietary recall steps) and any validation or usability studies(Reference Gazan, Vieux and Mora2). One researcher (BF) extracted and tabulated the data, which were subsequently checked by a second researcher (SM). Authors and/or owners of the recall tools were contacted with requests to provide missing data or verify information where needed.
Evaluation
Shortlisting, in-depth evaluation and consultation
Evaluation of the online 24-h dietary recall tools was divided into three steps including creation of a shortlist, in-depth evaluation and expert consultation. Tools were scored against a set of pre-defined criteria to shortlist those with the highest potential for use in a New Zealand nutrition survey. These pre-selection criteria were based on criteria used for the UK National Diet and Nutrition Survey(Reference Steer, Roberts and Amoutzopoulos8) with additional criteria related to specific requirements for a New Zealand survey (Table 1). Using a stepwise approach, the first five criteria were scored (0, 1 or 2 points each) and subtotalled. Any tools that did not meet criteria 3 or 4 or scored ≤ 4 points for criteria 1 to 5 were eliminated. The remaining tools were scored for criteria 6–8 and shortlisted if total scores were > 7 points. This cut-off score was based on advice from an expert advisory group (including eighteen members with expertise in nutrition and public health) and aimed to ensure essential criteria were met before advancing to the next evaluation stage. The scoring was conducted by one researcher (BF) and checked by a second (SM). Both have a background in public health nutrition and dietary assessment methods.
Table 1. Pre-selection criteria used to shortlist 24-h dietary recall tools
Shortlisted tools were then assessed against a larger set of evaluation criteria from the UK National Diet and Nutrition Survey selection process(Reference Steer, Roberts and Amoutzopoulos8,Reference Illner, Freisling and Boeing9) . Several criteria were added to determine the suitability of the tool for New Zealand by aligning with specific requirements for a national survey and priority ethnic groups. These added criteria were developed in consultation with expert advisors and key representatives of the Ministries of Health and Primary Industries. The final evaluation criteria were grouped into four categories: (1) organisational, logistical and financial aspects (e.g. ‘How many respondents can use the tool at the same time?’); (2) applicability (e.g. ‘Will the tool be suitable for the population groups of interest?’); (3) respondent, interviewer and research usability (e.g. ‘Can the tool be used offline?’); and (4) accuracy and precision (e.g. ‘Does the tool prompt for foods commonly consumed together?’). The full set of criteria can be provided upon request.
To evaluate the shortlisted 24-h dietary recall tools, the relevant details related to the criteria above were extracted from the updated scoping review. Additionally, relevant websites were searched, and tool developers or owners or corresponding authors of studies were contacted. The key strengths and limitations of each tool were identified.
In-person consultation with our nutrition and public health expert advisors was undertaken to guide the decision on which tool to select and adapt for use in a nutrition survey. Where needed, further information about the tools was collected to facilitate the discussions.
Results
Updated review of existing tools
After removing fifty-three duplicates, 644 records were screened for inclusion. Forty-six records were deemed eligible for full-text screening, from which fourteen new publications were included in the updated review. Six of the eighteen tools originally identified by Gazan et al. had new relevant publications since 2019. These fourteen publications described six studies on the Automated Self-Administered 24-h (ASA24) dietary assessment tool (n 2 validation(Reference Raffoul, Hobin and Sacco10,Reference Subar, Potischman and Dodd11) , n 3 usability(Reference Bekelman, Martin and Johnson12,Reference Bekelman, Johnson and Steinberg13,Reference Solbak, Robson and Siou14 ), n 1 both validation and usability(Reference Silveira, Jeng and Gower15)), two studies on Intake24 (n 2 usability(Reference Osadchiy, Poliakov and Olivier16,Reference Rowland, Rose and McLean17) ), two studies on Myfood24 (n 1 validation(Reference Koch, Conrad and Cade18), n 1 usability(Reference Ward, McLellan and Udeh-Momoh19)), two studies on R24W (n 2 validation(Reference Brassard, Laramée and Robitaille20,Reference Paradis21) ), one study on NutriNet-Salud (n 1 development(Reference Meléndez, Hercberg and Galán22)) and one study on Foodbook24 (n 1 usability(Reference Timon, Walton and Flynn23)). Data extracted on the development, validation and usability of each tool is not included in this short communication but can be provided upon request.
Evaluation
Selection of shortlist
Table 2 shows the scores for each of the 24-h dietary recall tools. Based on the scoring of the pre-selection criteria, three of the eighteen tools were shortlisted: ASA24 (9/10), Intake24 (10/10) and Myfood24 (9/10).
Table 2. 24-h dietary recall tools scored against the pre-selection criteria
ASA24, Automated Self-Administered 24-h dietary assessment tool; CANAA-W, Children’s and Adolescents’ Nutrition Assessment and Advice on the Web; CAPIS, Computer-Assisted Personal Interview System; FBQ, Web-based Food Behaviour Questionnaire; FoRC, Food Record Checklist; Myfood24, Measure Your Food on One Day; PAC24, Portuguese self-administered computerised 24-h dietary recall; R24W, web-based 24-h dietary recall; SACANA, Self-Administered Children, Adolescents, and Adult Nutrition Assessment; Web-SPAN, Web-Survey of Physical Activity and Nutrition.
The first five criteria were scored and subtotalled. Tools that did not meet criteria 3 or 4 or scored ≤ 4 for criteria 1–5 were eliminated. The remaining tools were scored for criteria 6–8.
Criteria: (1) evidence of validation of 24-h dietary recall tool; (2) evidence of use in previous (national) surveys or large studies; (3) the tool is available for use; (4) the tool is in English or can easily be adapted to English; (5) the tool can use images to portray portion size; (6) the tool can record (or can be easily adapted to record) additional details required to accurately code dietary data; (7) the tool can be updated to align with the expected requirements for a New Zealand national nutrition survey; and (8) the tool can automatically link food/drink consumption to food composition data.
Evaluation and consultation
The three shortlisted 24-h dietary recall tools were assessed against the evaluation criteria described above. Findings and key strengths and limitations were summarised (Table 3) and presented to expert advisory groups. Across the three tools, only a few differences were observed, including the tools’ offline use, use with children and costs associated with modifications, maintenance and research use. Expert advisors highlighted the need for offline use of the tool to allow for data collection in rural or remote communities in New Zealand where access to Wi-Fi and mobile data may be limited. Myfood24 has the option to collect data offline but only using an Android application, limiting the use in a national nutrition survey in which laptops are the most likely device used for data collection. Intake24 allows local installation of the tool on researchers’ laptops to enable offline data collection with the data synced to a server at a later stage. Contact with ASA24 confirmed that it is a web-based programme only and therefore not downloadable to be completed offline, with no future plans to develop this offline functionality.
Table 3. Overview of strengths and limitations for the shortlisted 24-h dietary recall tools
ASA24, the Automated Self-Administered 24-h dietary assessment tool.
Based on the best available information. Where needed, information was verified with authors/developers.
Given the nutrition survey will include participants from the age of 2 years, advisory group members preferred a tool that had been designed for and tested with a wide range of age groups, including younger children. All three shortlisted recall tools were created for use with people aged 10 years and older. However, evidence could only be found for adaptations to Intake24 to make it suitable for use with younger ages (1·5 years and over).
Modifications needed to create a tool suited for use in the New Zealand population would require substantial efforts and close collaboration for each shortlisted tool. Costs associated with this process, but also the maintenance and use of the dietary recall tools, however, varied significantly. Both ASA24 and Intake24 charge no licence fee, while Myfood24 has an annual licence fee and an additional fee per anticipated use of the system (e.g. based on the number of participants, recalls and time points). Furthermore, new foods and prompts can only be added by Myfood24, and additional charges apply. Adaptation of Myfood24 was therefore more restricted and costly compared with Intake24 and ASA24.
Feedback from the experts was used to recommend a preferred 24-h dietary recall tool. Representatives of the Ministries of Health and Primary Industries then reviewed the full evaluation report, including expert advice, and made a final decision on the tool to be used in a national nutrition survey. Intake24 was considered best suitable for use in a national nutrition survey for three main reasons: (1) it can be used offline; (2) it has been used or is being adapted for use in national surveys by countries similar to New Zealand, including Australia and the UK, thus providing opportunities to establish collaborations, learn from other user experiences and compare survey findings; and (3) Intake24 has been developed for and tested in both child and adult populations, which was a key requirement for a New Zealand national survey.
Conclusion
The rigorous evaluation process outlined in this short communication enabled the selection of a robust 24-h dietary recall tool for use in a New Zealand nutrition survey. It facilitated efficient and detailed data collection to determine the key strengths and limitations of online tools including their potential to be modified for the unique population demographics in New Zealand. This approach could be used as a guide for other countries when selecting a new 24-h dietary recall tool for use in a national nutrition survey.
Acknowledgements
Thank you to Jacqui Grey and Maria Maiquez for organising the meetings with the expert advisory groups and the Ministries. We would also like to thank our wider investigator group for their guidance, as well as the experts who provided critical advice to help select a 24-h dietary recall tool.
Financial support
The New Zealand Ministries of Health and Primary Industries funded this research. These funders played a role in the development of the evaluation criteria and the final decision to select Intake24 as the 24-h dietary recall tool for use in a New Zealand nutrition survey.
Competing interests
There are no conflicts of interest.
Authorship
B.F., S.M. and C.N.M. conceptualised the research design. B.F. conducted the database search; B.F. and S.M. completed the study selection, data extraction and evaluation; B.F. wrote the manuscript; and S.M. and C.N.M. edited and approved the final manuscript.
Ethics of human subject participation
NA.
Open access
