Accurate and valid assessment of dietary intake is a crucial component of high-quality nutrition research. Interview-administered 24-h dietary recalls, the standard reference method in dietary assessment, are costly and burdensome to administer making them impractical for inclusion in some large epidemiological studies(Reference Baranowski1–Reference Thompson, Dixit-Joshi and Potischman3). To reduce the cost and administrative burden, other dietary intake methods (e.g. FFQ and self-administered dietary recalls) are used; however, these measures have shortcomings such as not being able to ascertain meal patterns and having lower validity for some nutrients(Reference Baranowski1–Reference Thompson, Dixit-Joshi and Potischman3). Each of these methods is also prone to inaccuracies due to social desirability bias(Reference Baranowski1). Thus, the development and validation of new dietary intake assessment methods that limit participant burden, research cost and reduce bias are needed for both observational research and intervention studies.
Alongside recent advancements in technology, new innovative techniques for data collection, including the assessment of dietary intake, are under development(Reference Bruening, van Woerden and Todd4–Reference Subar, Kirkpatrick and Mittl6). Such methodologies include ecological momentary assessment (EMA), an assessment technique that utilises a set of real-time data capture tools (e.g. apps and wearable technology) in which study participants are repeatedly assessed during the course of their daily lives(Reference Stone and Shiffman7–Reference Stone, Shiffman, DeVries, Kahneman, Diener and Schwarz9). Researchers have begun to explore the use of EMA to assess dietary intake, positing that this method may offer a lower-cost, reliable, valid and feasible method of collecting dietary intake data from research participants(Reference Bruening, van Woerden and Todd4,Reference O’Connor, Ke and Dzubur5,Reference Dunton, Liao and Dzubur10) .
Given the growing popularity of EMA as a data collection method used within a variety of study designs (e.g. observational studies and interventions), it is important to carry out further development and evaluation of this new method; a logical starting point for evaluation is to compare data collected via EMA to a standard reference method of dietary intake assessment in free-living populations, such as interviewer-administered 24-h dietary recalls. To date, only three studies have evaluated the use of the EMA approach for assessing dietary intake(Reference Bruening, van Woerden and Todd4,Reference O’Connor, Ke and Dzubur5,Reference Spook, Paulussen and Kok11) . Overall, results from these comparison studies suggest moderate to high concordance between dietary intake collected via EMA and more traditional dietary recall methods(Reference Bruening, van Woerden and Todd4,Reference O’Connor, Ke and Dzubur5,Reference Spook, Paulussen and Kok11) . However, more research is needed to understand if similar levels of concordance exist when using EMA to assess dietary intake with younger children from racially/ethnically diverse and immigrant/refugee households.
The primary goal of the present study is to deepen our understanding of how dietary intake data collected via a brief EMA measure compares to that of data collected via the standard reference method for dietary recalls, an interviewer-administered 24-h dietary recall. Specifically, this paper aims to: (1) assess concordance between children’s intake of selected food groups as measured by parent’s completion of a brief EMA measure and 24-h dietary recalls collected via parent report using standard 3-d 24-h dietary recall protocols, and (2) explore differences in level of concordance between these two assessment types by important child and family characteristics, such as gender, race/ethnicity, socioeconomic status (SES); meal-level characteristics such as eating occasion, location of the meal and food preparation style (e.g. home-made v. restaurant meal); and type of EMA prompt (e.g. signal or event contingent). Findings from this paper will expand the current literature by being the first study, to our knowledge, to examine concordance between young children’s dietary intake, as reported by their primary caregivers, through a 24-h dietary intake recall and EMA.
Methods
Data for the current study are from phase I of Family Matters, a National Institutes of Health-funded study(Reference Berge, Trofholz and Tate12). Family Matters is a 5-year, two-phased, mixed-methods, observational study designed to identify novel risk and protective factors for childhood obesity in the home environments of racially/ethnically diverse and primarily low-income children; in-depth details regarding both phases of the study have been published elsewhere(Reference Berge, Trofholz and Tate12). The University of Minnesota’s Institutional Review Board Human Subjects Committee approved all protocols used in both phases of the Family Matters study; all adult participants provided written informed consent and parental consent for their children, and all children assented to the study.
Recruitment and data collection
Phase I of the Family Matters study was a cross-sectional study using mixed-methods to examine in-depth the home environments of low-income and minority households. For this phase, children (n 150) and their families were recruited between 2015 and 2016 from ten Minneapolis/St. Paul, Minnesota primary care clinics serving primarily low-income and diverse families. Children and their families were eligible to participate if the child was between the ages of 5 and 7 years, had a sibling between the ages of 2–12 years old living in the same home, were away from home during the day (e.g. school, day care and summer camp), lived with the parent/primary guardian full time and shared at least one meal/d with the parent/primary guardian. Study measures were completed by the parent/primary guardian who was most responsible for the care and feeding of the 5–7-year-old child; families participated in their preferred language.
Families were stratified by race/ethnicity to ensure an equal distribution of participants (n 25 per group) in the following categories: Caucasian, African-American, Native American, Latino, Somali and Hmong. Additionally, to ensure an equal representation of weight status, racial/ethnic groups were stratified so there was an equal distribution of overweight (≥85th BMI percentile) and normal-weight (5th–84th BMI percentile) children. In-depth details regarding recruitment and study design are published elsewhere(Reference Berge, Trofholz and Tate12).
Two in-home visits were conducted with families about 10 d apart; an in-person dietary recall was conducted at each home visit. In between-home visits, the parent completed 8 d of EMA and an additional 24-h dietary recall was collected via telephone. See Figure 1 for additional details on data collection methods and timing. The data used in this analysis (measure details included below) come primarily from the three 24-h dietary recalls completed by the parent over the course of this 10-d observation period, as well as 8 d of EMA(Reference Shiffman, Stone and Hufford13).
Components of in-home visits for the Family Matters study
Ecological momentary assessment measures
Multiple daily measures of EMA over 8 d were collected from parents. Standardised EMA data collection protocols from prior studies(Reference Stone and Shiffman7,Reference Shiffman, Stone and Hufford8,Reference Sallis, Owen, Fisher, Glanz, Rimer and Viswanath14,Reference Shiffman, Stone, Bachrach and Jobe15) were used in the study including: (1) signal contingent, (2) event contingent and (3) end-of-day EMA messaging. iPad minis equipped with data plans were provided to parents to enter responses to the EMA surveys during the 8-d observation period; parents were also given the option to respond to EMA prompts using their personal cell phone.
Signal contingent
EMA recordings were researcher-initiated and were used in a stratified random manner so that each parent was prompted via a text message to fill out a survey four times a day, within a 3-h time block (e.g. 07.00–10.00, 11.00–14.00, 15.00–18.00 and 19.00–22.00 hours). The timing of EMA prompts was adjusted for parent shift work and wake times to accommodate parent’s differing life situations. The average compliance rate to signals was 80·0 % (range 50·0 %–100·0 %). Event contingent EMA (henceforth, mealtime surveys) recordings were to be self-initiated by parents whenever an eating occasion that met study-specific criteria (parent completing the EMA survey present at the meal) occurred with the child. An end-of-day EMA recording was completed prior to sleep to capture any events not reported since the last recording and to get end-of-day measures. At the beginning of the signal-contingent and end-of-day recordings, participants were prompted to fill out an event contingent recording (e.g. mealtime survey) if they had an eligible eating occasion since the last recording but had not yet completed an event contingent survey for the eating occasion. Two-thirds of completed mealtime surveys were captured with these prompts. EMA mealtime survey recordings took an average of 3·1 (sd = 1·4) min to complete. Other details regarding the EMA component of the study have been published elsewhere(Reference Berge, Trofholz and Tate12,Reference Berge, Tate and Trofholz16) .
Dietary intake measured via ecological momentary assessment
The mealtime survey included a series of close-ended questions from the Healthfulness of Meal Index that was created for the study, which was adapted from the Healthy Eating Index categories(Reference Trofholz, Tate and Draxten17,Reference Guenther, Kirkpatrick and Reedy18) . The questions included whether the following ten categories of foods were served at the eating occasion: ‘fruit’; ‘vegetables’; ‘whole grains (e.g. whole wheat breads or cereals, brown rice, oatmeal and corn tortillas)’; ‘refined grains (e.g. white bread or cereals, flour tortillas and white rice)’; ‘dairy (e.g. milk, cheese, yogurt, milk alternate such as soya milk and ice cream)’; ‘meat protein (e.g. chicken, beef and seafood/fish)’; ‘beans, eggs, seeds, nuts and tofu’; ‘sugary drinks (e.g. pop, Kool-Aid, Capri Sun, Sunny Delight and sports drinks)’; ‘cake/cupcake/cookies or other baked goods’ and ‘candy (e.g. sweets, chocolate, Gushers and fruit snacks)’(Reference Trofholz, Tate and Draxten19,Reference Guenther, Casavale and Reedy20) . When asked about food served at the meal, participants were also given this guidance, ‘For a dish containing multiple foods (e.g. soups, sandwiches and casseroles), please select the main ingredients in the dish’. For example, if you had beef and vegetable soup, mark both: ‘meat protein’ and ‘vegetables’. For analysis, we combined the last two food categories into one category called ‘sweets’. After the respondent identified each of the foods served at the eating occasion, they were asked to indicate whether or not the target child ate each of the foods served (yes/no)(Reference Trofholz, Tate and Draxten19,Reference Guenther, Casavale and Reedy20) . Mealtime EMA surveys were intended only to capture meals when the child and at least the parent responsible for completing the EMA were present.
Eating occasion characteristics measured via ecological momentary assessment
The mealtime EMA survey also asked the respondent to indicate whether the eating occasion was a breakfast, lunch, dinner or snack; whether the occasion occurred on a weekday or weekend day and whether the meal was from a restaurant/fast food, pre-prepared or home-cooked. Finally, an indicator was created for whether the EMA recording of the eating occasion was initiated by the participant (i.e. event-contingent or self-generated) or was prompted at the beginning of the next signal-contingent survey.
24-h dietary recall measures
Three 24-h dietary recalls on the target child were carried out by study staff members trained and certified in the use of the Nutrition Data System for Research(Reference Schakel21). The 2015 software version was used, which was developed by the Nutrition Coordinating Center, University of Minnesota, Minneapolis, MN, USA. Recalls were conducted on non-consecutive days; two of the recalls were for weekdays, and one was for a weekend day.
Because young children are not considered reliable reporters of dietary intake, recalls were conducted with the target child’s parent (i.e. the same parent who completed EMA) with limited assistance from the target child. Parents were encouraged to turn to the target child for assistance with completing the recall when this made sense; for example, target children could provide details on what was served v. eaten during meals consumed outside the presence of the parent (e.g. school lunch). Parents were also provided with several additional tools to assist them in their reporting of the target child’s dietary intake. First, parents were provided a food amounts booklet, which assisted parents in reporting the amounts of foods consumed. School lunch and breakfast menus were utilised when appropriate, and older siblings were able to assist the parent in the recall if appropriate. Finally as recalls were prescheduled, parents were provided food record sheets in which to record the foods and drinks consumed by the target child the previous day. These sheets were important as the parent was not always with the child for the entire day, and the sheet prompted parents to record the child’s dietary intake.
Staff dietitians conducted quality assurance on 100 % of dietary recalls using a quality assurance protocol that mirrors the one used by Nutrition Coordinating Center(22); as a result of this quality assurance process, eleven recall days (1 recall day for eleven families) were excluded from the current analysis.
Eating occasion characteristics measured via 24-h dietary recalls
Meal type (e.g. breakfast, lunch, dinner or snack) and location of meal (e.g. home, school, day care and restaurant) were recorded. Information on whether the eating occasion occurred on a weekday or weekend day, and whether the dietary recall was collected in-person by phone (the first and last dietary recalls), was determined from the date of the recall.
Data analysis
Matching procedures
Among all families in the full sample (n 150), the days where a parent completed both EMA surveys and a dietary recall were identified. Within these matched days, eating occasions reported in the EMA survey were matched to eating occasions reported in the 24-h dietary recalls by eating occasion type (e.g. breakfast, lunch, dinner or snack). Foods that participants reported consuming during 24-h dietary recalls were aggregated into food subgroups by the Nutrition Data System for Research system. These subgroups were then combined to create food categories that were similar to the EMA food categories. For example, fruit juice, whole fruits, avocado, fried fruit and fruit-based savoury snacks were combined to create a Fruit category (See Supplemental Table 1 for a comparison between the nine EMA food categories and the Nutrition Data System for Research food subgroups).
Concordance analysis
Concordance in the foods at each eating occasion reported in both the EMA mealtime surveys and the dietary recalls (n 455 matched eating occasions) was evaluated in several ways. First, for each type of food queried in the EMA mealtime survey (e.g. fruit), the matched meals were coded into one of the following four mutually exclusive categories: (1) a food of that type was reported in both the 24-h dietary recall and the EMA (positive concordance); (2) a food of that type was NOT reported in the 24-h dietary recall or the EMA (negative concordance); (3) a food of that type was not reported in the 24-h dietary recall but was reported in the EMA (EMA only) or (4) a food of that type was reported in the 24-h dietary recall but not in the EMA (recall only).
Second, both concordance (i.e. overall sum of positive and negative concordance) and a Cohen’s kappa statistic were estimated for each food type(Reference Cohen23). Given the majority of prior research examining concordance between EMA dietary measures and dietary recalls has reported percentage concordance(Reference Bruening, van Woerden and Todd4,Reference O’Connor, Ke and Dzubur5) , the current study also reports percentage for comparison purposes. In addition, a kappa statistic is reported, which measures inter-rater agreement and is scaled such that a negative one kappa statistic indicates perfect disagreement, a zero kappa statistic indicates the amount of agreement that would be expected by chance and a one kappa statistic indicates perfect agreement. While the percent concordant measure is more readily understood, the kappa statistic takes into account that some concordance will occur by chance and thus it is considered a superior measure of concordance.
Third, the prevalence of each food type across all matched meals was calculated separately from the EMA surveys and from the 24-h dietary recalls. This approach evaluates the extent to which EMA survey data may underestimate or overestimate the frequency of consumption of each type of food assessed by the survey relative to the matching meals from the 24-h dietary recall. Unpaired t tests are used to test whether the prevalence from the EMA is significantly different from the prevalence obtained via 24-h dietary recall.
Analysis of correlates of concordance
To better understand what factors are related to higher concordance, regression analysis was used to examine which attributes of the eating occasion, EMA recording or family factors are related to the number of concordant foods and positive concordant foods by food type. Generalised estimating equations and within-estimation regression models are used to estimate linear models that adjust for the fact that multiple meals are observed per family. The within-estimator methods are identified from differences within families, holding all time-invariant family characteristics constant; thus, only attributes of the eating occasion, EMA recording or day (not family characteristics that do not vary by meal) are examined when using this model.
Analysis sample
Among the 150 parent–child dyads in the phase I Family Matters study, there were 185 d on which both a 24-h dietary recall and EMA data were collected; of note, 25 % of these 185 d (46 d) had only one meal match. Across those 185 d, there were more eating occasions reported via 24-h dietary recall (n 913) than via EMA surveys (n 560). This difference is likely in part due to differing protocols. Only child eating occasions observed by the parent were to be entered into the EMA mealtime survey, whereas the 24-h dietary recall protocol was designed to capture all eating occasions by the target child regardless of who was present. Of the 560 EMA-recorded eating occasions, 81 % (n 455) were matched to an eating occasion of the same type (e.g. breakfast) from the 24-h dietary recall. Of these 455 matched meals, there were twenty-seven second or third snacks and four second breakfasts reported in the same day. There were no second lunches or second dinners. Multiple reports of the same meal type were time-ordered within both data sets so that the second and third meals of one type in a day could be matched.
The remainder of the results presented will focus on the concordance of the foods reported at the 455 matched meals (occurring within 120 households). Demographic details on participants in the full sample and the analysis sample are included in Table 1. The families (n 120; thirty participants had no matched meals) included in this analysis sample were nearly as racially/ethnically as the full Family Matters sample (n 150). However, there were several marginal, but potentially important, differences between the two samples; specifically, individuals in the analysis sample were more educated (P = 0·05), had a higher income (P = 0·05) and were less likely to be foreign-born (P = 0·001) than those in the full Family Matters sample.
Sociodemographic characteristics of the diverse Family Matters study sample including Minnesota families with young children in 2015–2016*
TANF, Temporary Assistance for Needy Families; SNAP, Supplemental Nutrition Assistance Program.
* χ 2 tests were conducted to compare the sample with matched meals and the sample with no matched meals for all measures except for age, where a t test was used.
Results
Characteristics of matched eating occasions
Details on characteristics of the matched eating occasions can be found in Table 2.
Characteristics of matched meals (n 455 meals)
EMA, ecological momentary assessment.
A variety of eating occasions were represented within the analysis sample of matched meals (i.e. meals reported in both dietary recalls and EMA, matched by day and eating occasion type); 26·2 % (n 119) were breakfast meals, 16·0 % (n 73) were lunch meals, 29·7 % (n 135) were dinner meals and 28·1 % (n 128) were snacks. The bulk (81·5 %, n 371) of matched meals was eaten at home, and the meal preparation method varied ((home-cooked: 60 %, n 273), (pre-prepared: 30·5 %, n 139) and (restaurant meals: 9·5 %, n 43)).
Foods reported eaten during matched eating occasions
Details on foods reported eaten during matched eating occasions are included in Table 3.
Foods reported eaten in matched meals (n 455 meals)*
EMA, ecological momentary assessment; NDSR, Nutrition Data System for Research.
* Unpaired t tests are used to determine if the proportion of EMA meals containing the food is significantly different from the proportion of NDSR meals containing the food.
Of the nine food categories examined in this concordance study (fruit, vegetables, whole grains, refined grains, dairy, meat protein, non-meat protein, sugar-sweetened beverages and sweets), the average number of food categories participants reported their children eating at an eating occasion was 2·31 when reported using EMA, as compared with an average of 3·00 when reported using a 24-h dietary recall (P < 0·01). A similar pattern of difference in mean number of food categories reported was evident for each eating occasion type.
Concordance between children’s dietary intake data collected by a brief ecological momentary assessment measure and 24-h dietary recalls collected using standard 24-h dietary recall protocols
Figure 2 includes information on overall concordance between children’s dietary intake data collected by a brief EMA measure and 24-h dietary recalls collected using standard 24-h dietary recall protocols. Overall concordance (sum of positive and negative concordance) of reporting for eating occasions via EMA and 24-h dietary recall for each of the nine food categories ranged from 54 % for refined grains to 87 % for sweets (Fig. 1). Mostly consistent with the concordance percentages, the kappa statistics indicate moderate agreement for meat (kappa = 0·55); fair agreement for sweets (kappa = 0·38), beans/nuts (kappa = 0·37), dairy (kappa = 0·31), fruit (kappa = 0·31) and vegetables (kappa = 0·27); and little to no agreement for refined grains (kappa = 0·07), whole grains (kappa = 0·12) and sweetened beverages (kappa = 0·14)(Reference Cohen23).
Meal-level ingredient Kappa agreement
Differences in food category concordance within meals (Figs 1 and 2) are somewhat reflected in differences in prevalence of foods across eating occasions by data collection method (Table 3). In particular, the percentage of matched eating occasions where refined grains were reported eaten via EMA (20·7 %) is notably lower than that reported by 24-h recall (49·2 %; P < 0·01), as one would expect given the low concordance and kappa statistic for a given meal. Likewise, some items with high concordance within a meal have similar prevalence rates across meals; specifically, the percentage of matched meals where beans/nuts or sweets were reported to have been eaten via EMA was similar to the percentage of meals reported via by 24-h recall (beans/nuts: 16·3 % v. 20·7 %, P = 0·09; sweets: 9·9 % v. 13·6 %, P = 0·08). On the other hand, the percentage of meals where fruit and whole grains were eaten was similar between the two methods, 38 % of meals reported via either methods included fruit (P = 1·00) and 24–26 % of meals reported via either methods included whole grains (P = 0·40), despite only low/moderate concordance/agreement of fruit or whole grains within any given meal (Figs 1 and 2). Respondents reported consuming vegetables (P < 0·01), refined grains (P < 0·01), dairy (P < 0·01), meat (P = 0·02), beans/nuts (P = 0·09) and sweets (P = 0·08) more often when reporting intake via 24-h dietary recall as compared with EMA. However, respondents reported consuming sugar-sweetened beverages at more meals through EMA than via 24-h dietary recall (23·7 v. 14·7 %, P < 0·01).
Correlates of concordance
The numbers of concordant items are related to eating occasion, location of meal, survey characteristics and the race/ethnicity of the respondents (Table 4). Columns 1 and 2 of Table 4 display correlates with the number of overall (positive and negative) concordant items, where columns 3 and 4 show the correlates with the number of positively concordant items. Overall, concordance of intake was higher for breakfast and snacks compared with dinner (the reference category), but less positively concordant than dinner. This pattern of concordance is likely due to the fact that fewer foods are eaten at snacks and breakfasts as compared with dinners or lunches; indeed, in the current sample, snacks had on average 1·6 items, breakfasts have 2·3 items, lunches have 2·5 items and dinners have 3·0 items reported via EMA. Higher overall concordance was observed if the meal occurred at home relative to meals at school or meals in a restaurant. If the EMA survey was self-generated (as opposed to initiated by a prompt from the research via a signal-contingent survey), positive concordance is slightly lower. No statistically significant differences in concordance by dietary recall mode (in-person v. over the phone) were observed. Positive concordance was higher on the weekends. Finally, African-American, Hmong, Native American and Somali respondents had meals that were less concordant on average than meals reported by white families.
Correlates of concordant reporting (n 455 meals from 120 families)*
GEE, estimated using generalized estimating equations; Within, estimated using within-participant methods; EMA, ecological momentary assessments.
Coefficients reported and 95 % CI. *** P < 0·01, ** P < 0·05, * P < 0·1.
* The number of positively concordant items is the number of foods that were reported in both the 24-h dietary recall and the EMA, where the number of concordant items is the number of foods that were reported in both or neither method. Example interpretation: Breakfasts have 0·711 more concordant items than dinners, holding all else constant. Breakfast have 0·439 fewer positively concordant items than dinners, holding all else constant.
No difference in concordance was found by child weight status, gender of the respondent, receipt of Supplemental Nutrition Assistance Program (SNAP) benefits or number of children in the home. Two of the time-varying covariates included in the within-participant analysis vary by day (the indicators for in-person dietary recall and weekend), instead of by meal. The results are unchanged if those two day-level measures are omitted from the analysis.
Discussion
The primary goal of the present study was to understand how dietary intake data collected via a brief tool delivered via EMA compare to data collected via a standard interviewer-administered 24-h dietary recall; results provide important information to researchers considering utilising a similar EMA approach to gather dietary intake information within similar populations. Overall, data suggest that among meals reported in both the EMA and dietary recalls, concordance in reporting was reasonably good for some types of food, but only fair to low for others. Level of concordance was found to differ based on type of food, eating occasion and location of meal. Further development and evaluation are warranted for this new method that holds promise as a practical and potentially valid measure of the frequency of consumption of specific types of foods.
A number of findings from the current study aligned closely to previous studies that have explored a similar question, with some differences. For example, our study found a 87 % concordance in report of the consumption of sweets between both methods; separate studies by Bruening(Reference Bruening, van Woerden and Todd4) and O’Connor(Reference O’Connor, Ke and Dzubur5) found similarly high rates of concordance for sweets (Bruening: 91 %; O’Connor: 78·6 %). Concordance in the report of fruits and vegetables across these same three studies differed more widely, with Bruening reporting 94 % concordance, O’Connor reporting 64·9 % concordance, and the current study finding 65 % concordance. Of note, all three of these studies were conducted in different populations and used a slightly different brief screening tool to assess dietary intake via EMA. For example, Bruening asked college students to report on a total of five food items (sweets, salty snacks/fried side dishes, fruits and vegetables, and entrees)(Reference Bruening, van Woerden and Todd4), O’Connor asked elementary school-aged children to report on four food items (fruits or vegetables, chips or fries, pastries or sweets(Reference O’Connor, Ke and Dzubur5), soda or energy drinks), and the current study asked parents to report on child intake on nine total food items, modelled off of the Healthy Eating Index-2010 categories(Reference Guenther, Kirkpatrick and Reedy18,Reference Guenther, Casavale and Reedy24) . These differences in populations, methods and measures make direct comparison in concordance across all three studies challenging. To our knowledge, the current study is the first to report a kappa statistic as a means of comparing between these two methods of measuring dietary intake; because the kappa statistic takes into account that some amount of concordance will occur by chance, future research examining concordance should consider reporting this statistic.
Concordance (and kappas) between the two methods of measuring dietary intake was high in the current study for some food categories and low for others. First, discordance differed widely by type of food, with moderate levels of concordance for meat; fair concordance for sweets (e.g. cakes, cookies, pastries and candy), fruits and vegetables, beans/nuts and dairy; and low levels of concordance for grains (refined and whole) and sugar-sweetened beverages. These differences have direct implications for researchers seeking to measure intake of a specific food group; the pros and cons of using EMA v. a standard 24-h dietary recall must take into account the food group of primary interest, whether the amount of the particular food consumed is of interest, and if researchers wish to calculate specific nutrients consumed from the dietary intake data collected. For example, our results would suggest that EMA might be an appropriate choice for a study interested in knowing whether or not sweets were consumed, in any amount, at a particular meal or snack, or whether dairy, fruits or vegetables were served at any meal, whereas EMA would not be appropriate for a study interested in knowing whether refined grains were consumed. Using the current EMA protocol, estimating servings and nutrients consumed (e.g. servings of whole grains or grams of fibre consumed) is not possible.
Also of note, it was far more common, with two notable exceptions, for participants to report eating a food item via 24-h dietary recall and fail to report it via EMA, than the other way around; for example, participants were more likely to report consumption of vegetables, refined grains, dairy and meat, via 24-h dietary recall, as compared with EMA. Discordance in the report of fruits and whole grains, and to some extent, beans/nuts and sweets was split about evenly between EMA and 24-h dietary recalls, suggesting that while concordance between the two measures was not perfect, one measure was not more likely that the other to capture consumption of these food items. It is often suggested within the EMA literature that participants might be more apt to recall and report behaviours of interest (e.g. dietary intake) via EMA, as compared with recalling them at a later date or time, simply because EMA reduces the time lapse between engaging in the behaviour and reporting the behaviour. In contrast to this idea, while we are not able to know with certainty what participants in the current study ate, the current data would suggest that the 24-h dietary recalls were better able to capture the full scope of items eaten by participants than their report via EMA. It might be that participants benefit from the individual interview prompts associated with participating in a 24-h dietary recall.
Furthermore, there are challenges associated with asking people to report on intake of specific food categories; for example, a parent may not think of the tomato sauce in lasagna as a vegetable when they are asked via EMA whether a vegetable was served with the meal, whereas this information would be gathered during an interviewer-led 24-h dietary recall. Failing to report all foods eaten as a part of a mixed dish is an important issue as mixed dishes such as lasagna, spaghetti, burritos, stir fry dishes, etc. can contribute notably to vegetable intake. Researchers using EMA to collect dietary intake may want to also consider providing instructions to participants for how/if to classify ingredients that are present in small amounts (e.g. onions used as seasoning). A separate, but related, issue is that many research participants might not know how to properly categorise foods eaten when they asked to do so within an EMA protocol and this participant-led misclassification is likely to impact some food groups more than others (i.e. refined or whole grains, sugary drinks). Participant misclassification of these food items may have contributed, in part, to the lower concordance between the two dietary recall methods with regard to report of grains and sugar-sweetened beverages. Overall, it is crucial for researchers considering using EMA to collect dietary intake data to consider what type of research questions they are most interested in answering. Further, future research should pilot the use of different types of participant prompts and instructions designed to encourage the report of various food groups included in mixed dishes and reduce misclassification of certain foods to determine if the inclusion of such prompts and instructions might improve concordance between these two methods.
Results of the current study also showed that concordance of intake was highest for breakfast and snacks, as compared with other eating occasions; this higher concordance is likely due, at least in part, to the fact that participants reported eating fewer distinct items at breakfast (e.g. only cereal and milk) and snacks (e.g. only crackers). This finding suggests that assessment of dietary intake via EMA might be a good option for research studies with a focus on foods consumed during breakfast and snacks. Further, higher concordance was observed between the two measures if the eating occasion occurred at home. It might be that participants have an easier time remembering to report, and accurately reporting, their dietary intake via EMA when they are home, whereas they require more prompting to remember foods that are eaten outside of the home or ‘on the run’. Finally, only a small difference in concordance was seen by type of EMA report (event-contingent v. signal-contingent report); signal-contingent reports had slightly more positively concordant items than event-contingent reports. This finding suggests that unless it is important for a particular study to understand the timing of each eating occasion with extreme accuracy, it is not necessary to use event contingent surveys and instead dietary intake information could be gathered retrospectively via signal-contingent reports.
This paper sought to understand concordance in participant report of dietary intake of specific foods as reported via two different methodologies; thus, as discussed above, we restricted our analyses to only those meals we were able to match by meal type. That said, it is important to note that many meals were not included in our analyses because they were only reported via one dietary assessment methodology. A portion of these ‘unmatched meals’ can be explained by differences in participant instructions for the two dietary intake assessment methods; when completing a 24-h dietary recall, parents were instructed to report on all foods consumed by their child over the course of a full day, whereas for the EMA protocol, parents were instructed to report only foods their child consumed in the presence of the parent completing the EMA. This is major limitation of the current study design, which was not originally designed to assess how dietary intake assessed via EMA compares to diet assessed via interviewer-administered dietary recalls. Future studies comparing these two methodologies should use identical protocols for meal reporting.
Strengths of this study include the use of two time-linked dietary intake data sets which allowed for the capture of real-time snapshots of recent dietary intake (brief EMA dietary intake assessments), as well as detailed reports of full-day dietary intake (24-h dietary recalls). Further, the sample was diverse (i.e. race/ethnicity, SES and weight status), improving the generalisability of study findings to other, similarly diverse populations. In addition, kappas were used to calculate concordance, which is a more conservative measure which, to our knowledge, has not been used in other studies comparing dietary intake data collected via EMA and 24-h dietary recalls. This study is limited in that the design did not include direct observation of participant dietary intake; both EMA and 24-h dietary recalls are subject to measurement error stemming from retrospective recall bias and social desirability bias. Twenty-four-hour dietary recalls were prescheduled, and parents were asked to keep a food log introducing additional potential for measurement reactivity to impact data collected. This study seeks to examine concordance between two self-report measures of dietary intake, and errors in these methods are likely correlated (e.g. both likely subject to measurement error due to memory failure, social desirability bias, etc.). As a result, the level of agreement found between the two methods may over-state the accuracy of reporting via the EMA method. Future evaluations of the EMA method should consider using a comparison method that is not based on self-report such as meal observations or biomarker measures (e.g. serum carotenoids as a marker of fruit and vegetable intake).
Conclusions
The primary goal of the present study is to deepen our understanding of how dietary intake data collected via a brief EMA measure compares to that of data collected via the standard reference method of an interviewer-administered 24-h dietary recall. Overall, findings from this study and other similar studies suggest that dietary intake assessment via EMA shows promise and is appropriate for answering certain types of research questions; however, further development and evaluation of protocols should be pursued to continue to improve upon this methodology as a means of collecting dietary intake data. Researchers considering the use EMA to collect dietary intake should be aware of the current strengths and limitations of this methodology and must consider the primary goal(s) of their study (e.g. food groups and eating occasions of interest) before making a decision about the best approach to collecting dietary intake data.
Acknowledgements
Acknowledgements: The Family Matters study is a team effort and could not have been accomplished without the dedicated staff who carried out the home visits, including: Awo Ahmed, Nimo Ahmed, Rodolfo Batres, Carlos Chavez, Mia Donley, Michelle Draxton, Carrie Hanson-Bradley, Sulekha Ibrahim, Walter Novillo, Alejandra Ochoa, Luis ‘Marty’ Ortega, Anna Schulte, Hiba Sharif, Mai See Thao, Rebecca Tran, Bai Vue and Serena Xiong. These individuals have given permission for this manuscript and other manuscripts stemming from data collected as a part of the Family Matters study to be submitted for publication in peer-reviewed journals, including the Journal of the Academy for Nutrition and Dietetics. Financial support: Research is supported by grant no. R01 HL126171-04 from the National Heart, Lung and Blood Institute (PI: J.M.B.) and the first authors (K.A.L.) time is supported by grant no. K23-HD090324-01A1 from the National Institute of Child Health and Human Development (PI: K.A.L.). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Heart, Lung and Blood Institute, the National Institute of Child Health and Human Development or the National Institutes of Health. Conflict of interest: All authors have no financial disclosures to report. Authorship: K.A.L. assisted with the development of study concept and design; assisted with interpretation of the data; wrote the initial draft of the manuscript and coordinated revisions to the manuscript. A.F. assisted with the development of study concept and design; lead data analysis and assisted with writing and thorough review of the manuscript. A.T. assisted with the development of study concept and design; lead data acquisition; assisted with interpretation of the data and assisted with writing and thorough review of the manuscript. L.H. assisted with writing and thorough review of the manuscript. R.C. assisted with the development of study concept and design and assisted with writing and conducting a thorough review of the manuscript. D.N.-S. assisted with the development of study concept and design and assisted with writing and conducting a thorough review of the manuscript. J.M.B. is the principal investigator of the study; acted as a guarantor of the integrity of entire study; lead the development of study concept and design; assisted with data acquisition and assisted with writing and thorough review of the manuscript. Ethics of human subject participation: This study was conducted according to the guidelines laid down in the Declaration of Helsinki, and all procedures involving study participants were approved by the University of Minnesota Institutional Review Board. Written informed consent was obtained from all subjects/patients.
Supplementary material
For supplementary material accompanying this paper visit https://doi.org/10.1017/S1368980020001111
