Effect of an mHealth weight loss intervention on Healthy Eating Index diet quality: the SMARTER randomised controlled trial

In the few weight loss studies assessing diet quality, improvements have been minimal and recommended calculation methods have not been used. This secondary analysis of a parallel group randomised trial (regsitered: https://clinicaltrials.gov/ct2/show/NCT03367936) assessed whether self-monitoring with feedback (SM + FB) v. self-monitoring alone (SM) improved diet quality. Adults with overweight/obesity (randomised: SM n 251, SM + FB n 251; analysed SM n 170, SM + FB n 186) self-monitored diet, physical activity and weight. Real-time, personalised feedback, delivered via a study-specific app up to three times daily, was based on reported energy, fat and added sugar intake. Healthy Eating Index 2015 (HEI-2015) scores were calculated from 24-hour recalls. Higher scores represent better diet quality. Data were collected August 2018 to March 2021 and analysed spring 2022. The sample was mostly female (78·9 %) and white (85·4 %). At baseline, HEI-2015 total scores and bootstrapped 95 % CI were similar by treatment group (SM + FB: 63·11 (60·41, 65·24); SM: 61·02 (58·72, 62·81)) with similar minimal improvement observed at 6 months (SM + FB: 65·42 (63·30, 67·20); SM: 63·19 (61·22, 64·97)) and 12 months (SM + FB: 63·94 (61·40, 66·29); SM: 63·56 (60·81, 65·42)). Among those who lost ≥ 5 % of baseline weight, HEI-2015 scores improved (baseline: 62·00 (58·94, 64·12); 6 months: 68·02 (65·41, 71·23); 12 months: 65·93 (63·40, 68·61)). There was no effect of the intervention on diet quality change. Clinically meaningful weight loss was related to diet quality improvement. Feedback may need to incorporate more targeted nutritional content.

Overweight and obesity rates in the USA and across the world are high and continue to rise (1,2) .As such, extensive research has been conducted on developing and testing behavioural interventions for weight loss.Diet and exercise are recognised as key components of standard behavioural weight loss programmes with the US Preventative Services Task Force recommending such multicomponent interventions for adults with obesity (3) .However, many studies explicitly focus goals on energy restriction and low/moderate fat intake.While nutritional counselling in interventions may discuss other aspects of diet, such as increasing fruit intake, it is often couched in terms of helping with the energy and fat restriction goals.Few interventions report on dietary changes holistically; therefore, it is unclear if and how weight loss interventions affect diet quality.
Diet quality represents the healthfulness of an individual's overall dietary pattern.Poor diet quality is a major correlate of prevalent overweight and obesity (4) as well as weight gain over time (5,6) .In randomised controlled trials, improvements in diet quality have been shown to relate to weight loss separate from the effect of energy restriction (7,8) .Diet quality has also been shown to relate to weight loss maintenance postintervention (9,10) .
Importantly, diet quality measured using validated instruments (e.g.dietary recalls and food frequency questionnaires) and multiple indices (i.e.Healthy Eating Index, Alternate Healthy Eating Index, alternate Mediterranean Diet score and adherence to the Dietary Approaches to Stop Hypertension diet) has also been shown to relate to cardiometabolic risk factors other than weight (4,11) and to chronic disease independent of its effect on weight (12) .Indeed, women with obesity who have better diet quality are more metabolically healthy than those with poor diet quality (13) .Since weight loss can be difficult to achieve and weight regain is common (14) , diet quality change may provide an important additional endpoint for assessing the effectiveness of interventions for chronic disease prevention.
The provision of personalised dietary feedback may be particularly important for diet quality improvement.In a small pilot study among participants self-monitoring their diet using an app-based food diary (i.e.tracking foods as they were eaten throughout the day), who were not provided dietary feedback, diet quality worsened over 8 weeks.This was in comparison to participants using the Notepad app on the phone or paper-andpencil methods of self-monitoring who did receive feedback and who improved diet quality (15) .This secondary analysis expands on the potential importance of providing dietary feedback to adults with overweight or obesity that was suggested by the results of the pilot study.In a larger sample and over 12 months, we assessed the association between diet quality and a mHealth intervention of selfmonitoring (via Fitbit food diary) and personalised, automated feedback compared with self-monitoring alone.We hypothesised that the provision of feedback would result in improved diet quality over self-monitoring alone.We also assessed the relationship between diet quality improvement and weight loss with the hypothesis being that we would observe greater diet quality improvement among those with clinically meaningful weight loss compared with those without.In addition to establishing the importance of dietary feedback in a remote, scalable intervention, such an examination might help identify how the intervention can be refined for maximal benefit.

Study sample
The design of SMARTER, a 12-month parallel group randomised controlled trial, as well as the primary outcome of weight loss have been described elsewhere (16)(17)(18) (online Supplementary Fig. S1).Briefly, using randomisation software staff randomised adults with overweight or obesity with equal allocation stratified by race and gender to either a group which received individualised feedback messages based on self-monitoring data (SM þ FB) or to a self-monitoring only comparator (SM).Both groups were given energy, fat gram and physical activity goals.At baseline, all participants attended a one-on-one, 90minute dietary counselling session with a master's level registered dietitian who had prior experience in standard behavioural treatment.Both groups were instructed to weigh themselves daily on a study-provided digital scale, record all foods and drinks in the Fitbit food diary and wear a Fitbit activity tracker to monitor physical activity.The University of Pittsburgh institutional review board approved the protocol.Participants were recruited from the greater Pittsburgh, Pennsylvania area (August 2018-March 2020), and participants provided written informed consent.Neither assessors nor participants were blinded to treatment assignment due to the behavioural nature of the intervention.This analysis was conducted in Spring 2022.
Dietary feedback messages were delivered to a study-specific smartphone app up to three times daily while physical activity and weight messages were sent 3-4 times per week and once a week, respectively.Although there was no specific diet quality goal, suggestions to eat better were inherent in the feedback messages the intervention group received (e.g.'A balanced breakfast includes different food groups, for example, whole grain toast, nut butter, and fruit.').The type of message participants received was determined by algorithm conditions described previously (16) .Briefly, to select appropriate messages the algorithm used the energy, fat and added sugar intake reported in the Fitbit food diary along with consideration of the prescribed energy and fat goals.For example, if a participant recorded foods with fat content >30 % of the fat goal at breakfast, a message suggesting they limit fat for the remainder of the day may have been sent (e.g.'Choose lower fat foods as the day goes on to stay on track with your calorie and fat goal').
Most of the over 2000 unique dietary feedback messages in the message library focussed on energy, fat and added sugar intake, as well as the importance of self-monitoring foods.Supplementary Table S1 provides examples of messages that address each component of the Healthy Eating Index-2015 (HEI-2015).The HEI-2015 aligns with the Dietary Guidelines for Americans (DGA) 2015-2020 and was used as the measure of diet quality in this study (19) .Importantly, while some components were never addressed, such as sodium (Na), there were multiple possible messages addressing other components (e.g. the word 'fruit' appears in over 100 unique messages).

Measures
Demographic characteristics (e.g.race and ethnicity, income, education) and some clinical characteristics (i.e.blood pressure, high cholesterol, high trigylcerides [TAG] and smoking status) were self-reported at baseline.Height, weight, waist circumference and blood pressure were measured by trained staff at baseline and follow-up and BMI (kg/m 2 ) was calculated based on weight and height measurements.At-home weight data from the WIFI-enabled scale was used in lieu of staff-collected data after the start of the COVID-19 pandemic in the USA (March 2020).
Dietary data were collected at baseline, 6 and 12 months on two separate days (i.e. up to 6 recalls total per participant throughout the study period) to minimise within-person random error.The 6-month time point was selected because other weight loss studies have shown improvement through 6 months and then weight regain.Dietary intake was assessed at the 12-month time point as it was the end of the intervention period.Dietary recalls were collected using the Automated Self-Administered 24-hour recall (ASA-24) system managed by the National Cancer Institute (20) .ASA-24 performs well compared with intervieweradministered recall (21) with computer prompts imitating the multi-pass method.To minimise the potential for issues with usability, participants completed their first dietary recall while staff members trained in dietary recalls were present to answer questions.For subsequent recalls, which could be completed remotely, participants could reach out to staff for assistance.HEI-2015 scores were calculated from dietary recalls.The HEI-2015 has demonstrated construct validity, reliability and criterion validity (22) and is consistent with other dietary indices (23) .The HEI-2015 includes nine adequacy and four moderation components.For most components, intake per 1000 calories (1000 kcal = 4184 kJ) is scored.Values between the minimum and maximum score are scored proportionally.Component scores are summed to create an HEI total score with 100 being the best possible score (19) .Because the HEI is density-based, the correlation between HEI diet quality and diet quantity is low, suggesting a desirable independence between measures (22) .The average HEI-2015 score in the general population is 59 (24) , well below a score of 74 which would satisfy Healthy People 2020 objectives (25) .
There are various methods of HEI calculation.As the bivariate (26) and multivariate (27) methods require sufficiently large datasets, these methods were not viable for use in our sample.Therefore, HEI-2015 total and component scores were calculated using the population ratio method as it is recommended for the assessment of intervention effects (28) .The population ratio method provides a less biased estimate of a population's mean HEI score than the simple scoring algorithm or mean ratio method (29) .

Statistical analysis
The distributions of continuous variables were assessed for normality using histograms and normal probability plots.For continuous-type normally distributed variables, comparison of baseline characteristics between those with complete data at all time points and those with missing data at 6 and/or 12 months was conducted using pooled variance t tests when group variances were equal (i.e.male waist circumference, systolic and diastolic blood pressure, education) or separate-variance t tests with the Satterthwaite method approximation of degrees of freedom when group variances were not equal (i.e.female waist circumference).Wilcoxon rank-sum tests were applied to compare group-specific distributions when the distribution of continuous-type variables was not normal (i.e.age, BMI and body fat percentage).Chi-square tests were used to compare distributions of categorical variables (i.e.sex, race and ethnicity, employment, household income, self-reported high blood pressure, high cholesterol and high TAG, smoking status) between those with and without missing dietary data.All analyses were performed using SAS version 9.4 (SAS Institute, Inc.).
The population ratio method was used to calculate the mean intake of dietary constituents across all individuals prior to constructing ratios and scoring the HEI-2015.Bootstrap resampling was utilised to estimate 95 % CI for mean HEI-2015 total and subcomponent scores for each study time point by treatment group and by clinically meaningful weight loss status with 200 bootstrap resamples generated.To test whether there were differences in HEI 2015 total scores at each timepoint by study arm and weight loss categorisation, CI were compared for overlap.As recommended, radar plots were used to visualise component scores of the HEI-2015 (19,30) .
As there was no observed difference in mean weight loss, the primary outcome of the parent study, between study arms at either 6 or 12 months (17,18) ; the study arms were combined when assessing the diet quality over time of (1) those achieving clinically meaningful weight loss (≥ 5 % of baseline weight) at the 6-month time point compared with those without clinically meaningful weight loss at the 6-month time point and (2) those achieving clinically meaningful weight loss at the 12-month time point compared with those without clinically meaningful weight loss at the 12-month time point.
A sensitivity analysis was performed comparing those participants in the SM þ FB group who viewed less than the median percentage of FB messages over 12 months compared with those who viewed greater than or equal to the median percentage of FB messages.The percentage of FB messages viewed was calculated as the number of messages viewed divided by the number of intended messages.For participants completing the entire study, the number of messages they were supposed to receive was 1095 messages (i.e., 3 messages a day × 365 days).

Results
Of the 502 participants enrolled (SM n 251; SM þ FB n 251), 356 were included in this secondary data analysis as retention at 12 months was 78•5 % (18) and because some participants retained in the study did not contribute dietary data at followup.Participants in this complete case analyses (Table 1) were mostly female (78•9 %), white (85•4 %), middle-aged (median = 51•0 years) and had obesity (median BMI = 33•1).Participants with complete dietary data (n 356) who were included in this analysis were defined as having ≥ 1 dietary recall at all timepoints.Analysed participants were significantly older (P < 0•0001), more likely to identify as white (P = 0•01), more likely to report high blood pressure (P < 0•01), and high cholesterol (P < 0•01) and had higher systolic blood pressure (P = 0•04) and lower BMI (P = 0•03) compared with those who were missing dietary data at 6 and/or 12 months.The percentage of participants with complete dietary data did not differ by treatment assignment (SM = 67 quality (∼2 points) have been observed at 6-and 12-month time points in the cellphone-based arm of a behavioural intervention despite specific emphasis on following the Dietary Approaches to Stop Hypertension (DASH) eating pattern, regular prompts to self-monitor and communication with 'buddies' (31) .In a workplace intervention that included automated feedback, 6-, 12-and 24-month improvements in HEI-2015 total scores were similarly small (2•2, 1•8 and 1•6 points respectively) (32) .Additionally, in an internet-based weight loss intervention with energy and fat goals, self-monitoring and automated feedback, HEI-2015 total scores improved by approximately 4 points over 3 months (33) .However, as diet quality improvements have been seen to lessen over the course of an intervention (34,35) , additional follow-up time might have shown attenuated results.Limited engagement with digital self-monitoring might have precluded improvement in diet quality in our study.Dietary selfmonitoring declined curvilinearly over time with only about half of days being self-monitored over 12 months on average (36) , and the median percentage of feedback messages intervention participants viewed over the 12 months was low (18) .Such issues with engagement might underly the lack of differences over time as well as the lack of differences between groups since the number of diet-related feedback messages a participant receives is related to weight loss only indirectly through self-monitoring adherence (37) .If self-monitoring can be improved and feedback better received, results might be stronger as evidenced by the fact that we saw a signal towards greater improvement in diet quality among SM þ FB participants who viewed more messages.This aligns with our previously reported results showing that percentage of feedback messages viewed is related to weight loss (18) .
The single session with a dietician at baseline in SMARTER may not have been sufficient for inducing diet quality change, and there were no specific goals for altering macronutrient intake, other than fat intake.Many behavioural weight loss programs such as ours, draw from the Diabetes Prevention Program (DPP).However, diet quality improvements in the DPP were small despite a high level of coach contact.This may be because only one intervention session directly addressed healthy eating independent of weight loss (7,38) .Indeed, it may be that improvements in diet quality were small in this study because there was no explicit focus on altering components of diet quality identified by the HEI.In a study in which sessions were specifically focussed on recommendations from the DGA, the number of sessions attended was positively associated with HEI-2005 scores (35) .Other reasons for the lack of improvement in diet quality in this study may be related to the fact that the intervention targeted individual-level behaviour change only.Multilevel interventions that simultaneously target individuallevel and socio-ecological factors may be warranted.Diet quality change in a mHealth intervention Despite the lack of a significant difference between SM þ FB and SM groups, the 5-6 point improvement from baseline that we observed among participants with clinically meaningful weight loss at 6 months was similar to what has been suggested as a likely meaningful difference between groups (28) .Our results were similar when assessing weight loss status at 12 months as 76•0 % of those who lost ≥ 5 % of baseline weight at 12 months had already lost 5 % of baseline weight by the 6-month time point.
While it is important to note a clinically meaningful cut point for improvement in diet quality has not been established and scores from different versions of the HEI (and different indices) are not directly comparable, the relationship we observed between weight loss and HEI improvement is broadly similar to that seen in other studies.In a small pilot study with a 16-week intervention period, women who achieved clinically meaningful weight loss improved HEI-2005 total scores by 8•3 points (39) .In both the Weight Optimization Revamping Lifestyle using the Dietary Guidelines (WORLD) (35) and Daughters and Mothers Against Breast Cancer (DAMES) (8) studies, which used the HEI-2005, relationships between change in diet quality and weight loss were also observed.Additionally, among rural breast cancer survivors who maintained lost weight (≥ 5 % of baseline weight) v. those who did not, there was a 4•6-point difference in Alternate Healthy Eating Index (AHEI) scores at 18 months (9) .Such accumulating evidence of a relationship between diet quality and weight loss reinforces the need for thorough consideration of diet quality in intervention design and analysis.
A scoping review of mobile-based interventions concluded there was inadequate examination of dietary behaviour change and its relationship to health outcomes (40) .Indeed, a systematic review of the use of the HEI in eighteen studies aimed at weight loss identified limitations in the assessment of diet (41) .A major strength of this study is that it is one of the few to explore this question rigorously by using multiple 24-h dietary recalls, which are less biased than food frequency questionnaires and appropriate calculation of the Healthy Eating Index.Additional strengths of the study include the randomised design, relatively large sample size and the intervention's scalability.

Limitations
There are some limitations to this analysis.Reporting errors cannot be ruled out as recalls were not unannounced, thus reducing one of the main advantages of using dietary recalls (42) .Similarly, a little less than half of participants had both a weekday and weekend dietary recall at baseline with less than a third contributing both a weekday and weekend recall at 6 and 12 months.Most participants contributed two non-consecutive recalls at baseline with only about half doing so at 6 months and 12 months.Collecting data on both weekdays/weekends and on non-consecutive days may reduce error as weekday eating has been shown to differ from weekend eating (43) , and consecutive days of recall tend to be correlated (44) .A final limitation is that our analysis included only those participants with complete data.Likely those with missing data had little to no improvement in diet quality due to their disengagement with the study.Therefore, had they been included, results might have been attenuated.

Conclusions
In conclusion, minimal diet quality improvement was observed in this mHealth intervention among participants with overweight and obesity.When there is excessive energy intake, as with those with overweight/obesity, a focus on addressing low scores for refined grains, added sugars and/or saturated fats might be most beneficial to improving diet quality scores for two reasons.First, the general population has particularly low scores on these components.Second, as described by Krebs-Smith and colleagues, focus on these components would lead to improvements in all component scores that are density-based because refined grains, added sugars and saturated fats contribute a large number of energies (19) .In SMARTER, no messages addressed sodium and few addressed grains (refined or whole) or saturated fat specifically.Therefore, more focussed feedback messages might help induce changes in diet quality.Personalisation of features besides feedback, such as delivery or timing, (45) increasing self-efficacy and culturally tailoring nutritional feedback have also been suggested as ways of increasing the benefits of remote, personalised interventions (46) .However, because diet quality improvement in weight loss trials has not been extensively explored, components of interventions necessary for producing diet quality improvement remain unclear.As such, weight loss researchers may consider assessing improvements in diet quality such that clarity can be achieved.Widening focus when designing interventions to include improvement not only for weight and other cardiometabolic outcomes, but also for health behaviours such as diet quality, may benefit participants having difficulty achieving or maintaining weight loss.

Supplementary material
For supplementary material/s referred to in this article, please visit https://doi.org/10.1017/S0007114523001137

Table 2 .
HEI-2015 total and component scores by treatment group and time point

Table 3 .
HEI-2015 scores at each time point by weight loss status at 6 months