Expanding the dissemination and implementation of evidence-based programmes that address nutrition-related health issues and/or behaviours is necessary to improve population health. These interventions have the potential to reach and impact a large proportion of the population, including those at risk of developing nutrition-related health conditions and those already diagnosed. This potential reach is particularly important for medically underserved and health disparate communities, including those in rural Appalachia, USA, where access to evidence-based lifestyle interventions is limited.
Understanding participant engagement can provide evidence of the external validity necessary to speed the translation of effective community-based lifestyle interventions into practice. Yet engagement, or the extent to which and how participants become involved with intervention activities( Reference Leung, Chan and Sea 1 , Reference Nobles, Perez and Skelton 2 ), is infrequently explored. Engagement includes considerations such as attrition and dropout (i.e. loss of participants, often determined by participants not completing final data collection activities) and attendance at or completion of intervention activities (e.g. classes and calls)( Reference Nobles, Perez and Skelton 2 ). A recent review of community-based lifestyle interventions targeting diet and physical activity behaviours found that 83 % of studies reported attrition but only 66 % reported attendance rates( Reference Stoutenberg, Stanzilis and Falcon 3 ).
Given the lack of general reporting of engagement indicators, it is unsurprising there is a lack of understanding regarding the more nuanced aspects of engagement, such as how participant factors impact engagement and relationships between engagement and outcomes. Although there have been a number of systematic reviews assessing the influence of participant-level factors (e.g. demographics, behaviours, weight status, health) on engagement in lifestyle interventions targeting weight-related behaviours( Reference Leung, Chan and Sea 1 , Reference Moroshko, Brennan and O’Brien 4 – Reference Burgess, Hassmen and Pumpa 6 ), there are no definitive conclusions about which factors may be the most influential. However, among the factors that have been explored, older age is most consistently associated with greater engagement. It is important to note that these reviews are limited by small numbers of studies exploring individual variables and inconsistency in how engagement was defined and measured. Additionally, although reviews suggest greater engagement in an intervention’s activities leads to more favourable outcomes, the relationship between engagement and outcomes for specific interventions is rarely reported( Reference Durlak and DuPre 7 ). This means the actual dose–response relationship for many interventions is unknown, and it is only an assumption that greater engagement in the intervention’s activities will lead to more favourable outcomes. Importantly, when these more in-depth explorations of engagement are conducted, they often examine by total engagement and not by individual component( Reference Thomson, Tussing-Humphreys and Goodman 8 ) although many interventions have multiple components.
Greater understanding of participant engagement is important for interventions targeting sugar-sweetened beverage (SSB) intake. Due to the high levels of SSB intake( Reference Popkin, Armstrong and Bray 9 – Reference Rosinger, Herrick and Gahche 11 ) and the preventable health conditions associated with high intake( Reference Malik, Popkin and Bray 12 ), there is a strong need to translate these interventions into practice to reduce intake to the recommended amount of ≤8 ounces (237 ml) of SSB daily( Reference Popkin, Armstrong and Bray 9 ) or to a clinically significant reduction of ≥12 ounces (355 ml) of SSB daily( Reference Chen, Caballero and Mitchell 13 ) (i.e. US fluid ounces; 1 US fl. oz = 29·57 ml).
SIPsmartER is a multicomponent community-based intervention targeting SSB intake. It was developed for adults living in rural Appalachia, USA, and was designed in consideration of the needs of those with lower incomes, educational attainment and health literacy status. SIPsmartER is one of only two known interventions proven to produce significant reductions in SSB intake among adults( Reference Vargas-Garcia, Evans and Prestwich 14 – Reference Zoellner, Hedrick and You 18 ). SIPsmartER has been shown to reduce SSB intake significantly (−19 v. −5 ounces (–562 v. –148 ml), P = 0·001) and BMI (−0·21 v. 1·0 kg/m2, P < 0·05) compared with a matched contact comparison condition( Reference Zoellner, Hedrick and You 18 ). Additionally, SIPsmartER has been designated as a Research-Tested Intervention/Program by the National Cancer Institute( 19 ). Although rates of attrition and activity completion were reported in the outcome paper( Reference Zoellner, Hedrick and You 18 ) and a mixed-methods summative evaluation identified barriers to participation( Reference Bailey and Hill 20 ), an in-depth exploration of participant engagement with SIPsmartER has not been conducted.
The purpose of the present study was to explore participant engagement in SIPsmartER in order to support its translation into practice. Specifically, the current paper describes patterns of participant engagement within intervention activities, explores relationships between participant baseline characteristics and three engagement variables (completion of 6-month health screening, class attendance, call completion) and investigates the relationship between engagement and SSB outcomes. We hypothesize participants will have consistent patterns of engagement across components of the programme (e.g. those with high class attendance will have high call completion)( Reference Thomson, Tussing-Humphreys and Goodman 8 ). We anticipate that engagement will be positively associated with greater age( Reference Leung, Chan and Sea 1 , Reference Moroshko, Brennan and O’Brien 4 ). Finally, we expect that greater participation in SIPsmartER, as measured by the engagement indicators, will be associated with greater reductions in SSB intake( Reference Durlak and DuPre 7 ).
Methods
The present study is a secondary analysis of data from the SIPsmartER arm of the Talking Health trial( Reference Zoellner, Chen and Davy 17 , Reference Zoellner, Hedrick and You 18 ). All study procedures were approved by the Institutional Review Boards of Virginia Tech and the University of Virginia. Participants provided written informed consent and received $US 25 and $US 50 in gift cards for completing the baseline and 6-month health screenings.
Participant recruitment
Participants from rural south-west Virginia, USA, an area with documented health disparities( 21 ) and high rates of SSB intake( Reference Zoellner, Krzeski and Harden 22 ), were recruited into the trial through passive (i.e. non-direct interactions with potential participants, such as flyers) and active (e.g. direct interactions with potential participants, such as in-person recruitment at local clinics and stores) methods( Reference Zoellner, Chen and Davy 17 , Reference Estabrooks, You and Hedrick 23 ). Participants had to be >18 years old, speak English, consume ≥837 kJ (≥200 kcal) from SSB daily, not have contraindications to physical activity and have access to a telephone. Participants were randomly assigned into SIPsmartER or the matched contact comparison condition. A total of 155 participants were enrolled between June 2012 and June 2014 into one of eight cohorts, which were determined by geographic location.
SIPsmartER design
SIPsmartER consisted of two health screenings, three group classes and twelve support calls( Reference Zoellner, Chen and Davy 17 , Reference Porter, Chen and Estabrooks 24 ). Participants completed health screenings before and immediately following the intervention. Group classes (~2 h) occurred in weeks 1, 7 and 19, and consisted of activities to build behaviour-specific content knowledge and skills, including action planning. If participants missed a class, materials were mailed to them and a trained research assistant attempted to complete the lesson with them by telephone. A teach-back call occurred the week after the first group class. This supportive call allowed the participant to review content from the first class, identify current SSB intake and complete a personal action plan with a trained research assistant( Reference Porter, Chen and Estabrooks 24 ). The eleven interactive voice response (IVR) calls took place weekly (weeks 4–6) and biweekly (weeks 9–18 and 21–26). During these <10 min calls, participants identified their ounces of SSB intake, completed an action plan and received a motivational message.
SIPsmartER’s retention protocol was designed to support (re)engagement in intervention components. Participants were sent reminder postcards and texts prior to classes and health screenings. The IVR system and research staff contacted participants up to three times to complete calls during times the participant had designated as convenient. If participants missed two activities in a row, a research assistant contacted them to complete the next activity and support re-engagement. Participants could schedule their 6-month health screening. If they were unable to schedule themselves, they were assigned a time slot. Participants who missed a health screening appointment were called within 30 min of the missed appointment to reschedule.
Data collection and measures
The present study incorporates demographic, anthropometric, engagement and behavioural data collected during the recruitment process, baseline and 6-month health screenings, and/or during SIPsmartER’s implementation. Ten of the eleven demographic characteristics were included as independent variables in the analyses because they have been previously associated with participant engagement indicators and/or health behaviour outcomes( Reference Leung, Chan and Sea 1 , Reference Moroshko, Brennan and O’Brien 4 – Reference Burgess, Hassmen and Pumpa 6 , Reference Thomson, Tussing-Humphreys and Goodman 8 , Reference Bailey and Hill 20 , Reference Fitzpatrick, Jeffery and Johnson 25 – Reference Williams, Coday and Somes 29 ).
Demographics
Nine demographic variables were included in the current analysis. Eight were collected during the recruitment and screening process: (i) age (continuous variable); (ii) gender (male or female); (iii) race (six categories based on the National Institutes of Health protocol); (iv) highest level of educational attainment (six categories, including: completed Grade 8, some high school, completed high school, some college, college graduate and graduate school); (v) employment (nine categories, including: employed full-time, employed part-time, self-employed, on disability, out of work <1 year, out of work >1 year, homemaker, student and retired); (vi) annual household income (twelve categories ranging in $US 5000 increments from <$US 5000 to ≥$US 50 000); (vii) marital status (six categories, including: married, divorced, widowed, separated, never married and member of an unmarried couple); and (viii) number of children under 18 years living at home (continuous variable). The health literacy variable was collected during the baseline health screening using the Newest Vital Sign, a validated health literacy measure( Reference Weiss, Mays and Martz 30 ).
Anthropometry
Participant BMI was calculated using height and weight data collected during the baseline health screening. Height was measured twice using a portable stadiometer. Weight was assessed using a calibrated digital Tanita scale (model SC-331S). Participants removed shoes, outer clothing (e.g. sweatshirts), belts and items from their pockets before stepping on to the scale.
Engagement
Participant completion of the three group classes, twelve calls and 6-month health screening were tracked by research staff throughout the intervention.
Behavioural data
Participant SSB intake, the primary outcome of the trial, was measured using the validated BEVQ-15( Reference Hedrick, Savla and Comber 31 ) at both baseline and 6-month health screenings. This measure was administered using an audio-assisted computer program( Reference Riebl, Paone and Hedrick 32 ).
Data analysis
Data analysis was conducted using the statistical software package IBM SPSS Statistics version 23 (2014). Descriptive statistics (counts, frequencies, means and standard deviations) were used to describe the study sample and participant engagement in intervention activities.
Three engagement variables were used: (i) completion of 6-month health screening (6-month completion, yes or no); (iii) number of classes attended (class attendance, 0–3); and (iii) number of calls completed (call completion, 0–12).
Three SSB outcome variables were created: (i) change in SSB intake between 6-month and baseline health screenings (SSB ounce reduction = 6-month intake subtracted from baseline intake); (ii) 12 ounce (355 ml) or greater decrease from baseline to 6-month health screening (reduced ≥12 ounces, yes or no); and (iii) 8 ounces (237 ml) or less daily intake recommendation met at 6-month health screening (achieved ≤8 ounce intake, yes or no).
Age and baseline SSB intake (as total daily ounces) were modelled in continuous form. Six demographic variables were either changed from continuous to categorical form or were collapsed into smaller categories. Recategorization of the variables was performed due to small numbers of participants in some categories or to allow for more meaningful interpretation of the data. Race was collapsed into ‘White’ or ‘non-White’ categories. Education was collapsed into ‘high school or less’ or ‘some college or more’. Employment was collapsed into three categories: ‘full/part-time employment’, ‘no employment’ or ‘other’. Other included students and retirees. Annual income was collapsed into ‘earning <$US 20 000’ or ‘earning ≥$US 20 000’. Marital status was collapsed into ‘partner in the home’ or ‘no partner in the home’. Number of children in the home was collapsed into ‘child(ren) in the home’ or ‘no child in the home’. Health literacy scores from the Newest Vital Sign test were dichotomized into ‘high health literacy’ and ‘low health literacy’ using standard scoring techniques( Reference Weiss, Mays and Martz 30 ). Participant BMI (kg/m2) was categorized into three categories: underweight/normal weight (BMI < 25), overweight (25 ≤ BMI > 30) and obese (BMI ≤ 30).
Chi-square tests of association, Pearson correlation coefficients, Spearman rank correlations, t tests and one-way ANOVA with Tukey’s post hoc tests were used to assess bivariate associations between engagement, SSB intake, and baseline participant characteristics and measures. While all participants (n 155) were included in engagement analyses, only the 109 participants who completed the 6-month health screening and were not pregnant at either the baseline or 6-month health screening were included in SSB intake analyses. Pregnant participants (n 4) were removed because their baseline and/or 6-month SSB intake could be impacted by factors other than the intervention, such as trying to make more healthful beverage decisions because of their pregnancy.
Generalized linear models were used to test for significant effects of demographic variables on engagement variables and for significant effects of demographic and engagement variables on SSB intake outcomes. All potential explanatory variables, both demographic characteristics and engagement indicators, were initially included in the models. Using a backwards stepwise process, variables were iteratively removed until only significant variables remained in the final models. Call completion and SSB ounce reduction were modelled using normal distributions. Class attendance was modelled using a multinomial distribution, while 6-month completion, reduced ≥12 ounces and achieved ≤8 ounce intake were modelled using binomial distributions.
Results
Participant characteristics for the complete sample are presented in Table 1. Participants’ mean age was 41·5 (sd 13·4) years and mean daily SSB consumption was 42·9 (sd 30·9) ounces (1268·7 (sd 913·8) ml) at baseline. The majority of participants were female (81 %), White (91 %) and earned less than $US 20 000 per annum (61 %). Per census data, enrolled participants were similar to the general population for all demographic factors, except for having a lower mean income, a somewhat higher educational attainment and a lower proportion of males( Reference Zoellner, Chen and Davy 17 ).
Table 1 Demographics of the rural participants (n 155) from south-west Virginia, USA, enrolled in SIPsmartER, a behavioural intervention targeting sugar-sweetened beverage (SSB) intake, June 2012–June 2014
† n 154 (due to missing baseline data).
Patterns of engagement
Patterns (frequency distributions) of participant engagement with intervention activities are presented in Fig. 1. The 6-month health screening was completed by 74 % of participants. The majority of participants attended all three classes (51 %) and completed seven or more calls (54 %). Mean class attendance and call completion were 2·0 (sd 1·2) and 6·4 (sd 3·7), respectively. Additionally, participants who completed the 6-month health screening attended significantly more classes (2·5 v 0·9, P < 0·001) and completed more calls (7·5 v. 3·3, P < 0·001) than those who did not. Class attendance and call completion were positively associated (Spearman rank correlation = 0·626, P < 0·001).
Fig. 1 Distribution of participant engagement across intervention components: (a) distribution of health assessment completion, (b) distribution of class attendance rate and (c) distribution of call completion rates, among rural participants (n 155) from south-west Virginia, USA, enrolled in SIPsmartER, a behavioural intervention targeting sugar-sweetened beverage intake, June 2012–June 2014
Relationships between engagement and participant baseline characteristics
Results from the generalized linear models predicting engagement are presented in Table 2. Significant bivariate associations are described in the text.
Table 2 Final generalized linear model results for engagement variables among the rural participants (n 155) from south-west Virginia, USA, enrolled in SIPsmartER, a behavioural intervention targeting sugar-sweetened beverage (SSB) intake, June 2012–June 2014
*P ≤ 0·05, **P ≤ 0·01, ***P ≤ 0·001.
† Initial models included all independent variables: age, gender, race, educational attainment, employment, household income, marital status, children in home, health literacy, baseline SSB intake, baseline BMI category. Only final models are presented. Only variables significant at the 0·05 level were retained in final models.
‡ One participant missing baseline SSB intake was excluded from the analysis.
Completion of 6-month health screening
Participants who completed the 6-month health screening had a higher mean age than participants who did not complete the screening (43·0 v. 36·0 years, P = 0·002). Proportionally more White participants completed the health screening compared with non-White participants (77 v. 43 %, P = 0·031). Proportionally more participants classified as obese completed the health screening compared with participants classified as underweight/normal weight (80 v. 56 %, P = 0·038).
Age and race significantly contributed to the final model which explained 12 % of variance (χ 2 = 13·71, P = 0·001). For a 1-year increase in age, the odds of completing the 6-month health screening increased by 4 % (OR = 1·04; 95 % CI 1·01, 1·07; P = 0·009). For White participants, the odds of completing the 6-month health screening were almost four times the odds of non-White participants (OR = 3·47; 95 % CI 1·06, 11·34; P = 0·039).
Class attendance
Participant age was positively associated with class attendance (Spearman rank coefficient = 0·36, P < 0·001). Mean class attendance was higher among White participants compared with non-White participants (2·1 v. 1·5 classes, P = 0·070) and higher among participants without children in the home compared with participants with children in the home (2·2 v. 1·9 classes, P = 0·042).
Only age was a significant predictor in the final model which explained 16 % of variance (χ 2 = 24·58, P < 0·001). For a 1-year increase in age, the odds of attending an additional class increased by 6 % (OR = 1·06; 95 % CI 1·04, 1·09; P < 0·001).
Call completion
Participant age was positively associated with call completion (Spearman rank coefficient = 0·33, P < 0·001) while baseline SSB intake was negatively associated with call completion (Spearman rank coefficient = −0·20, P = 0·011). Mean call completion was higher among participants earning ≥$US 20 000 per annum compared with those earning less (7·3 v. 5·8 calls, P = 0·018) and higher among participants without children in the home compared with participants with children in the home (7·2 v. 5·6 calls, P = 0·007).
Age, income and children in the home significantly predicted call completion in the final model which explained 17 % of variance (F = 9·56, P < 0·001). For every 1-year increase in age, the number of calls completed increased by approximately one-tenth (e.g. for every 10-year increase in age, call completion increased by about one; β = 0·08; P < 0·001). Participants earning ≥$US 20 000 per annum completed 1·5 more calls compared with participants earning <$US 20 000 per annum (β = 1·53; P < 0·009). Participants with children in the home completed 1·2 fewer calls compared with participants without children in the home (β = −1·24; P < 0·033).
Relationships among sugar-sweetened beverage outcomes, engagement and participant baseline characteristics
The mean change in SSB intake from baseline to the end of the intervention was −26·1 (sd 35·6) ounces (–771·9(sd 1052·8) ml). Regarding the specific benchmarks, 56 % (n 61) of participants had a ≥12 ounce decrease in SSB intake and 53 % (n 58) met the ≤8 ounce recommendation for SSB intake at intervention end. Results from the generalized linear models are presented in Table 3, with significant bivariate associations described in the text.
Table 3 Final generalized linear model results for sugar-sweetened beverage (SSB) outcome variables† among the rural participants (n 155) from south-west Virginia, USA, enrolled in SIPsmartER, a behavioural intervention targeting SSB intake, June 2012–June 2014
*P ≤ 0·05, **P ≤ 0·01, ***P ≤ 0·001.
† US fluid ounces (1 US fl. oz = 29·57 ml).
‡ Initial models included all independent variables: age, gender, race, educational attainment, employment, household income, marital status, children in home, health literacy, baseline SSB intake, baseline BMI category, class attendance, calls completion. Only final models are presented. Only variables significant at the 0·05 level were retained in final models.
Sugar-sweetened beverage ounce reduction
Mean reduction in SSB intake was larger among participants without children in the home compared with participants with children in the home (−33·4 v. −18·7 ounces (–987·8 v. –553·0 ml), P = 0·030). Baseline SSB intake was negatively associated with change in SSB intake (Pearson correlation coefficient = 0·74, P < 0·001).
In the generalized linear model, baseline SSB intake and class attendance significantly contributed to the model which explained 70 % of variance (F = 124·39, P < 0·001). For every 1 ounce increase in baseline SSB intake, change in intake decreased by approximately 1 ounce (β = −0·90; P < 0·001). For each class attended, participants decreased intake by about 6 ounces (β = −6·12; P = 0·003).
Reduced ≥12 ounces
Mean baseline SSB intake was higher among participants who reduced their SSB intake by ≥12 ounces compared with participants who did not (58·3 v. 22·5 ounces (1724·1 v. 665·4 ml), P < 0·001). Mean class attendance also was higher among participants who reduced their SSB intake by ≥12 ounces compared with participants who did not (2·6 v. 2·2 classes, P = 0·021).
The final model was significant and explained 5 % of variance (χ 2 = 3·92, P = 0·048). Only the presence of a partner in the home contributed to the model. Participants without a partner were 2·2 times more likely to reduce their SSB intake by ≥12 ounces (OR = 2·21; 95 % CI 1·00, 4·92; P = 0·05).
Achieved ≤8 ounce intake
Proportionally more participants who earned ≥$US 20 000 per annum achieved the ≤8 ounce daily SSB intakerecommendation compared with participants who earned <$US 20 000 per annum (50 v. 33 %, P = 0·079). Proportionally fewer participants with a partner in the home achieved this recommendation compared with participants without a partner in the home (50 v. 67 %, P = 0·079). Mean baseline SSB intake was lower among participants who achieved ≤8 ounce intake compared with participants who did not (35·1 v. 51·1 ounces (1038·0 v. 1511·2 ml), P = 0·011). Mean call completion was higher among participants who achieved ≤8 ounce intake compared with those who did not (8·2 v. 6·8 calls, P = 0·027).
Baseline SSB intake and total calls were significant predictors in the final model which explained 8 % of variance (χ 2 = 6·77, P = 0·009). For a 1 ounce increase in baseline SSB intake, the odds of achieving ≤8 ounce intake decreased by approximately 2 % (OR = 0·98; 95 % CI 0·97, 1·00; P = 0·015).
Discussion
There have been other studies exploring engagement in lifestyle interventions, including in-person, group-based programmes targeting general nutrition and physical activity behaviours( Reference Thomson, Tussing-Humphreys and Goodman 8 ) and behaviours associated with hypertension( Reference Thomson, Landry and Zoellner 27 ); a multicomponent (group sessions and one-on-one counselling) programme for overweight/obese type 2 diabetic patients( Reference Fitzpatrick, Jeffery and Johnson 25 ); and a web-based weight-loss programme( Reference Neve, Collins and Morgan 33 ). However, the present study is the first known to conduct an in-depth exploration of participant engagement in an effective intervention targeting SSB intake among adults. Findings from the study inform the future dissemination of SIPsmartER into practice. Additionally, these findings in conjunction with other explorations of engagement in community-based lifestyle interventions( Reference Leung, Chan and Sea 1 , Reference Moroshko, Brennan and O’Brien 4 – Reference Burgess, Hassmen and Pumpa 6 ) provide insight for future studies assessing patterns of engagement and relationships among participant baseline characteristics, engagement and outcomes in community-based, behavioural interventions.
Patterns of participation
As hypothesized, participants were consistent in their levels of engagement across intervention components (i.e. those with high class attendance also had high call completion rates). This is aligned with previous findings that participants engage with each component within a multicomponent trial at similar rates; that is, either high engagement with all or low engagement with all( Reference Thomson, Landry and Zoellner 27 ).
Engagement in SIPsmartER is similar to what has been reported in the literature. Participation in SIPsmartER classes and calls was 56 %, and reviews suggest attendance is roughly 50–60 % in lifestyle interventions addressing weight-related behaviours( Reference Leung, Chan and Sea 1 , Reference Lemstra, Bird and Nwankwo 34 ). The attrition rate in SIPsmartER was 26 %, which is also within the 0 to >30 % range of attrition from other community-based lifestyle interventions( Reference Stoutenberg, Stanzilis and Falcon 3 ). The greater frequency of completion of the health screening (73·5 %), compared with classes and calls, is most likely related to it being incentivized while the classes and calls were not.
Pertaining to patterns of engagement among classes and calls, while 51 % of participants attended all three classes, the remaining participants were about equally distributed among the zero, one or two class attendance categories. However, for the IVR calls, no distinctive pattern was apparent and there appeared to be two primary ‘peaks’ in the distribution of calls: completing three calls (12 %) and completing nine calls (14 %). We can speculate potential reasons for inconsistencies in engagement with the IVR calls: (i) participants’ ability to consistently complete calls may have been impacted by their schedule and issues with phone service( Reference Bailey and Hill 20 ); (ii) participants could have stopped completing calls when they had reached their goal; and (iii) participants may have felt compelled to complete all or as many calls as possible as part of their enrolment in SIPsmartER.
Participant baseline characteristics and SIPsmartER engagement
The present study’s findings indicate that, as hypothesized, increasing age was associated with greater engagement for each of the three engagement variables, and age retained its significant associations in the presence of other explanatory variables. It is likely older participants may have more flexibility in their schedules (e.g. retired or children are grown) that allows them to attend classes and complete calls. Also, older participants may be more concerned about the effects of excessive SSB intake on chronic health conditions, such as diabetes, heart disease and obesity. This finding adds to existing literature that older age is associated with greater attendance in lifestyle interventions targeting weight-related behaviours( Reference Leung, Chan and Sea 1 , Reference Burgess, Hassmen and Pumpa 6 ). It also suggests that more strategies to support the engagement of younger participants should be developed and/or implemented in SIPsmartER and other similar interventions.
Three other participant baseline characteristics were significantly associated with engagement variables in the presence of other explanatory variables: race, presence of children in the home and income. However, each of these baseline characteristics significantly impacted only one of the three engagement variables. This lack of consistency among predictive baseline characteristics reflects findings from the literature( Reference Leung, Chan and Sea 1 , Reference Burgess, Hassmen and Pumpa 6 ) but also suggests that different participant characteristics are more influential for different components. This finding may support the identification of component-specific barriers and guide the development of strategies and considerations to support greater engagement. First, although race did not significantly impact engagement with classes and calls, non-White participants were significantly less likely to attend the 6-month health screening than White participants. Although the Appalachian counties targeted by SIPsmartER are known to be majority White (92 %), this finding highlights the need for researchers to be mindful that potential racial differences could impact completion of data collection activities during future implementations of SIPsmartER in the region. Second, participants with children in the home completed fewer calls than those without children. This participant-level factor did not impact either of the other engagement variables. Although the calls were relatively short (<10 min), duties associated with childcare may have contributed to those with children having less predictable schedules to answer and complete the calls. Third, income level impacted completion of calls. Lower-income participants could have had less consistent access to telephone service due to phone service being turned off, waiting for new cell phone minutes to start at the beginning of the month, purposefully rationing talk minutes on their cell phone at the end of the month and changing phones more frequently.
Importantly, our findings provide further evidence that SIPsmartER engaged its target audience. Although SIPsmartER may be applicable for all adults who consume SSB, it was developed to specifically address the needs of rural adults who consume SSB and who have low health literacy, low socio-economic status and/or low educational attainment( Reference Thompson, McNeel and Dowling 35 – Reference Sharkey, Johnson and Dean 39 ). None of the three engagement variables were impacted by health literacy or education and only the call completion variable was influenced by income; therefore, there is evidence to suggest that SIPsmartER engaged its target population beyond just enrolling them into the intervention( Reference Estabrooks, You and Hedrick 23 ).
Baseline characteristics and engagement on SIPsmartER sugar-sweetened beverage outcomes
Two demographic and one engagement variables significantly contributed to SSB outcome models. Higher baseline SSB intake and higher class attendance predicted SSB ounce reduction, while lower baseline SSB intake predicted achievement of ≤8 ounce intake. Not having a partner in the home predicted achieving a reduction of ≥12 ounces of SSB.
Although predictive in two models, baseline SSB influenced achievement of outcomes differently. This finding is logical as it most likely would require a less drastic change (i.e. allow for more daily SSB intake and require less of an overall SSB decrease) for participants with higher baseline intakes to have a greater SSB ounce reduction than to achieve ≤8 ounce intake. This pattern would be opposite or less pronounced for someone with lower baseline intake.
Not having a partner in the home could have increased the likelihood of whether a participant was able to reduce his/her SSB intake by ≥12 ounces as the participant may have had fewer intrapersonal-level barriers to making this change. We can only speculate on why this variable was predictive of only one of the behavioural outcomes. It could be that achieving an SSB reduction of ≥12 ounces is less dependent on baseline intake than the other SSB outcome variables. Importantly, there is mixed evidence on how being part of a married/cohabitating couple influences dietary behaviour change and this evidence often centres on couples in which one has a chronic disease that requires dietary modification, such as diabetes( Reference Albanese, Huffman and Celano 40 ).
Findings regarding engagement do not fully reflect our hypothesis about the relationship between engagement and outcomes, as class attendance predicted one of three SSB outcomes (SSB ounce reduction) while call completion was not predictive of any outcome. There are possible reasons why the results were different from our hypothesized relationships.
Greater call completion may not be related to improved SSB intake because the nature of the calls – personalized and majority automated – may have led participants to complete calls when they felt it necessary to support their personal changes and, by doing so, to determine their own effective dose. For example, some may have completed calls when they were actively working to change their SSB intake and stopped once they felt they achieved their goals. This nature of the calls differed from that of the classes, which involved interaction with other participants and research staff; therefore, participants, regardless of success at reaching their intake goals, may have felt compelled to attend as many of the classes as possible.
The lack of influence of either class attendance or call completion on achievement of the ≤8 ounces recommendation could be due to a number of factors. One, while the classes and calls were intentionally designed to help participants both reduce intake and achieve the daily recommendation of ≤8 ounces of SSB, they may have been more effective at the former. Although the classes and calls explicitly discussed the recommendation and encouraged participants to reach it by the end of the programme, participants had the flexibility during the classes and calls to set their intake goals at the amount that worked best for them, which could have been higher than 8 ounces. Two, class attendance or call completion may not be the best engagement variables to measure SIPsmartER’s impact on achievement of ≤8 ounce intake. That only 8 % of the variance is explained by the model suggests that there are most likely other factors that influenced the achievement of this outcome. It possible one of these factors was a different type of engagement variable. Of the four types of engagement indicators that have been identified (attrition, attendance, self-monitoring behaviours and dietary adherence)( Reference Leung, Chan and Sea 1 ), class attendance and call completion directly reflect attendance but only indirectly reflect self-monitoring (due to participants using self-monitored behaviour to report their current intake and on action plans completed during classes and calls). So, it is possible that another engagement variable related to SIPsmartER that more directly measured self-monitoring (e.g. completion of weekly diaries) or dietary adherence (e.g. whether participant set their weekly intake goal at ≤8 ounces) may have been more predictive of achieving ≤8 ounce intake than class attendance and call completion.
Limitations
There are a few limitations to the present study. The generalizability of the study results is limited because the sample was predominantly White, middle-aged and female. However, the racial composition of the sample is relatively representative of the targeted region and larger Appalachia( Reference Estabrooks, You and Hedrick 23 ). Also, there may be variables more predictive of either engagement (e.g. personal factors, distance to intervention location, pregnancy, health conditions, transportation) or SSB outcomes (e.g. temporal changes in SSB, different engagement measures such as frequency of self-monitoring) that were either not collected during the trial or were too small to be explored in the present study.
Future directions
Findings provide practical considerations for the future implementation of SIPsmartER. They will be used to identify strategies or intervention adaptations that could support greater engagement at the 6-month health screening, classes and calls. For example, based on what was learned in this trial, we could hold focus groups with lower-income participants to identify how to increase call completion rates within the context of potential limited phone access. These findings also offer suggestions for future research that explores relationships among engagement, outcomes and participant characteristics. First, future research should examine engagement by intervention component, not just as a whole, as predictors of engagement may vary by component. Second, the execution of studies exploring engagement should be an a prioridecision. Doing this will increase the likelihood that these studies systematically collect the necessary predictors and engagement outcomes.
Conclusions
Our study provides evidence that greater engagement in the class component of SIPsmartER, but not the call component, influenced achievement of SSB outcomes. Additionally, our study indicates that SIPsmartER was able to engage its target population beyond recruitment and identifies several participant-level characteristics (e.g. age, race, children in the home, income, race) which might have impacted engagement in SIPsmartER. Overall, our study findings can be applied to inform future translation of SIPsmartER, and other multicomponent evidence-based programmes of similar design and intent, into practice.
Acknowledgements
Acknowledgements: The authors acknowledge the Talking Health team members who have been instrumental in the development and implementation of the trial. Financial support: This work was supported by the National Institutes of Health/National Cancer Institute (NIH/NCI; J.M.Z. principal investigator, grant number 1R01CA154364-01A1). NIH had had no role in the design, analysis or writing of this article. Conflict of interest: None. Authorship: K.J.P. collected data, conceptualized the current study, analysed data and wrote/edited the manuscript. J.L.T. conceptualized the current study, analysed data and edited the manuscript. J.M.Z. conceptualized the parent study and the current study, provided feedback on data analysis and edited the manuscript. Ethics of human subject participation: This study was conducted according to the guidelines laid down in the Declaration of Helsinki and all procedures involving human subjects were approved by the Virginia Tech and University of Virginia Institutional Review Board committees. Written informed consent was obtained from all participants.
