Introduction
Online grocery shopping has increased dramatically in recent years. Online food and beverage sales in the US nearly tripled from 2017 to 2021, from $12⋅7 billion to $34⋅2 billion, and now comprise 10% of all US grocery sales(Reference Pasquali1). Online grocery shopping is expected to continue to grow, with industry projections estimating that by 2024–2026, Americans will spend more than $47⋅6 billion on online groceries, or approximately 20% of all grocery sales(Reference Pasquali1). Although online grocery shopping is more popular among higher-income households, online grocery shopping is growing rapidly among all demographic groups(Reference Conway2), and policymakers and retailers are actively seeking to expand access to online groceries to new populations(3–5).
As online grocery shopping becomes more popular, it will be important to understand how policies and interventions (from marketing practices(Reference Moran, Headrick and Perez6) and restrictions(Reference Eshoo, Schakowsky, Rush, Jones, Roybal-Allard and San Nicolas7) to warning labels(Reference Pomeranz, Mozaffarian and Micha8–Reference Corvalán, Reyes and Garmendia11) to corporate initatives(Reference Taillie, Ng and Popkin12,Reference Acosta13) ) implemented in online stores affect consumer behaviour in this setting. Randomised trials are important tools for determining the causal impact of nutrition interventions on behaviour. However, conducting trials with real-world online grocers is not always feasible because retailers are often not willing or able to randomly assign consumers to different interventions. To address these challenges, we developed a naturalistic online grocery ordering platform designed to mimic a real-world online shopping experience while allowing researchers to implement nutrition interventions(Reference Rummo, Higgins and Chauvenet14). We have previously described the development of this store (see details below)(Reference Rummo, Higgins and Chauvenet14), but little is known about how consumers behave in and respond to naturalistic online grocery stores. Without information on consumer experiences in these settings, it is difficult to know whether naturalistic online grocery stores are a feasible, acceptable, and potentially ecologically valid venue for conducting nutrition research.
To address these gaps, the objectives of the present study were to (1) examine the extent to which participants found the naturalistic online store to be realistic and acceptable and (2) describe participants’ food and beverage selections in the naturalistic online grocery store and examine the extent to which these selections mirror their choices in a real online grocery store.
Methods
Participants
From November 2021 to March 2022, study staff recruited participants using Craigslist advertisements. To be eligible, participants needed to be aged ≥18 years, live in the US, be able to read and write in English, have shopped for groceries online during the past 12 months, and have access to the Internet (see Supplementary Table S1 for the eligibility screener). Supplementary Figure S1 shows the number of people screened, enrolled, and included in analyses.
Setting
Participants completed online study visits with a member of the research team using the videoconferencing software Zoom. At each visit, participants completed a shopping trip in one of two stores: a previously developed naturalistic (‘mock’) online grocery store (Lola's Grocery) or a popular real online grocery store (Walmart.com). Details on the development of the naturalistic store have been published previously(Reference Rummo, Higgins and Chauvenet14). Briefly, the naturalistic store mimics the appearance and functionalities of real online grocers, including browsing, search, product pages, sorting and filtering, and checkout. It offers approximately 23 000 products sold at a top US online grocery retailer(Reference Rummo, Higgins and Chauvenet14), with products shown with images, price, and nutrition information. Researchers can manipulate aspects of the store (e.g., changing product prices, display order, and labels). Fig. 1 depicts an example page from the naturalistic online store.
Sample page from the naturalistic online grocery store.
Procedures
Participants completed two online study visits (~30–60 min each) separated by ~1 week. During the first visit, research staff obtained informed consent from participants using an electronic survey. Additionally, during the first visit, research staff instructed participants to make a list of foods and beverages they typically buy each week when grocery shopping because most US households shop for groceries using a list(Reference Smith, Berning and Yang15,16) . Participants made the list using an online document (a Google Doc); study staff saved this list as a PDF and shared it with participants via Zoom chat immediately before both shopping tasks so participants could reference it while they shopped, if they chose.
During both study visits, participants selected foods and beverages in the naturalistic online store or the real online store and responded to survey questions about their experience shopping in that store. Store order (naturalistic v. real) was randomised within participants. Before the participants began shopping, research staff provided them with the following instructions: to select foods and beverages they typically buy each week using their list. The same instructions were used for both the naturalistic and the real online store. Participants then shopped on their own (i.e., without screensharing or involvement from the research team); one research staff member remained in the Zoom room with video and microphone off while the participant shopped to troubleshoot any technical difficulties in real time. For the shopping trip in the naturalistic online store, participants navigated to the naturalistic online store using a link provided by study staff. They placed items they wished to select in their cart. No payment was required. When participants were finished shopping, they were automatically re-directed to an online survey programmed in Qualtrics, which they completed on their own. The naturalistic online store automatically captured data on all products selected by participants, including information on product name, brand, quantity, and price. For the shopping trip in the real online store, participants navigated to the store's online grocery ordering platform using a link provided by study staff and logged into the study team's account. The account was pre-set to display items available at a St. Louis, MO store (selected because the grocery cost of living in St. Louis is similar to the US average(17) and because prices in the naturalistic store were set using St. Louis grocery prices(Reference Rummo, Higgins and Chauvenet14)). When they were finished shopping, participants responded to a survey programmed in Qualtrics, which they completed on their own. No payment was required. Research staff then logged into the study account and recorded the items in the participant's cart, including name, brand, quantity, and price. Because data collection coincided with frequent product shortages (e.g., stemming from the COVID-19 pandemic), participants notified staff of items they wished to purchase that were out of stock; staff recorded these in real time.
Participants received a $30 electronic gift card for completing the first visit and a $50 gift card for the second visit. The University of North Carolina and Harvard Longwood Campus Institutional Review Boards approved all study procedures.
Measures
We assessed participants’ experiences in the store using survey measures. Survey measures included acceptability of the stores (e.g., satisfaction with number of options), acceptability of the study protocol (e.g., willingness to participate in another research study using the stores), and realism of the stores (e.g., extent to which the stores felt real) (Supplementary Table S2). Participants responded to these measures after both shopping trips (i.e., for both the naturalistic and real stores).
We also examined participants’ spending in both stores on fifteen food and beverage categories that are often targeted by food policies or are relevant for health (e.g., sugar-sweetened beverages, fruits and vegetables; Supplementary Table S3). Participants’ selections were categorised using product descriptions and nutrition information. Because the naturalistic online store did not offer alcohol, we excluded alcohol selections from the real online store from our analyses (five items selected by three participants).
Analysis
To characterise acceptability, we estimated the proportion of participants who endorsed positive statements about their experience in the naturalistic online store. We also examined these outcomes for the real online store to provide a baseline against which the naturalistic store could be compared.
Next, we calculated means and SDs of expenditures in each food category for each store. To understand whether participants shopped similarly in the two stores, we calculated Pearson's correlation coefficients (rs), comparing expenditures in each category from the naturalistic online store to the real store. Correlations of 0⋅30–0⋅49 and ≥0⋅50 were considered moderate and strong, respectively(Reference Akoglu18–Reference Cohen20). We also examined whether expenditures differed between the naturalistic and real stores using paired t tests. We focused on proportional expenditures in each category to address potential inflation in the real online store that would not have occurred in the naturalistic store (which kept fixed prices throughout the study); we also report absolute expenditures (i.e., dollars) to provide context for estimates of proportional expenditures. All analyses were conducted in 2022–2023 using Stata MP version 17.1 (StataCorp LLC, College Station, TX, USA).
Results
The average age in the sample was 39⋅5 years (sd = 11⋅0). The sample was diverse in terms of race/ethnicity, education, and income (Table 1). For example, 43% of participants identified as white, 38% as Black or African American, and 13% as Latino(a) or Hispanic. About one-third (30 %) had educational attainment of some college or less and more than half (59%) reported a household income of less than 200% of the Federal Poverty Level.
Sample characteristics, n 144 US adults
Note. Missing data ranged from 0⋅0% to 0⋅7%.
Results suggested the study protocol was feasible. Study staff enrolled 157 participants in ~4 months (mid-November to early March, excluding winter holidays). Retention was high: of the 157 participants who were enrolled and completed their first study visit, 154 (98%) completed the second visit (Supplementary Figure S1).
Participants’ survey responses indicated that the naturalistic online store offered a highly acceptable setting for nutrition research. For example, all participants (100%) indicated they would be willing to participate in another research study that required shopping in the naturalistic online store (Fig. 2). Likewise, nearly all (98%) reported they were able to find most things, almost everything, or everything on their shopping list in the naturalistic store. Most (83%) reported that the naturalistic online store was easy or very easy to use. Nearly all participants reported that they would pick similar products in the real world as they selected in the naturalistic online store (97%), that their selections in this store were similar to their usual purchases (95%), and that the naturalistic online store felt like a real online store (92%). On all process evaluation dimensions, the naturalistic online store performed similarly to the real store (Fig. 2).
Acceptability of the naturalistic and real online grocery stores, n 144 US adults.
Mean total spending in the online naturalistic store was $139⋅02 (sd = 83⋅96), similar to the real store (mean = $156⋅04, sd = 85⋅75) (Supplementary Table S4). Across food categories, meat and seafood had the highest absolute mean spending in both stores (approximately $30–32) while eggs had the lowest (approximately $2–3).
Analyses found moderate to strong correlations between spending on food categories in the naturalistic online store compared with spending in the real online store (range of correlation coefficients, r = 0⋅36–0⋅67, all P-values < 0⋅001, Table 2). The highest correlations were observed for meat and seafood (r = 0⋅67) and bread (r = 0⋅67), followed by dairy (r = 0⋅66) and fruits, vegetables, and legumes (r = 0⋅64) (all P-values < 0⋅001). There were few significant differences in mean expenditures on the fifteen food categories between the two stores; the only exceptions were that participants spent slightly less on eggs (1⋅6% v. 1⋅9%, P = 0⋅02) and salty snacks (4⋅6% v. 6⋅3%, P = 0⋅001) in the naturalistic store than the real store and slightly more on low-calorie drinks and juice (9⋅7% v. 7⋅4%, P = 0⋅002).
Comparison of proportional expenditures on fifteen food and beverage categories in a naturalistic online store and a real online store, n 144 US adults
Note. Definitions of food categories are provided in Supplementary Table S3. Purchases of alcohol were excluded (five items selected by three participants in the real online store) from analyses because alcohol was not available in the naturalistic online store.
Discussion
The present study of US adults found that a naturalistic online store developed for research is highly acceptable to participants. For example, the large majority of participants in this study reported that the naturalistic store was easy to use and that they were satisfied with the number of options in the store. All participants also indicated they would be willing to participate in another study using the naturalistic online store. Moreover, participants rated the naturalistic online store similarly to the real online store, suggesting that the naturalistic store does not have major acceptability shortcomings relative to conducting studies with a real online grocer. Acceptability and feasibility measures in the present study also compared favourably with reactions to other naturalistic grocery stores used for research(Reference Waterlander, Jiang and Steenhuis21,Reference Hall, Higgins and Grummon22) .
Results also suggested that naturalistic store effectively mimicked a real online shopping experience. First, nearly all participants reported that the naturalistic online store felt like a real store and that their purchases in this store were similar to the groceries they usually buy. Second, participants appeared to behave in similar ways in the naturalistic online store as they do at real-world grocers. In our study, participants selected similar foods and beverages in the naturalistic online store as they did in the real online store, with correlations in spending on food categories in the moderate-to-strong range (correlations of 0⋅36–0⋅67) and few significant differences between stores. Importantly, although the present study assessed hypothetical purchases, participants’ food and beverage selections in the naturalistic store also mirrored nationally representative data on real-world grocery purchases. For example, when simulating their usual weekly shopping trip, participants in this study purchased a mean of $139 from the naturalistic online, which is within the range of average weekly spending on food-at-home among households with and without children ($106–160)(Reference Wunsch23). Although differences in food categories examined between this study and national data preclude comprehensive comparisons, when these comparisons were possible, purchases in the naturalistic online store aligned well with national estimates. For example, fruit and vegetable purchases comprised 20⋅0% of total spending in the naturalistic online store, very similar to the estimated 19⋅8% of total at-home food expenditures on fruits and vegetables among households of similar size in the nationally representative Consumer Expenditure Surveys (CES)(24). Likewise, spending on meat and seafood (21% in this study v. 21% in CES), dairy (7% v. 9%), eggs (2% v. 1%) and cereal (2% v. 4%) in the naturalistic store were all similar in this study compared with national estimates(24). Together, these results suggest that the naturalistic online store could offer an ecologically valid setting for nutrition research.
The strengths of the present study include the use of a novel online grocery store resembling a real online grocer and the diverse sample. Limitations include that all participants had shopped for groceries online in the previous 12 months, so results may not generalise to consumers who never shop online. However, online grocery shopping has risen dramatically in recent years and is projected to comprise 20% of all grocery sales by 2024–2026(Reference Pasquali1). Another limitation is that participants did not purchase their selections. This did not differ between the naturalistic and real online stores, but selections in this study could differ from real-stakes purchases. However, selections in this study aligned closely with national data on food purchases from the CES, perhaps lessening this concern. Future research will establish the feasibility of delivering groceries from naturalistic online grocery stores.
Conclusions
Online grocery shopping is increasingly common, creating an urgent need to understand the impact of nutrition policies and interventions in online settings. Naturalistic online grocery stores offer a promising avenue for research studies seeking to evaluate how nutrition interventions affect online food purchasing behaviour.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/jns.2023.21.
Acknowledgements
A. H. G., P. E. R., M. G. H., L. M. J. and L. S. T. designed and conceptualised the study and developed the study measures. A. C. T., V. N., C. E. P. and V. L. contributed to data collection. A. H. G., A. C. T. and V. N. contributed to data curation and statistical analyses. C. E. P. and V. L. provided project management. A. H. G. provided supervision and drafted the initial manuscript. All authors provided critical feedback on the manuscript draft and approved the final manuscript.
A. H. G. was supported by grants from the National Institutes of Health (K01 HL158608 and T32 HL098048). M. G. H. was supported by a grant from the National Institutes of Health (K01 HL147713). The funders had no role in the design, analysis or writing of this article.
The authors declare that they have no conflicts of interest.
Open access
