The food environment (FE), encompassing both formal and informal settings, is the space where consumers interact with the food system. Availability, affordability, convenience, promotion, quality and sustainability of food and beverages are factors that can influence the FE(Reference Downs, Ahmed and Fanzo1). Informal food outlets are selling spots that offer ready-to-eat fresh or packaged food and beverages in public spaces. These establishments lack access to essential amenities such as water, toilets, shelter and electricity and normally operate without government regulations(Reference Skinner2). Informal food outlets could include mobile vendors, kiosks, wet markets and street vendors(Reference Downs, Ahmed and Fanzo1,Reference Fellows and Hilmi3) . Street food outlets (SFO) regularly offer ready-to-eat items, with a growing presence of packaged ultra-processed products(Reference Fellows and Hilmi3–Reference Long-Solís5). On the other hand, formal food outlets are those regulated by formal governance structures. In these spaces, sellers can advertise their location and prices. Formal food outlets include supermarkets, supercentres or megastores, retailers and restaurants(Reference Downs, Ahmed and Fanzo1). In recent years, the evaluation of the FE has gained increasing significance in public health since limited access to healthy foods in low-income areas(Reference Pinheiro, Quintiliano-Scarpelli and Flores6–Reference Pérez-Ferrer, Auchincloss and de Menezes13) has been associated with the rise of overweight and obesity(Reference Pérez-Ferrer, Auchincloss and de Menezes13–Reference Rosales Chavez, Bruening and Ohri-Vachaspati18).
While current FE assessments predominantly focus on the density or proximity of food outlets within communities, limited research has been done on the variables of availability, variety, promotion, price and nutritional quality of foods and beverages within stores(Reference McKinnon, Reedy and Morrissette19,Reference Lytle and Sokol20) . With certain tools and according to the variables mentioned above, a nutritional healthfulness (NH) index, i.e. the quality that establishments have for offering healthy and non-healthy products, can be estimated(Reference Duran, Roux and Maria do Rosario7,Reference Franco, Roux and Glass10,Reference Borges, Gabe and Jaime21–Reference Martins, Cremm and Leite24) . Other studies considered healthy products like fruit and vegetables, non- or low-fat milk or whole wheat bread and non-healthy products such as soft drinks, sugar-sweetened nectar/juice, chocolate-filled cookies, highly processed foods and ultra-processed foods and beverages. However, the evaluation of FE is mainly centred on the formal environment(Reference Downs, Ahmed and Fanzo1). In contrast, assessment tools of the NH for informal FE, such as SFO, remain limited(Reference Medina, Piña-Pozas and Aburto25,Reference Akhter and Cameron26) .
The assessment of the psychometric properties of tools is essential to ensure the consistency and accuracy of measurements, support the result interpretation and ensure generalisation(Reference Zerafati-Shoae, Taghdisi and Azadbakht27,Reference Lytle28) . Among the literature reviewed, ten studies were identified detailing the psychometric properties of tools applied in at least one type of informal establishment (e.g. open-air food markets, mobile stands and street vendors). These studies assessed inter-rater and test–retest reliability through Cohen’s kappa(Reference Duran, Roux and Maria do Rosario7,Reference Duran, Lock and do Latorre11,Reference Martins, Cremm and Leite24,Reference Hosler and Dharssi29–Reference Chávez, Jehn and Lee33) , intraclass(Reference Duran, Lock and do Latorre11,Reference Martins, Cremm and Leite24,Reference Hosler and Dharssi29) and Pearson’s correlation indexes(Reference Borges and Jaime30). Validity evaluation employed methods like internal consistency(Reference Borges, Gabe and Jaime21,Reference Martins, Cremm and Leite24) , face(Reference Byker Shanks, Jilcott Pitts and Gustafson34), content(Reference Borges and Jaime30) and construct validity(Reference Duran, Lock and do Latorre11,Reference Borges, Gabe and Jaime21,Reference Martins, Cremm and Leite24,Reference Byker Shanks, Jilcott Pitts and Gustafson34) . However, it is important to note that most of these studies use the same tool for assessing FE in formal and informal settings. While having a comprehensive tool covering both FE might seem advantageous, it is essential to recognise the need for assessment tools specific to each type of establishment since what may be offered can vary, and accurate healthfulness classification requires a tailored approach.
To our knowledge, no tool has been exclusively validated for measuring the NH of SFO, a crucial gap, especially in low- and middle-income countries where the informal FE is widespread(Reference Downs, Ahmed and Fanzo1). Therefore, this study aims to develop and evaluate the validity and reliability of the Street Food and Beverage Tool (SFBT).
Methods
Street food outlets
In this study, SFO were defined as outlets that offer ready-to-eat foods and beverages prepared and/or sold by vendors in the streets and public places(Reference Fellows and Hilmi3), lack essential services such as access to water, toilets, shelter and electricity and normally operate without government regulations(Reference Skinner2). SFO were classified based on their physical structure, construction and material characteristics, location, stored quantity, temporality, occupation of physical space in relation to human use of space, mobility and legislation(Reference Villanueva35) as fixed (those that are permanent, located on public roads, and when opened, can occupy more of the public space), semi-fixed (those whose structure presents feasibility of assembling and dismantling and that are placed on public roads), mobile setups (those that continually move from one place to another) and extensions (those semi-fixed elements that extend from a legally approved structured (e.g. home) towards the public road, usually as a table containing ready-to-eat food or beverages)(Reference Fellows and Hilmi3).
Tool development and pilot testing
The SFBT was developed in 2021 at the National Institute of Public Health Mexico by an internal technical group of experts in nutrition, FE, anthropology and urban planning, most of them from this institution, with a main focus on measuring the NH of SFO. The instrument’s items were developed in two stages. In the first stage, a series of items was proposed based on a systematic review of instruments that measured the NH of the informal food outlets(Reference Medina, Piña-Pozas and Aburto25). With the internal research group (CM, CN, TA, CG, AG, JCH, UL, LH and VB – eight with a bachelor’s degree in Nutrition), an iterative process was undertaken to select and improve the final items. Then, pilot testing was performed in a convenience sample of SFO located in four different states of Mexico and within different contexts in Mexico City. In total, 134 SFO were evaluated around Morelos, Toluca, Campeche and Mexico City schools. In the last state, 339 SFO were evaluated around parks, eighty around subway stations and 130 around public hospitals. A list of food and beverages found during the pilot phase was created, and the products were classified as healthy or unhealthy for daily consumption based on Gaona-Pineda et al. 2018(Reference Gaona-Pineda, Martínez-Tapia and Arango-Angarita36), the EAT-LANCET’s sustainable food recommendations(Reference Willett, Rockström and Loken37), the NOVA classification(Reference Monteiro, Cannon and Levy38) and the internal technical group’s agreement. In total, 113 items were identified. Then, inter-rater and test–retest reliability tests were performed, with most items achieving strong and perfect agreement results (data not shown). Since several of these items were Mexican dishes or specific to the Mexican context, the 113 items were categorised into healthy and unhealthy food and beverage groups to facilitate their use in other contexts. Then, a NH index was developed. This index was based on the availability (i.e. presence) of the above-described food or beverage groups. The final index used to determine the NH of each SFO was as follows:
$$\eqalign{ & {\rm{NH - index \;(\% ) = (number \;of \;available \;healthy \;groups/ }} \cr & {\rm{(number \;of \;available \;healthy \;groups + }} \cr & {\rm{number\;of \;available\;unhealthy\;groups))}} \cr} $$
The second stage involved strengthening the tool items based on feedback provided by an external technical group of five researchers free of conflicts of interest from the National Institute of Public Health, the National Institute of Statistics and Geography and the Institute of Nutrition from Central America and Panama, with previous experience in epidemiology, international food and nutrition policy, FE evaluation and urban health.
Content validity
To assess the content validity of the SFBT (i.e. the degree to which the items capture the desired content)(Reference Lytle28), each member of the external technical group evaluated the adequacy and pertinence of the tool items using an adapted online questionnaire based on Rubio et al. 2003(Reference Rubio, Berg-Weger and Tebb39). The questionnaire asked about the adequacy and pertinency of each item, considering the following options: (a) not a lot adequate/pertinent = 1, (b) not very adequate/pertinent = 2, (c) more or less adequate/pertinent = 3, (d) adequate/pertinent = 4 and (e) very adequate/pertinent = 5.
In addition, the experts provided comments or suggestions on modifying, substituting or eliminating specific items. The Content Validity Index was estimated as the proportion of experts that considered each item as adequate/pertinent or very adequate/pertinent. Satisfactory or strong agreement among the experts was considered when the average content validity index was ≥0·80(Reference Rubio, Berg-Weger and Tebb39).
Construct validity
The estimated NH index was compared by income areas to assess the construct validity (i.e. the degree to which the tool’s measurement is consistent with the theoretical hypotheses) of the SFBT(Reference Lytle28).
The NH index was classified as none healthfulness (none-NH) if the availability percentage of healthy groups was = 0 %, low healthfulness (low-NH) if the availability percentage of healthy groups was >0 % to <51 % and high healthfulness (high-NH) if the availability percentage of healthy groups was ≥51 %. According to previous studies(Reference Pinheiro, Quintiliano-Scarpelli and Flores6–Reference Pérez-Ferrer, Auchincloss and de Menezes13,Reference Rosales Chavez, Bruening and Ohri-Vachaspati18) , SFO located around schools in higher-income areas would have a higher availability of foods and beverages recommended for daily consumption than SFO in low-income areas. The tool was considered valid if the hypothesis was congruent with the expected results. The Social Development Index (SDI) 2020 served as a proxy variable of income area. The SDI is calculated based on the method of Unsatisfied Basic Needs, which is part of the Integrated Poverty Measurement. This index, frequently used to measure poverty in Mexico City, evaluates some sociodemographic dimensions, including social security, health, housing, education, durable goods and energy, which are described in detail on the EVALUA Ciudad de México web page(40). For this study, the SDI was used and categorised into low-SDI areas (i.e. very low and low), middle-SDI areas (i.e. medium) and high-SDI areas (i.e. high). The total SFO was estimated by counting the number of SFO per block and was used as a continuous variable in the analysis. Meanwhile, ‘residents per block,’ expressed as a continuous variable, was obtained from the Census of Population and Housing from the National Institute of Statistics and Geography in 2020(41). Both variables, SDI and ‘residents per block,’ were used as covariates(Reference Duran, Roux and Maria do Rosario7). The school was selected using stratified cluster sampling; the strata were schools, and clusters were defined by geographic areas surrounding the schools (see online supplementary material, Supplemental File S1). A sample of 60 schools was selected by systematic sampling with equal probability where schools were sorted by a socio-economic index computed on the school neighbourhoods(42). With the selected school being the centroid, all street outlets within an Euclidean buffer of 500 m2 were evaluated(43).
After reading the manual and having five hours of theoretical and six hours of practical training, three raters with a bachelor’s degree in health, nutrition and geography evaluated 1581 SFO using the SFBT programmed in RedCap. The measurement was conducted by observing the readiness of the products, focusing on the main dishes and not accounting for the various combinations of toppings that consumers could add. For products prepared on-site, raters referred to the menu. When the menu was unavailable, raters asked the vendor about the available preparation options for sale. An informed consent form was read to the vendors, and 1419 agreed to participate. Data were collected from September 30, 2022, to June 23, 2023, between 8:00 and 15:00 hrs.
Descriptive statistics were used to characterise the sample, and a multinomial regression model was used to estimate construct validity, using the NH index as the dependent variable and SDI as the independent variable. The goodness-of-fit test, model specification and the independent variables’ multicollinearity were evaluated.
Inter-rater and test–retest reliability
SFO sell food or beverages in any location, including high-foot-traffic locations(Reference Fellows and Hilmi3). An example of this in the Mexican context is the subway station exit(44). The inter-rater (i.e. the degree to which measures are repeatable between two or more evaluators and over time)(Reference Lytle28) and test–retest reliability (i.e. the consistency of a measurement when replicated over time)(Reference Lytle28) of the NH index and group availability were evaluated in thirty-eight subway exits. For subway station selection, two stations (Pantitlán and Tacubaya) had the highest yearly affluence and were selected with probability proportion according to the passenger’s affluence of stations per route line in 2021(44). The rest of the stations (150, n 132·5 million passengers per year) worked as a cluster for sampling (further details are described in see online supplementary material, Supplemental File S1). After selecting a subway station, interviewers randomly selected two SFO located to the right of the exits up to 50 metres from each selected station.
Data were collected from July 25 to August 4, 2023, between 9:00 and 16:00 hrs. During the first visit, an informed consent form was read to the vendors, and their participation was verbally confirmed. Following the training and measurements detailed in the construct validity section, two evaluators with bachelor’s degrees in nutrition collected the information for the reliability tests. This procedure assessed the same 80 SFO on two occasions simultaneously, one week apart and at the same hour of the day. Vendors that decided not to participate once or twice were excluded from the evaluation.
Descriptive statistics were used to characterise the sample. The percentage of agreement, Cohen’s kappa index and Pearson’s correlation (r) were used to assess inter-rater and test–retest reliability. For categorical variables (group availability; yes/no), kappa was classified as null when the index was 0–0·20, minimum 0·21–0·39, weak 0·40–0·59, moderate 0·60–0·79, strong 0·80–0·90, almost perfect >0·90, and perfect = 1(Reference McHugh45). The continuous variable (NH index) underwent a square root transformation to achieve a normal distribution. All analyses were performed using the statistical program STATA version 13.
Results
Street food and beverage tool
The SFBT comprises 15 items and six sub-items divided into two sections. The first section focuses on the characteristics of the SFO. It includes questions directed to vendors, such as whether they have been evaluated previously, their days of operation, whether they are selling in other locations and whether they have been authorised to photograph their products. It also includes items assessed by observation, such as the type of outlet; surroundings near the outlet (i.e. the place closest to the assessment area, such as parks, hospitals, schools, and bus stops); the name of the outlet, if applicable; address; structural features (i.e. whether the establishment had a roof, seating or tables for customers, the availability of lighting and the type of fuel used for cooking, such as gas, electricity, charcoal, or firewood); categories of products offered (i.e. foods, beverages or a mix with non-food products); types of preparation; (i.e. homemade or industrialised ready-to-eat, fully or partially prepared on-site); hygiene practices of vendors; and outlet hygiene (i.e. presence of a garbage can and gel-type sanitiser).
The second section estimates the availability of healthy and unhealthy groups and the nutritional quality of the SFO expressed as an NH index. Nine groups were classified as recommended or healthy (i.e. daily consumption is unrelated to chronic diseases and the ingredients are environmentally sustainable; preparation method: plain or natural, if cooked, without deep frying; ingredients: foods that have one or more than two ingredients; without alcohol; red or processed meat; sugar or artificial sweeteners), and nine were classified as not recommended for daily consumption or unhealthy (i.e. daily consumption of these food groups is associated with an increased risk of overweight and obesity, as well as other chronic diseases, and the ingredients can limit the environment’s sustainability; preparation method: can include deep frying; ingredients: foods that have two or more ingredients; with added sugar or artificial sweeteners; red or processed meat; alcohol). The recommended group included the following: 1.-natural/sparkling water; 2.-natural fruit and vegetable juices; 3.-sugar-free milk, coffee, tea and other beverages; 4.-preparations without red or processed meat; 5.-fruit; 6.-vegetables; 7.-cereals with or without dairy products; 8.-non-fried-snacks; 9.-plain yoghurt. The not-recommended group included 1.-ultra-processed beverages with added sugar; 2.-homemade beverages with added sugar; 3.-alcoholic beverages; 4.-preparations with red or processed meat, 5.-deep-fried dishes, 6.-sweets and desserts, 7.-sweet bread and pastries; 8.-fried salty snacks and other ultra-processed foods; 9.-fruit-flavoured yoghurts. Table 1 describes some examples and definitions of the groups mentioned above. The complete SFBT is available in English and Spanish in see online supplementary material, Supplemental File S2.
Classification and description of food and beverage groups recommended and not recommended for daily consumption
Content validity
Based on the external experts’ comments, an average value of 0·99 was obtained for the SFBT’s adequacy and pertinence. The experts’ comments are described in Table 2.
Content validity results: comments by experts and changes made
* CVI: Content Validity Index.
Construct validity
The analysis was conducted on a sample of 1066 SFO with complete data on SDI and resident distribution per block. As shown in Table 3, there is a higher percentage of none-NH (0 % of NH) in low-SDI areas (45·9 %, 95 % CI 41·7, 50·2 %) compared to high-SDI areas (30·9 %, 95 % CI 25·8, 36·6 %). On the other hand, the percentage of low-NH (>0 % to <51 % of NH) was significantly lower in low-SDI areas (42·0 %, 95 % CI 37·9, 46·2 %) compared to high-SDI areas (56·6 %, 95 % CI 50·8, 62·3 %). These findings suggest that the observed differences across SDI categories were notable in the none and low-NH, while high-NH (≥51 % of NH) remained consistent across all SDI categories.
Multinomial regression analysis for categories of the nutritional healthfulness index by social development index categories
Bold values: P < 0.05.
SDI: Social development Index.
NH index: None 0 %, Low >0 % to <51 %, High ≥51 %.
RRR: relative risk ratio.
* Adjusted model by the total of street food outlets and the total of residents per buffer (continuous variable).
After adjusting for covariates, the multinomial regression model showed that SFO in high-SDI areas were more likely to be classified as having low-NH compared to none-NH, relative to those in low-SDI areas (relative risk ratio = 1·98, 95 % CI 1·43, 2·74, P < 0·000).
Inter-rater and test–retest reliability
Table 4 shows the inter-rater and test–retest reliability of the presence of recommended and not-recommended food and beverage groups for daily consumption at the 80 studied SFO. Based on inter-rater reliability results, strong to almost perfect kappa values (0·83–0·96) were achieved in six groups (natural/sparkling water; fruit; ultra-processed beverages with added sugar; homemade beverages with sugar; preparations with red or processed meat; sweets and desserts). Ten groups achieved moderate kappa values (0·62–0·78) for week 1, and two groups (preparations with red or processed meat and fruit-flavoured yoghurts) remained with moderate kappa values (0·72–0·75) in week 2. Although not shown, a positive correlation value ranging from 0·77 to 0·91 was observed for the NH index in both raters.
Inter-rater and test–retest reliability of the presence of recommended and not-recommended food and beverage groups for daily consumption
Average between rater 1 and rater 2, week 1.
* Percentage of present groups at street food outlets.
n 80 street food outlets.
On the other hand, test–retest results showed that 13 groups achieved moderate to almost perfect kappa values (0·62–0·91) for both raters. Two groups (vegetables and sugar-free milk, coffee, tea and other local beverages) obtained weak kappa values (0·51–0·55), and the fruit-flavoured yoghurts group showed weak to perfect kappa values (0·49–1). A positive correlation value ranging from 0·73 to 0·95 was observed for the NH index. Finally, the plain yoghurt group exhibited zero agreement in both inter-rater and test–retest reliability measurements, and the alcoholic beverages group was not observed during the evaluation.
Discussion
Results from this study showed a 15-item and 6-sub-item tool designed to estimate an NH index of SFO. The tool demonstrated a satisfactory content validity (0·99), and construct validity results indicated a greater availability of healthy foods and beverage groups in higher-SDI areas. The NH index showed a positive correlation.
Content validity
Based on previous studies, there is a lack of tools that estimate content validity(Reference Medina, Piña-Pozas and Aburto25). AuditNOVA(Reference Borges and Jaime30) reported slightly lower content validity (0·91) than SFBT (0·99). These differences could be attributed to the fact that AuditNOVA, in addition to the measurement of adequacy/pertinence, also evaluated the relevance and clarity of the questions(Reference Borges and Jaime30). Another reason could be the greater number of items in AuditNOVA since it evaluates both the informal and formal environments, which could reduce content validity.
Construct validity
This study confirmed the hypothesis, revealing a higher NH index in areas with higher SDI, consistent with national and international findings(Reference Pinheiro, Quintiliano-Scarpelli and Flores6–Reference Pérez-Ferrer, Auchincloss and de Menezes13). Previous studies, one in Chile(Reference Pinheiro, Quintiliano-Scarpelli and Flores6) and another in Mexico(Reference Bridle-Fitzpatrick8), have yielded similar findings regarding the healthfulness of SFO and income disparities around schools. Another study conducted in Mexico City, focusing exclusively on formal FE, reported that 30·5 % of food oases (defined as ‘territory with the best possible access to fresh food for a healthy diet’)(Reference González-Alejo, Frejomil and Rosales-Tapia9) were found in a low degree of marginalisation, 14·3 % in the moderate and 13·7 % in the high degree(Reference González-Alejo, Frejomil and Rosales-Tapia9). Finally, a study of SFO in Mexico City(Reference Rosales Chavez, Bruening and Ohri-Vachaspati18) aligns with the findings presented in this study, which reported that healthy products (e.g. fruit and vegetables) were found in middle-SDI areas, though the results were not statistically significant. In contrast, this study found significant differences, with a lower proportion of low-NH and a higher proportion of none-NH in low-SDI areas than SFO in high-SDI areas.
Inter-rater reliability
The results for the NH index (0·77–0·91) were similar to the healthy score reported by NEMS-S Brazil (0·98)(Reference Martins, Cremm and Leite24). Kappa values for food and beverage groups such as fruits, canned soda or ultra-processed beverages or sugary drinks, and bottled water in SFBT were also similar to NEMS-S Brazil(Reference Martins, Cremm and Leite24), AuditNOVA(Reference Borges and Jaime30), ESAO-S(Reference Duran, Lock and do Latorre11) and FROST(Reference Hosler and Dharssi29). However, moderate kappa values were found for sweet bread and pastries, fried salty snacks and other ultra-processed foods and vegetables, in comparison with the almost perfect or perfect kappa values of the NEMS-S Brazil(Reference Martins, Cremm and Leite24), ESAO-S(Reference Duran, Lock and do Latorre11) and FROST(Reference Hosler and Dharssi29).
The differences observed can be explained by the fact that these studies report formal and informal FE evaluations altogether. In the formal FE, the products and their ingredients are signposted and visible to consumers(Reference Downs, Ahmed and Fanzo1). The similarities in bottled water, ultra-processed beverages and fruits may be due to these products being visible to raters. In contrast, some products within the SFO may not be fully visible. This is because street products are sometimes partially or fully covered by vendors against sun exposure(Reference Long-Solís5), insects, animals or due to lack of space. Additionally, SFO pose a distinct challenge as they frequently prepare dishes on the spot, hindering precise observation and measurement of the complete range of products being offered(Reference Downs, Ahmed and Fanzo1,Reference Fellows and Hilmi3,Reference Organization46) . Consequently, this may lead to the introduction of random measurement errors. Another distinction observed was that most of the tools described earlier consist of an extensive list of food and beverages, compared to the SFBT, except for the AuditNOVA(Reference Borges and Jaime30) a tool that categorises food groups based on NOVA classification(Reference Monteiro, Cannon and Levy38), like what is done in some groups of the SFBT.
When comparing results from the SFBT with those obtained from other tools used to evaluate only SFO in the United States(Reference Lucan, Maroko and Sanon31,Reference Lucan, Maroko and Patel32) , Lucan 2022(Reference Lucan, Maroko and Patel32) and 2015(Reference Lucan, Maroko and Sanon31) reported an ‘exceptionally high’ and ‘complete’ inter-rater reliability, respectively. Reasons for the differences could be the way tools are used to measure food and beverage availability. For instance, Lucan 2022 uses three food and beverage groups: healthful, less-healthful and neither healthful nor less-healthful. On the other hand, Lucan 2015 has only two groups: produce and non-produce products. In contrast, the SFBT encompasses 18 food and beverage groups. When comparing the SFBT with another tool used in the Mexican context (SFSAT)(Reference Chávez, Jehn and Lee33), similar kappa values for yoghurt, fruits, soft drinks, flavoured water and hard candy were reported. However, differences were observed in cookies and pastries and cooked meals. Variations in inter-rater reliability may be due to differences in the number of items and product classification (e.g. cookies and pastries, gorditas de nata and pan dulce, among others, are considered in the sweet bread and pastries group in the SFBT, while in the SFSAT these products are reported separately in the Snacks group). Furthermore, the differences between SFBT and SFSAT can be observed in group classification (e.g. 10 groups v. 18 groups for the SFBT) and the detailed observation of food preparation (e.g. deep-fried) and ingredients (e.g. red meat) required for the NH index construction of the SFBT v. the no requirement of detailed observation for the SFSAT(Reference Chávez, Jehn and Lee33).
Test–retest reliability
The NH index showed a positive correlation ranging from 0·73 to 0·95, similar to those values reported by NEMS-S Brazil (0·98)(Reference Martins, Cremm and Leite24). Additionally, similar kappa values for the availability of water, fruits and vegetable groups were found when compared to AuditNOVA(Reference Borges and Jaime30). However, almost perfect or perfect kappa values were observed in the total score of NEMS-S Brazil(Reference Martins, Cremm and Leite24) as well as in the availability of fruits, vegetables and canned soda groups for NEMS-S Brazil(Reference Martins, Cremm and Leite24), ESAO-S(Reference Duran, Lock and do Latorre11) and AuditNOVA(Reference Borges and Jaime30) compared to the weak to moderate kappa values obtained in SFBT. Differences in the kappa values or the correlation among the tools(Reference Duran, Lock and do Latorre11,Reference Martins, Cremm and Leite24,Reference Borges and Jaime30) could be because the tools mentioned above can measure informal and formal settings. In other words, these tools were used to measure SFO, as well as supermarkets, convenience stores and restaurants. The measurement in the formal environment could produce higher kappa values since products are visually located, the aisles are named, and the products are arranged. For restaurants, standardised menus are publicly available.
The research on the healthfulness of informal FE has received comparatively less attention than formal FE, primarily due to the methodological complexities associated with assessing some of these outlets, specifically street food establishments(Reference Lucan, Varona and Maroko47), and the absence of tools specifically designed for informal FE evaluation(Reference Medina, Piña-Pozas and Aburto25,Reference Akhter and Cameron26) . Zerafati-Shoae et al. note that despite more than two decades of formal FE assessments, there is still a lack of standardised methodology for designing and evaluating the reliability and validity of these tools(Reference Zerafati-Shoae, Taghdisi and Azadbakht27). Consequently, information on informal FE is considerably more limited, given that the prevalence of SFO is lower in high-income countries, where most of these assessment tools have been developed(Reference Downs, Ahmed and Fanzo1).
Therefore, one of the limitations is that the criterion validity of the SFBT was not estimated(Reference Lytle28), due to the absence of gold standard measurements(Reference Medina, Piña-Pozas and Aburto25,Reference Akhter and Cameron26) . On the other hand, the inter-rater reliability values highlighted the need to review items with null kappa values (i.e. the plain and flavoured yoghurt) and provide more comprehensive training for evaluators. However, the variations could also be attributed to the challenges presented in the SFO. Another study conducted in the Mexican context reported the same values for yoghurt inter-rater reliability(Reference Chávez, Jehn and Lee33). The availability of street foods could vary during the evaluation because restocking or demand fluctuations could not be halted due to the evaluation process, even though the evaluators were simultaneously in the same SFO(Reference Chávez, Jehn and Lee33). Therefore, it was possible that while one rater had the product available to answer the questionnaire, the vendor might have sold out of the product, preventing another rater from observing it. Additionally, as mentioned before, some products within the SFO may not be fully visible. Finally, this tool requires evaluators to observe products consistently, especially in the case of prepared meals and street foods that vary widely in their ingredients, preparation methods and consumption. They can range from simple snacks with one or two ingredients to complex meals with multiple components requiring separate processing(Reference Fellows and Hilmi3).
A strength of this tool is the NH index construction. It was based on places where the tool was applied, on input from various stakeholders, including nutritionists (raters and internal and external groups), and on recent studies or recommendations promoting healthy and sustainable diets(Reference Gaona-Pineda, Martínez-Tapia and Arango-Angarita36–Reference Monteiro, Cannon and Levy38). Furthermore, the proposed foods and beverages groups, used to estimate the NH index in each SFO, consider the products sold, their ingredients and the preparation method. They were sufficiently general and held potential applicability in other countries, contexts and settings (schools, corporate areas and hospitals), particularly in those places where SFO are prevalent(Reference Downs, Ahmed and Fanzo1). This tool could engage local community members in discussions about their FE to collaborate with decision-makers to consider possible interventions and establish healthier environments, thereby improving the well-being of their populations. More research is needed to investigate the effect of these spaces on human health outcomes. This can be translated to better policies that create an impact. Finally, the methodology employed for assessing the reliability and validity of the SFBT is based on the latest findings in the scientific literature(Reference Medina, Piña-Pozas and Aburto25–Reference Lytle28).
Conclusion
This study demonstrated that the SFBT is reliable and valid for estimating the availability of healthy and unhealthy groups and the nutritional quality of the SFO expressed as an NH index. Evaluating these spaces in future studies will provide insights into potential healthy or less healthy informal FE, leading to interventions aimed at improving the availability and accessibility of recommended daily consumption foods and beverages in SFO.
Supplementary material
For supplementary material accompanying this paper visit https://doi.org/10.1017/S1368980024002581
Acknowledgements
We would like to acknowledge the contribution of Yenisei Ramírez Toscano, Ana Gabriela Ortega Ávila, Nancy López-Olmedo, Monica Mazariegos and Sofía Rincon Gallardo Patino for being the external technical group who reviewed the tool´s items. The contributions of Ana Laura González for compiling the spatial measurement data, Mildred Moreno Villanueva for creating the SFO definitions, and Ana María Martínez Cano for reviewing the grammar of the first version of the manuscript. Finally, we extend our recognition to Irving Contreras and Magdiel Barrios the personnel who carried out surveys.
Financial support
This study was supported by Bloomberg Philanthropies (project number 43 003) and the CONACYT-PRONACE-SALUD study (project number 2256, CI:1767). The funders had no role in the study design, data collection, and analysis, decision to publish, or preparation of the manuscript.
Competing interests
There are no conflicts of interest.
Authorship
Conceptualisation U.L.,C.M.; Data curation; C.G., U.L. Formal analysis U.L.,C.M., T.A., C.G.; Funding acquisition C.M.,S.B.; Investigation U.L.; Methodology U.L., C.M. and M.R.; Project administration C.M.; Resources S.B.; Software U.L.; Supervision C.M. and T.A.; Validation C.M. and T.A.; Visualisation C.M. and S.B.; Witting-Original draft U.L., C.M., T.A.; Writing – Review and editing C.G., L.H., C.N., J.C.H., V.B., A.O., M.R., S.B., C.M., T.A., U.L. The author(s) read and approved the final manuscript.
Ethics of human subject participation
This study was conducted according to the guidelines laid down in the Declaration of Helsinki, and all procedures involving research study participants were approved by the Instituto Nacional de Salud Publica’s institutional review board. Verbal informed consent was obtained from all subjects. Verbal consent was witnessed and formally recorded.
Open access
