Validation of semiquantitative FFQ administered to adults: a systematic review

Objective: To conduct a systematic review of studies for the validation of semiquantitative FFQ (SFFQ) that assess food intake in adults. Design: The authors conducted a systematic search in PubMed for articles published as late as January 2020 in Spanish, English, French and Portuguese. Individual searches (twelve in total) paired three hyphenated and non-hyphenated variations of ‘semiquantitative food frequency questionnaire’ with both ‘validity’ and ‘validation’ using the ‘all fields’ and the ‘title/abstract’ retrieval categories. Independent extraction of articles was performed by four authors using predefined data fields. Setting: We searched for original SFFQ validation studies that analysed general diet composition (nutrients with or without food groups or energy analysis) in healthy adults, in any setting, and that also reported correlation coefficients. Participants: Healthy adults. Results: Sixty articles were included. The preferred comparison standard for validation was food records (n 37). The main correlation coefficients used were Pearson’s (n 41), and validity coefficients varied from −0·45 to 1. Most correlation coefficients were adjusted by energy (twelve studies presented only crude values). The elements mentioned most frequently were energy, macronutrients, cholesterol, SFA, PUFA, fibre, vitamin C, Ca and Fe. Conclusions: Although all these SFFQ are reported as validated, coefficients may vary across groups of foods and nutrients. Based on our findings, we suggest researchers to consult our revision before choosing a SFFQ and to review important issues about them, such as their validation, number of items, number of participants, etc. Systematic Review Registration: PROSPERO number CRD42017064716. Available at: http://www.crd.york.ac.uk/PROSPERO/display_record.asp?ID=CRD42017064716.

SFFQ is considered to be a relatively inexpensive, quick and easy-to-implement method that can provide in-depth insights into food and nutrient intake and dietary patterns (1,4) .
Unlike qualitative FFQ, SFFQ include specific portion sizes in their questions or items on food intake frequency (3) and they also require weighted responses regarding these portions. Their overall aim is to obtain estimates of nutrient intake, which may help to identify dietary deficiencies or excesses.
However, because a SFFQ that has been developed in a particular society and culture may not be applicable elsewhere, first it must be validated in the population for which it has been designed (1,5) , given that validity is a continuous variable which may range from no validity to very high (3) . Besides, because SFFQ responses are based on memories of previous eating habits, administering them to children and the elderly may be particularly problematic. Hence, they are generally and should preferably be administered to healthy adults when validating the instrument for use on the general population (5) .
Because of the considerable variety of available SFFQ, it may be challenging to select the best one for a context. Thus, this systematic review of SFFQ validation studies to assess food intake in adults is intended to serve as an up-to-date reference tool that will help researchers to validate SFFQ or choose those best suited to the needs of whatever specific studies they wish to conduct on particular populations.

Methods
This study is a systematic review that followed criteria for the search and selection of articles stipulated by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement (6) . Also, the protocol for this study was registered in the International Prospective Register of Systematic Reviews (PROSPERO) with registration number CRD42017064716.

Literature search
Two authors (E.S.-R. and M.F.B.-O.) performed a systematic search in the PubMed bibliographic index for articles in four languages. Individual searches (twelve in total) paired all possible combinations of the multi-word terms 'semiquantitative food frequency questionnaire', 'semiquantitative food frequency questionnaire' and 'semiquantitative food-frequency questionnaire' with 'validity' and 'validation'. Each of these searches was performed using both the 'all fields' and the 'title/abstract' retrieval categories. Results included articles published up to the date of the last search (31 January 2020).

Study selection
Articles were chosen based on six inclusion criteria: (1) be original SFFQ validation articles; (2) analyse nutrient intake, with or without energy or food groups analysis, and without an exclusive focus on specific nutrients (e.g. folic acid or Fe); (3) include healthy adults with no particular nutritional needs (pregnant women, athletes, etc.); (4) include individuals without cognitive impairment (when elderly subjects are studied); (5) include analyses with correlation coefficients (Spearman's, Pearson's and intraclass); (6) be published in English, Spanish, French or Portuguese. Exclusion criteria were studies that (1) validate SFFQ designed for specific diseases (e.g. cancer) or conditions (e.g. post-myocardial infarction); (2) report only food group analyses; (3) describe data only in tertiles, quartiles or quintiles without reporting correlation coefficients and (4) include data reported in a previous paper.
Abstracts obtained using these search criteria were assessed for eligibility according to inclusion criteria and retrieved through the University of Guadalajara's virtual library and databases, open access links or Google Scholar. If a full-text paper could not be retrieved through these means, it was obtained by contacting the paper's corresponding author or through the payment of applicable fees. Related articles identified when searching fulltext papers were also retrieved, after confirming their adherence to the inclusion criteria and their presence in PubMed. The full text of these studies was also assessed for eligibility. Any doubts about the eligibility of studies were resolved through discussions with a third author (B.V.). Table 1   For each SFFQ, the following characteristics were analysed: author and year of publication, country in which the questionnaire was developed, geographical region, number and sex of participants, minimum and maximum ages or age range (difference between the extremes), number of items, number of response categories, visual support to identify portion sizes and how SFFQ were administered (by interview or self-administration). For the analysis of data used in the validation, the following was recorded: validation methods (records and recalls) and the number of times they used; units of analysis (energy, nutrients and food groups) with the lowest and highest correlation coefficient values that were produced (Pearson's, Spearman's, intraclass and Rosner's); and the specific values for energy, carbohydrates, proteins, fat and nutrients that were mentioned most frequently across studies (Ca, Fe, etc.). To determine which nutrients were most frequently reported, a matrix was created in Excel into which the nutrients reported in the studies were captured and where the seven most mentioned nutrients were highlighted. Regarding reproducibility, the intervals between questionnaire administrations and correlation coefficients (Pearson's, Spearman's, intraclass and Rosner's) were identified.

Quality assessment
Risk of bias analysis was conducted applying elements of three tools: Newcastle-Ottawa scale for cohort studies (7) , the Cochrane evaluation tool (8) and some aspects considered in the Strobe statement (9) . The Newcastle-Ottawa scale was used to assess: the representativeness of the sample (not volunteers); the assurance of exposure (data obtained from the instruments used to validate the SFFQ were mainly through reminders or records, not selfreporting); whether the analyses were controlled by at least one additional variable (e.g. energy); the method used to assess the results of the SFFQ (preferably by interview); the duration of the follow-up to validate the tool (ideally 6-12 months) and whether it was representative of the habitual diet; and the number of dropouts during the follow-up phase (ideally <30 % of the population) with explanations of the reasons. Cochrane's tool was used to assess the risk of notification bias (considered as low when the results were consistent with those described in the methodology section). Finally, based on the Strobe Statement, some aspects of the methodology (description of the location, dates, recruitment periods, and eligibility criteria; a detailed description of the SFFQ application method; the way in which the sample size was determined; a description of how variables were addressed in the analysis) and the results (description of participant characteristics and confounding variables and reports of other analyses such as de-attenuation) were evaluated. A total of fourteen factors related to the risk of bias were assessed in each of the studies.

Studies' and participants' characteristics
The study with the fewest participants was that by Nath and Huffman (38) , which was conducted on twenty Cuban immigrants to the USA. The largest sample, found in a study from France, consisted of 1863 participants (29) . No justification was found for the sample sizes used in the validations.

Quality of the studies
Most studies met at least half of the risk of bias and quality criteria that were assessed, and nine studies met less than half of the established criteria. The studies that met most of the criteria were Klipstein et al. (14) , Chen et al. (54) , Ocké et al. (20) , Gunes et al. (70) and Whitton et al. (62) , in which twelve of the fourteen criteria were met.
It is important to notice that the nutrients mentioned most frequently across studies, in addition to energy and macronutrients, were cholesterol, SFA, PUFA, fibre, vitamin C, Ca and Fe. PUFA had the lowest coefficient, which was −0·10 (55) (Pearson's, crude), while vitamin C showed the highest coefficient value of 0·98 (63) (Pearson's, or Spearman's, crude). These data are detailed in Supplemental Table 2 in the online supplementary material.

Discussion
The main objective of this review is to provide detailed data on the validation of SFFQ. A total of sixty SFFQ that met the selection criteria were found from several geographical regions. Europe is the region with the highest number of published studies (n 25) (5,12, , while the number of studies from Latin America (37,40,45,47,52) has increased significantly in recent years. These data may indicate that the number of epidemiological assessments of diet in Latin America has probably increased recently since food-related health problems have become more prevalent in that region.
More than half of the studies (81·1 %) included male and female subjects. Some authors had specific reasons for selecting subjects of only one sex. For example, Willett et al. (42) selected women because their study targeted female nurses; in the case of SFFQ administered only to men (18,39,46) , no reasons were given for this exclusive selection, with the exception of the study on male health professionals (39) .
Regarding age, the broadest age ranges were reported in the studies by Gazan et al. (29) and Shatenstein et al. (41) . The latter had the broadest range, with 64 years of difference between the lowest and highest values. The existence of such a broad age range in the SFFQ validation process supports the administration of this questionnaire to older adults and younger populations. Notice that it was decided to include articles with minimum ages below 18 years because the mean age was not influenced by these values (22,54,68) .
The Pearson's (r) and Spearman's (ρ) correlation coefficients measure the degree of linear association between two variables, the former being more suitable for normal distribution and the latter for non-normal distribution. However, neither of these coefficients provides information about the degree of agreement observed, nor about the presence of systematic differences between measurements or instruments (71)(72)(73)(74) . The intraclass correlation coefficient is usually used for this purpose in reproducibility studies (72,74,75) . However, it has been proposed as an alternative method to evaluate the agreement observed between methods in validation studies (32,52,76) . Taking these observations into consideration, it is suggested that validation studies include, in addition to the mean values of each method, the Pearson's or Spearman's correlation coefficients and, if desired, the intraclass correlation coefficient.
Based on the analysis of previous proposals (3,76,80,81) and our observations, and taking into consideration that the essential reason to validate an instrument is to confirm that it evaluates variables adequately, we recommend taking into consideration specific elements to validate SFFQ that were designed to evaluate global food intake in adults, such as the number of items; the number and sex of participants for validation; administration of the SFFQ (interview or self-reporting); seasonal fluctuations in dietary intake; visual support material for use during questionnaire administration; number of applications of the reference method (dietary records or 24-h recalls); unit of analysis: energy, nutrients and food groups; the use of unadjusted, adjusted and de-attenuated correlation coefficients for the validity analysis (depending on whether the distribution of variables is normal); the Bland-Altman plots; a reproducibility analysis between questionnaire administrations; identification of the statistical package used to perform statistical analyses; and identification of the software used to perform the nutritional analysis.
Finally, we note that when SFFQ validation studies are conducted, validity results are not always favourable for all nutrients or food groups evaluated. Ideally, improvements should be made to these SFFQ, after which they should be revalidated. At least, the limitations of these instruments should be acknowledged, and their results should be interpreted with caution. However, we believe that following the recommendations regarding the limitations of other studies and those we have discussed here will lead to better validation results.
Among the strengths of our study are the inclusion of questionnaires produced in several geographical regions and continents, and that its search period was not limited. Hence, our results include papers that are among the oldest available in the database up to those published as late as January 2020. Furthermore, we have included articles written in English, Portuguese, French and Spanish that were found using systematic search processes. We also extracted data directly from tables to prevent data omission. The scoring system we used is intended to help researchers select those SFFQ that would be the most complete and suitable for the objectives and target populations of their studies.
However, our review also has limitations. One of these is that PubMed was the only search engine used, thus resulting in the exclusion of relevant papers not retrievable through it. We nonetheless decided to only include articles found in PubMed because it is a proven source of articles from highly regarded scientific journals. Also, it may be questionable to have included studies validated only against dietary assessment tools and not studies focused on nutrients for which there might be unbiased biomarkers such as urinary potassium, urinary nitrogen (proteins) and doubly labelled water (energy). However, biomarkers have limitations, which include the fact that recovery markers (those that refer to a measure of absolute intake per 24 h) are not available for most nutrients and may, therefore, provide limited information. Therefore, we consider that biomarkers may be more suitable in studies focused on specific nutrients and not on the general diet, as in our study.
As future perspectives, the creation and validation of new food-frequency consumption questionnaires are justified, since we are living in the personalised nutrition era. Having specific tools for diverse population groups and diverse purposes will support research and application of new knowledge. Besides, it is desirable that these questionnaires not only focus on assessing food but also processed or prepared products, to not limiting existing evidence between food/nutrition and health risks.

Conclusions
The characteristics of and validation processes for different SFFQ can vary substantially, even within individual countries. Therefore, the composite components of SFFQ should be carefully reviewed when being selected. Having described the parameters of and results from different validations, we conclude that even in cases where all SFFQ are reported as validated, their coefficients may vary. The results of this analysis show that even in cases where correlation coefficients range from poor to excellent, validation may still be feasible provided that overall results are interpreted with caution.