As stated in the foreword⁽Reference Willett¹⁾ to articles by Henriquez-Sanchez et al. ⁽Reference Henriquez-Sanchez, Sanchez-Villegas and Doreste-Alonso²⁾ and others, this collection presents valuable work that identifies quality features of dietary validation studies and supports the use of FFQ in future research. The authors' substantial effort and results are appreciated; however, inconsistencies in classifying studies as including supplements, assigning quality scores and presenting complete validity data may have limited their findings. The paper by Henriquez-Sanchez et al. ⁽Reference Henriquez-Sanchez, Sanchez-Villegas and Doreste-Alonso²⁾ will be primarily examined, but the comments may also apply to others in the collection.

Henriquez-Sanchez et al. ⁽Reference Henriquez-Sanchez, Sanchez-Villegas and Doreste-Alonso²⁾ report that 23–31 % of validation studies included dietary supplements in their analyses, but there were no large differences in correlation coefficients between studies with or without dietary supplements. This finding is surprising, given reports that the inclusion of supplements improves intake assessment and validity correlations⁽Reference Boucher, Cotterchio and Kreiger^3, Reference Messerer, Johansson and Wolk⁴⁾. Although Henriquez-Sanchez et al. observed that correlations improve when supplement data from both FFQ and reference methods are included, the inconsistent use of this criterion when reviewing the studies may account for the results just mentioned. Individual studies may describe FFQ as listing supplements, but it cannot be assumed that these data were collected in the reference methods or included in the validity analyses, unless explicitly stated by the original authors. Further, it is erroneous to report a validation study as including supplements if these data were not specifically collected in both dietary methods and also used in the analysis. Four studies in the review of Henriquez-Sanchez et al. serve to illustrate this point. Flagg et al. ⁽Reference Flagg, Coates and Calle⁵⁾ describe their FFQ as including supplements, but do not specify collecting these data in recalls, and even state that they were excluded from validity correlations. Similarly, Jain et al. ⁽Reference Jain, Howe and Rohan⁶⁾ describe both the FFQ and records as including supplements, but state that these were not included in their analysis. The study that was assigned the highest quality score⁽Reference Sudha, Radhika and Sathya⁷⁾ describes its FFQ as including supplements, but does not explicitly state that these were included in the recalls or analysis. Although none of the three studies describes supplements in their analyses, all were evaluated by Henriquez-Sanchez et al. as including supplements. In contrast, another study reports including supplements in the FFQ, recalls and validity analysis⁽Reference Boucher, Cotterchio and Kreiger³⁾, yet was classified as excluding these data.

Inconsistent assignment of study quality scores by Henriquez-Sanchez et al. and by the research group evaluating minerals⁽Reference Serra-Majem, Pfrimer and Doreste-Alonso⁸⁾ was also evident, despite having similar group members, study objectives and the same evaluation criteria. When the list of studies using recalls or records for vitamins A, C, D and E⁽Reference Henriquez-Sanchez, Sanchez-Villegas and Doreste-Alonso²⁾ was compared to those for minerals⁽Reference Serra-Majem, Pfrimer and Doreste-Alonso⁸⁾, eighteen of the same studies were differently scored by the two research groups, many by at least one point⁽Reference Riboli, Elmstahl and Saracci^9–Reference Fornes, Stringhini and Elias¹¹⁾. Although seemingly small, these differences were large enough to shift most studies into the next category of quality, described by Serra-Majem et al. ⁽Reference Serra-Majem, Andersen and Henriquez-Sanchez¹²⁾. A quality score to weight study findings seems appropriate⁽Reference Willett¹⁾, but to be fully useful, guidelines for its derivation must be clear and applied consistently and with equal rigour across studies. Shortcomings in this process might explain the inconsistent scoring between groups.

Finally, the introductory paper in this collection reports that the aim was to review all published validation studies⁽Reference Serra-Majem, Andersen and Henriquez-Sanchez¹²⁾. However ambitious, some validation studies⁽Reference Subar, Thompson and Kipnis^13–Reference Wirfalt, Jeffery and Elmer¹⁶⁾ seem to have been missed or excluded from the reviews⁽Reference Henriquez-Sanchez, Sanchez-Villegas and Doreste-Alonso^2, Reference Serra-Majem, Pfrimer and Doreste-Alonso⁸⁾. There is also incomplete reporting of nutrients from included studies. Relevant nutrients were sometimes not reported, or were included only in one review, when both vitamins and minerals were listed in the original studies. Examples include Osowski et al. ⁽Reference Osowski, Beare and Specker¹⁷⁾ (included in mineral review only, but original study also reported vitamin D) and Boucher et al. ⁽Reference Boucher, Cotterchio and Kreiger³⁾ (included for vitamin C, but original study also reported vitamins A, D and E; not included in mineral review, but original study reported Fe, Ca, Se and Zn). Although data in the reviews were adequate for certain evaluations, missing data from excluded studies or incomplete reporting would have strengthened the findings overall and might have meaningfully contributed to those areas where the data are limited. Validity data were reported as being particularly sparse for vitamins B₆, B₁₂, niacin and vitamin D⁽Reference Henriquez-Sanchez, Sanchez-Villegas and Doreste-Alonso²⁾, as well as for Se, Zn and iodine⁽Reference Serra-Majem, Pfrimer and Doreste-Alonso⁸⁾, in addition to the general lack of studies including supplements. Most studies just cited as missing or incompletely reported, included data for at least one of these limited nutrients – two reported supplements in their analyses⁽Reference Boucher, Cotterchio and Kreiger^3, Reference Mares-Perlman, Klein and Klein¹⁵⁾.

In conclusion, although examples of inconsistencies in the article by Henriquez-Sanchez et al. have been identified, the full extent of these problems and their effects on quality scores, study weighting and findings cannot be determined without a more complete review of all studies. This may be a worthwhile exercise for the authors to consider undertaking given the important impact of their work on future research, including guiding decisions about supplement data collection, analysis and reporting in validation studies.

The author has no conflict of interest. There was no funding associated with this letter.