Fish consumption has been suggested as a protective factor for the development of various cancers; however, results from studies investigating this association have been inconsistent, ranging from statistically significant associationsReference Kato, Akhmedkhanov, Koenig, Toniolo, Shore and Riboli1, Reference Terry, Lichtenstein, Feychting, Ahlbom and Wolk2 to no clear association at allReference English, MacInnis, Hodge, Hopper, Haydon and Giles3, Reference Nothlings, Wilkens, Murphy, Hankin, Henderson and Kolonel4. Possible reasons for this could be either a lack of detail collected by questionnaires used to measure fish consumption and therefore the failure to accurately capture the fish consumption of study participants, or a tendency for investigators to use a single measure to represent overall fish intake in statistical modelling despite having collected detailed information on the various types of fish and seafood consumed. However, there is no published evidence that the distinction among types and styles of fish items is important if ultimately the research question being asked is whether fish, as a food group, has a protective effect on the development of disease.
We recently examined data collected as part of a large population-based case–control study for an association between fresh and preserved fish consumption and prostate cancer (Mina K, Fritschi L, Johnson KC, The Canadian Cancer Registries Epidemiology Research Group, 2007, unpublished). Information on fish intake was collected by means of two questions about frequency of consumption of a given amount of fresh/frozen/canned fish (fresh fish) and salted/smoked/dried fish (preserved fish). The purpose of the current paper is to compare the estimates of absolute intake of fish from these two ‘summary’ questions regarding fish consumption with estimates from a multi-item, semi-quantitative food-frequency questionnaire (FFQ) from which aggregate measures of fresh and preserved fish consumption were generated, and to determine if these measures are comparable in terms of validity (using an independent biomarker) and retest reliability.
Methods
Recruitment
Data collected on 109 participants as part of an Australian-based study to validate a thorough FFQ on fish consumption were used5. Healthy (defined as no history of heart disease, cancer (not including skin cancer), severe inflammatory diseases, emphysema or asthma, diabetes, severe gastrointestinal disease or mental illness) volunteers aged 21–75 years from the Perth metropolitan area were recruited to complete the developed FFQ and provide a fasting blood sample for analysis of omega-3 polyunsaturated fatty acids (PUFA), between March and November 2005. Recruitment was facilitated by a short advertisement circulated by local fish markets, radio broadcasts and websites, state-wide and local newspapers, and staff and student mailing lists at The University of Western Australia. Potential participants (n = 175) were screened by telephone or email to ensure that they met health and age requirements and, if eligible, were posted a study pack that included the fish consumption FFQ and blood collection request form.
All participants recruited prior to 31 July (n = 107) were re-sent the FFQ three months after completing the original. Seventy-one participants returned a completed second FFQ.
FFQ design
The content of the new FFQ was based on an English translation of an existing section of a validated dietary FFQ on fish consumption from NorwayReference Hjartaker, Lund and Bjerve6. This FFQ was chosen as a template because it contained detailed questioning on fish and seafood consumption and had been previously tested for validity. Due to differences in food availability and the design of the current study, adjustments to formatting, content and participant instructions were made based on consultation with nutrition, omega-3 PUFA and fishing industry experts, an informal assessment of the availability of items on local supermarket shelves and the results of a small pilot study, in order to develop a locally appropriate FFQ.
The final new FFQ consisted of three sections (demographic and health information, fish consumption, changes in consumption over time). The section on fish consumption consisted of multiple tables covering 71 individual food items, grouped as fresh/frozen fish (20 species), processed fish and seafood (18 items including canned, salted, smoked and dried fish, fish spreads and pre-prepared meals), fresh seafood (seven types of molluscs and crustaceans), omega-3-fortified foods (eggs, milk, bread, margarine) and omega-3 supplements. Information on both frequency of consumption and portion size was collected for each food item; however, the method of collecting information on portion size varied. For fresh and frozen fillets, a single question was included, supported by photographs of fish portions, asking participants to estimate their usual serving size. For other food items, participants were asked to indicate whether they ate a small, medium or large serving. The medium serving size was defined in words (for example, half a cup); a small serving size was defined as half or less of a medium size, and a large serving was one-and-a-half or more of a medium serving size. Frequency of consumption was indicated by choice of one of nine categories ranging from never to two or more servings per day.
The new FFQ also contained two summary questions (replicated from the Canadian case–control study FFQReference Johnson, Mao and Argo7) regarding fresh fish and preserved fish intake. Due to the different origin of these questions, the style of the summary questions was different to that of the individual item questions. The summary questions asked how often a specific amount of the foods was eaten (100 g of fresh/frozen/canned fish and 50 g of smoked/salted/dried fish). The summary questions were included in the new FFQ for the purpose of testing their validity with an independent biomarker, and for this reason were not modified to match the style of the majority of questions in the new detailed FFQ. Participants were specifically requested not to refer to their answers in the more detailed sections of the FFQ when responding to the summary questions.
Participants were asked to respond to all questions regarding their fish consumption two months earlier, to coincide with the omega-3 fatty acid levels indicated by measuring erythrocyte membrane fatty acids (discussed below). The repeat FFQ asked about fish consumption five months earlier so that both FFQs referred to the same time period. To make this conceptually easier, participants were given a calendar month to refer to when completing the FFQ.
Blood processing and analysis
Erythrocyte membrane eicosapentaenoic acid (EPA) was chosen as the omega-3 biomarker used to validate the questions on fish consumption, because EPA is specific to the consumption of fish as a food groupReference Arab8 and has been demonstrated to be an appropriate biomarker for the validation of FFQs regarding fish consumptionReference Godley, Campbell, Miller, Gallagher, Martinson and Mohler9, Reference Dallongeville, Yarnell, Ducimetiere, Arveiler, Ferrieres and Montaye10. Specifically, erythrocyte membranes were chosen as the medium for measures of EPA because samples are relatively easy to obtain by phlebotomy (compared with adipose tissue sampling which requires biopsy), but thought to be less affected than plasma levels by recent consumptionReference Arab8.
Participants were instructed to have their blood sample taken within two weeks of completing the questionnaire to ensure that the diet recorded in the questionnaire was likely to reflect that indicated by omega-3 biomarkers. While the lifespan of an erythrocyte is 120 days, the time period of consumption that is reflected by erythrocyte EPA may be as recent as 1 month ago, although levels may take as long as 6 months to plateau after a change in omega-3 PUFA intakeReference Katan, Deslypere, van Birgelen, Penders and Zegwaard11. We therefore chose a period of 8–10 weeks to allow for this uncertainty. Fasting blood samples were taken by trained phlebotomists at metropolitan collection centres, transferred to a central laboratory, and processed according to a specified protocol. Samples were collected in 9 ml tubes containing ethylenediaminetetraacetic acid and centrifuged at 1500 g for 10 min at 4°C. The plasma fraction was removed and retained in separate tubes for analysis. Next, 4 ml of 0.9% saline was added to the erythrocyte fraction and inverted gently several times, then centrifuged again at 1500 g for 10 min at 4°C. The packed, washed erythrocyte fractions and the retained plasma fractions were then stored at −80°C until analysis.
Processed samples were stored until transferred to a separate laboratory for PUFA analysis. Plasma (0.5 ml) or erythrocytes were extracted with chloroform–methanol (2:1, 5 ml). Heptadecanoic acid (17:0) as internal standard was added to the total lipid extracts and fatty acid methyl esters were prepared by treatment with 4% H2SO4 in methanol at 90°C for 20 min. Samples were analysed by gas–liquid chromatography using a Hewlett-Packard model 5980A gas chromatograph. The column was a BPX70 (25 m × 0.32 mm, 0.25 μm film thickness; SGE) with a temperature programme from 150 to 210°C at 4°C min−1 and using N2 as the carrier gas at a split ratio of 30:1. Peaks were identified by comparison with a known standard mixture. Individual fatty acids were calculated either as a relative percentage with the evaluated fatty acids set at 100% or as absolute amounts based on the internal standard addedReference Mori, Burke, Puddey, Watts, O’Neal and Best12. Analysed samples were then returned to the central laboratory for storage.
Data entry and analysis
Data from the FFQs were entered as categorical variables into a Microsoft® Access database designed specifically for the project, and then converted to servings per week and grams per week. Missing frequency values were assumed to represent zero consumption and were coded accordingly. Missing values for serving sizes were replaced with the medium serving size (two participants), or with the mean serving size in the case of fresh/frozen fish fillets (five participants) because participants tended to choose a serving size larger than the medium option for fish fillets.
Blood sample analysis data were received as a Microsoft® Excel spreadsheet and transferred into the same Microsoft® Access database.
In order to compare the information collected by the summary questions with that collected by the more detailed questioning, aggregate variables were generated by summing either servings per week or grams per week variables representing equivalent fish items to those in the summary questions. When generating aggregate measures, fresh and canned fish items included all canned fish (not seafood) and fresh/frozen fillets of fish (44 items). In order to reproduce results as closely as possible to the summary question, other items from the new FFQ such as fish dishes, stew, fish fingers, raw fish and fish fillings were not included in the aggregate measure. Smoked, salted and dried fish items included hot and cold smoked salmon and dried or salted fish (three items). Smoked seafood was not included in the aggregate measure.
Validity was tested using linear regression analysis of erythrocyte membrane EPA percentage areas (relative percentage of fatty acids) and the summary and aggregate estimates. Fish intake measures, in grams per week, and EPA levels were assessed for skewness and then log-transformed (according to log[x + 1]) to account for positively skewed distributions. Regression models included variables for age, sex, smoking status (never or ever), alcohol consumption (grams per week) and body mass index (BMI) to adjust for individual variation in energy intake. For comparability with other studies using biomarkers as a validation tool, Spearman’s correlation coefficients were also calculated. Retest reliability was conducted using two-way, mixed-model (consistency) intra-class correlation coefficients (ICCs). SPSS version 14 (SPSS Inc.) and STATA version 9 (StataCorp) statistical packages were used.
Ethics approval for this study was obtained from the Human Research Ethics Committee of The University of Western Australia.
Results
Fifty-seven per cent of participants were women and the mean age was 51 years (standard deviation (SD) 15). Study participants were generally of European descent (78%), well-educated (79% with a tertiary education or higher) and were current non-smokers (97%).
Fresh fish was consumed by more participants (94% as per summary question) than preserved fish (38% as per summary question). Mean servings per week and grams per week from both the aggregate and summary measures demonstrated that fresh fish was also consumed more frequently and in greater amounts than preserved fish (Table 1). Both measures indicated that participants consumed higher amounts of fish than Australian per capita estimates13.
Mean (range) fresh and preserved fish consumption estimates (n = 109)Footnote †
† Non-log-transformed data.
The summary measure for fresh fish underestimated both the frequency of servings (by 35%) and grams per week (by 45%) given by the aggregate of individual items (Table 1). Conversely, the summary question for preserved fish overestimated both the frequency of servings and grams per week consumption suggested by the aggregate measure by 2.5 to three times. Inclusion of canned smoked salmon (smoked flavour) and canned smoked seafood into the aggregate measure did not alter the discrepancies observed (results not shown here).
Spearman’s correlation coefficients between aggregate and summary measures (Table 2) for both grams and servings of fish per week were very similar for preserved fish. The correlation for grams of fresh fish per week was slightly lower than for servings per week.
Correlation of aggregate with summary measures (Spearman’s ρ)Footnote †
† Non-log-transformed data.
Examination of the skewness of data for erythrocyte membrane EPA levels and fish intake measures demonstrated positively skewed distributions of the raw data for omega-3 levels and justified the use of log-transformed EPA values for analysis. The mean for non-transformed erythrocyte membrane EPA was 1.46 (SD 0.70); the geometric mean was 1.37 (SD 0.31).
Spearman correlation coefficients (Table 3) for summary estimates with erythrocyte membrane EPA were greater than those for aggregate estimates. In contrast, the R 2 values from multiple linear regression analyses (which adjust for age, sex, smoking and BMI) for summary estimates with erythrocyte membrane EPA were slightly lower than those for aggregate estimates. The regression coefficients for all four measures were statistically significant and were of similar magnitude except for the summary measure of grams of preserved fish per week, which was smaller than the aggregate measure.
Correlation (Spearman’s ρ) and multiple linear regression analysis† of erythrocyte membrane eicosapentaenoic acid with aggregate and summary measures
* P < 0.05, **P < 0.01.
† Adjusted for age, sex, smoking, body mass index and alcohol intake.
‡ Coefficient from the multiple linear regression model.
§ R 2 value from the multiple linear regression model.
ICCs for both the summary question and the aggregate measures derived from multiple questions on fish consumption are presented in Table 4. The ICCs for all four measures indicate good reliability.
Retest reliability of aggregate and summary measures of fish consumptionFootnote †
† Using non-log transformed data.
Discussion
Our analysis demonstrates that fresh and preserved fish summary and aggregate measures are comparable in terms of validity and reliability, but differ in the absolute estimates of intake that they provide. Instinctively, one would expect that aggregate measures would overestimate summary measures of fish intake because of the number of items used to generate the aggregates. We found this to be the case for fresh fish consumption estimates in the current study. A small pilot study to test a fish consumption questionnaire containing detailed and summary questions also found that an aggregate overestimated fish consumption relative to a summary measureReference Cavan, Gibson, Cole and Riedel14. This is of practical importance given that studies comparing estimated intakes with biomarkers of energy have demonstrated a tendency for FFQs and food records to typically underestimate consumptionReference Westerterp and Goris15, Reference Black and Cole16. This underestimation could be amplified by the use of abbreviated questioning, and has implications for the usefulness of dietary estimates collected by abbreviated FFQs for the purpose of examining associations between absolute dietary intakes and disease.
On the other hand, we found that the summary questions overestimated preserved fish consumption relative to the aggregate measure. This observation may be a result of a flaw in our questionnaire, for example the absence of an important item contributing to preserved fish consumption, although piloting and testing of the questionnaire did not reveal any missing items. It could also suggest that summary questions tend to underestimate foods that are consumed regularly and in large amounts by most people, but overestimate foods that are not consumed regularly or commonly.
There may however be a flaw in the methods used to collect information on frequency and portion size in our questionnaire. If either style over- or underestimates usual amounts, so that respondents cannot opt for an appropriate amount that reflects their intake, then they may systematically over- or underestimate their consumptionReference Cypel, Guenther and Petot17. Inspection of portion size data for individual items in our questionnaire provides some evidence of this occurring, with participants indicating consumption of smaller portions of preserved fish and larger portions of fresh fish than the portion sizes specified in the summary question. This would mean, as was observed in our estimates in Table 1, that the summary question overestimates preserved fish consumption relative to the aggregate measure, and underestimates fresh consumption.
We perhaps also would have expected that, if both styles of questioning consistently estimate consumption in grams per week (relative to one another), servings per week would have correlated more poorly between the two styles than grams per week. However, our findings of a higher correlation for servings of fresh fish would suggest that participants are disregarding the specified portion size in the summary question, instead of lowering or increasing their frequency to account for a portion size that is smaller than they would usually consume. An alternative reason for these observed differences in correlations could be the larger number of items and portion sizes used to generate the aggregate measure for grams of fresh fish per week, resulting in greater variation of consumption estimates.
Our results for Spearman correlation coefficients, for the aggregate and summary measures for fish consumption with erythrocyte membrane EPA, fall within the range of correlations published in the literature (0.16–0.65)Reference Hjartaker, Lund and Bjerve6, Reference Godley, Campbell, Miller, Gallagher, Martinson and Mohler9, Reference Andersen, Solvoll and Drevon18–Reference Sasaki, Ushio, Amano, Morihara, Todoriki and Uehara26, suggesting that both are valid methods of ranking participants according to overall fish consumption. In terms of relative validity of the two methods, the correlation coefficients suggest that the summary questions are relatively better at ranking according to fish intake. However, there is a less marked difference between the two types of measure when assessed using multiple regression analysis, and these are more appropriate indicators of validity because they are adjusted for factors (age, sex, smoking status, alcohol intake and BMI) that could account for much of the inter-subject variation in energy and nutrient intakes. Validity studies using regression analysis and adjusting for similar factors have been published to a lesser extent than correlation coefficientsReference Hjartaker, Lund and Bjerve6, Reference Svensson, Akesson, Nilsson and Skerfving21, Reference Amiano, Dorronsoro, deRenobales, RuizdeGordoa and Irigoien27–Reference Wakai, Ito, Kojima, Tokudome, Ozasa and Inaba30, and the two we identified that reported R 2 values demonstrated comparable results to oursReference Hjartaker, Lund and Bjerve6, Reference Bonaa, Bjerve and Nordoy29, indicating that both summary and aggregate measures are valid measures of overall fish consumption.
As part of a validity study of the original Norwegian questionnaire, Hjartaker et al.Reference Hjartaker, Lund and Bjerve6 compared summary questions on lean fish for dinner and fatty fish for dinner with more comprehensive questions and found that the estimates from the summary questions did not significantly correlate with either the overall dietary estimates of total omega-3 PUFA or plasma phospholipid omega-3 PUFA. This suggests that the summary questions on lean and fatty fish in this instance were not valid methods for ranking consumption. While participants were prompted as to what fish are considered fatty, there may have been some difficulty or confusion in grouping these fish together in order to estimate consumption, resulting in the relatively poor validity of this style of questioning. Other studies that have assessed the validity of shortened dietary FFQs with the aid of biomarkersReference Rohrmann and Klein31–Reference Resnicow, Odom, Wang, Dudley, Mitchell and Vaughan33 have demonstrated that with careful choice of food items, correlations between food consumption and biomarkers can be preserved, therefore indicating that abbreviated questionnaires are as effective as more comprehensive questionnaires in ranking according to overall consumption of foods.
ICCs for both the summary questions and the aggregate measures of fish consumption are comparable with or higher than those reported in the literatureReference Mannisto, Virtanen, Mikkonen and Pietinen34, indicating that both methods of assessing fish consumption are reliable. The lower retest reliability observed for the aggregate smoked/dried/salted measure may have resulted from confusion between hot and cold smoked fish, although misclassification between these two items should not have affected the reliability of an aggregate measure.
It is necessary to mention here that the similarity in validity and reliability of summary and aggregate measures of fish consumption may be a result of both measures being employed within the same questionnaire, and that the act of responding to more detailed questioning primed the participants to respond more thoughtfully to the summary questions. While a study comparing brief and detailed questionnaires on separate occasions found both to be valid measures of fruit and vegetable intakeReference Serdula, Coates, Byers, Mokdad, Jewell and Chavez35, it may be useful to apply our different methods of measuring fish consumption on separate occasions to more appropriately assess the comparative validity and reliability.
We would like to point out that the results of the validity and reliability analyses may not be applicable to the general population because the participants in this study were healthy, well-educated volunteers who consume higher-than-average amounts of fish, and are likely to be knowledgeable about fish or interested in the potential health benefits of fish. Due to the reference periods used in this study, it is difficult to predict the relative validity of these methods when asking about long-ago fish consumption, and we would caution that our observations may only be relevant to estimation of recent fish consumption in the setting of a prospective study.
Our study of summary and aggregate estimates of fish consumption indicates that employing more extensive questioning on individual food items does not add any information if the ultimate aim is to rank individuals according to overall consumption of fresh or preserved fish, and therefore brief questioning is a better choice for this purpose in order to reduce participant burden. However, the use of a more detailed questionnaire may be necessary in order to assess consumption of oily fish if there is concern that study participants are unable to accurately respond to a summary question. We would also note that the methods used to collect information on portion size have the potential to affect estimates of absolute intake, and not in a predictable manner. Finally, as other studies have found, caution needs to be taken when choosing styles of questions to be used to generate absolute estimates of fish intake.
Acknowledgements
Sources of funding: This project was funded by the Western Australian Department of Fisheries. K.M. is supported by a National Health and Medical Research Council (NHMRC) Postgraduate Public Health Scholarship. L.F. is supported by an NHMRC Career Development Award.
Conflict of interest declaration: No conflict of interest to declare.
Authorship responsibilities: K.M. was responsible for the study design, recruitment and coordination of subjects, data entry and analysis, and synthesis of the paper. L.F. and M.K. provided advice on study design and methods, statistical methods, interpretation of results, and editorial assistance.
Acknowledgements: Omega-3 PUFA analyses were conducted by Dr Trevor Mori, School of Medicine and Pharmacology, The University of Western Australia. Database design and support were provided by Robin Mina, School of Population Health, The University of Western Australia.
