The Internet is now the single largest source of health information and is used by many consumers interested in healthy living. Also, a large number of websites provide nutritional information on the Internet. According to a survey by the American Dietetic Association (ADA), 24 % of Internet users access websites to get food and nutritional information(1).
The Mediterranean diet is a traditional dietary pattern in the Mediterranean area. Its main characteristics include an abundance of plant foods (fruits, vegetables, wholegrain cereals, nuts and legumes), olive oil as the principal source of fat, fish and poultry consumed in low-to-moderate amounts, relatively low consumption of red meat and moderate consumption of wine, normally with meals(Reference Hu2).
Recently, the Mediterranean diet has become recognized as one of the healthiest dietary patterns because research has shown its effectiveness in preventing the development of cardiovascular and other chronic diseases(Reference Mente, de Koning and Shannon3). Many websites with information on the Mediterranean diet are available on the Internet. Consumers interested in a healthy dietary pattern, not only patients with particular health concerns, seek and obtain knowledge about diet on the Internet. To protect consumers from being misled, it is important that website information be reliable. The poor quality of Internet information about treatment of and medication for diseases has been shown previously(Reference Eysenbach, Powell and Kuss4), but we could not identify any articles related to the quality of Internet information on the Mediterranean diet.
The aim of the present study was to evaluate the quality of Internet information available concerning the Mediterranean diet and to determine the relationship between quality and source of the websites. We used using the DISCERN rating instrument, the Health On the Net Foundation's (HON) code principles, and Journal of the American Medical Association (JAMA) benchmarks. The DISCERN instrument has been developed to help patients judge the quality of written information concerning treatment choices(Reference Charnock, Shepperd and Needham5) and to find evidence-based and unbiased information. JAMA benchmarks(Reference Silberg, Lundberg and Musacchio6) were created to assure the quality of medical information on the Internet. HON(7) is a not-for-profit international organization with a mission to guide non-medical users and medical practitioners to useful and reliable online health information. These have frequently been used as tools to evaluate the quality of health information websites(Reference Griffiths and Christensen8–Reference Bedell, Agrawal and Petersen18).
Methods
Website selection and data extraction
We conducted a cross-sectional assessment of website information about the Mediterranean diet. Keyword searches and website evaluations were done in February 2010 by the first author. The keyword ‘Mediterranean diet’ was entered into six commonly used English-language search engines: Google, Yahoo, MSN, AOL, Ask and Altavista(19). The first thirty results from each search engine were taken for the initial evaluation, because results lower in the relevancy lists are often duplications of those higher in the lists(Reference Hargrave, Hargrave and Bouffet9) and 90 % of search engine users click on a result within the first three pages of search results(20).
Duplicate URL were excluded, and the first pages of each website were reviewed for relevance for analysis. Websites were excluded if they were inaccessible (invalid address); required an access fee or login; were linked to other sites; were open forums, chat rooms or message boards; were personal web pages; were selling diet or supplement products; or were for professionals, such as journal websites. Recipe websites, dictionary sites and portal news sites were also excluded. Moreover, sites that included only the term ‘Mediterranean diet’ without specific information about the Mediterranean diet or sites that included articles copied from other sites were excluded. To review the website pages as they were found on the date of the search, we used the web capture feature in Acrobat® Reader 9 (Adobe Systems Incorporated, San Jose, CA, USA) to archive each page and copied website pages onto compact disk (CD ROM).
Evaluation of website quality
Affiliations were determined on the basis of the information provided by the site and divided into the following six categories: (i) institutional (e.g. government, hospital or university); (ii) non-pharmaceutical commercial (e.g. sponsored sites or private medical sites that did not sell either diet or supplement products); (iii) pharmaceutical (e.g. drug manufacturer, pharmaceutical research); (iv) charitable (e.g. non-profit organization); (v) support (e.g. patient support groups); or (vi) alternative medicine (e.g. unorthodox medicine).
Quality of websites was evaluated with the DISCERN instrument(Reference Charnock, Shepperd and Needham5), JAMA benchmarks(Reference Silberg, Lundberg and Musacchio6) and the HONcode principles(7). These quality indicators have been used frequently for studies that evaluated health information websites and whether their validity was reliable(Reference Griffiths and Christensen8–Reference Bedell, Agrawal and Petersen18, Reference Khazaal, Chatton and Cochand21, Reference Fallis and Fricke22).
The DISCERN instrument is a validated rating tool that is freely available online and the instruction handbook can be downloaded from the website (www.discern.org.uk). Consumers without previous knowledge of the subject can easily use it. A series of fifteen questions are asked about the website content. The user rates each question on a 5-point Likert scale from ‘no’ to ‘yes’, with the higher score considered the better score. A ‘no’ answer is given 1 point and a ‘yes’ answer 5 points, with scores of 2 to 4 indicating varying degrees of dissatisfaction or satisfaction with the website content. The total DISCERN score for the overall quality of the information and the website content is divided into five categories: excellent (63–75), good (51–62), fair (39–50), poor (27–38) and very poor (15–26).
The JAMA benchmarks require that a website should disclose each of the following four concepts: (i) authorship of medical content; (ii) attribution or references; (iii) currency (date of update); and (iv) ownership, sponsorship, advertising policies or conflicts of interest. Each website was rated according to whether the information provided met these four benchmarks.
The HONcode principles consist of the following 8-point code of conduct for websites that provide health information: (i) authoritative, (ii) complementarity, (iii) privacy, (iv) attribution, (v) justifiability, (vi) transparency, (vii) financial disclosure and (viii) advertising policy. The HONcode logo is available for placement on websites based on an honour system; its presence indicates that the website adheres to all of the HONcode principles. In our analysis, we checked whether information on a website satisfied each principle and displayed the HONcode logo.
Readability of content
We used the built-in readability statistics in the word processing software Word 2007 (Microsoft® Corporation, Redmond, WA, USA). Microsoft provides two ways to judge the readability of a given text, the Flesch Reading Ease score and the Flesch–Kincaid Grade Level score. The Flesch Reading Ease score rates text on a 100-point scale, with a higher score indicating that the content is easy to understand. It is suggested that to achieve ‘plain English’ a score should be between 60 and 70(Reference D'Alessandro, Kingsley and Johnson-West23). The Flesch–Kincaid Grade Level score rates a text on US school grade level (i.e. a score of 8 means that an eighth grader can understand the text). It is suggested that health information should aim for a grade level of 6 to 8(Reference D'Alessandro, Kingsley and Johnson-West23). We copied and pasted the main text of a website's information into our word processor before rating readability.
Food group components of the Mediterranean diet
To identify website information on the content of the Mediterranean diet, we extracted articles that included the composition of food groups of the Mediterranean diet from each website's main text and sorted the food group components. Trichopoulou et al. presented nine food groups – (i) vegetables, (ii) legumes, (iii) fruits and nuts, (iv) cereal, (v) fish, (vi) meat and poultry, (vii) dairy products, (viii) alcohol, and (ix) foods with a high monounsaturated-to-saturated lipid ratio, which mostly refers to olive oil – as comprising the Mediterranean diet(Reference Trichopoulou, Costacou and Bamia24), and this grouping has been used in many epidemiological studies(Reference Scarmeas, Stern and Mayeux25–Reference Gao, Chen and Fung29). We referred to these nine food groups but separated ‘fruits and nuts’ into ‘fruits’ and ‘nuts or seeds’ because fruits and nuts were presented as separate items in several websites. Also, we separated ‘meat and poultry’ into ‘meat, red meat, meat products’ and ‘poultry or white meat’ for the same reason. In addition, we observed that other foods (e.g. eggs, honey or sugar and water) were described as components of the Mediterranean diet in several websites. Ultimately, fourteen food groups were used in our study.
If the article described the recommended amount, gave a rough indication of food group components or indicated frequency of consumption by various phrases, we categorized the degree of recommended intake according to quantity by three descriptors (high, medium, and low) and the degree of recommended frequency of intake into four descriptors (daily, weekly, monthly and infrequently).
Information about the relationship of the Mediterranean diet to diseases or health
An examination was done in two steps to evaluate whether the website information included articles about relationships between the Mediterranean diet and diseases or health. As the first step, relevant words regarding disease or health appearing in the title of studies were extracted by PubMed search (spring, 2010) using ‘Mediterranean diet [TI]’ query. Then we recorded the presence or absence of the relevant words in each website by use of the built-in word search feature of the software program.
Researchers
Website extraction was completed within 3 weeks and data evaluation was done within 8 weeks of the initial identification of websites. Two of our investigators (R.H. and K.S.) independently reviewed and evaluated each website. Any discrepancy between the reviewers was resolved by consensus.
Statistical analysis
Simple descriptive statistics were used to analyse the data. Differences in distribution between variables were calculated using Pearson's χ2 test. We used ANOVA to compare scores by categories of websites. Post hoc testing was performed by Dunnett's T3. Results of all statistical tests were considered significant at P < 0·05. The analyses were conducted using the SPSS statistical software package version 17·0 (2008; SPSS Inc., Chicago, IL, USA).
Results
Figure 1 shows details of the website search. By using the first thirty results from each of the six search engines, a total of 180 websites were included for the initial assessment. Of the 180, ninety-six (53 %) websites were duplicates (i.e. the same website listed more than once) and were excluded at the first screening, which left eighty-four sites to be evaluated. Fifty-two (62 %) websites were excluded by the criteria described in the Methods section, leaving thirty-two to be evaluated. Of the thirty-two sites assessed, the sources were institutional for eight (25 %), non-pharmaceutical commercial for twelve (38 %) and support for twelve (38 %). No websites provided by pharmaceutical, charitable and alternative medicine sources were identified.
Website selection process for evaluation of quality of information on the Mediterranean diet
The mean scores for the total of thirty-two websites for each of the DISCERN questions are shown in Table 1. The median score for all questions combined was only 2 out of a possible 5. No question had a mean score of 4 or above. The lower scoring questions were those that asked if the website described what would happen without treatment, clarified that there was more than one possible treatment choice, and described the risks of treatment and effects on quality of life.
Combined results of the fifteen DISCERN questions from all thirty-two Mediterranean diet websitesFootnote *
* 5-point Likert scale scoring system. For a definite ‘no’, 1 should be given and if ‘yes’, 5 should be given; 2–4 points should be given if the publication being considered meets the criterion in question to some extent, with the lower score being less favourable.
Table 2 shows results for the DISCERN score, JAMA benchmarks and HONcode principles. The mean total DISCERN score for all thirty-two websites was 33·8, which was rated as ‘poor’. Twenty-two websites (69 %) were categorized as ‘poor’ or ‘very poor’ according to the DISCERN score. When the websites were categorized by source, the support sites scored 39·3, which indicated a ‘fair’ rating. The institutional and non-pharmaceutical commercial sites scored 36·0 (poor) and 27·0 (poor), respectively; therefore, scores for these two categories were significantly lower than the support site score (ANOVA and post hoc Dunnett's T3 test: P = 0·031, between institutional and non-pharmaceutical commercial; P = 0·004, between support and non-pharmaceutical commercial). The distribution of DISCERN score ratings showed that no site was rated ‘excellent’ and that only one site was rated ‘good’. However, thirteen sites were rated ‘poor’ and nine sites were rated ‘very poor’.
Results of examination by quality rating and readability assessment tools of thirty-two Mediterranean diet websites according to source categoryFootnote *
* Values are means (range) for DISCERN score, Flesch Reading Ease score and Flesch-Kincaid Grade Level score. Values are number of websites for DISCERN score rates, JAMA benchmarks and Display of HONcode logo.
† ANOVA was used to compare means. The χ2 statistic and χ2 test for trend of Display of HONcode logo were used to compare proportions. All statistical tests were two-sided.
‡ Total score of fifteen questions (scale of 1 to 5, higher being better). Total range is 15–75.
§ The total DISCERN scores were grouped into categories as excellent (63–75), good (51–62), fair (39–50), poor (27–38) and very poor (15–26).
∣ ∣The Journal of the American Medical Association (JAMA) benchmarks consist of four concepts. Values are total number of websites that satisfied different numbers of JAMA benchmarks.
¶ The Health On the Net Foundation's HONcode has eight principles. Values are means for total number of principles adhered to per website.
** Total range is 0–100, with a higher score indicating that the content is easy to understand. The recommended score for health literature for adults is suggested to be from 60 to 70.
†† Total range is 1–12. Rating is on a US school grade level. The recommended score for health literature for adults is suggested as between grades 6 and 8.
‡‡ Post hoc test was performed by Dunnett's T3. Significant difference between institutional and non-pharmaceutical commercial (P = 0·031), support and non-pharmaceutical commercial (P = 0·004).
§§ Post hoc test was performed by Dunnett's T3. Significant difference between non-pharmaceutical commercial and support (P = 0·011).
According to the JAMA benchmarks, two sites (6 %) met three concepts, one site (3 %) met two concepts, twelve (38 %) met one concept, and seventeen (53 %) met none of the concepts. No website achieved all four concepts.
With regard to the HONcode principles, only one site achieved all eight principles, two sites achieved seven principles, three sites achieved six principles and two sites achieved five principles. Twenty-four sites (75 %) achieved fewer than five principles. Of the thirty-two websites examined, eight displayed the HONcode logo. A mean of 3·2 (range 0–8) HONcode principles was achieved for the total of thirty-two websites, with the mean for support sites being significantly higher than that for the non-pharmaceutical commercial sites (ANOVA and post hoc Dunnett's T3 test: P = 0·011, between non-pharmaceutical commercial and support).
Table 2 shows the mean Flesch Reading Ease score and Flesch–Kincaid Grade Level for the websites examined. Although the recommended Flesch Reading Ease score for plain English is between 60 and 70, we found that only twelve websites achieved such a score. The recommended Flesch–Kincaid Grade Level for health literature for adults is suggested to be US school grades 6 to 8; none of the websites had reading levels above grade 10. Lower mean scores for Flesch Reading Ease scores and higher mean scores for the Flesch–Kincaid Grade Level were concentrated in the institutional category. Distribution of the Flesch–Kincaid Grade Level scores differed significantly among three source categories. A negative correlation between the two scoring systems was observed (P < 0·001).
All websites were reviewed to determine whether they described the recommended amount or frequency of intake of food group components of the Mediterranean diet, and nineteen articles were extracted (59 %; 95 % CI, 41–77 %). In the nineteen articles, fourteen food groups were mentioned. In comparisons of whether the article described recommended amounts or frequency of intake, we did not find significant differences in the DISCERN score, distribution of the DISCERN score rate and distribution of source categories. We categorized the degree of recommended intake and frequency of intake based on the descriptors used for each food group (quantity: high, medium and low; frequency: daily, weekly, monthly and infrequently). Table 3 shows the distribution of the recommended amounts or frequency of intake per food group. Recommended amounts of vegetables, fruits, cereals and legumes were mostly described as ‘high’. ‘Low’ frequently described the recommendation for meat. Four websites recommended ‘daily’ and twelve websites recommended ‘medium’ for intake of alcohol. Of the nineteen websites making recommendations for olive oil, seven indicated ‘high’, but the remaining twelve websites did not include a recommended amount. Nine websites indicated ‘high’ and three websites ‘low’ for nuts. Various descriptors were applied for fish and dairy products.
Distribution of descriptors of recommended intake of food group components for the Mediterranean dietFootnote *
* All values indicate number of websites.
† Descriptors representing how much or how often the food should be eaten were recorded.
‡ Articles refer to the food group but not to the recommended frequency or quantity.
§ Fourteen food groups were extracted from nineteen articles that described food group components of the Mediterranean diet.
We extracted twenty-five relevant words about disease or health. Twenty-six (81 %) website articles included at least one word. The words and their frequency of appearance were cancer (n 19), coronary (n 6), heart (n 26), cardiovascular (n 12), myocardial (n 2), obesity (n 7), weight (n 21), diabetes (n 10), metabolic (n 4), metabolic syndrome (n 3), cognitive (n 1), Alzheimer (n 7), stroke (n 6), brain (n 6), hypertension (n 6), kidney (n 1), liver (n 1), asthma (n 2), allergy (n 3), arthritis (n 2), depression (n 2), postmenopausal (n 1), longevity (n 6), chronic (n 13) and neurological (n 1).
Discussion
The concept of the Mediterranean diet originated from the Seven Countries Study initiated by Ancel Keys in the 1950s. The study showed that, despite a high fat intake, the population of the island of Crete in Greece had very low rates of CHD and certain types of cancer and had a long life expectancy(Reference Hu2). Since many website articles included information about the Seven Countries Study, we observed the appearance of the words ‘heart’ and ‘cancer’ frequently among the websites. Moreover, in an examination of the effects of three diets (i.e. low-fat, Mediterranean and low-carbohydrate), it was suggested that a Mediterranean diet was as effective as a low-fat diet for weight loss(Reference Shai, Schwarzfuchs and Henkin30). This finding might be related to the observation of the frequent appearance of ‘weight’ in the website articles. The appearance of many other relevant words about disease or health in our examination of these websites suggested that the Mediterranean diet was expected to be effective in preventing the development of many diseases.
In the websites that we evaluated, various components of food groups were mentioned and many descriptors were used with regard to recommended amounts. Some websites presented food group components with no recommended amount or frequency of intake. Kafatos et al. described the food groups assessed in a Cretan population as part of the Seven Countries Study in 1991(Reference Kafatos, Kouroumalis and Vlachonikolis31). Thereafter, in 1995 Trichopoulou et al. first presented eight food groups that comprised the Mediterranean diet (fish was not included, due to its consumption depending on proximity to the sea)(Reference Trichopoulou, Kouris-Blazos and Wahlqvist32), but in 2003 fish was included, resulting in nine food groups(Reference Trichopoulou, Costacou and Bamia24). This grouping has been used in many epidemiological studies(Reference Scarmeas, Stern and Mayeux25–Reference Gao, Chen and Fung29). In the present study, we observed the notation of other food groups (eggs, honey or sugar and water) in several websites.
Although identifying the impact or risk of each food in the development of a particular disease is difficult in using dietary pattern analysis, scored categories of dietary pattern based on Trichopoulou et al.'s food groups(Reference Trichopoulou, Costacou and Bamia24) were used for analysis in studies describing the relationship of the Mediterranean diet and diseases(Reference Scarmeas, Stern and Mayeux25–Reference Gao, Chen and Fung29). Therefore, in the present study, inclusion of a food group as part of the Mediterranean diet appeared to be made based on its perceived effectiveness in comparison with a reference dietary pattern and construction of the components of the Mediterranean diet was affected by the purpose of the website. A general agreement on a definition of food groups within the Mediterranean diet was not found on these websites; neither was there agreement on recommended intake of components of the Mediterranean diet.
Of thirty-two websites providing information about the Mediterranean diet, the majority were rated as low in quality according to the criteria of our examination. Although no website had a level of readability that could be considered difficult, not all achieved an appropriate level of readability. The quality of websites was likely to be influenced by the website source. Institutional and support websites tended to provide information of fair quality, but non-pharmaceutical commercial websites tended to provide low-quality information. This is consistent with the conclusions of a previous study that nutritional websites of governmental (‘gov’), educational (‘edu’) and non-profit (‘org’) organizations are generally recommended as providers of first-rate and reliable health information(Reference Griffiths and Christensen33).
Using the Internet to obtain information on the Mediterranean diet does not require the time or cost of meeting with a dietitian, is easier than seeking out results of current studies, and is useful for consumers to self-regulate their dietary behaviour in comparison with face-to-face dietary counselling by a dietitian. But website information provided without a personal nutritional assessment cannot provide tailored dietary instruction. Also, the exact Mediterranean diet components are not yet established and the results of studies mentioned in these websites cannot be used directly by consumers, as they may be unreliable. Moreover, numerous websites that vary in quality according to source and content were found on the Internet in the present study. Unfortunately, consumers are likely to arbitrarily read and use biased information on the effectiveness of a specific food, so these actions increase health risks and inhibit the benefits of medication. Therefore, consumers should take care in incorporating such information into their daily lives by taking into account the source and purpose of the website as well as readability. It is important to increase health literacy among consumers so that they may decide on the quality of the available websites. As health literacy may not be increased in the near future, it is also important for consumers to get advice from experts.
The efficacy of Internet nutritional education or Internet nutritional intervention programmes has been studied in recent years(Reference Oenema, Brug and Dijkstra34–Reference Moore, Alsabeeh and Apovian37). Generally, it is expected that the Internet will be useful in effecting dietary behavioural changes. For that purpose, Papadaki and Scott indicated the need for increasing the opportunity for interaction between a website and its users and the development of applications that can provide tailored dietary advice by experts or dietitians(Reference Papadaki and Scott38). These needs indicate the future direction in the development of the ability of the Internet to meet a variety of needs with regard to nutrition.
Several limitations must be considered. First, the results of the present study are somewhat limited in that part of the evaluation is subjective. However, these were balanced through the use of some important objective assessments (such as the Flesch Reading Ease score and the Flesch–Kincaid Grade Level score). Second, we have only assessed websites in English, and it is possible that websites in other languages have characteristics different from the ones evaluated in our study. Third, our sample size was only thirty-two, so that results may not be generalized. Fourth, the rapidly changing nature of information available on the Internet means that our results are truly valid only for the period over which the study was conducted. Some of our findings about a specific website may change over time, and some sites may not stay in existence.
Conclusions
The websites containing information on the Mediterranean diet were of poor quality. Information presented on the websites varied as to the recommended amounts and frequency of intake of each component of the food groups comprising the Mediterranean diet. Consequently, patients or consumers who are interested in the Mediterranean diet should get advice from physicians or dietitians. Considering that so many people use the Internet as a source of health information, nutrition information websites (also including information on the Mediterranean diet) should be designed according to the HONcode principles or be evaluated using DISCERN prior to their launch on the Internet. Furthermore, health education for consumers is needed because physicians or general practitioners rarely have sufficient time to talk to their patients about nutrition.
Acknowledgements
H. Sone and S.K. are recipients of a Grant-in-Aid for Scientific Research (20300227) and Postdoctoral Research Fellowship (202965), respectively, both from the Japan Society for the Promotion of Science (JSPS). This work was also financially supported by the Ministry of Health, Labor and Welfare, Japan. The sponsors had no influence over the design and conduct of the study or approval of the manuscript. The researchers were fully independent in pursuing this research. None of the authors had a conflict of interest. Authors’ contributions were as follows: R.H., K.S., K.K., H. Shimano and H. Sone designed the research; Y.I., S.K., C.H., A.S. and Y.H. conducted the research; R.H. and K.S. evaluated and analysed the data; R.H., Y.Y., M.A. and H. Sone wrote the manuscript. All authors critically reviewed the manuscript. R.H. and H. Sone had access to all data at all times and had the final responsibility to submit the manuscript for publication. The authors thank Ms Satomi Fukuya for her excellent secretarial assistance.
