A good diet before and during pregnancy(1) is important because suboptimal maternal nutrition can result in poor infant outcomes, such as neural tube defects, low birth weight and infant hypovitaminosis D(2). Managing weight gain in pregnancy reduces the risk of adverse short- and long-term maternal and neonatal outcomes, including high maternal postpartum weight retention and the development of obesity in mothers and children(Reference Ho, Flynn and Pasupathy3).
The UK National Health Service (NHS) has recommendations on healthy eating in pregnancy(1), as well as foods to avoid(4) and supplement use(5) – notably for folic acid and vitamin D. The advice on foods to avoid consists of a list of foods to avoid or consider avoiding, avoidance of high-fat, high-sugar containing foods, restricting alcohol consumption and on avoiding soil ingestion by washing fruits and vegetables(4). Diet-related advice in pregnancy is lengthy and complicated. A recent review on fish intake concluded that the complexity of the guidelines may lead to pregnant women reducing their intake or not eating fish at all(Reference Taylor, Emmett and Emond6). Funnell et al.(Reference Funnell, Naicker and Chang7) showed adherence to taking folic acid pre-conception is low and less than 10 % of eligible women accessed free vitamins in the UK. Lucas et al.(Reference Lucas, Charlton and Yeatman8) suggested women in developed countries do not receive adequate written or verbal nutrition advice from reputable sources, that is, healthcare professionals. In a global online survey of 613 participants, 49 % reported dissatisfaction with the information given by healthcare professionals and 47 % reported a lack of time to ask questions as key reasons to use the Internet(Reference Lagan, Sinclair and Kernohan9).
Internet access and use is increasing, especially via mobile devices. Pregnancy advice-seeking has evolved with younger men and women utilising the Internet and social media(Reference Walker, Blumfield and Truby10,Reference Bland, Dalrymple and White11) . Narasimhulu et al.(Reference Narasimhulu, Karakash and Weedon12) noted that benefits for pregnant women using the Internet included immediacy of response, anonymity and convenience compared with making an appointment or phoning a busy healthcare professional. They also noted that the main topics searched were pregnancy complications, fetal development and diet and nutrition. However, both the study by Narasimhulu et al.(Reference Narasimhulu, Karakash and Weedon12) and the systematic review by Sayakhot et al.(Reference Sayakhot, Carolan-Olah and Steele13) noted that in most cases women do not ‘close the loop’ by discussing Internet findings with their healthcare professional.
Evidence supports the concept that Internet-sourced information increases women’s confidence in making decisions and results in better health-related behaviours. In their UK study, Funnell et al.(Reference Funnell, Naicker and Chang7) found pre-pregnancy Internet use was associated with better adherence to supplement advice and higher fruit and vegetable intake. However, a US study found that while books (used by 61 % of participants) and the Internet (used by 58 % of participants) were the most popular sources of information, only face-to-face physician advice reduced the odds of exceeding the official gestational weight gain guidelines (0·55 (95 % CI 0·35, 0·88, P = 0·01))(Reference Mercado, Marquez and Abrams14).
Despite the Internet’s convenience, information on diet and nutrition related to pregnancy may be unreliable and out of date. In a study examining accuracy of online pregnancy-related information in Australia, Storr et al.(Reference Storr, Maher and Swanepoel15) found that 40 % of 693 Internet pages contained accurate information, 37 % contained inaccurate information and 23 % contained a mix of both. A recent review of pregnancy smartphone apps found that overall they do not consistently provide accurate and useful nutrition information(Reference Bland, Dalrymple and White11).
Internet information on diet and nutrition in pregnancy may be difficult to read and comprehend. In England, 15 % of adults (5·1 million people) lack basic literacy skills(16) and 42 % do not have the literacy skills to discuss a medical condition with a doctor or specialist(17). UK government advice is that all public Internet information is written to a minimum Flesch Readability Ease (FRE) score of 60(18). This score assesses reading ease, based on the number of words in a sentence and the number of syllables per word(Reference Flesch19). A high score means that the text is easier to read than a low score. A score between 60 and 80 should be understandable for an individual aged 12–15 years old. In the Australian study by Storr et al.(Reference Storr, Maher and Swanepoel15), the mean FRE of pregnancy Internet pages was 52, described as ‘fairly difficult’.
The Internet is a popular source of information. The purpose of the present study was to examine the accuracy and readability of Internet pages offering pregnancy-related diet and nutrition information when searching the Internet from the UK.
Methods
A Search Intent Modelling(20) (SIM) report that identified Internet search phrases used in the UK for pregnancy-related diet and nutrition searches during 2017 was available to the first author while employed by a commercial organisation(21). The SIM report created by a London media agency categorised 4·1 million searches of which 63 % related to pregnancy. These search phrases echoed popular topics identified by other studies, such as fetal development, diet and nutrition, exercise and employment issues(Reference Lagan, Sinclair and Kernohan9,Reference Sayakhot, Carolan-Olah and Steele13) . Searches performed on diet and nutrition that reached at least ten searches per month were reported and these comprised 194 phrases. There was much similarity and overlap within the 194 phrases, for example, pregnancy diet, pregnancy meal plan, pregnancy diet plan, foods to eat when pregnant and healthy pregnancy diet. The top twelve search phrases were those occurring more than 1000 times per month, with the monthly reported total usage of these search phrases being: foods to avoid when pregnant – 5400; pregnancy vitamins – 4400; folic acid pregnancy – 3600; what not to eat when pregnant – 2900; foods to avoid during pregnancy – 2900; pregnancy diet – 2900; prenatal vitamins – 2400; vitamin D pregnancy – 1900; vitamin A pregnancy – 1600; what to eat when pregnant – 1600; low iron in pregnancy – 1300; and pregnancy food – 1300. The present study performed Internet searches using these twelve phrases identified in the SIM report that were grouped into three themes; foods to avoid; foods to eat; and supplement use. Each phrase was entered into the Google browser bar and searched on different days between August and December 2018. Google was used as it is the most popular search engine, powering 75 % of global searches(22). Searches were performed while signing out of a personal Google account to ensure anonymity and to avoid the first author’s history of Internet use influencing the content served. After entering each phrase into the Google browser bar, the first two pages of multiple millions of hits were reviewed, which yielded approximately twenty results for each phrase. Only the first two pages of each search were included because a previous study had shown individuals rarely look beyond the first two Internet pages when searching for health information(Reference Storr, Maher and Swanepoel15). Internet pages only offering dietary supplements for sale or pregnancy-related services were excluded, as were links to videos, duplicate pages and Internet pages offering no nutrition or dietary information. The three NHS pages containing the reference advice – foods to eat in pregnancy(1), foods to avoid(4), and supplement use(5) – were excluded from the analysis. Pages that met the inclusion criteria were analysed for accuracy against the NHS advice and for readability by copying the text and pasting into an online tool(23) to calculate the FRE score. No prior assumption was made about a relationship between the accuracy of content and its readability.
Three checklists were created in Microsoft Excel summarising the NHS advice on the three themes: foods and beverages to avoid during pregnancy (including advice on high-fat, high-sugar containing foods, alcohol, and advice regarding washing fruits and vegetables), foods to eat in pregnancy, and supplements in pregnancy (vitamins in general, folic acid, vitamin D, vitamin A and Fe). The twelve search phrases were entered into separate Google searches. Each page that met the inclusion criteria was opened and examined and the advice compared with that of the NHS. Results were recorded on the spreadsheet as ‘accurate’, ‘inaccurate’ or ‘missing’. A page was deemed fully accurate where all the advice was accurate and complete compared with the NHS advice, inaccurate where all the information provided was incorrect, mixed where the page included both accurate and inaccurate advice, and no relevant information where the page offered nutrition or dietary advice not stated in the NHS advice.
The following information was also captured on the three spreadsheets for each Internet page: country of origin; type of originating organisation grouped as government, charity/not-for-profit, publisher (broadcast or magazine publishers) and commercial (selling products or services); date of creation, if stated, and other languages offered. This approach to capturing results was piloted independently by the authors using three Internet pages for each theme in August 2018 and, with minor amendments, was found to be suitable for recording the Internet page results.
Statistical analysis
Data were entered into Microsoft Excel 2011 and imported into IBM SPSS version 25 for analysis. Descriptive statistics were calculated as percentages of pages with fully accurate and complete, fully inaccurate, mixed, or no relevant information. Percentage accuracy for the total number of individual items of advice offered, excluding missing advice, overall and by dietary theme were also calculated. Median page readability (FRE score) was assessed for all pages examined, and by dietary theme.
Spearman’s correlation was used to investigate the relationship between percentage accuracy and readability of advice. Because the data were not normally distributed, between-theme comparisons were performed by Kruskal–Wallis tests with Bonferroni correction to reduce the chances of obtaining false-positive results (type 1 errors) when multiple pairwise tests were performed. Non-parametric Mann–Whitney U tests were used to test for two-sample differences in medians. Percentage accuracy and readability of Internet pages were also explored with reference to origin of Internet page (UK and non-UK) and type of organisation (commercial and not-for-profit).
Significance was set at the 0·05 level.
Results
The Internet searches identified 240 pages for review, the top twenty results from the twelve search phrases. In all, 110 (46 %) pages were duplicate pages of those already examined, offered only goods or services for sale, or links to videos with no written content, and were excluded. The three search phrases related to foods to avoid identified thirty Internet pages for inclusion, the three phrases for foods to eat identified twenty-eight Internet pages and the six phrases for vitamins and minerals identified seventy-two pages.
Overall, of the 130 pages examined, eighty-three pages (64 %) contained a mix of accurate and inaccurate advice, twenty-three pages (18 %) were complete and accurate, twenty-one pages (16 %) were entirely inaccurate and three pages (2 %) lacked any relevant advice. Only 33 % of the 130 Internet pages had an FRE score of 60 or above (the recommended readability ease for public Internet pages(18)).
The overall median percentage accuracy (lower quartile, upper quartile) of all items of advice was 83 (48, 100) and the median readability FRE score of the 130 Internet pages examined was 55 (46, 61). There was a significant weak positive correlation between overall accuracy and overall readability, and between accuracy and readability for the advice on supplements, but not for the advice on foods to avoid or foods to eat (Table 1).
Percentage accuracy of all advice items and readability (Flesch Readability Ease (FRE) score), and correlation between accuracy and readability by dietary theme
(Median values and lower and upper quartiles)
LQ, lower quartile; UQ, upper quartile.
Internet pages on the theme of foods to avoid in pregnancy scored highest for readability, with 60 % of the thirty pages scoring an FRE of 60 or above; 61 % of the twenty-eight Internet pages dealing with foods to eat in pregnancy had an FRE of below 60, most classified as ‘fairly difficult’ to read. The seventy-two Internet pages on supplements were the most difficult to read: 81 % had an FRE score below 60; 45 % scored 30–49 defined as ‘difficult’ to read; and 10 % scored less than 29, defined as ‘very difficult’ to read. The NHS Internet pages on these themes had an FRE score of 65 each for foods to avoid(4) and foods to eat(1), and 54 for supplements(5).
The pairwise differences in the percentage accuracy by dietary theme were not statistically significant, but this was not so for differences in readability between the three dietary themes. Internet pages on foods to avoid were easier to read than pages on supplements (median FRE difference 10·0, 95 % CI 6·0, 14·0, P < 0·000). Pages on foods to eat were also easier to read than those on supplements (median FRE difference 9·0, 95 % CI 5·0, 14·0, P < 0·000).
Geographical origin, organisation type, date of content and languages of Internet pages
Seventy-three (56 %) Internet pages originated in the UK and forty-two pages (32 %) in the USA. The remaining fifteen pages originated in Canada (four pages), India (three pages), Australia (three pages), Switzerland (two pages), Spain (one page), Germany (one page) and Ireland (one page). Internet pages originating in the UK were significantly more accurate (median percentage difference 33·0, 95 % CI 16, 40, P < 0·000) and readable (median FRE difference 8·0, 95 % CI 5·0, 12·0, P < 0·000) than those from outside the UK.
Almost half (48 %) of the Internet pages were provided by publishing organisations, commercial organisations contributed 27 %, charities 22 % and governments outside the UK 3 %. When type of organisation was grouped as commercial (publishing and commercial) and not-for-profit (charities and government), those provided by commercial organisations were significantly less accurate than those from not-for-profit organisations (median percentage difference –8·0, 95 % CI –29, 0·0, P < 0·02). Readability, however, did not differ significantly.
Seventy-two Internet pages (55 %) showed a date of creation, but only twenty-five (19 %) were created in 2018. Of the pages, 110 pages (85 %) did not offer an alternative language to English. Where another language was offered, Spanish was the most common (fifteen of twenty pages). Of the thirty Internet pages containing advice on foods to avoid, eighteen were dated (2003 (one page), 2016 (six pages), 2017 (two pages), 2018 (nine pages)) and four offered alternative languages. Two Internet pages offered Spanish as an alternative language, both from the USA. One Irish government site offered some content in the Irish language and one Indian site offered Hindi as an alternative to English. In terms of the theme of what to eat in pregnancy, twelve of the twenty-eight Internet pages gave a date of creation (2008 (one page), 2014 (one page), 2015 (one page), 2016 (two pages), 2017 (four pages) and 2018 (three pages)). Two US Internet pages offered an alternative language (Spanish). Of the seventy-two Internet pages on supplements, forty-two (58 %) gave a content date. The earliest was 1987 for advice on vitamin A(24). Four were dated from 2000 to 2010, and a further five from 2011 to 2015. Twelve Internet pages were dated 2016, seven dated 2017 and the remaining thirteen dated 2018. Alternative languages were more numerous on Internet pages dealing with supplements. Spanish was the most common (eleven Internet pages, ten from the USA and one from the World Health Organization) with French, Chinese and Arabic each offered on two Internet pages. Russian and Portuguese languages were alternatives on one Internet page each.
Foods to avoid in pregnancy
None of the thirty Internet pages on foods to avoid in pregnancy gave advice on all twenty-two NHS recommendations(4). Twenty-nine pages offered a mix of accurate and inaccurate advice and one page had no relevant advice. Overall, 45 % of all the advice was accurate, 46 % was missing and 9 % was inaccurate. Excluding missing advice, 84 % was accurate (Table 2). More specifically, advice on eight of the twenty-two topics was 100 % accurate. The advice most likely to be inaccurate (52 % of Internet pages) was on consumption of raw eggs: the recommendation to avoid raw and lightly cooked eggs in pregnancy due to risk of salmonella was changed in October 2017 by the UK Food Standards Agency(25), and this advice had not been updated in over half of the Internet pages examined. One of the four items of advice on fish was frequently incorrect: that to avoid more than four cans of tuna per week which was incorrect on 36 % of Internet pages. One Internet page erroneously offered a list of ten fruits to avoid in pregnancy(26) and contained no accurate advice.
Compliance of Internet pages with National Health Service (NHS) recommendations for foods and beverages to avoid during pregnancy(4)
(Numbers and percentages)
Six of the thirty Internet pages originated in the USA where dietary advice(27) about foods to avoid differs from that in the UK. US Internet pages advised avoiding smoked seafood, fresh fruit juice from out-of-home outlets, energy drinks, raw eggs and raw sprouts (bean sprouts), for example, the advice from Internet MD(28). Some Internet pages were clear that they originated in the USA, for example, The American Pregnancy Association(29). Internet pages with unclear origins were common and can be difficult to identify, especially when the same organisation is returned in searches in both UK and US versions, for example, BabyCentre UK(30) and BabyCentre US(31) that only differ in the domain extensions ‘.co.uk’ and ‘.com’, respectively.
Foods to eat in pregnancy
Overall, none of the twenty-eight Internet pages accurately offered the nine items of advice on foods to include in pregnancy(1). Twenty-seven pages offered a mix of accurate and inaccurate advice and one page was entirely inaccurate. Overall, 43 % of all the advice was accurate, 42 % was missing and 15 % was inaccurate. Excluding missing advice, 74 % of advice was accurate (Table 3).
Compliance of Internet pages with National Health Service (NHS) recommendation for foods to eat during pregnancy(1)
(Numbers and percentages)
Advice on eating a variety of foods, having a healthy breakfast and using the Eatwell Guide(32) was 100 % accurate when offered. However, advice about breakfast and the Eatwell Guide was only given by eight (29 %) of the twenty-eight Internet pages. The advice most likely to be inaccurate was on protein – 88 % of occurrences. The most common error was advising ‘plenty’ rather than ‘some’ protein in the diet. Advice on 45 % of Internet pages about dairy products failed to reflect the NHS advice regarding the benefit of choosing low-fat dairy products. As with ‘foods to avoid’, many websites offered advice based on US health guidelines, differing from UK advice, for example, the American Pregnancy Association that advises an extra 1255·2 kJ/d (300 kcal/d) throughout pregnancy in an article dating from 2015(33).
Supplements in pregnancy
Seventy-two Internet pages were examined for advice on vitamins and Fe. Twenty-seven pages offered a mix of accurate and inaccurate advice, twenty-three were accurate, twenty were entirely inaccurate and two had no relevant advice. Overall, 50 % of all the advice was accurate, 21 % was missing and 29 % was inaccurate. When missing advice was excluded, 63 % of the advice was accurate (Table 4).
Compliance of Internet pages with National Health Service (NHS) recommendations for supplements during pregnancy(5)
(Numbers and percentages)
Advice about taking vitamin D in pregnancy was mostly accurate (67 %), but that for vitamin A and Fe was inaccurate on over 50 % of pages. Twenty-eight of the seventy-two Internet pages on supplements originated in the USA, and these focused on four nutrients: folic acid, Fe, Ca and vitamin D. Folic acid followed by Fe were the most frequent US supplements advised, and when vitamin D was mentioned the recommended daily dose was higher (15 µg) than the UK recommendation (10 µg)(34). Of note, US sites such as Healthline(35) advised that vitamin A is safe when in a multivitamin supplement which is inconsistent with UK advice.
Discussion
The present study assessed the accuracy and readability of Internet advice on diet and supplement use in pregnancy in 2018 based on popular Internet search phrases identified in 2017. Internet page contents were compared with NHS advice for accuracy, and readability was assessed using the FRE score. Only 18 % of the 130 Internet pages examined were fully complete and accurate and 67 % did not meet the recommended FRE score of 60 or above for public websites. There was a weak positive correlation between the accuracy and readability of all Internet pages as well as for those specifically about supplements, but not for the pages on foods to eat or foods to avoid during pregnancy. None of the thirty pages on to foods to avoid or the twenty-eight pages on foods to eat in pregnancy were fully accurate and complete. When accuracy of advice was calculated as a proportion of all advice offered, median accuracy was 83 %. The median readability of all 130 Internet pages was 55, which is deemed ‘fairly difficult’, and there was a wide variation in FRE scores, from 8 (very difficult) to 79 (fairly easy). Two of the three reference NHS websites scored above 60 for readability, but the page on supplements scored below 60. Most (56 %) Internet pages originated in the UK, with the second largest group from the USA (32 %). Advice from websites originating outside the UK was more likely to be inaccurate and more difficult to read than that from the UK. The largest providers of Internet pages examined were publishers, followed by commercial organisations. Only 19 % of Internet pages with a date of creation were created in 2018. This is an important finding because the absence of a date of creation and a prominent search engine position (in first twenty results), indicates the Internet is promoting information of indeterminate age. Most (85 %) Internet sites did not offer an alternative language to English.
The combined UK NHS pregnancy advice on foods to avoid, foods to eat and supplements comprises thirty-six items. This lengthy advice is often shortened on Internet pages in an arbitrary way such as ‘seven foods to avoid in pregnancy’(36). Few Internet sites offer the complete NHS advice for the three themes examined, omissions occured for different items of advice on different sites. However, where advice is offered, it is largely accurate. This gap between the completeness of NHS and other advice may be due to non-NHS sites being out of date compared with NHS advice. Additionally, non-NHS sites are not necessarily written by those who understand the evidence base for diet and nutrition in pregnancy and may make incorrect assumptions about the most important recommendations or state the NHS advice incorrectly. This is compounded by the difficulty for the user in knowing the geographical origin of the website, which is relevant given that recommendations can vary by country. Another may be that Internet sites offering information to attract users with a view to selling products may choose to highlight only the relevant offical advice regarding their products. The selective replication of NHS advice and lack of consistency of advice suggests that Internet users will experience confusion about what is really important and may be reluctant to make positive behaviour changes. This is much to be regretted given that pregnancy is a key ‘teachable moment’ for dietary improvements(Reference Da Costa, Zelkowitz and Bailey37).
The accuracy of the Internet pages in the present study (18 %) was lower than that found by Storr et al.(Reference Storr, Maher and Swanepoel15) in Australia where 40 % of 693 Internet pages examined offered accurate advice. The latter study assessed Internet page content against the 2013 Australian Dietary Guidelines(38) and, although 73 % of the Internet pages originated outside of Australia, a higher level of accuracy was observed compared with this UK study. Commercial Internet pages made up 49 % of the Internet pages in the Australian study compared with 76 % in the present study. Not-for-profit organisations contributed 16 % of Internet pages in Australia, compared with 24 % in the UK. There was no difference in accuracy between commercial and not-for-profit Internet pages in the Australian study, unlike the present study where not-for-profit Internet pages (regardless of geographical origin) were significantly more accurate than those provided by commercial organisations. In the present study, many websites offered minimal advice and consisted mainly of supplement advertisements and reviews, as was the case with a commercial site from the UK(39).
The present study used similar methods to that used in the larger Australian study(Reference Storr, Maher and Swanepoel15), but differed in that search phrases based on SIM(20) were available for the present study. Other studies have highlighted the importance of the Internet for dietary advice in pregnancy but did not assess the quality of the information sources(Reference Lagan, Sinclair and Kernohan9,Reference Storr, Maher and Swanepoel15) . A strength of the present study is that it is the first of its kind to assess both the accuracy and readability of Internet page content against UK dietary recommendations in pregnancy using current popular search terms. There are also limitations, given that websites returned via a search phrase will vary from day to day. Hence, the present study is a snapshot of the advice available online using Google as the sole search engine during 2018. FRE is a guide to readability but only analyses text and does not take into account other ways to improve comprehension, such as tables, images and clear layout of information. In addition, it does not examine the use of jargon which can be a barrier to the readability of text. There is also a danger of observer bias since the analysis was undertaken by one person.
Two-thirds of the Internet pages examined in the present study did not meet the recommended readability ease for public information. Dietary advice may thus be inaccessible to pregnant women with low literacy skills. Only one in three of the 130 Internet pages examined would be accessible to someone with a low literacy skill. Given the overall weak and positive correlation between Internet page readability and accuracy, there is no guarantee that more readable Internet pages are accurate. Previous studies have linked poor literacy to poor healthcare outcomes, so ease of comprehension must be considered by those writing about diet and pregnancy for the Internet, such as by adding images and diagrams to text.
The Internet has an important role to play in the provision of health advice before and during pregnancy. Information via the Internet, such as the advice to take folic acid, can be particularly useful pre-pregnancy and in early pregnancy, as women may not have met with a healthcare professional. In the rapidly changing digital information environment, with multiple sources of dietary advice, healthcare professionals should direct pregnant women to accurate and comprehensible information. Those who publish Internet advice for pregnant women should consult with appropriately qualified health professionals, that is, registered nutritionists and dietitians. Steps to monitor the quality of information online should be considered by government and regulators of advertising. Internet-sourced health information has great potential to reach and influence women in pregnancy, but it must be accurate, up to date and easy to read.
Acknowledgements
Nestlé Nutrition provided registration fees for A. S. to complete the MSc Public Health programme at Southampton University while the author was in their employment. Nestlé Nutrition had no role in the design, analysis or writing of this article.
P. N. supervised this project. A. S. and P. N. agreed the methods, A. S. undertook the Internet searching, performed the statistical analysis and wrote the paper. P. N. contributed to the writing of this paper.
