In 2008, the European Food Safety Authority (EFSA) began reviewing the proposed health benefit claims on all foods.

To date, none of the 164 claims of the benefits of probiotic or prebiotic products submitted to EFSA and reviewed by the Panel on Dietetic Products, Nutrition, and Allergies (NDA) have been accepted (see Table 1). Those who are not aware of either the research supporting specific probiotics and prebiotics or the NDA review process may come to the fallacious conclusion that probiotics or prebiotics have not been shown to have health benefits. Without doubt, fraudulent or exaggerated claims are being made for some products. However, the scientists and clinical investigators belonging to the Board of Directors of the International Scientific Association for Probiotics and Prebiotics (ISAPP) are concerned that claims supported by solid scientific evidence are also being rejected. They are further concerned that there is a lack of clarity regarding the criteria – from study design through wording of the claim – for a dossier suitable for a positive regulatory opinion. One unintended consequence of the current review process may well be that the responsible companies studying the physiological effects of their probiotic or prebiotic products will decide that continued investment into this line of research is not cost-effective if, in the end, evidence supporting product benefits deemed valid by the scientific community cannot be communicated to the consumer.

Table 1 European Food Safety Authority (EFSA) opinions on questions about health claims for food products including probiotics or prebiotics under articles 13.1, 13.5 or 14 of EC Regulation 1924/2006*

* Data obtained from EFSA web site on February 2011. http://registerofquestions.efsa.europa.eu/roqFrontend/questionsListLoader?panel = NDA&foodsectorarea = 26.

† A cause and effect relationship has been established between the consumption of the food and the health outcome. There is a favourable opinion for live yogurt cultures and improved lactose digestion, but the term ‘probiotic’ was not used to define the food constituent⁽⁷⁾.

‡ Insufficient evidence or no cause and effect established.

§ Question withdrawn by the applicant.

Certainly, evaluation of evidence to support claims is not a simple process. The NDA scientists must implement challenging legislation and assess a flood of dossiers providing evidence, which in the nature of all research could always be improved. But the process is difficult for industry scientists, too, who must prepare a dossier in support of a claim with only general guidance from the NDA. A successful dossier requires not only compelling studies on efficacy, but also specification of a physiological effect that will be considered by the NDA as beneficial and a claim that is worded to accurately reflect the science but also be in compliance with regulations. Some recent documents have been drafted by the NDA to provide guidance on their interpretation of what constitutes beneficial effects and acceptable outcome measures (http://www.efsa.europa.eu/en/consultationsclosed/call/nda100928.pdf), but many questions remain unanswered. This opinion is reflected in a letter (http://www.gut-health.eu) by the European scientists expressing dissatisfaction with the process. As of 23 February 2011, 148 scientists have signed this letter.

One overriding concern with the review process is the standard of evidence required by the NDA. The legislation states that ‘Health claims should only be authorised for use in the Community after a scientific assessment of the highest possible standard’. However, this seems to be interpreted by the NDA to mean that the evidence (as opposed to the assessment) must meet the highest possible standard. A more realistic standard is expressed in article 6 of the EC Regulation 1924/2006, which states that health claims shall be based on and substantiated by ‘generally accepted scientific evidence’. Thus, regulators have indicated a definite roadmap: generally accepted scientific evidence is not the same as the notion that evidence must be based on a restrictive number of criteria established by a closed group of individuals. Generally accepted scientific evidence is a well-established concept, and is the basis for the peer review process of scientific journals, evaluation of grant applications or scientific productivity of researchers, and grading recommendations in evidence-based medicine. In the latter case, this means that findings of a single randomised control study with narrow CI can constitute level 1b of evidence and invoke a recommendation of top, Grade A, intervention. In practice, this means that the recommendation should be applied unless there is a specific reason for not doing it⁽¹⁾.

An example of implementing the ‘highest possible standard’ is apparent when the NDA rejected the validity of an independently conducted study published in the British Medical Journal ⁽Reference Hickson, D'Souza and Muthu²⁾ to support a claim that a probiotic food could reduce Clostridium difficile toxin in the gut and reduce the risk of acute diarrhoea in patients receiving antibiotics⁽³⁾. One concern expressed by the NDA panel judgement of the trial was with study blinding. Although the products were not identified to the patients, the bottle shapes were different for the placebo and the test product, but only for product sent home with a subset of discharged patients. It is unclear how the NDA expected this small imperfection to influence the level of C. difficile toxin in faeces. Importantly, the staff who conducted the toxic analysis on the stool samples from patients who had diarrhoea remained fully blinded to the test group assignment⁽Reference Hickson, D'Souza and Muthu²⁾. An additional criticism of the study was that C. difficile toxin was measured only in patients with diarrhoea and not in all study participants. However, it is common practice in a hospital environment to assay toxin only when diarrhoea occurs. The authors of this study concluded ‘Consumption of a probiotic drink… can reduce the incidence of antibiotic-associated diarrhoea and C. difficile-associated diarrhoea. This has the potential to decrease morbidity, healthcare costs and mortality if used routinely in patients aged over 50.’⁽Reference Hickson, D'Souza and Muthu²⁾ Yet the NDA deemed the evidence in this study as not of sufficiently high quality to support communication of the protective effect of the product against antibiotic-associated diarrhoea and C. difficile toxin production. Treatment costs per patient for C. difficile-associated diarrhoea are on average $3669 in the USA⁽Reference Kyne, Hamel and Polavaram⁴⁾ and £4000 in the UK⁽Reference Wilcox, Cunniffe and Trundle⁵⁾. The availability of a safe, dietary approach to reduce the morbidity of such conditions should be hailed, not suppressed.

Another concern is that the panel appears to conclude that unless all studies support the relationship being claimed, the totality of evidence is not compelling. However, we should not conclude that a study with a primary end point that does not reach statistical significance represents contradictory evidence. Studies on foods are often characterised by small magnitudes of effects and insufficient power to detect them at an acceptable level of statistical significance, placebos that may not be fully inert, high variability in study group subjects due to confounding factors such as background diet and individual microbiota, and a study group that is healthy, which makes it difficult to measure physiologically relevant changes. On the other hand, results from study populations that are more susceptible are considered irrelevant by the panel for the general population. Considering these factors, an underpowered study that does not yield statistically significant positive results should not be used to negate the outcomes of other positive studies. Assessing the preponderance of evidence, including magnitudes of effect observed in studies lacking statistical significance (although such findings clearly cannot be accorded the same weight as statistically significant results), is a more reasonable approach.

Many of these difficulties could be resolved to the ultimate benefit of the European Union Community. A mechanism for pre-application meetings should be instituted. This would enable companies to gain NDA feedback on a research approach before launching expensive and time-consuming studies. Increased use of scientific experts to augment the NDA panel could provide the expertise and perspective needed for a greater diversity of viewpoints and better balance to the evaluation process. And finally, the NDA should adjust its approach on what it requires as the standard of evidence. A requirement of evidence of the ‘highest possible standard’ may be unrealistic for a functional food that is proposed to maintain health or reduce risk of disease. In keeping with the legislation, EFSA should strive for an assessment process of the ‘highest possible standard’. This would be a process that correctly evaluated the degree of support for a claim that properly interpreted studies, that evaluated the strengths and weaknesses of studies to determine their true worth, and that, overall, had mechanisms in place to assure that the spirit of the legislation is upheld. (It would also include proper distinctions between ‘probiotic’ and ‘prebiotic’, which were confused in the NDA opinion on lactulose⁽⁶⁾.) Such mechanisms would include the pre-application meetings and use of ad hoc experts.

In the end, such changes would prevent the use of unsubstantiated claims on food products in the European marketplace while allowing communication of science that is suitably compelling. Finally, such changes would provide industry with a clear roadmap to understanding what is required to gain approval for a health claim on food, so that further investment in research is encouraged.

ISAPP is a non-profit scientific society that brings together independent academic and industrial scientists involved in research on fundamental and applied aspects of probiotics and prebiotics, to forward its mission of fostering high-quality research and scientific communication in the fields of probiotics and prebiotics. ISAPP is supported by contributions from members of its Industry Advisory Committee.