Skip to main content Accessibility help
×
Home
Hostname: page-component-684899dbb8-7wlv9 Total loading time: 0.55 Render date: 2022-05-22T01:49:07.971Z Has data issue: true Feature Flags: { "shouldUseShareProductTool": true, "shouldUseHypothesis": true, "isUnsiloEnabled": true, "useRatesEcommerce": false, "useNewApi": true }

EMA and FDA psychiatric drug trial guidelines: assessment of guideline development and trial design recommendations

Published online by Cambridge University Press:  30 April 2021

Kim Boesen*
Affiliation:
Nordic Cochrane Centre, Rigshospitalet, Dept. 7811, Copenhagen, Denmark Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, CA, USA Meta-Research Innovation Center Berlin (METRIC-B), Berlin Institute of Health, Charité Universitätsmedizin, Berlin, Germany
Peter C. Gøtzsche
Affiliation:
Institute for Scientific Freedom, 2970 Copenhagen, Denmark
John P. A. Ioannidis
Affiliation:
Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, CA, USA Meta-Research Innovation Center Berlin (METRIC-B), Berlin Institute of Health, Charité Universitätsmedizin, Berlin, Germany Departments of Medicine, of Epidemiology and Population Health, of Biomedical Data Science, and of Statistics, Stanford University, Stanford, CA, USA
*
Author for correspondence: Kim Boesen, E-mail: kim.boesen@charite.de
Rights & Permissions[Opens in a new window]

Abstract

Aims

The European Medicines Agency (EMA) and the US Food and Drug Administration (FDA) produce guidelines for the design of pivotal psychiatric drug trials used in new drug applications. It is unknown who are involved in the guideline development and what specific trial design recommendations they give.

Methods

Cross-sectional study of EMA Clinical Efficacy and Safety Guidelines and FDA Guidance Documents. Study outcomes: (1) guideline committee members and declared conflicts of interest; (2) guideline development and organisation of commenting phases; (3) categorisation of stakeholders who comment on draft and final guidelines according to conflicts of interest (‘industry’, ‘not-industry but with industry-related conflicts’, ‘independent’, ‘unclear’); and (4) trial design recommendations (trial duration, psychiatric comorbidity, ‘enriched design’, efficacy outcomes, comparator choice). Protocol registration https://doi.org/10.1101/2020.01.22.20018499 (27 January 2020).

Results

We included 13 EMA and five FDA guidelines covering 15 psychiatric indications. Eleven months after submission, the EMA had not processed our request regarding committee member disclosures. FDA offices draft the Guidance Documents, but the Agency is not in possession of employee conflicts of interest declarations because FDA employees generally may not hold financial interests (although some employees may hold interests up to $15,000). The EMA and FDA guideline development phases are similar; drafts and final versions are publicly announced and everybody can submit comments. Seventy stakeholders commented on ten guidelines: 38 (54%) ‘industry’, 18 (26%) ‘not-industry but with industry-related conflicts’, six (9%) ‘independent’ and eight (11%) ‘unclear’. They submitted 1014 comments: 640 (68%) ‘industry’, 243 (26%) ‘not-industry but with industry-related conflicts’, 44 (5%) ‘independent’ and 20 (2%) ‘unclear’ (67 could not be assigned to a specific stakeholder). The recommended designs were generally for trials of short duration; with restricted trial populations; allowing previous exposure to the drug; and often recommending rating scale efficacy outcomes. EMA mainly recommended three arm designs (both placebo and active comparators), whereas FDA mainly recommended placebo-controlled designs. There were also other important differences and FDA's recommendations regarding the exclusion of psychiatric comorbidity seemed less restrictive.

Conclusions

The EMA and FDA clinical research guidelines for psychiatric pivotal trials recommend designs that tend to have limited generalisability. Independent and non-conflicted stakeholders are underrepresented in the guideline development. It seems warranted with more active involvement of scientists and independent organisations without conflicts of interest in the guideline development process.

Type
Original Article
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0), which permits unrestricted re-use, distribution and reproduction, provided the original article is properly cited.
Copyright
Copyright © The Author(s), 2021. Published by Cambridge University Press

Background

Reports have found that newly authorised medicines often have questionable or no added patient-relevant benefits compared to older treatments (Motola et al., Reference Motola, Ponti, Poluzzi, Martini, Rossi, Silvana, Vaccheri and Montanaro2006; van Luijn et al., Reference Van Luijn, Gribnau and Leufkens2010; Prescrire, 2019; Wieseler et al., Reference Wieseler, McGauran and Kaiser2019; Erhel et al., Reference Erhel, Scanff and Naudet2020). In specialities like oncology, this seems to be particularly prevalent (Kim and Prasad, Reference Kim and Prasad2015; Davis et al., Reference Davis, Naci, Gurpinar, Poplavska, Pinto and Aggarwal2017; Gyawali et al., Reference Gyawali, Hey and Kesselheim2019; Naci et al., Reference Naci, Davis, Savovic, Higgins, Sterne, Gyawali, Romo-Sandoval, Handley and Booth2019). In a cohort of drugs approved by the German drug regulatory agency between 2011 and 2016, the Institute for Quality and Efficiency in Health Care (IQWiG) judged that 10% had substantial benefits compared to already available treatments, while 58% had no proof of added benefits (Wieseler et al., Reference Wieseler, McGauran and Kaiser2019). Out of 18 newly approved drugs in the combined field of psychiatry/neurology, only one was judged to have substantial added benefits compared to already available treatments (Wieseler et al., Reference Wieseler, McGauran and Kaiser2019).

Systematic reviews of commonly used psychiatric drugs, particularly of antidepressants for depression (Jakobsen et al., Reference Jakobsen, Katakam, Schou, Hellmuth, Stallknecht, Leth-Møller, Iversen, Banke, Petersen, Klingenberg, Krogh, Ebert, Timm, Lindschou and Gluud2017; Munkholm et al., Reference Munkholm, Paludan-Müller and Boesen2019) and of central stimulants for attention deficit hyperactivity disorder (ADHD) (Storebø et al., Reference Storebø, Ramstad, Krogh, Nilausen, Skoog, Holmskov, Rosendal, Groth, Magnussen, Moreira-Maia, Gillies, Rasmussen, Gauci, Zwi, Kirubakran, Forsbøll, Simonsen and Gluud2015; Punja et al., Reference Punja, Shamseer, Hartling, Urichuk, Vandermeer and Nikles2016; Castells et al., Reference Castells, Blanco-Silvente and Cunill2018; Cândido et al., Reference Cândido, Menezes de Padua, Golder and Junqueira2021), have highlighted potential problems with low generalisability of psychiatric drug trials. Methodological limitations include small sample sizes, short trial durations, restricted trial populations in terms of allowed psychiatric comorbidity, risk of withdrawal effects due to previous exposure to the drug of interest, and use of surrogates and rating scales rather than patient-relevant outcomes.

Researchers have advocated for psychiatric pivotal trials, i.e. clinical trials conducted with the purpose of obtaining a marketing authorisation for a new drug or a new indication, with active comparators (i.e. already approved medications instead of placebo only) and a focus on hard, functional outcomes such as hospitalisation admissions and suicide rather than symptom rating scales (Barbui et al., Reference Barbui, Cipriani, Lintas, Bertelé and Garattini2007; Barbui and Garattini, Reference Barbui and Garattini2007; Barbui and Bighelli, Reference Barbui and Bighelli2013). An assessment of the evidence base for new psychiatric drug approvals in Europe categorised the general evidence as being ‘poor’ (Erhel et al., Reference Erhel, Scanff and Naudet2020), and the European Medicines Agency (EMA)'s guidelines for designing pivotal psychiatric drug trials have been highlighted as important to improve the usefulness of these trials (Barbui and Bighelli, Reference Barbui and Bighelli2013). On a more general note, it has been argued that the threshold for new drug approvals has been lowered to accommodate the pharmaceutical industry's interests (Davis et al., Reference Davis, Lexchin, Jefferson, Gøtzsche and McKee2016).

Pivotal trials are often designed in a bilateral agreement between the pharmaceutical company and the drug regulator (Fig. 1). The EMA and the US Food and Drug Administration (FDA) publish guidelines on how to design and conduct these pivotal trials for new drug approvals and these guidelines offer a roadmap in supporting the design of pivotal trials. EMA publishes ‘Clinical Efficacy and Safety Guidelines’ (EMA, 2020a), which the agency ‘strongly encourages’ their applicants to follow. These guidelines ‘reflect a harmonised approach of the EU Member States and the Agency on how to interpret and apply the requirements for the demonstration of quality, safety and efficacy []’ (EMA, 2020b). The FDA publishes ‘Guidance Documents’ (FDA, 2021). These documents are ‘intended to assist the pharmaceutical industry’ in designing and conducting pivotal trials, but they are not ‘legally binding or enforceable’ (FDA, 2005). The documents ‘represent current Agency thinking, sponsor submissions that conform to current guidance should be considered acceptable’ (FDA, 2005). Pharmaceutical companies and regulatory agencies may communicate at any stage of medicines development, e.g. EMA provides ‘scientific advice’ (EMA, 2021a).

Fig. 1. The role of regulatory guidelines in pivotal trial designs. (1) The sponsor approaches the relevant drug regulator with the purpose of getting a new drug authorised. (2) Sponsor and regulator discuss (and often agree) on a drug development program. (3) The design and conduct of the pivotal trials will often follow design recommendations outlined in regulatory guidelines. (4) The sponsor conducts the pivotal trials. 5) The acquired data are submitted to the regulators as a Marketing Authorisation Application (EMA) or a New Drug Application (FDA) using the Common Technical Format. (6) The regulator reviews the data, and, if necessary, steps 2, 3 and 4 are repeated to obtain additional information. (7) The regulator decides whether to authorise or reject the application. Figure based on FDA (2019d); EMA (2021b).

Some of EMA's trial design recommendations (choice of comparator, outcomes, and patient population) for psychiatric drug trials have previously been described (Barbui and Bighelli, Reference Barbui and Bighelli2013). However, the development and content of EMA and FDA guidelines for psychiatric drug trials have never been systematically assessed. We wanted to assess (1) the guideline committee members and their declared conflicts of interest; (2) the organisation of commenting phases on draft and concept papers; (3) the stakeholders who comment on guideline draft and concept papers; and (4) the EMA and FDA design recommendations for five trial characteristics in pivotal psychiatric drug trials.

Methods

Inclusion of drug regulatory agency guidelines

We applied the following inclusion criteria: (1) the guideline (draft or final version) should be published by the EMA or FDA on how to design pivotal trials for drug approval; (2) the guideline should cover all aspects of pivotal trial design (i.e. what the FDA calls ‘umbrella guidance’) and not address specific trial aspects only, e.g. inclusion criteria or specific outcome measures; and (3) the guideline should cover psychiatric diagnoses as defined in current or previous versions of the Diagnostics and Statistics Manual of Mental Disorders (DSM) or the International Classification of Diseases (ICD). We included the newest guideline (draft or final version) if more than one guideline was published on the same subject. See eMethods for details.

Guideline committee members' conflicts of interest

The guideline committee members and/or EMA and FDA employers that are responsible for drafting and revising the guidelines are not publicly disclosed. Similarly, the committee members' declared conflicts of interest are not publicly available. We therefore filed Freedom of Information Act requests to the EMA and FDA to access the committee member lists and their declared conflicts of interest. We wanted to report the proportion of committee members with declared conflicts of interest for each guideline.

Guideline development

We assessed the development of draft guidelines in regards to, (1) public availability of draft versions for commenting; (2) active recruitment of specific stakeholders to comment on drafts; (3) applied restrictions on who may comment on the documents; (4) announcement of drafts and final guidelines, e.g. on social media, in newsletters or in relevant journals.

We searched the EMA and FDA websites for information. It was not necessary to contact the regulators for additional information. See eMethods for details.

Stakeholder comment documents

EMA publishes dedicated documents with stakeholder comments submitted on draft guidelines along the final guidelines. The stakeholder comments and EMA's corresponding replies are collated in a single document and made available for each guideline (EMA, 2020a).

FDA manages and operates public correspondence on federal regulations and other rule-making documents through the Docket's Management (US Government, 2020). All official US Federal documents, such as drafts or final Guidance Documents, have their own designated entry, called a Docket Folder Summary, where the public can submit and also view submitted comments on the Guidance Documents.

For each included stakeholder document related to a specific guideline, we counted the total number of stakeholders, total number of comments, and total number of comments by each stakeholder. We categorised the stakeholders according to their conflicts of interest as ‘industry’ (e.g. pharmaceutical companies or other industry), ‘not-industry but with industry-related conflicts’ (e.g. organisations, associations or individuals reporting financial conflicts of interest related to pharmaceutical companies), ‘independent’ (organisations, associations and individuals with declared no conflicts of interest related to the pharmaceutical industry) or ‘unclear financial relationship’ (there was insufficient information to identify and/or categorise the stakeholder). We searched for information on Google, PubMed, OpenPaymentsData, and stakeholder websites regarding funding, annual budgets, and disclosed conflicts of interest. For organisations and associations, we focused on board and executive members. We did not define a lower monetary limit for conflicts of interest. See eMethods for details.

Trial design recommendations

For all included research guidelines, we extracted recommendations regarding five trial design characteristics: (1) duration of follow-up, also after the randomised phase ended; (2) exclusion criteria related to psychiatric comorbidity; (3) allowing previous exposure to the drug or drug class, which may be called an ‘enriched design’ or ‘enrichment strategy’ (FDA, 2019a); (4) efficacy outcomes (primary and secondary); and (5) choice of comparator (e.g. placebo, active comparator, or both).

We wanted to put the trial design recommendation in a clinical context for each of the psychiatric diagnoses and corresponding guidelines, i.e. what is the natural history of the different conditions in terms of expected symptom duration; what are the most common comorbid psychiatric diagnoses and how common are they; and whether previous exposure to the drug might introduce potential withdrawal effects and also bias the sample of participants in favour of those who tolerate the drug. We extracted this basic clinical epidemiology information from the guidelines, when available. We further searched the Core Outcome Measures in Effectiveness Trials (COMET) database (COMET, 2020) for standardised sets of outcome measures for the individual indications, and we assessed the most recent systematic review (Eiring et al., Reference Eiring, Landmark, Aas, Salkeld, Nylenna and Nytrøen2015) of patient preferences for outcome measures in psychiatric drug trials.

See the Supplementary material for differences between the protocol (Boesen et al., Reference Boesen, Gøtzsche and Ioannidis2020) and the final manuscript.

Results

Overview of results

We searched the EMA and FDA websites in February 2020 and identified 13 eligible EMA Clinical Efficacy and Safety Guidelines and five FDA Guidance Documents (Table 1). The 13 EMA guidelines were published between 2005 and 2017; the panic disorder guideline could not be downloaded. All EMA guidelines were adopted (i.e. final) versions, two (insomnia and schizophrenia) had been revised once, and the depression guideline had been revised twice. Concept papers for upcoming revisions were published for the depression and bipolar disorder guidelines. The five FDA guidelines were published between 2015 and 2019; four were draft versions and one (opioid use disorder) was a final guideline. The 18 guidelines covered 15 unique indications (depression, ADHD, and alcohol dependence were covered by both agencies). The full dataset is available here (https://osf.io/3xcdu/?view_only = 957edc6293894497ba7aaf5bb8fc8205).

Table 1. Overview of included regulatory research guidelines

Listed in order of publication year.

a A concept paper for a revision of the guideline has been published.

b The link did not work so we could not include the guideline. Links to the guidelines can be found in Supplement 1.

Guideline committee members' conflicts of interests

The EMA informed us they were not able to respond within their regular response time caused by EMA's relocation to Amsterdam. After 11 months from submission, EMA had not processed our request of access to committee member lists. See correspondence with EMA (Supplement 2).

The FDA Freedom of Information Team informed us during a telephone meeting that, (1) federal employees are not allowed to have financial conflicts of interest; (2) Guidance Documents are authored internally in the Agency by divisions or offices and not by individual employees; and (3) therefore the FDA is not in possession of conflicts of interest declarations related to Guidance Document development (see summary of the meeting in Supplement 1). We partly confirmed statement (1) from FDA's website (FDA, 2017a), where it is stated that FDA employees are generally prohibited from holding financial interests, e.g. stocks, in any FDA ‘significantly regulated organisation’. However, there are exceptions to the rules. For instance, only designated FDA employees, called ‘confidential filers’, are required to disclose conflicts of interest. Employees that are not required to disclose their financial conflicts of interest may hold financial interests up to US$15,000 under certain conditions (FDA, 2017a). We found some information relevant to statements (2) and (3) in the FDA Manuals of Policies and Procedures (MAPPs) ‘Developing and Issuing Guidance’ (FDA, 2005) and ‘Developing indication-specific Guidances’ (FDA, 2014), specifically that offices and divisions are responsible for writing the Guidances (FDA, 2014). See correspondence with the FDA in Supplement 2.

Guideline development

EMA and FDA produce their research guidelines in a similar fashion, summarised in Table 2.

Table 2. Overview of commenting phases on regulatory guidelines

Exact references and verbatim statements can be found in the full dataset (https://osf.io/3xcdu/?view_only=957edc6293894497ba7aaf5bb8fc8205).

EMA guidelines

The development of EMA's Clinical Efficacy and Safety Guidelines is primarily described in the guideline ‘Procedure for European Union guidelines and related documents within the pharmaceutical legislative framework’ (EMA, 2009), to which stakeholder comments have also been published (EMA, 2005). There are three stages where stakeholders may comment on the guidelines during the development: (1) first, a concept paper, which outlines the overall purpose of the guideline, is made available for commenting for 2–3 months. It is unclear if, and when, a concept paper is written with input from outside stakeholders or comments are actively solicited (EMA, 2009; section 4.4); (2) then a draft guideline is made available for comments, usually for 3–6 months. Also for this stage, it is unclear if and when comments are actively solicited (EMA, 2009; section 4.6); finally (3) the adopted guideline is published and may be commented on at any given time. EMA announces all documents for public consultation on a designated website called ‘Open consultations’ (EMA, 2020c).

FDA Guidance Documents

The Guidance development is described in the FDA Manual of Policies and Procedures ‘Developing and Issuing Guidance’ (FDA, 2005) and ‘Developing indication-specific Guidances’ (FDA, 2014), the Code of Federal Regulation, Part 10, Section 10.115 (CFR 2019), and FDA's report on Guidance Documents (FDA, 2011). There are two stages, possibly three if the draft development is included, where stakeholders may comment on Guidance Documents: (1) First, FDA develops a draft Guidance. This can be done in collaboration with individuals or organisations outside FDA, e.g. through input from workshops or Advisory Committees. Stakeholders may also submit draft proposals directly to FDA; (2) a draft Guidance Document is made publicly available for comments usually for 60 days through announcement (called a ‘notice of availability’) in the Federal Register (Federal Register, 2020), the official journal of the United States Government, and a designated website (FDA, 2020a); (3) the final Guidance Document is published and announced in the Federal Register and online and may be commented on at any given time. Every year, FDA also publishes a ‘Guidance Agenda’, a list of guidances they plan to publish or revise during the year (FDA, 2020b).

Stakeholder comments

There were stakeholder comments available for six of the 13 EMA research guidelines and four of the five FDA guidelines. Seventy stakeholders submitted a total of 1014 comments, of which 947 comments could be assigned to a stakeholder. For 67 comments the stakeholder was unknown, as EMA's post-traumatic stress disorder (PTSD) guideline did not assign the individual comments to the stakeholder. There were 52 unique stakeholders of which two, International Society for CNS Clinical Trials and Methodology and Lundbeck, commented on both EMA and FDA documents. See the full dataset for details.

EMA guidelines

Thirty-eight stakeholders made 771 comments on six EMA guidelines. There were 25 unique stakeholders; 19 stakeholders commented on one guideline and six stakeholders commented on two or more guidelines, Lundbeck and the European Federation of Pharmaceutical Industries and Associations (five guidelines), Merck Sharp & Dome (three), and European College of Neuropsychopharmacology, International Federation of Association of Pharmaceutical Physicians, and Hoffman-La Roche (two each). We categorised 25 (66%) stakeholders as ‘industry’, nine (24%) as ‘not-industry but with industry-related conflicts’ and four (10%) as ‘independent’ (Table 2). On the comment level, we categorised 534 (76%) comments as ‘industry’, 148 (21%) as ‘not industry but with industry-related conflicts’, 22 (3%) as ‘independent’ and 67 were not categorised (Table 3).

Table 3. Stakeholders categorised by conflicts of interest

FDA Guidance Documents

Thirty-two stakeholders made 243 comments on four FDA Guidance Documents. There were 29 unique stakeholders; 26 stakeholders commented on one document and three stakeholders, Lundbeck, Anthem Inc., and International Society for CNS Clinical Trials and Methodology, commented on two Guidance Documents each. We categorised 13 (41%) stakeholders as ‘industry’, nine (28%) as ‘not-industry but with industry-related conflicts’, two (6%) as ‘independent’ and eight (25%) as ‘unclear’ (Table 2). On the comment level, we categorised 106 comments (44%) as ‘industry’, 95 (39%) as ‘not-industry but with industry-related conflicts’, 22 (9%) as independent and 20 (8%) as ‘unclear’ (Table 4).

Table 4. Stakeholder comments categorised by conflicts of interest

a The comments were not assigned to the individual stakeholders.

b Calculated as the proportion of assigned comments, not the total number of comments.

Trial design recommendations

The EMA guidelines provided recommendations for all five trial characteristics, i.e. trial duration, psychiatric comorbidity, ‘enriched design’, efficacy outcomes, and choice of comparator in all guidelines, except ‘enriched design’ (alcohol dependence) and psychiatric comorbidity (bipolar disorder). None of the five FDA Guidance Documents contained recommendations for all five trial characteristics; for example, previous exposure to relevant medications was described in one guideline only (opioid use disorder). The trial recommendations for each guideline are summarised in Supplement 1, eTable 1–15.

Overlapping guidelines

EMA and FDA published overlapping guidelines on ADHD (eTable 3), depression (eTable 5), and alcohol dependence (eTable 10). The three FDA guidelines were draft versions and did not contain recommendations on ‘enriched design’ or on psychiatric comorbidity in the ADHD guideline. The overlapping guidelines concurred on trial durations for depression and alcohol dependence, e.g. EMA recommended 4–8 weeks and FDA 6–8 weeks in depression trials, and both agencies recommended a 6-month randomised withdrawal design to assess long-term outcomes. The EMA guidelines seemed more restrictive regarding psychiatric comorbidity, whereas the FDA depression and alcohol dependence guidelines emphasised the inclusion of relevant comorbidity. The guidelines agreed on the use of rating scales in ADHD and depression trials as efficacy outcomes and on the use of full abstinence or no heavy drinking for alcohol dependence.

Trial duration

EMA recommended short- and long-term trial durations in all 12 available guidelines. Short-term durations ranged from 2 to 14 weeks, with the exception of alcohol dependence (3–6 months) and premenstrual dysphoric syndrome (6 months). Long-term durations ranged from 3 to 15 months, of which seven guidelines recommended the randomised withdrawal design, three guidelines (ADHD, insomnia, alcohol dependence) recommended a placebo-controlled trial or the randomised withdrawal design, and two guidelines did not recommend any particular design (bipolar disorder and premenstrual dysphoric disorder).

The FDA recommendations were more varied; 6–8 weeks for depression, 24 weeks for ‘low sexual interest and/or arousal in women’, 6 months for alcohol dependence, and for ADHD they only gave a recommendation for long-term safety trials of 12 months to assess the effect on growth in children.

Psychiatric comorbidity

EMA listed extensive exclusion criteria related to psychiatric comorbidity in ten of the 12 available guidelines. Two guidelines (bipolar disorder and depression) did not specify exclusion criteria.

The three FDA Guidance Documents (depression, alcohol dependence, ‘low sexual interest and/or arousal in women’) that contained recommendations on psychiatric comorbidity were less restrictive than EMA's guidelines and stated that unnecessary restrictions should be avoided.

Enriched design

All EMA guidelines, except alcohol dependence, specified that participants could have been exposed to relevant medication prior to enrolment and there should be a ‘washout’ of such drugs.

Only one FDA Guidance Document (opioid use disorder) referred to previous exposure by stating, ‘patients should be either new entrants to treatment or stable on other treatments’.

Efficacy outcomes

Ten EMA guidelines recommended symptom-rating scales as efficacy measures. The alcohol dependence guideline recommended full abstinence or no heavy drinking, and the insomnia guideline recommended self-rated sleep, polysomnography, and quality of life. Several guidelines stipulated the use of ‘responder analyses’, i.e. dichotomising the participants as ‘responders’ or ‘non-responders’ based on their treatment response on symptom rating scales using arbitrary thresholds.

The FDA Guidance Documents recommended rating scales for two indications (depression and ADHD), no heavy drinking or full abstinence (alcohol dependence), urine toxicology (opioid use disorder), and number of satisfying sexual events or self-rating of sexual interests, arousal, and distress (low sexual interest and/or arousal in women).

Choice of comparator

Ten EMA guidelines recommended three-arm trial designs with both placebo and an active comparator. Two guidelines (alcohol dependence and premenstrual dysphoric syndrome) recommended a two-arm placebo design, but also mentioned three-arm designs as a viable option.

Four FDA Guidance Documents recommended a two-arm design with placebo (ADHD, depression, low sexual interest and/or arousal in women, and alcohol abuse), and one guideline recommended either placebo or an active comparator (opioid use disorder). The alcohol abuse guideline also mentioned the possibility of a three-arm trial design.

Discussion

Principal findings

To the best of our knowledge, this is the first study to systematically assess the development and content of regulatory research guidelines for psychiatric drugs. Some EMA trial design recommendations have previously been assessed, mainly focusing on the choice of comparator (Barbui and Bighelli, Reference Barbui and Bighelli2013). These regulatory guidelines are intended for industry, but the dearth of non-industry stakeholders was surprising considering the guideline process being apparently open to public comments. The FDA launched in 2010 a ‘Transparency Initiative’ (FDA, 2011) and one of the objects was to improve stakeholder involvement. FDA's stakeholder composition was indeed more varied than EMA's, although still dominated by stakeholders with conflicts of interest. EMA stated in their procedure guideline (EMA, 2009) that they would develop procedures to ensure proactive consultation of patients and patient organisations. For this cohort of guidelines, there was an absence of such stakeholders.

The guidelines recommend the design of clinical trials with limited generalisability, which may contribute to research waste (Chalmers and Glasziou, Reference Chalmer and Glasziou2009). Most recommended trial durations were short and both agencies recommended the randomised withdrawal design for long-term trials. In such a trial, participants who benefit from a drug either in a placebo-controlled trial or in an open-label phase are randomised to continued active drug or placebo (FDA, 2019a; p. 18). Abrupt stopping of psychotropic drugs can lead to withdrawal symptoms, e.g. after use of antidepressants (RCP, 2019) or central stimulants, which carries an FDA black box label warning about the risk of drug dependence (FDA, 2017b). If these drugs are not tapered off slowly, the withdrawal symptoms may be misinterpreted as symptom deterioration and superiority of the drug over placebo. In FDA's opioid use disorder guideline it was stated, ‘Sponsors can conduct trials in patients already stable on other treatments’. This indicates that participants who are adequately treated before the trial may be randomised to placebo and thereby risk experiencing withdrawal symptoms. In EMA's schizophrenia guideline it was stated, ‘typically a few days will be appropriate’ for washing out prior antipsychotic medication during a placebo lead-in, which seems to be insufficient to avoid withdrawal symptoms.

All but two EMA guidelines allowed participants with previous exposure to the drugs to be included in the pivotal trials. Before their inclusion, such participants should ‘wash out’ the drug, i.e. stop the treatment for a certain amount of time. The sample would be ‘enriched’ because participants with a previous negative response would not be allowed to participate, either due to explicit exclusion criteria and/or because participants with a negative response would not want to participate. This trial design detail may seem trivial but it can likely lead to substantial overestimation of the benefits and underestimations of the harms.

Most EMA guidelines recommended extensive exclusion criteria regarding psychiatric comorbidity. The obsessive-compulsive disorder (OCD) guideline described that the population should be ‘pure’ OCD but the guideline also highlighted, ‘lifetime co-morbidity rates of other psychiatric diseases in patients with OCD range from 75 to 84%’. The ADHD guideline mentioned, ‘another Axis I disorder’, ‘severe comorbid symptoms such as depression and anxiety’ or ‘primary axis II disorder’ as exclusion criteria. Such exclusion criteria may reduce the trials' generalisability, e.g. most patients diagnosed with ADHD (Surman et al., Reference Surman, Monuteaux, Petty, Faraone, Spencer, Chu and Biederman2010) or depression (Zimmerman et al., Reference Zimmerman, Balling, Chelminski and Dalrymple2019) in a clinical setting are not eligible for inclusion in pivotal clinical trials. The FDA Guidance inclusion criteria were less restrictive and emphasised the inclusion of participants with comorbidities. In 2019, FDA published a draft guideline entitled ‘Enhancing the diversity of clinical trial populations’ (FDA, 2019b) and the Agency seems cognisant of the problem of restricted trial populations, ‘Sponsors should adopt practices for determining eligibility criteria that will allow the clinical trial population to reflect the diversity of the patients who will be using the drug if the drug is approved’ (FDA, 2019b).

We found that EMA and FDA mostly recommended surrogate outcomes. A recent analysis (Hey et al., Reference Hey, Kesselheim, Patel, Mehrotra and Powers2020) similarly reported that 21 of 27 FDA anti-infective Guidance Documents recommended surrogate outcomes rather than patient-relevant outcomes such as mortality.

Finally, we noted that EMA mostly recommended three-arm trial designs with both placebo and active comparator. In contrast, the FDA mostly recommended two-arm placebo-controlled designs, particularly notable for ADHD and depression trials, where many approved medications are available. EMA has previously been encouraged to require comparative evidence prior to regulatory approval (Sorenson et al., Reference Sorenson, Naci, Cylus and Mossialos2011; Barbui and Bighelli, Reference Barbui and Bighelli2013; Wieseler et al., Reference Wieseler, McGauran and Kaiser2019), and such efforts seem warranted for FDA as well.

Implication for practice and research

We identified several cases where the agencies seemed to prioritise the interests of the applicants rather than the public. For instance, FDA wrote in their opioid use disorder guideline, ‘There is great public health interest in assessing additional, clinically meaningful endpoints such as reduction in hospitalizations, emergency department visits, overdose, and death, as well as improvements in the ability to resume work, school, or other productive activity. Though understanding these outcomes would be highly valuable, the Agency recognizes that evaluating these outcomes could require larger trials than those usually conducted for marketing approval’ (FDA, 2019c). If the agencies wish to serve the public, the required pivotal trials should reflect the interests of the public and thus address real-world patient populations and patient-centred outcomes.

FDA informed us that their employees cannot have conflicts of interest. The FDA policy allows non-designated employees to have assets up to US$15 000 (FDA, 2017a). Eleven months after we submitted our request regarding access to the committee members' conflicts of interest, EMA had not processed it. It would be useful for regulatory agencies to make such information publicly available to increase transparency.

Study limitations

The sample of guidelines was small and there was an uneven distribution of stakeholders and comments. There were challenges in counting the number of FDA comments due to the non-uniform format, and we were unable to access a few guidelines and stakeholder documents. We recognise the large variations in the stakeholder comments and that different criteria for quantifying the comments would yield different counts. However, count differences would likely be minor and unlikely to change the overall results. We were also unable to identify several of the individual commenters who submitted comments on the FDA Guidance Documents, since their affiliations were not disclosed. We also did not compare draft and final guidelines to assess if, and how, the submitted stakeholder comments might have served specific industry interests. We believed that such analyses would be too subjective.

It was a challenge to separate the stakeholders according to their conflicts of interest, especially patient groups and organisations. Some evidence points to a dose–response relationship between conflicts of interest and drug prescription patterns (Perlis and Perlis, Reference Perlis and Perlis2016), and a recent systematic review reported that industry funding of patient organisations is common (Fabbri et al., Reference Fabbri, Parker, Colombo, Mosconi, Barbara, Frattaruolo, Lau, Kroeger, Lunny, Salzwedel and Mintzes2020). We did therefore not define a lower monetary threshold, but categorised stakeholders as ‘not-industry but with industry-related conflicts’ if we identified any conflicts of interest.

Conclusion

The EMA and FDA clinical research guidelines for psychiatric pivotal trials recommend designs that tend to have limited generalisability. Independent and non-conflicted stakeholders are underrepresented in the development phases and current guidelines emphasise trials with limited scope that may not offer much clinical value. EMA and FDA should reconsider their guideline development and find ways to promote greater involvement of the public and independent stakeholders. This may require greater advertisement of the guideline process to the public and active invitation of large numbers of scientists and organisations that are known to have no conflicts of interest to comment on the guidelines.

Supplementary material

The supplementary material for this article can be found at https://doi.org/10.1017/S2045796021000147.

Data

The project protocol was made available on 27 January 2020 on MedRxiv (https://www.medrxiv.org/content/10.1101/2020.01.22.20018499v1). The full dataset is available from the Open Science Framework (https://osf.io/3xcdu/?view_only = 957edc6293894497ba7aaf5bb8fc8205). Questions regarding the dataset should be referred to the corresponding author.

Acknowledgements

We thank the Meta-Research Innovation Center at Stanford (METRICS) lab meeting participants (Nov 2019) for comments on the draft protocol.

Author contributions

KB conceived the idea, collected and analysed data, corresponded with the drug regulatory agencies, and wrote the first draft. JPAI and PCG contributed to the design of the study, commented and supervised the final manuscript. All authors critically revised the final manuscript. All authors approved the final version. KB is the guarantor.

Financial support

METRICS is supported by a grant from the Laura and John Arnold Foundation and METRIC-B is supported by a visiting Einstein fellowship in the Berlin Institute of Health to JPAI from the Einstein Foundation and the Stiftung Charité.

Conflict of interest

None.

Ethical standards

Not applicable.

Footnotes

*

Current address: Meta-Research Innovation Center Berlin (METRIC-B), Berlin Institute of Health, Charité Universitätsmedizin, QUEST Center for Transforming Biomedical Research, Charité Platz 1, 10117 Berlin, Germany

References

Barbui, C and Bighelli, I (2013) A new approach to psychiatric drug approval in Europe. PLoS Medicine 10, e1001530.CrossRefGoogle ScholarPubMed
Barbui, C and Garattini, S (2007) Regulatory policies on medicines for psychiatric disorders: is Europe on target? The British Journal of Psychiatry 190, 9193.CrossRefGoogle ScholarPubMed
Barbui, C, Cipriani, A, Lintas, C, Bertelé, V and Garattini, S (2007) CNS drugs approved by the centralised European procedure: true innovation or dangerous stagnation? Psychopharmacology 190, 265268.CrossRefGoogle ScholarPubMed
Boesen, K, Gøtzsche, PC and Ioannidis, JPA (2020) FDA and EMA clinical research guidelines: assessment of trial design recommendations for pivotal psychiatric drug trials (Protocol). MedRxiv 27 January 2020. Available at https://www.medrxiv.org/content/10.1101/2020.01.22.20018499v1 (accessed 13 Jan 2021).CrossRefGoogle Scholar
Cândido, RCF, Menezes de Padua, CA, Golder, S and Junqueira, DR (2021) Immediate-release methylphenidate for attention deficit hyperactivity disorder (ADHD) in adults. Cochrane Database of Systematic Reviews 1, CD013011.Google ScholarPubMed
Castells, X, Blanco-Silvente, L and Cunill, R (2018) Amphetamines for attention deficit hyperactivity disorder (ADHD) in adults. Cochrane Database of Systematic Reviews 8, CD007813.Google ScholarPubMed
CFR (2019) Code of Federal Regulations, Title 21, Volume 1. Section 10.115 Good guidance practice. Revised 1 April 2020. Available at https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfcfr/CFRSearch.cfm?fr = 10.115 (Feb 2021).Google Scholar
Chalmer, I and Glasziou, P (2009) Avoidable waste in the production and reporting of research evidence. Lancet (London, England) 374, 8689.CrossRefGoogle Scholar
COMET (2020) COMET initiative. Core Outcome Measures in Effectiveness Trials. Available at http://www.comet-initiative.org/.Google Scholar
Davis, C, Lexchin, J, Jefferson, T, Gøtzsche, P and McKee, M (2016) ‘Adaptive pathways’ to drug authorisation: adapting to industry? BMJ 354, i4437.CrossRefGoogle Scholar
Davis, C, Naci, H, Gurpinar, E, Poplavska, E, Pinto, A and Aggarwal, A (2017) Availability of evidence of benefits on overall survival and quality of life of cancer drugs approved by European Medicines Agency: retrospective cohort study of drug approvals 2009–13. BMJ 359, j4530.CrossRefGoogle ScholarPubMed
Eiring, Ø, Landmark, BF, Aas, E, Salkeld, G, Nylenna, M and Nytrøen, K (2015) What matters to patients? A systematic review of preferences for medication-associated outcomes in mental disorders. BMJ Open 5, e007848.CrossRefGoogle ScholarPubMed
EMA (2005) European Medicines Agency. Overview of comments received on draft guideline ‘Procedure for EU guidelines and related documents within the pharmaceutical legislative framework’. 24 June 2005. Available at https://www.ema.europa.eu/en/documents/other/overview-comments-received-draft-guideline-procedure-eu-guidelines-related-documents-within_en.pdf (Feb 2021).Google Scholar
EMA (2009) European Medicines Agency. Procedure for European Union guidelines and related documents within the pharmaceutical legislative framework. 30 January 2009. Available at https://www.ema.europa.eu/en/documents/scientific-guideline/procedure-european-union-guidelines-related-documents-within-pharmaceutical-legislative-framework_en.pdf (Feb 2021).Google Scholar
EMA (2020a) European Medicines Agency. Clinical efficacy and safety guidelines. Available at https://www.ema.europa.eu/en/human-regulatory/research-development/scientific-guidelines/clinical-efficacy-safety-guidelines (Feb 2021).Google Scholar
EMA (2020b) European Medicines Agency. Scientific guidelines. No date. Available at https://www.ema.europa.eu/en/human-regulatory/research-development/scientific-guidelines (Feb 2021).Google Scholar
EMA (2020c) European Medicines Agency. Open consultations website. Available at (https://www.ema.europa.eu/en/news-events/open-consultations).Google Scholar
EMA (2021a) European Medicines Agency. Scientific advice and protocol assistance. No date. Available at https://www.ema.europa.eu/en/human-regulatory/research-development/scientific-advice-protocol-assistance (Jan 2021).Google Scholar
EMA (2021b). European Medicines Agency. Obtaining an EU marketing authorisation, step-by-step. No date. Available at https://www.ema.europa.eu/en/human-regulatory/marketing-authorisation/obtaining-eu-marketing-authorisation-step-step (Feb 2021).Google Scholar
Erhel, F, Scanff, A and Naudet, F (2020) The evidence base for psychotropic drugs approved by the European Medicines Agency: a meta-assessment of all European Public Assessment Reports. Epidemiology and Psychiatric Sciences 29, e120.CrossRefGoogle ScholarPubMed
Fabbri, A, Parker, L, Colombo, C, Mosconi, P, Barbara, G, Frattaruolo, MP, Lau, E, Kroeger, CM, Lunny, C, Salzwedel, DM and Mintzes, B (2020) Industry funding of patient and health consumer organisations: systematic review with meta-analysis. BMJ 368, I6925.CrossRefGoogle ScholarPubMed
FDA (2005) Center for Drug Evaluation and Research. Developing and issuing guidance. MAPP 4000.2. September 2005. Available at https://www.fda.gov/media/71702/download (Feb 2021).Google Scholar
FDA (2011) US Food and Drug Administration. Report on good guidance documents. Improving efficiency and transparency. December 2011. Available at https://www.fda.gov/media/82644/download (Feb 2021).Google Scholar
FDA (2014) Center for Drug Evaluation and Research. Developing indication-specific guidance. MAPP 4000.10 Rev. 1. November 2014. Available at https://www.fda.gov/media/71717/download (Feb 2021).Google Scholar
FDA (2017a) US Food and Drug Administration. Fact sheet: prohibited financial interests for FDA employees. October 2017. Available at https://www.fda.gov/about-fda/ethics/fact-sheet-prohibited-financial-interests-fda-employees (Feb 2021).Google Scholar
FDA (2017b) US Food and Drug Administration. Concerta prescribing information. January 2017. Available at https://www.accessdata.fda.gov/drugsatfda_docs/label/2017/021121s038lbl.pdf (Feb 2021).Google Scholar
FDA (2019a) US Food and Drug Administration. Enrichment strategies for clinical trials to support determination of effectiveness of human drugs and biological products guidance for industry. March 2019. Available at https://www.fda.gov/media/121320/download (Feb 2021).Google Scholar
FDA (2019b) US Food and Drug Administration. Enhancing the diversity of clinical trial populations – eligibility criteria, enrollment practices, and trial designs. Guidance for industry. Draft guidance. June 2019. Available at https://www.fda.gov/media/127712/download (Accessed 12 March 2020).Google Scholar
FDA (2019c) Center for Drug Evaluation and Research. Opioid use disorder: developing depot buprenorphine products for treatment guidance for industry. Feb 2019. Available at https://www.fda.gov/media/120090/download (Feb 2021).Google Scholar
FDA (2019d) New drug application. Current as of 6 Oct 2019. Available at https://www.fda.gov/drugs/types-applications/new-drug-application-nda (Feb 2021).Google Scholar
FDA (2020a) US Food and Drug Administration. Newly added guidance documents. Updated 29 June 2020. Available at https://www.fda.gov/drugs/guidances-drugs/newly-added-guidance-documents. (Feb 2021).Google Scholar
FDA (2020b) US Food and Drug Administration. CDER guidance agenda. New & revised draft guidances documents planned for publication in calendar year 2021. January 2021. Available at https://www.fda.gov/media/134778/download (Feb 2021).Google Scholar
FDA (2021) US Food and Drug Administration. Search for FDA guidance documents. 2 Feb 2021. Available at https://www.fda.gov/regulatory-information/search-fda-guidance-documents (Feb 2021).Google Scholar
Federal Register (2020) The daily journal of the United States Government. Available at https://www.federalregister.gov/agencies/food-and-drug-administration.Google Scholar
Gyawali, B, Hey, SP and Kesselheim, AS (2019) Assessment of the clinical benefit of cancer drugs receiving accelerated approval. JAMA Internal Medicine 179, 906913.CrossRefGoogle ScholarPubMed
Hey, SP, Kesselheim, AS, Patel, P, Mehrotra, P and Powers, JH (2020) US Food and Drug Administration recommendations on the use of surrogate measures as end points in new anti-infective drug approvals. JAMA Internal Medicine 180, 131138.CrossRefGoogle ScholarPubMed
Jakobsen, JC, Katakam, KK, Schou, A, Hellmuth, SG, Stallknecht, SE, Leth-Møller, K, Iversen, M, Banke, MB, Petersen, IJ, Klingenberg, SL, Krogh, J, Ebert, SE, Timm, A, Lindschou, J and Gluud, C (2017) Selective serotonin reuptake inhibitors versus placebo in patients with major depressive disorder. A systematic review with meta-analysis and trial sequential analysis. BMC Psychiatry 17, 58.CrossRefGoogle ScholarPubMed
Kim, C and Prasad, V (2015) Cancer drugs approved on the basis of a surrogate end point and subsequent overall survival. An analysis of 5 years of US Food and Drug Administration approvals. JAMA Internal Medicine 175, 19921994.CrossRefGoogle ScholarPubMed
Motola, D, Ponti, FD, Poluzzi, E, Martini, N, Rossi, P, Silvana, MC, Vaccheri, A and Montanaro, N (2006) An update on the first decade of the European centralized procedure: how many innovative drugs? British Journal of Clinical Pharmacology 62, 610616.CrossRefGoogle ScholarPubMed
Munkholm, K, Paludan-Müller, AS and Boesen, K (2019) Considering the methodological limitations in the evidence base of antidepressants for depression: a reanalysis of a network meta-analysis. BMJ Open 9, e024886.CrossRefGoogle ScholarPubMed
Naci, H, Davis, C, Savovic, J, Higgins, JPT, Sterne, JAC, Gyawali, B, Romo-Sandoval, X, Handley, N and Booth, CM (2019) Design characteristics, risk of bias, and reporting of randomised controlled trials supporting approvals of cancer drugs by European Medicines Agency, 2014–16: cross sectional analysis. BMJ 366, I5221.CrossRefGoogle ScholarPubMed
Perlis, RH and Perlis, CS (2016) Physician payments from industry are associated with greater medicare part D prescribing costs. PLoS ONE 11, e0155474.CrossRefGoogle ScholarPubMed
Prescrire (2019) Drugs in 2019: a brief review. Prescrire International 29, 110111.Google Scholar
Punja, S, Shamseer, L, Hartling, L, Urichuk, L, Vandermeer, B and Nikles, J (2016) Amphetamines for attention deficit hyperactivity disorder (ADHD) in children and adolescents. Cochrane Database of Systematic Reviews 2, CD009996.Google Scholar
RCP (2019) Royal College of Psychiatrists. Position statement on antidepressants and depression. May 2019. Available at https://www.rcpsych.ac.uk/docs/default-source/improving-care/better-mh-policy/position-statements/ps04_19---antidepressants-and-depression.pdf?sfvrsn = ddea9473_5 (accessed 12 March 2020).Google Scholar
Sorenson, C, Naci, H, Cylus, J and Mossialos, E (2011) Evidence of comparative efficacy should have a formal role in European drug approvals. British Medical Journal 343, d4849.CrossRefGoogle ScholarPubMed
Storebø, OJ, Ramstad, E, Krogh, HB, Nilausen, TD, Skoog, M, Holmskov, M, Rosendal, S, Groth, C, Magnussen, FL, Moreira-Maia, CR, Gillies, D, Rasmussen, KB, Gauci, D, Zwi, M, Kirubakran, R, Forsbøll, B, Simonsen, E and Gluud, C (2015) Methylphenidate for children and adolescents with attention deficit hyperactivity disorder (ADHD). Cochrane Database of Systematic Reviews 11, CD009885.Google Scholar
Surman, CBH, Monuteaux, MC, Petty, CR, Faraone, SV, Spencer, TJ, Chu, NF and Biederman, J (2010) Representativeness of participants in a clinical trial for attention-deficit/hyperactivity disorder? Comparison with adults from a large observational study. Journal of Clinical Psychiatry 7, 16121616.CrossRefGoogle Scholar
US Government (2020) Regulations.gov. Available at https://beta.regulations.gov/.Google Scholar
Van Luijn, JCF, Gribnau, FWJ and Leufkens, HGM (2010) Superior efficacy of new medicines? European Journal of Clinical Pharmacology 66, 445448.CrossRefGoogle ScholarPubMed
Wieseler, B, McGauran, N and Kaiser, T (2019) New drugs: where did we go wrong and what can we do better? BMJ 366, I4340.CrossRefGoogle ScholarPubMed
Zimmerman, M, Balling, C, Chelminski, I and Dalrymple, K (2019) Have treatment studies of depression become even less generalizable? Applying the inclusion and exclusion criteria in placebo-controlled antidepressant efficacy trials published over 20 years to a clinical sample. Psychotherapy Psychosomatics 88, 165170.CrossRefGoogle ScholarPubMed
Figure 0

Fig. 1. The role of regulatory guidelines in pivotal trial designs. (1) The sponsor approaches the relevant drug regulator with the purpose of getting a new drug authorised. (2) Sponsor and regulator discuss (and often agree) on a drug development program. (3) The design and conduct of the pivotal trials will often follow design recommendations outlined in regulatory guidelines. (4) The sponsor conducts the pivotal trials. 5) The acquired data are submitted to the regulators as a Marketing Authorisation Application (EMA) or a New Drug Application (FDA) using the Common Technical Format. (6) The regulator reviews the data, and, if necessary, steps 2, 3 and 4 are repeated to obtain additional information. (7) The regulator decides whether to authorise or reject the application. Figure based on FDA (2019d); EMA (2021b).

Figure 1

Table 1. Overview of included regulatory research guidelines

Figure 2

Table 2. Overview of commenting phases on regulatory guidelines

Figure 3

Table 3. Stakeholders categorised by conflicts of interest

Figure 4

Table 4. Stakeholder comments categorised by conflicts of interest

Supplementary material: File

Boesen et al. supplementary material

Boesen et al. supplementary material 1

Download Boesen et al. supplementary material(File)
File 83 KB
Supplementary material: PDF

Boesen et al. supplementary material

Boesen et al. supplementary material 2

Download Boesen et al. supplementary material(PDF)
PDF 850 KB
You have Access
Open access
2
Cited by

Save article to Kindle

To save this article to your Kindle, first ensure coreplatform@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

EMA and FDA psychiatric drug trial guidelines: assessment of guideline development and trial design recommendations
Available formats
×

Save article to Dropbox

To save this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you used this feature, you will be asked to authorise Cambridge Core to connect with your Dropbox account. Find out more about saving content to Dropbox.

EMA and FDA psychiatric drug trial guidelines: assessment of guideline development and trial design recommendations
Available formats
×

Save article to Google Drive

To save this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you used this feature, you will be asked to authorise Cambridge Core to connect with your Google Drive account. Find out more about saving content to Google Drive.

EMA and FDA psychiatric drug trial guidelines: assessment of guideline development and trial design recommendations
Available formats
×
×

Reply to: Submit a response

Please enter your response.

Your details

Please enter a valid email address.

Conflicting interests

Do you have any conflicting interests? *