‘Using scientific evidence to ensure clinical effectiveness’ is one of the pillars of clinical governance (Reference Scally and DonaldsonScally & Donaldson, 1998). The highest levels of evidence for treatment are randomised controlled trials (RCTs) and systematic reviews and meta-analyses of these RCTs. However, over the past two decades, several studies have been published in general medical journals that have demonstrated an association between the financial support of a RCT and its outcome (Reference DavidsonDavidson, 1986; Reference Rochon, Gurwitz and SimmsRochon et al, 1994; Reference Yaphe, Richard and KnishkowyYaphe et al, 2001; Reference Kjaergard and Als-NielsenKjaergard & Als-Nielsen, 2002). More recently this association has also been investigated in relation to RCTs of psychotropic drugs (Reference Wahlbeck and AdamsWahlbeck & Adams, 1999; Reference Freemantle, Anderson and YoungFreemantle et al, 2000; Reference MoncrieffMoncrieff, 2003).
To our knowledge, our study is the first of its kind looking directly at the association between the support and outcome of RCTs published in mainstream psychiatry journals.
Method
We reviewed all RCTs published in Acta Psychiatrica Scandinavica (APS), American Journal of Psychiatry (AJP), Archives of General Psychiatry (AGP) and British Journal of Psychiatry (BJP) between July 1998 and June 2003. The journals were searched as full text on the internet (APS, AJP and BJP on KA24 database, and AGP on Proquest database) through the http://www.hilo.nhs.uk/website. We also hand searched all issues of BJP for this time period to check if we were missing any RCTs by searching the journals electronically, but we did not find any additional trials.
We included only original clinical trials that compared the efficacy and/or the side-effects of a drug with any treatment including placebo, which had a control group and which assigned patients randomly to groups. Studies comparing non-pharmacological treatments only, analysing pooled, subgroup or follow-up data from previously published RCTs, appearing in journal supplements, and comparing different doses or durations of the same drug were all excluded.
Study support
Data were extracted on details of (1) financial support, (2) whether the manufacturer of the experimental drug had provided the study medications and (3) whether one or more of the authors was an employee of the manufacturer, while being masked to the study outcome.
-
• If any type of support from the manufacturer of the experimental drug was declared, the study was classified as manufacturer-supported.
-
• If there was no mention of any kind of manufacturer support the study was classified as non-manufacturer-supported. This included trials which had received support from sources other than the manufacturer.
Study outcomes
Study outcomes were independently assigned as positive or negative depending on whether the outcome would promote the prescribing of the experimental drug or not. The following specific criteria were used.
-
• If the experimental drug was more efficacious than alternative treatment and had acceptable side-effects, or if there was no difference in the efficacy of the two treatments and the experimental drug had fewer side-effects or was less expensive, the study was classified as having a positive outcome.
-
• If the experimental drug was less efficacious than alternative treatment or had unacceptable side-effects, or if there was no difference in the efficacy of the two treatments and the alternative treatment had fewer side-effects or was less expensive, the study was classified as having a negative outcome.
Statistical analysis
We used chi-squared tests to compare the difference in proportion of negative and positive outcomes between manufacturer-supported and non-manufacturer-supported trials, and to analyse the association of different types of support with study outcome. We used logistic regression to determine which variable among all types of support and different journals was the best predictor of outcome. All the analyses were performed with the Statistical Package for the Social Sciences (SPSS) version 12.0 for Windows.
Distribution of study randomised controlled trials in the four journals according to manufacturer support and positive outcome
| Journal | Number of studies | Manufacturer support present (%) | Positive outcome (%) |
|---|---|---|---|
| Acta Psychiatrica Scandinavica | 11 | 9 (81.8) | 9 (81.8) |
| American Journal of Psychiatry | 99 | 71 (71.7) | 86 (86.9) |
| Archives of General Psychiatry | 53 | 37 (69.8) | 49 (92.5) |
| British Journal of Psychiatry | 25 | 21 (84.0) | 20 (80.0) |
| Total | 188 | 138 (73.4) | 164 (87.2) |
Results
Our search yielded 306 RCTs. Of these, 91 RCTs (30%) compared non-pharmacological treatments only and were therefore excluded. Of the remaining 215 RCTs, which had evaluated at least one drug, we excluded a further 25 RCTs; 10 because they compared different doses, durations or blood levels of the same drug, 12 because they compared neither efficacy nor side-effects, 1 because the drug of interest was not evaluated, and 2 because there was no specific drug of interest. There were 2 studies whose outcomes could not be classified as positive or negative even after extensive discussion and were therefore excluded from the analyses. The remaining 188 RCTs were entered in the study.
There was disagreement on outcomes of 3 studies (2%). After discussion the differences were resolved in each case; 164 studies (87%) were classified as having a positive outcome and 24 (13%) as having a negative outcome.
There were 138 studies (73%) which declared receiving support from the manufacturer of the experimental drug. Of these, 107 (57%) had received financial support, 58 (31%) had an employee author and 34 (18%) mentioned receiving medications; 93 studies (50%) declared receiving funding from non-industry sources and 6 (3%) did not declare any support.
Chi-squared tests showed a significant difference in proportion of manufacturer-supported (125/138, 91%, 95% CI 88-93) and non-manufacturer-supported (39/50, 78%, 95% CI 72-84) trials having a positive outcome (χ2=5.21, d.f.=1, P=0.02, odds ratio=0.37, 95% CI=0.15-0.89).
Among the subtypes of support, a significantly higher proportion of trials with an employee author had a positive outcome than trials without (97%, 95% CI 94-99 v. 83%, 95% CI 80-86) (χ2=6.53, d.f.=1, P=0.01, odds ratio=0.12, 95% CI=0.02-0.77). The proportion of trials with a positive outcome was also higher in trials with manufacturer-supplied medications and financial support than without, but the difference was not statistically significant (medications, 94%, 95% CI 90-98 v. 86%, 95% CI 83-89, P=0.18; financial support, 90%, 95% CI 87-93 v. 84%, 95% CI 80-88, P=0.24).
The results of the full logistic regression model, in which we entered all the six variables at the same time, are presented in Table 2. Having an employee author was the best predictor of a positive outcome.
Factors associated with a positive outcome: simple multiple logistic regression analysis
| Variables | Odds ratios1 (95% Cl) | P |
|---|---|---|
| Financial support | 0.37 (0.07-1.74) | 0.21 |
| Medications | 0.20 (0.04-1.02) | 0.053 |
| Employee author | 0.12 (0.02-0.61) | 0.01 |
| Non-manufacturer support | 0.37 (0.08-1.74) | 0.20 |
| No support | 0.26 (0.02-3.94) | 0.32 |
| Journal | 0.95 (0.56-1.64) | 0.86 |
Discussion
Our finding of difference in outcomes of manufacturer-supported and non-manufacturer-supported trials is consistent with several previous studies. Davidson (Reference Davidson1986) reviewed all clinical trials published in five general interest medical journals in 1 year and found that there was a statistically significant association between the source of funding and outcome of a study (P=0.002). Rochon et al (Reference Rochon, Gurwitz and Simms1994) found that manufacturer-associated non-steroidal anti-inflammatory drug (NSAID) was comparable or superior to the comparison drug in all 56 trials of NSAIDs in the treatment of arthritis. Yaphe et al (Reference Yaphe, Richard and Knishkowy2001) found that negative outcomes were significantly less likely to be found in industry-supported studies than non-industry-supported studies. Kjaergard & Als-Nielsen (Reference Kjaergard and Als-Nielsen2002) found that authors’ conclusions significantly favoured experimental interventions if financial competing interests were declared.
In psychiatry, Wahlbeck & Adams (Reference Wahlbeck and Adams1999) reported that in a Cochrane review of trials comparing clozapine with typical antipsychotics, studies sponsored by the manufacturer of clozapine were associated with more favourable outcomes for clozapine. In a meta-regression analysis, Moncrieff (Reference Moncrieff2003) reported that trials which declared receiving some financial support from the manufacturer of clozapine showed a greater benefit of clozapine over conventional antipsychotics. In another meta-regression analysis, Freemantle et al (Reference Freemantle, Anderson and Young2000) found that the most important structural predictor of RCT outcome was trial sponsorship, although this finding was not statistically significant.
Multiple hypotheses have been put forward to explain the association between trial support and outcome. These include: publication bias (Reference Kjaergard and Als-NielsenKjaergard & Als-Nielsen, 2002); pharmaceutical companies selecting for study drugs that have been previously shown to be efficacious (Reference DavidsonDavidson, 1986); selective release and publication of data by pharmaceutical companies (Reference Rochon, Gurwitz and SimmsRochon et al, 1994; Reference Blumenthal, Campbell and AndersonBlumenthal et al, 1997; Reference RennieRennie, 1997; Reference Nathan and WeatherallNathan & Weatherall, 1999); multiple publications from the same trial (Reference GøtzscheGøtzsche, 1989; Reference Huston and MoherHuston & Moher, 1996), biased interpretation of results (Reference Rochon, Gurwitz and SimmsRochon et al, 1994; Reference Friedberg, Saffran and StinsonFriedberg et al, 1999); and pharmaceutical companies influencing study designs or reporting ensuring that the results favour their drug (Reference Bero and RennieBero & Rennie, 1996; Reference Johansen and GøtzscheJohansen & Gøtzsche, 1999; Reference SaferSafer, 2002).
There are some limitations to the conclusions that can be drawn from our study. First, studies of this kind can only demonstrate association, and not causation. Second, authors may not be disclosing conflicts of interests completely, thus resulting in any study similar to this one being based on incorrect or incomplete information. There is some evidence to support this assertion (Reference Lewison, Dawson and AndersonLewison et al, 1995; Reference SmithSmith, 2001; Reference Henderson, Howard and WilkinsonHenderson et al, 2003). Third, there is the need to make subjective judgements in a study of this kind. We tried to deal with this by one reviewer making a masked assessment of support, two reviewers separately assigning outcomes based on explicit criteria, one of them being masked to data on support, and estimating level of agreement between these two reviewers.
Conclusion
The primary question this study raises is how valid and safe our evidence on treatment is considering the findings that almost three-quarters of RCTs had received some support from the manufacturer of the experimental drug, and that the outcomes of trials supported and not supported by manufacturer of the experimental drug were significantly different.
Open access
eLetters
No eLetters have been published for this article.