Hostname: page-component-6b989bf9dc-jks4b Total loading time: 0 Render date: 2024-04-14T07:57:20.575Z Has data issue: false hasContentIssue false

Abortion and mental health: guidelines for proper scientific conduct ignored

Published online by Cambridge University Press:  02 January 2018

Julia H. Littell
Graduate School of Social Work and Social Research, Bryn Mawr College, Bryn Mawr, Pennsylvania, USA. Email:
James C. Coyne
Department of Psychiatry, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
Rights & Permissions [Opens in a new window]


Copyright © Royal College of Psychiatrists, 2012 

In her review of research on the mental health effects of abortion, Coleman Reference Coleman1 stated:

‘In this highly politicised area of research it is imperative for researchers to apply scientifically based evaluation standards in a systematic, unbiased manner when synthesising and critiquing research findings. If not, authors open themselves up to accusations of shifting standards based on conclusions aligned with a particular political viewpoint. Moreover, the results may be dangerously misleading and result in misinformation guiding the practice of abortion.’

However, Coleman failed to follow well-accepted scientific standards for the conduct and reporting of systematic reviews and meta-analyses. Further, Coleman’s failure to state her obvious conflicts of interest in this review raises serious questions about biases in her analysis. Hence, the review is open to serious questions about the author’s scientific standards, methods, political viewpoints, and potentially misleading conclusions.

Widely accepted standards for systematic reviews and meta-analyses are contained in the published AMSTAR, MARS, MOOSE, and PRISMA statements. Reference Shea, Grimshaw, Wells, Boers, Andersson and Hamel2Reference Moher, Liberati, Tetzlaff and Altman5 None of these standards were cited or followed by Coleman. AMSTAR is the only validated instrument for assessing the methodological quality of systematic reviews and meta-analysis. We assessed Coleman’s review according to the AMSTAR statement, and found that it failed to meet any of the eleven basic requirements for systematic reviews and meta-analysis included in AMSTAR.

Following AMSTAR, specific flaws of the Coleman review are as follows:

  1. 1 there was no public a priori design

  2. 2 there was no duplicate study selection or duplicate data extraction

  3. 3 the author did not describe the search strategy in sufficient detail

  4. 4 the review was limited to published studies, contrary to all published standards

  5. 5 a list of excluded studies was not provided

  6. 6 the author did not provide sufficient descriptive information on included studies, including demographic characteristics of participants

  7. 7 the scientific quality of included studies was not documented

  8. 8 scientific quality of included studies was not considered in formulating conclusions

  9. 9 appropriate methods were not used in combining the findings of studies (Coleman clearly violated the rule for avoiding dependencies in meta-analysis, when she synthesised 36 effects from 22 studies in Fig. 1)

  10. 10 the likelihood of publication bias was not assessed

  11. 11 conflicts of interest and sources of support were not acknowledged (no financial disclosures were made and no other potential conflicts were acknowledged).

An article in the British Journal of Psychiatry Reference Maj6 calls attention to the importance of non-financial conflicts of interest in the psychiatric literature. Coleman has at least two types of conflict of interest here. Among the most important of such conflicts is an agenda-driven bias, by which authors seek to influence legislation and social policy. David Reardon is a co-author with Coleman on seven articles included in the review and an author on an additional study in the review that does not involve Coleman as a co-author. Reardon is quite explicit about his agenda to instil fear of abortion as a way of facilitating passage of anti-abortion legislation. Reference Reardon7

Coleman is the first author on 6 studies and co-author on 5 additional studies in her review; thus, she authored or co-authored fully half of the 22 studies included. According to the Cochrane Handbook, Reference Higgins and Green8 this is another potential conflict of interest, since it may ‘unduly influence judgements made in a review (concerning, for example, the inclusion or exclusion of studies, assessments of the risk of bias in included studies or the interpretation of results)… This should be disclosed in the review and, where possible, there should be an independent assessment of eligibility and risk of bias by a second author with no conflict of interest.’ Coleman did not obtain an independent assessment of the studies she authored or co-authored, nor did she acknowledge these conflicts in the review.

Coleman’s conclusion that the results of the studies in her review are ‘quite consistent’ (p. 183) is belied by visual inspection of the Forest plots, which include non-overlapping confidence intervals. Coleman should have reported results of heterogeneity tests (chi-squared and I 2), which probably would have shown significant heterogeneity in results across studies (presumably that is why she chose the random effects model).

Some of the commentaries on Coleman’s review appear to be uninformed by current scientific standards for reviews. Comments by Fergusson et al are particularly misleading. Faced with variations in the methodological quality of available studies, it is essential for reviewers to weed out weaker studies. Valid conclusions can only be based on valid studies.

It is unclear how this paper got through peer review at the Journal. It appears that peer reviewers and the Editor ignored published standards for systematic reviews and meta-analyses. Given the serious methodological flaws contained in Coleman’s review and the author’s failure to report obvious conflicts of interest, we believe the article should be retracted.


1 Coleman, PK. Abortion and mental health: quantitative synthesis and analysis of research published 1995–2009. Br J Psychiatry 2011; 199: 180–6.Google Scholar
2 Shea, BJ, Grimshaw, JM, Wells, GA, Boers, M, Andersson, N, Hamel, C, et al. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol 2007; 7: 10.CrossRefGoogle ScholarPubMed
3 APA Publications and Communications Board Working Group on Journal Article Reporting Standards. Reporting standards for research in psychology: Why do we need them? What might they be? Am Psychol 2008; 63: 839–51.Google Scholar
4 Stroup, DF, Berlin, JA, Morton, SC, Olkin, I, Williamson, GD, Rennie, D, et al. Meta-analysis of observational studies in epidemiology: a proposal for reporting. JAMA 2000; 283: 2008.Google Scholar
5 Moher, D, Liberati, A, Tetzlaff, J, Altman, DG. Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLOS Med 2009; 6: e1000097.CrossRefGoogle ScholarPubMed
6 Maj, M. Non-financial conflicts of interests in psychiatric research and practice. Br J Psychiatry 2008; 193: 91–2.Google Scholar
7 Reardon, DC. A defense of the neglected rhetorical strategy (NRS). Ethics Med 2002; 18: 2332.Google Scholar
8 Higgins, JPT, Green, S (eds). Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0. The Cochrane Collaboration, 2011.Google Scholar
Submit a response


No eLetters have been published for this article.