National Health Service (NHS) Research Outputs Database

The Research Outputs Database was constructed by The Wellcome Trust (Reference Dawson, Lucocq and CottrellDawson et al, 1998) and then maintained by the Centre for Information Behaviour and the Evaluation of Research, City University. It covers the full range of research publications, including basic and clinical sciences, in the peer-reviewed journals contained in the Science Citation Index and Social Science Citation Index databases. The Research Outputs Database contains bibliographic information from biomedical papers with a UK address, including details of funding acknowledgements. The NHS Research Outputs Database, a subset of the Research Outputs Database, has been constructed and contains details of papers from England that involve some element of NHS financial support (Wellcome Trust & NHS Executive, 2001).

The NHS papers have been identified using a filter for England that identifies them via one or more of the following: characteristics of the name of the author's institution, for example ‘hospital’; the institution's postcode; or the funding acknowledgements on the paper.

Journal impact factors

Journal impact factors were obtained from the 2001 edition of the on-line Journal Citation Reports from the Institute for Scientific Information. The journal impact factor is ‘a measure of the frequency with which the “average article” in a journal has been cited in a particular year or period. The annual JCR impact of a journal... is calculated by dividing the number of current year citations to the source items published in that journal during the previous two years’ (Reference GarfieldGarfield, 1994). A ranking order of journals within the ‘psychiatry’ category based on journal impact factors was also taken from the 2001 Journal Citation Reports for each of the citation indices (i.e. the Science Citation Index and the Social Science Citation Index).

Questionnaire survey

Survey findings

A total of 560 questionnaires (47%) were completed and returned. The return rates for psychiatrists treating each of the three groups were: child, 49%; adult, 38%; old age, 52%. Those psychiatrists with some academic commitment formed 26% of respondents.

A substantial number of recipients added more journals to the questionnaire, bringing the total number of journals from the original 32 up to 156. Those journals that were not listed on the original questionnaire but appear in the summary tables have been marked with an asterisk. The 560 respondents ticked or added journal names on 3215 occasions. Out of these, 13 (0.4%) related to 10 unverified journal names that nevertheless were included within the database.

The difference in the median number of journals read by psychiatrists with academic commitments (ten journals or more) and those without (three journals) was statistically significant (Kruskal–Wallis; \batchmode \documentclass[fleqn,10pt,legalpaper]{article} \usepackage{amssymb} \usepackage{amsfonts} \usepackage{amsmath} \pagestyle{empty} \begin{document} \({\chi}_{1}^{2}\) \end{document} =7.823, P=0.005). The percentage of ‘non-academic’ psychiatrists reading three journals or fewer was higher across all three age groupings but especially so in the adult group where the figure was 40%. Overall, approximately twice as many academics compared with non-academics read at least ten journals.

Tables 1 and 2 detail the specific journals that psychiatrists read and have ranked first, second or third with regard to their clinical work. A striking consistency was found at the top of each table, both across all age groups and between academic and non-academic psychiatrists. The two most prominent journals across the board were the British Journal of Psychiatry followed by the BMJ.

Table 1

Percentage of psychiatrists reading selected journals with regard to their clinical work (all journals read by at least 10% of psychiatrists in one or more category)

Journal	Patient age-group to whom psychiatric services are provided
	All groups			Child			Adult			Old age
	±	+	—	±	+	—	±	+	—	±	+	—
British Journal of Psychiatry	97	97	97	97	97	97	99	100	98	95	93	96
BMJ	89	90	89	85	90	84	91	88	92	93	91	93
American Journal of Psychiatry	38	48	34	25	32	22	50	73	41	39	44	38
Journal of Child Psychology and Psychiatry and Allied Disciplines	35	41	32	96	100	94	3	5	3	0	0	1
International Journal of Geriatric Psychiatry	30	27	31	0	0	0	3	2	3	80	93	76
Psychological Medicine	29	41	25	21	36	15	36	46	31	32	44	28
Lancet	22	30	19	10	15	7	22	41	15	34	42	31
Age and Ageing	16	14	18	0	0	0	2	0	3	43	47	43
Acta Psychiatrica Scandinavica	15	20	13	5	10	4	20	34	15	18	19	18
* Journal of the American Academy of Child and Adolescent Psychiatry	14	15	14	39	37	40	1	0	1	0	0	0
Journal of Neurology, Neurosurgery and Psychiatry	13	19	11	3	7	1	11	22	7	24	33	22
Hospital Medicine (previously British Journal of Hospital Medicine)	12	11	12	8	10	7	9	10	9	14	14	14
* Advances in Psychiatric Treatment	11	6	13	14	8	16	13	2	17	7	5	8
* Clinical Child Psychology and Psychiatry	9	10	8	24	24	24	1	0	1	0	0	1
Journal of the Royal Society of Medicine	9	12	7	8	10	7	10	17	7	8	12	8
Biological Psychiatry	8	14	6	5	12	5	9	22	5	8	9	8
Schizophrenia Research	7	12	5	3	5	1	16	32	10	3	2	3
* Archives of General Psychiatry	6	15	3	3	7	1	14	41	4	3	0	4
Journal of Affective Disorders	6	12	3	1	3	0	11	32	3	6	9	6
Journal of Psychopharmacology	6	10	5	3	5	2	9	20	6	6	7	6
International Journal of Eating Disorders	5	5	4	11	14	9	5	0	6	0	0	0
* Journal of Family Therapy	5	7	4	13	15	12	0	0	0	0	2	0
* Psychiatric Bulletin	5	7	4	3	0	4	9	10	8	4	14	2
Psychopharmacology	5	6	4	2	2	1	5	10	3	7	7	7
International Clinical Psychopharmacology	4	5	4	1	0	1	6	15	3	5	2	6
* Child Abuse and Neglect	3	4	2	7	10	5	0	0	0	0	0	0
* Journal of Adolescence	3	3	3	9	7	10	0	0	0	0	0	0
Social Psychiatry and Psychiatric Epidemiology	3	4	2	0	0	0	7	12	4	2	2	1
* International Psychogeriatrics	2	4	1	0	0	0	0	0	0	5	14	3
* Ageing and Mental Health	1	4	0	0	0	0	0	0	0	3	14	1
* Journal of the American Geriatrics Society	1	3	0	0	0	0	0	0	0	3	12	1

Table 2

Percentage of psychiatrists ranking selected journals first, second or third with regard to their clinical work (all journals ranked by at least 10% of psychiatrists in one or more category)

Journal	Patient age-group to whom psychiatric services are provided
	All groups			Child			Adult			Old age
	±	+	—	±	+	—	±	+	—	±	+	—
British Journal of Psychiatry	81	76	83	71	73	70	89	78	93	85	79	86
BMJ	56	50	58	38	42	36	66	54	71	64	53	66
Journal of Child Psychology and Psychiatry and Allied Disciplines	28	32	27	78	78	78	3	2	3	1	0	1
International Journal of Geriatric Psychiatry	26	23	27	0	0	0	2	0	3	69	79	66
American Journal of Psychiatry	15	20	13	9	12	7	27	46	20	13	12	14
* Journal of the American Academy of Child and Adolescent Psychiatry	10	12	9	27	29	26	1	0	1	0	0	0
* Advances in Psychiatric Treatment	7	5	8	6	5	7	11	2	15	6	5	6
* Clinical Child Psychology and Psychiatry	6	2	7	16	5	20	0	0	0	0	0	0
Age and Ageing	5	4	5	0	0	0	1	0	1	13	14	13
Psychological Medicine	5	7	5	4	5	3	10	12	9	2	2	3
* Archives of General Psychiatry	3	10	1	2	5	0	9	29	2	1	0	1
Biological Psychiatry	2	5	1	1	2	1	5	15	2	1	2	1
Journal of Psychopharmacology	2	3	1	0	0	0	5	10	3	1	0	1

Table 1 shows that for adult psychiatrists these two journals dominated their reading habits. Both were cited by over 90% of the sample whereas the third most commonly cited journal (the American Journal of Psychiatry) was read by only 50%. Although both the British Journal of Psychiatry and the BMJ were still cited by about 90% of the child and old age psychiatrists, specialised journals became more prominent.

The importance of these specialised journals for child and old age psychiatrists was even more clearly reflected in their ranking of the journals (Table 2). Within each category there were only a small number of journals (between four and seven) ranked in the top three in importance for their clinical practice by more than 10% of psychiatrists (i.e. only a few are widely viewed as important; see Table 2).

Relationship of readership to the NHS Research Outputs Database and journal impact factors

Table 3 contains the same 31 journals as those that appear in Table 1: those journals read by 10% or more of psychiatrists in any category (by patient age or academic commitments). The numbers of psychiatry papers in the journals between 1990 and 1999 were identified through the NHS Research Outputs Database. The British Journal of Psychiatry is clearly the journal with the largest number of papers in the psychiatry section of the NHS Research Outputs Database and also is ranked as the most important to clinical practice overall. The pattern needs to be interpreted with caution because only a relatively small proportion of BMJ papers are related to psychiatry and the Archives of General Psychiatry publishes comparatively few papers per year.

Table 3

All 31 journals read by at least 10% of psychiatrists in one or more category and ranked by the percentage of psychiatrists that read them, their journal impact factors (JIFs) and various ranking methods of JIF

Journal	% Psychiatrists (all categories) reading the journal	NHS (England) psychiatry publications 1990-1999	JIF 2001	Ranking of the 31 journals by JIF	Position in ISI ranking of the 81 psychiatry journals in SCI by JIF
British Journal of Psychiatry	97	1049	4.1	6	8
BMJ	89	168	6.6	4	-
American Journal of Psychiatry	38	41	6.9	3	2
Journal of Child Psychology and Psychiatry and Allied Disciplines	35	64	2.8	13	91
International Journal of Geriatric Psychiatry	30	335	1.8	19	38
Psychological Medicine	29	299	3.1	9	15
Lancet	22	74	13.3	1	-
Age and Ageing	16	52	1.7	20	-
Acta Psychiatrica Scandinavica	15	136	2.1	16	29
* Journal of the American Academy of Child and Adolescent Psychiatry	14	15	3.6	7	11
Journal of Neurology, Neurosurgery and Psychiatry	13	101	3.0	11	16
Hospital Medicine (previously British Journal of Hospital Medicine)	12	79	0.3	27	-
* Advances in Psychiatric Treatment	11	N/A	N/A	N/A	N/A
* Clinical Child Psychology and Psychiatry	9	N/A	N/A	N/A	N/A
Journal of the Royal Society of Medicine	9	69	0.7	24	-
Biological Psychiatry	8	61	5.5	5	4
Schizophrenia Research	7	75	3.6	7	12
* Archives of General Psychiatry	6	29	12.0	2	1
Journal of Affective Disorders	6	114	1.9	17	36
Journal of Psychopharmacology	6	63	2.6	14	20
International Journal of Eating Disorders	5	100	1.9	17	35
* Journal of Family Therapy	5	12	0.5	26	-
* Psychiatric Bulletin	5	N/A	N/A	N/A	N/A
Psychopharmacology	5	51	3.1	9	14
International Clinical Psychopharmacology	4	126	2.3	15	26
* Child Abuse and Neglect	3	1	1.2	21	-
* Journal of Adolescence	3	14	0.8	23	-
Social Psychiatry and Psychiatric Epidemiology	3	79	1.2	21	331
* International Psychogeriatrics	2	N/A	N/A	N/A	N/A
* Ageing and Mental Health	1	32	0.6	25	-
* Journal of the American Geriatrics Society	1	4	2.9	12	-

The data on journal impact factors are presented in a number of ways in Table 3, including (in the final column) their position in the Institute for Scientific Information ranking of psychiatry journals, which does not include general medical journals such as the BMJ or the Lancet.

Few key journals

For respondents, it appears that a small number of journals are very important for the dissemination of information with a bearing on clinical practice. Furthermore, the numbers of psychiatrists who read three journals or fewer are most marked in the adult group, which is approximately three times as large as the other two categories in the membership of the Royal College of Psychiatrists. Adjustment for this factor would give overall figures for psychiatrists reading three journals or fewer of 27%, with 11% for academics and 34% for non-academics. If a small number of journals are of greatest importance to clinicians, then research findings published in these journals have greater potential to result in benefit to patients.

The journals that were found to be the most important to clinical psychiatrists – the British Journal of Psychiatry and the BMJ – are both available as part of membership to the Royal College of Psychiatrists and the British Medical Association, respectively. Most psychiatrists, therefore, will receive them without subscription. Because our sample was provided by the Royal College of Psychiatrists from lists of its Members and Fellows, then all will receive the British Journal of Psychiatry. Approximately 80% of practising doctors are members of the British Medical Association and therefore automatically will receive the BMJ. In addition, the BMJ is freely accessible via the internet. These two journals were noticeably ahead of all the other journals in terms of readership. The British Journal of Psychiatry also contained a significant proportion of the total papers in the whole NHS Research Outputs Database mental health field. It published more than three times as many articles funded in some way by the NHS as the International Journal of Geriatric Psychiatry in second place. Rafferty et al (Reference Rafferty, Traynor and Lewison2000) found that one dominant journal in the field of nursing contained far more publications than any other (46% of the total), followed by a second containing 6.5%.

Comparisons with journal impact factors

The journal impact factor has been used as an indicator of the quality of research published within journals (Reference Schwartz and Lopez HellinSchwartz & Lopez Hellin, 1996). However, our study shows that the correlation between the perceived importance attributed by clinical practitioners in the field of psychiatry and the journal impact factor is neither a simple nor a consistent relationship (see Fig. 1). This was found for clinicians both with and without academic commitments.

Fig. 1

Percentage of psychiatrists reading selected journals v. journal impact factor 2001 (all journals read by at least 10% of psychiatrists in one or more category, as in Table 3). Note that the Archives of General Psychiatry was not listed on the original questionnaire.

The journal impact factor scores of the British Journal of Psychiatry and the BMJ are reasonably high. The British Journal of Psychiatry was positioned eighth out of 81 within the field of psychiatry, and the BMJ, although obviously not listed in the psychiatry journals, had a journal impact factor that would have put it in third position. Of the top ten journals in the field of psychiatry, according to journal impact factors, only four (Archives of General Psychiatry, Biological Psychiatry, the American Journal of Psychiatry and the British Journal of Psychiatry) were found in the 31 journals with a clinical readership of 10% or more in any one category (by patient age-group or academic commitments) and the first three of these were found to be of significantly greater importance to academics in the adult group than to any other category of psychiatrist.

In this discussion an inevitable limitation has to be considered. As discussed later, national bias has been found in both publication trends and readership of journals (Reference Grant, Cottrell and CluzeauGrant et al, 2000; Reference Schein, Paladugu and SutijaSchein et al, 2000), therefore it was felt that a standard list of journals that were most important as outlets for UK psychiatry publications would seem an appropriate starting point for this survey. Including a comparatively small number of such journals enabled a manageable list to be included in the questionnaire, but allowing clinicians to add journal names inevitably created two populations of journals. It would seem reasonable to assume that those journals included within the questionnaire were more likely to be ticked as read than those not included. The Archives of General Psychiatry, which has the highest journal impact factor in the field of psychiatry, was not one of the journals listed on the questionnaire owing to the small number of UK papers published in it. This absence from the questionnaire might have reduced the numbers of psychiatrists referring to the Archives of General Psychiatry but would not account for the marked differences found between academics and non-academics and between the adult group compared with the other two patient age categories. It is possible of course that papers in journals such as this have an important indirect, rather than direct, influence on clinical practice in the UK through their impact on guidelines, etc.

The Journal of Child Psychology and Psychiatry and Allied Disciplines is widely read and the most highly rated by child psychiatrists, but its journal impact factor would put it in 17th position if it were included in the Science Citation Index listing for psychiatry in the Journal Citation Reports. The International Journal of Geriatric Psychiatry is similarly widely read within its patient age-group of psychiatrists but is poorly rated by journal impact factor, being positioned 38th in the psychiatry list. Clinical Child Psychology and Psychiatry and Advances in Psychiatric Treatment feature quite prominently in the results of this survey but neither was included in the original questionnaire because neither is listed by the Science Citation Index.

Survey findings in context

Previous research has raised several relevant issues, including the significance of country of publication and the relevance of journal impact factor to readership patterns. Grant et al (Reference Grant, Cottrell and Cluzeau2000) examined UK clinical guidelines to determine the flow of information from basic research to clinical practice and the nationality of papers cited in UK clinical guidelines. They found that UK authors of clinical guidelines cite UK publications in a greater proportion (25%) than is found in world biomedical literature (10%). Schein et al surveyed 1000 Fellows of the American College of Surgeons and found that they were only interested in American journals, despite the fact that an international survey by e-mail had found a UK journal, the British Journal of Surgery, to be the ‘best’ general surgical journal in the world (Reference Schein, Paladugu and SutijaSchein et al, 2000). Furthermore, journal impact factor was not a consideration for these surgeons when selecting journals to read. Lewison (Reference Lewison2002) examined the relationship between the importance of journals to researchers and to the users of research in a series of medical sub-fields. He found significant variations. In the more clinical subjects such as nursing there was virtually no correlation between their perceived relative importance and the citation score of the journal.

The findings of this survey indicate that in terms of nationality of journals read, and ranked first, second or third, although some of the American journals are of considerable significance there is a clear bias towards journals published in the UK. This bias may, however, have been exaggerated by the choice of journals listed in the original questionnaire.

The study examines what psychiatrists read and perceive as important to their clinical practice, and it covers a large number of publications. Adopting a broad approach in a brief questionnaire inevitably means that some issues were not explored. The term ‘read’ has not been examined, just as the different sections and article types in the journals have not been analysed individually. Further studies, possibly on a journal-by-journal basis, would provide more information in this area (Reference TyrerTyrer, 2003). Also, there may have been some differences in the respondents’ interpretation of the questions asked, which possibly will have had some effect on the findings. A further limitation that has not been examined here involves the variation in journal availability to psychiatrists, with expensive ones available only to some psychiatrists. Research on these issues would provide additional information on the factors involved in translating biomedical research into clinical practice.

By itself, transmission through journals is not seen as a major way of securing the implementation of research findings (Reference Coomarasamy, Gee and PublicoverCoomarasamy et al, 2001) but clinicians do believe that journals are their main source of information (Reference Schein, Paladugu and SutijaSchein et al, 2000). Assessment of the impact of journals is an important part of a wider stream of work being developed to examine the value of research (Reference Buxton and HanneyBuxton & Hanney, 1996) and, in turn, such analysis (Reference Hanney, Soper and BuxtonHanney et al, 2003) is being linked to work on how best to implement research findings (Reference Grimshaw, Shirran and ThomasGrimshaw et al, 2001). Perhaps greater recognition should be given to researchers who publish in the journals that are of greatest importance in disseminating research, irrespective of their journal impact factors.

Clinical Implications

1. ▪ Researchers aiming to inform clinical practice should consider targeting their findings to journals widely read by clinicians.

2. ▪ The incentives to researchers to disseminate research to clinicians should be increased.

3. ▪ Journals should be concerned with identifying ways to enhance their perceived clinical relevance.

Limitations

1. ▪ The two journals ranked highest for readership were ‘subscription’ journals that are routinely received by most respondents.

2. ▪ The response rate was only 47%.

3. ▪ The study is cross-sectional and therefore is unable to track associations between reading habits and changes in journal impact factors.