Training matters The right stuff ? A prospective controlled trial of trainees ' research

Books and courses on the research process are becoming big business in psychiatry, as well as other specialties (Freeman & Tyrer, 1989; Mathie et al, 1989). There is certainly a market: it is increasingly clear to trainees that having undertaken and, cru cially, completed some aspect of research is viewed favourably by appointment committees. The present career structure, with its bottleneck at senior regis trar level, has put the onus on general trainees to demonstrate some research experience on their C.V. How well do such research courses work? Indeed, can the efficacy of such a course be measured in any way objectively? There are problems involved in teaching something as elusive as research motiv ation. Researchers acknowledge that research can be difficult compared to clinical work, for a variety of more or less subtle reasons. It is proactive rather than reactive. There is little to show for it for a consider able time. It is highly personalised: your own ideas are on show. Recently I was asked to set up a research course for trainees. In planning the course I therefore took the opportunity to set out to answer several questions. Is research important to the career of trainees and, if so, what aspect is important? What are trainees' atti

Books and courses on the research process are becoming big business in psychiatry, as well as other specialties (Freeman & Tyrer, 1989;Mathie et al, 1989).There is certainly a market: it is increasingly clear to trainees that having undertaken and, cru cially, completed some aspect of research is viewed favourably by appointment committees.The present career structure, with its bottleneck at senior regis trar level, has put the onus on general trainees to demonstrate some research experience on their C.V.
How well do such research courses work?Indeed, can the efficacy of such a course be measured in any way objectively?There are problems involved in teaching something as elusive as research motiv ation.Researchers acknowledge that research can be difficult compared to clinical work, for a variety of more or less subtle reasons.It is proactive rather than reactive.There is little to show for it for a consider able time.It is highly personalised: your own ideas are on show.
Recently I was asked to set up a research course for trainees.In planning the course I therefore took the opportunity to set out to answer several questions.Is research important to the career of trainees and, if so, what aspect is important?What are trainees' atti tudes about research?If negative, can a research course change these in a measurable way?What do the trainees want out of such a course and do they feel they get it?And, most importantly, do courseattenders actually end up doing more research and getting more publications, compared with trainees not attending such a course?

Does research matter for trainees?
Trainees believe that evidence of their having under taken some research is important to senior registrar appointment committees.As a small test of the val idity of this assumption I examined retrospectively the curricula vitae of applicants to a recent vacancy on the Charing Cross senior registrar scheme.There were 32 applicants.The initial shortlisting had been carried out by consultant-grade members of the appointments committee, as is usual practice.Individual shortlists had then been combined using a points system by a member of the personnel department to produce a consensus.After some early withdrawals which forced an extended shortlist, 11 candidates were shortlisted at some stage: 21 were not.
I compared the CVs of these two groups on a range of variables which might be expected to influence likelihood of shortlisting.These included months since MRCPsych, gender, training on a teaching hos pital rotation or not, other higher degrees, and being a local trainee.I also looked at two aspects of research involvement: the number of research proj ects in which the applicant had taken part, and whether or not the applicant had one or more articles (including letters) published (or in press).
The results were clear enough.Only one of the variables significantly predicted likelihood of being shortlisted: the presence of a publication.Of the 15 applicants with a publication, eight (53%) had been shortlisted.Of the 17with no publications, only three (18%) had been shortlisted (Fisher's exact, P=0.04, one-tailed).The qualities looked for in a higher trainee are far broader than the presence or absence of a publication.But any applicant with such quali ties will not be given a chance to show them unless he or she reaches the interview stage.And this retro spective analysis suggests that having a publication (and not simply being involved in some research project) is important in getting the trainee to the interview stage.

Trainees' research: a controlled intervention study
The basic skeleton of the research course was planned, and a maximum often general trainees was envisaged.The course was advertised across the A prospective controlled trial of trainees training scheme three months before the starting date.Only trainees at SHO or registrar grade with Part I MRCPsych were eligible.Sixteen applications were received.Of these, half were chosen randomly for course attendance and the other eight were assigned to a waiting list for the following year's course.Neither group was aware of the randomised procedure until after follow-up had been completed, six months later.
Ten 2{ hour sessions at weekly intervals were planned.Each session was split into two parts.The first 90 minutes was led by different invited experts, each on a particular research topic: epidemiology, biological research, clinical trials, data analysis and so on.The discussion was focused around two or three key papers chosen in advance by the expert, and presented by the trainees.The second part of the session was designated a "project workshop" whose structure, apart from being facilitated by the same trainer (myself) each week, was left open until the beginning of the course.
At the start of the course, each of the eight attenders was asked to complete a three-part questionnaire.The first part enquired about research experience, and any publications, to date.The second part asked the trainees to rate their responses (from 'strongly agree' to 'strongly disagree') to ten statements chosen to examine research-related atti tudes, including such items as "Research is easier than clinical work", "Statistical analysis is a mystery to me", "I find it easy to get research ideas", "Publi cations are an important part of a trainee's C.V.", "Research improves clinical skills", and so on.The third part of the questionnaire listed eight possible aims of a research course, and asked the trainee to indicate the four which he or she thought most important from their own point of view.From then on.the weekly workshop sessions were geared to ful fil those aims which had collectively been considered to be the most important.The eight applicants assigned to the waiting-list condition were also given the questionnaire.
The study set out to test the following hypotheses: (a) negative attitudes about aspects of research could be changed usefully by the research course (b) expressed aims could be successfully met by the research course (c) at follow-up, course-attenders would be engaged in more research projects, and have more publications printed or in press, than the non-attending control group.There was no significant difference at the outset between the group attending the research course and the waiting-list control group in terms of research experience or number of publications.Three of the eight attenders already had one publication, v. two of the eight non-attenders.Four trainees in each group stated they were actively engaged in a research project; the others had no experience of research.
At the start of the course, three of the attitude items scored particularly highly.At least half the trainees strongly disagreed with the statement "I get enough time to do research", and at least half strongly agreed with "Statistical analysis is a mystery to me".All trainees indicated strong agreement with the statement "Publications are an important part of a trainee's C.V." These attitudes were reflected in the choice of aims the trainees thought the course should be addressing.All eight course attenders thought that one of the main aims of the course should be to get attenders a publication.The second most commonly expressed aim was to get started on a research project.
Accordingly, the workshop was oriented towards these goals.Current journals were examined and trainees encouraged to think critically about selected articles.Fashionable topics for research were identi fied.A list of a dozen specific hypotheses was drawn up and trainees constructed individual protocols around these.Emphasis was placed on original ideas testable with simple projects likely to take no more than a year of trainee's spare work time.The work shop was conducted in a collaborative way, with trainees working in pairs or small groups and ideas discussed for strengths and weaknesses.

Evaluation of the research course
At the end of the course, attending trainees once more completed the attitude questionnaire.The scores for individual items were compared to the initial scores.Of the change possible on individual items, there was less than 10% change at follow-up on five of the ten items.
Change was statistically significant on the last two items (Wilcoxson sign-rank, P< 0.05, two-tailed).
Did the course meet the aims set by the trainees?At the end of the course, each attender was given back the aims questionnaire which he or she had com pleted at the start, and asked to indicate whether or not each expressed aim had been fully met.Of the two aims expressed by the majority, six of the eight trainees stated that both aims had been fully met.
Changing negative attitudes and fulfilling subjec tive aims are all very well, but a more stringent test of the course was made at three-month follow-up.Both the course-attending group and the waiting-list con trol group were followed up with a repeat of the first part of the initial questionnaire, about research activity and publications.In the six month interval between the two evaluations, all eight of the attenders had started and continued to pursue a research project, compared with two of the nonattenders (Fisher's exact, P=0.003, one-tailed).
Furthermore, six of the eight attenders either had a new publication or one in press, compared to none of the non-attenders (Fisher's exact, P=0.003, onetailed).Thus, the research course seemed to work, not only as judged by the trainees' own criteria obtained at the onset and by the change in some negative attitudes about research, but also by the improved performance on objective indicators of research activity by course-attenders compared to non-course-attenders. and the presence or absence of a publication is one of the few convenient ways to distinguish applicants for a post.The more important reason is that trainers generally believe that evidence of completed research and publications says more about a trainee than a good reference ever can.It is said to communicate the impression that a trainee has several virtues: a spirit of critical enquiry, the ability to observe and question, an interest in keeping abreast of the current literature, and qualities of organisation and persistence, all of which are seen to reflect a commitment to the chosen speciality.The validity of these assumptions is clearly open to question.Nonetheless, the implementation of Achieving a Balance will shift these issues still further down the career ladder.

Comment
Judging from the evidence of the initial shortlisting study, the attenders at the course can expect in future to have a better chance of being interviewed for the senior registrar job they want.Before and after the research course, trainees were unanimous in their strong agreement that publications were an import ant aspect of a trainee's C.V. Talking to colleagues, most trainers also seem to believe this, mainly for two reasons.The less important reason is that, at the end of their general training, many trainees on paper are very similar in terms of approved clinical experience,