CLINICAL TUTORS' SUB COMMITTEE CONFERENCE ON ASSESSMENT OF POSTGRADUATE TRAINING IN PSYCHIATRY

areas of psychiatry (e.g. drug effects and the borderland of psychiatry and neurology). Problemsolving and decision-making were more difficult to test. In the USA sample case histories had been presented to candidates, who were asked to choose between alternative courses of action. Such tests were complicated to construct and score and were vul nerable to examination strategyâ€”for example, the wording of later problems tended to suggest the correct answer to earlier ones. In the testing of clinical skill he felt that patients given as clinical cases should be under active treatment and have 'live' problems;

This Conference was held at the Scientific Societies Lecture Theatre, Savile Row, London on 5th February 1979. The President, PROFESSOR D. A. POND, was in the Chair.
The first speaker was DR JOHN STOKES,the chief architect of the revision of the MRCP examination. Dr Stokes defined the aims of assessment procedures as being to evaluate a trainee's standing in four areasâ€" fund of information, problem-solving and decisionmaking, clinical skill and attitudes. Multiple-choice questions were the best way of testing factual know ledge, but nothing else; and in his experience psychiatric MCQ's tended to be confined to limited areas of psychiatry (e.g. drug effects and the borderland of psychiatry and neurology). Problemsolving and decision-making were more difficult to test. In the USA sample case histories had been presented to candidates, who were asked to choose between alternative courses of action. Such tests were complicated to construct and score and were vul nerable to examination strategyâ€"for example, the wording of later problems tended to suggest the correct answer to earlier ones. In the testing of clinical skill he felt that patients given as clinical cases should be under active treatment and have 'live' problems; the examiner should be present throughout the history taking. The candidate's interview with the patient could be videotaped and judged later, but this was difficult and time-consuming, and the equipment might be off-putting. In evaluating attitudes Dr Stokes felt that examiners tended to look for people like themselves. Trainers'judgements were important and could be graded and classified. He drew attention to the need to assess the ability of trainees to work in teams.
In discussion, Dr Stokes said he felt that essay questions were not very useful at the postgraduate level ; they tended to be marked on factual content rather than on ability to write good English or organize thoughts, which is what essays are supposed to test.
PROFESSORH.J. EYSENCK, speaking on 'Psychology in the Training of Psychiatrists' reminded the meet ing of Sir Aubrey Lewis' view that psychology is a most important part of a psychiatrist's training. In recent years the quality and quantity of instruction in psychology given to psychiatric trainees had deterior ated. Examination questions were less searching and trainees less interested. At the same time the contribu tion of psychologists to psychiatry had greatly increased. He felt that psychiatrists needed to know more about, for example, modern research on memory, the theoretical basis of conditioning procedures, personality, motivation and intelligence testing. He appreciated the difficulty of adding to existing psychiatric training but thought that at least a year's full time training in psychology was needed to enable a psychiatrist to become a fully effective behaviour therapist. He emphasized the importance of continuous monitoring of trainees' performance and of experiments to assess the reliability and validity of examinations.
PKOH-SSORHFNRY WALTONdrew attention to the great variation in the effects of training programmes on psychiatric trainees. Many factors known to be important were not available to an equal degree in all programmes. There was evidence, for example, that trainees gain considerably from fellow students, the peer group ihus forming an important component of training.
The tutor's aim should be that all trainees accepted into the programme should complete their training and become psychiatrists: selection, therefore, was a crucial preliminary of training. Tutors had the respon sibility to manage the learning process; weaknesses in trainees were a challenge to be met, and resources must be mobilized to overcome them. He distinguished between sumrnative evaluation, e.g. accreditation of specialists, and formative evalua tionâ€"the in-course assessment for purposes of instruction, which was a part of training. The student had to know in advance what was expected of him. This was conveyed by instructional objectives which told him what he must know and be like at the end of each course and at the end of the training programme. Feed-back had to be given at each stage. Professor Walton described a number of approaches which had been used for defining training objectives. These included the critical incident technique, an approach calling for a large sample of experienced psychiatrists to stipulate the main clinical demands occurring in ordinary practice. An epidemiological approach was another method used, and was based on study of the range of clinical problems presented by patients, which an adequately trained psychiatrist should manage. He felt that the College could adopt either or both of these approaches to define training objec tives.
Although careful selection of trainees was essential and contributed more to the outcome of training than any other variable, selection techniques for entry to psychiatric training were perfunctory as compared, for example, with those for the Army. The General Medical Council now viewed the concept of the preregistration year divided into two six-month periods as obsolete. The pre-registration period, like the medical school phase, should be used for general medical education, because all doctors now obtained prolonged postgraduate training in the specialty they selected. If psychiatry came to be widely used as an option in the pre-registration period, that would have important implications for recruitment.
Professor Walton next reviewed examination pro cedures. Separate areas needing evaluation included factual knowledge, clinical skills and professional attitudes. MCQ_ examinations had been more satis factory than might have been anticipated from the alleged 'softness' of facts in psychiatry. Their main weakness was that when test items were straight forward diey tended to test only rote learning, and not problem solving. He agreed fully with Dr Stokes' views about essay questions and was surprised that tutors rarely seemed to give trainees the chance to practise this technique in the course of training. For essay questions to be used adequately, model answers should be provided and answers marked independently by several examiners. The require ments were met in the Membership examination. He considered short answer questions preferable to essays; they extended the range of knowledge tested and were scored more reliably. Oral examinations were of ancient origin and could be very informative, but inter-examiner variation was notorious and they could deteriorate into a factual exercise (better applied in MCQJ. The College was right to have pairs of oral examiners; the guide prepared for examiners was critically important in all parts of an examination. Videotaped interviews could be used in place of actual patients in clinical examining, but the weakness then was that the clinical information was not elicited by the candidate.
Professor Walton then dealt with the place of continuous assessment, evaluation by instructors at the end of each clinical attachment. Instructors must be informed clearly of the evaluative dimensionsâ€"he suggested rating scales, with additional space for tutors to make narrative comment about trainees. The student had to be told his instructor's opinion of him, unless there were special reasons for not doing so. Trainees clearly unsuitable for a career in psychiatry should be told this as soon as possibleâ€"for example, after two adverse six-monthly reports.
In discussion, PROFESSOR A. H. CRISPsaid that at St. George's Hospital trainees had been given die right to assess consultantsâ€"their main complaint was of insufficient time with the consultant.
DR J. L. T. BIRLF.Ydescribed the assessment pro cedures in use at the Maudsley Hospital. Trainees at the Maudsley were selected by a committee meeting twice yearly. Trainees were entering psychiatry earlier, and the majority did not have the MRCP (UK), which he did not regard as an important qualification for a psychiatrist. Progress was reviewed every six months in such areas as clinical skills, relations with colleagues, willingness and ability to take advice, and administra tive skill. Reports were discussed between trainer and trainees, and the Vice-Dean saw all the trainees at regular intervals. An old-fashioned 'dressing-down' was sometimes useful. Dr Birley emphasized the need to keep consultants and tutors up to date and deplored the tendency to see postgraduate education as a process of saturating trainees with facts rather than one of awakening their curiosity.
In discussion, the importance of good psychiatric experience in G.P. Vocational Training Schemes was emphasized. These schemes have proved a good recruiting ground for psychiatry. The possibility of requiring MRCPsych. candidates to submit a 'log book' of experience was discussed. DR T. H. BEWLEY feared that this might lead trainees to do only the minimum required. Another suggestion was that essay questions should be sent out several weeks before hand and candidates allowed access to books and that this might lead to better testing of the ability to marshal thoughts and express them in good English. The PRESIDENT, however, felt that this would be open to abuse.
The Conference, which was attended by about 100 Clinical Tutors and others, provided a stimulating survey of the current situation in an area of increasing importance. The Scientific Societies Lecture Theatre is an exceptionally pleasant and well appointed venue and might well be considered for future College Meetings.