Mixed-methods evaluation of a leadership development programme for higher specialist trainees

learning. knowing; Aims and method Newly appointed consultant psychiatrists have reported that management and leadership is an area for which they are unprepared. Our aim was to evaluate the impact of a leadership development programme based on the principles of ‘action learning’ for higher trainees. A questionnaire survey was sent to 54 trainees and consultants who had attended the programme. Qualitative interviews were conducted with 15 participants using the concept of maximum variance sampling. Results The relevance of the topics covered, the opportunity for free discussion and increased understanding of National Health Service policy were rated highest. At the end of a thematic analysis, themes were organised into four major categories: (a) lack of engagement with management during training; (b) the lasting impact of the leadership development programme; (c) understanding the larger organisational context; and (d) transition to consultant psychiatrist. Clinical implications The ﬁndings suggest that programmes such as this can be successful in raising awareness and increasing engagement in medical leadership and management. Declaration of interest None.

Modern healthcare is increasingly complex and management and leadership are now a key part of a doctor's professional work regardless of specialty and setting. It is a requirement as laid out in Good Medical Practice 1 and Management for Doctors. 2 Newly appointed consultant psychiatrists have consistently reported that this is an area of their work for which they feel most unprepared. 3 A report by The Commission for Health Improvement 4 identified the difficulties in securing doctors' interest in management, particularly for those working in mental health, but reported that services do better when doctors get involved.
A review of the literature suggests that trainees are still experiencing difficulties acquiring management skills despite some innovative initiatives. 5,6 For example, Hewson et al 7 and Spurgeon & Vassilas 8 introduced management courses. Feedback suggested that these initiatives were well received, but the samples in both studies were relatively small, made up of newly appointed consultants only 7 and as the evaluations took place straight after training the authors were unable to examine any lasting impact.
The High Quality Care for All: NHS Next Stage Review report 9 pledged to incorporate leadership development and training into both undergraduate and postgraduate curricula. The Medical Leadership Competency Framework 10 and associated Medical Leadership Curriculum 11 set out a minimum set of management and leadership skills that all doctors now have to acquire to be able to fulfil their practitioner roles effectively. Along with Mental Health: New Ways of Working for Everyone 12 and the reforms laid out in the Health and Social Care Bill 2011, 13 it is likely that the role of the consultant psychiatrist will change again and that the years ahead will be very different. For this reason it is imperative that our future psychiatrists are equipped with the necessary skills to function efficiently and effectively within an ever increasing complex healthcare system. 14 The aim of this evaluation was to examine the impact of the leadership development programme using a combination of research methods, including a questionnaire survey and qualitative in-depth interviews.

Action learning
Reg Revans 16 coined the term action learning to heal the split between thinking and doing to help us to understand how human beings can work together more effectively. In the late 1920s as a doctoral student of astrophysics and then a fellow at Cambridge University, he worked alongside eight Nobel Prize winners. These experiences fuelled strong beliefs that underpin the key principles of action learning. These include: (a) learning starts from not knowing; (b) Aims and method Newly appointed consultant psychiatrists have reported that management and leadership is an area for which they are unprepared. Our aim was to evaluate the impact of a leadership development programme based on the principles of 'action learning' for higher trainees. A questionnaire survey was sent to 54 trainees and consultants who had attended the programme. Qualitative interviews were conducted with 15 participants using the concept of maximum variance sampling.

Results
The relevance of the topics covered, the opportunity for free discussion and increased understanding of National Health Service policy were rated highest. At the end of a thematic analysis, themes were organised into four major categories: (a) lack of engagement with management during training; (b) the lasting impact of the leadership development programme; (c) understanding the larger organisational context; and (d) transition to consultant psychiatrist.
Clinical implications The findings suggest that programmes such as this can be successful in raising awareness and increasing engagement in medical leadership and management.
Declaration of interest None.

EDUCATION & TRAINING
Mixed-methods evaluation of a leadership development programme for higher specialist trainees Christine Healey, 1 David Fearnley, 2 Mandy Chivers, 2 Ovais Wadoo, 3 Peter Kinderman 1 people who take responsibility in a situation have the best chance of taking actions that will make a difference; (c) learning involves both what is taught and read and a questioning insight; and (d) learning should be greater than the rate of change. 15

Objectives of the management leadership programme
The Mersey Care National Health Service (NHS) Trust specialist registrar leadership development programme was initiated in 2006 in an attempt to raise awareness and increase engagement in medical leadership and management. The programme was run by two of the authors (D.F. and M.C.) so did not involve any additional costs to the Trust. Based on the principles of 'action learning' 15 the programme consisted of bi-monthly meetings for trainees who were in their final 2 years of training in psychiatry, as part of the Mersey Deanery training scheme. Each meeting was facilitated by two of the authors (D.F. and M.C.), and included a guest speaker who would describe their area of work, and then a more structured action learning set would take place to take forward ideas. An action learning approach was encouraged in this way. The guest speakers varied considerably and include a senior manager at a major utility company, various Mersey Care NHS Trust managers (deputy director of finance, director of service user and carer involvement, deputy director of human resources chief executive and clinical director), the medical director for hospices and a journalist. Trainees were also able to lead the discussion as guest speakers on a few occasions. They selected topics connected to leadership and undertook some research before leading the next meeting.

Questionnaire
Based on the findings of Hewson et al 7 and Spurgeon & Vassilas 8 a questionnaire was drawn up to examine the impact of the leadership development programme. The questionnaire consisted of 15 items broken down into two sections. Section one consisted of ten items on a five-point Likert scale that rated the style, relevance and amount of personal involvement in the programme. Section two consisted of five items on a seven-point Likert scale that covered five aspects of management appreciation.

Qualitative interview
A semi-structured interview schedule was drawn up on the basis of the results from the questionnaire. This would allow additional questions to be asked on the basis of responses given. The schedule consisted of 19 items covering the following domains: (a) perspectives on the leadership development programme; (b) perspectives on management and leadership; and (c) perspectives on the transition to consultant psychiatrist. Interviews were conducted face to face and audio-recorded. The transcripts were subject to a thematic analysis. 17

Recruitment
One of the authors, C.H., sent the questionnaire electronically and by post to 54 trainees and consultants who had attended the programme. Thirteen had left the Trust and could not be traced. For the qualitative interviews we aimed to recruit a minimum of 15 participants from the 54 who attended the programme using the concept of maximum difference sampling. 18 This involves deciding beforehand what elements of a population to include in a sample. One of the main advantages of using this sampling technique is that it seeks to uncover a full range of views rather than focus on just typical or average cases. It also ensures that most or all of the perceptions that might be important are uncovered. 19 For this study the sample would include at least one consultant who had completed the programme: (a) 36 months previously; (b) 24 months previously; (c) 12 months previously; (d) a currently acting consultant; (e) as well as a specialist trainee year 6 (ST6) and year 5 (ST5). Most qualitative studies aim for a minimum of 15 participants. 19 For the purpose of this study a larger sample would have been too labour intensive and time consuming. A letter of invitation was sent electronically and by post by C.H. to all the trainees who had been sent the survey earlier.

Procedure
As this was an evaluation of an educational initiative formal National Research Ethics Service ethical approval was not required. To improve the response rate to the questionnaire participants were informed that answers would be aggregated and were advised to post back the questionnaire anonymously. The only information requested was on grade. C.H. conducted and transcribed the qualitative interviews. Participants were reassured that the final transcripts would contain nothing that might identify them personally before being viewed by other members of the research team.

Analysis
Data from the questionnaires were entered into predictive analytics software version 18 (PASWS v18 on Windows) and subject to summary statistics and standard hypothesis tests. Data from the qualitative interviews were subject to an inductive thematic analysis. 17 C.H. and P.K. performed and led the analysis. Through its theoretical freedom, thematic analysis provides a flexible and useful research tool, which can potentially provide a rich and detailed, yet complex account of data. A theme was identified if it captured something important in relation to the interview questions. 18 To improve the trustworthiness of the analysis the first five transcripts were coded by C.H. and sorted into initial themes. The transcripts were then passed on to the research team for feedback. This involved researchers from different disciplinary backgrounds including two consultant psychiatrists (D.F. and O.W.), a clinical psychologist (P.K.), a speech therapist (M.C.) and a social worker (C.H.). This prevented the personal or disciplinary biases of a single researcher from dominating the analysis. 20 Modifications were made to the interview schedule on the basis of feedback received. 20 The next five transcripts along with initial categories and confirmatory or disconfimatory evidence for initial themes were circulated to the team before embarking on the final five interviews. At the end of the analysis the themes were organised into four major categories through consensus between all researchers. 21

Questionnaire
The overall response rate to the questionnaire was 21 responses from a possible 41 respondents (51%); 67% were higher specialist trainees and 33% were consultant psychiatrists. All questions had a 100% response rate.
In section one of the questionnaire, the relevance of the topics covered and the opportunity for free discussion were rated highest, with a mean and median score of four (Table 1). This was followed by the appropriateness of the written material, the opportunity to discuss local services, understanding the management roles of a consultant and an increased understanding of the wider organisational context. The trainees rated questions five and seven higher than the consultants and this difference reached statistical significance (Mann-Whitney U-test P50.5).
In section two of the questionnaire, increased understanding of the NHS policy and awareness of factors affecting the Trust was rated highest ( Table 2). The trainees rated question 13 and 15 higher than the consultants and this difference reached statistical significance (U-test P50.5).

Qualitative interviews
A total sample of n = 15 consisting of higher trainees and consultants were recruited for the in-depth interviews (Table 3). At the end of the thematic analysis the themes were organised into four major categories.
Theme one: a lack of engagement with management during training The early years of training were described as being dominated by the need to pass exams and build clinical skills. The need for more hands-on practical management experience was a constant theme for both the trainees and the consultants. Opportunities for this seemed to depend on individual placements and supervisors. 'If the person mentoring you is quite influential you will get a project where you will be able to develop and flourish in many ways.' (acting consultant)  A number of trainees felt that the Medical Leadership Competency Framework 10 and Medical Leadership Curriculum 11 too prescriptive and based too much on competencies. The reduction in special interest sessions from four a week to two in which to achieve a number of competencies was viewed as difficult. 'We need all these things in our portfolio, but it's not explicit how we should go about it. There is a sense that people are just ticking off activities. The role of supervision on placements should be about learning something you can apply on a wider basis.' (trainee) Theme two: the lasting impact of the leadership development programme There were many examples of action learning being incorporated into everyday practice. Respondents talked of sharing their difficulties and learning from and with other colleagues.
'Much less of a traditional model here where the consultant has all the answers. We share problems and solutions. Shared responsibility of the decision-making. It consolidates team support.' (consultant) But this was not always possible because of the expectations placed on consultants from junior staff.
'When I go to the ward, that sense of hierarchy is much more obvious. People can be quite deferential to you which I find uncomfortable. If it's the wrong decision you make, you can be left feeling quite exposed. If you have debate and other people have an opinion it feels a lot safer.' (consultant) Action learning was viewed positively by the majority of participants. The sessions allowed them to practice the skills they would need in the future.
'It was a good exercise [action learning sets]. They are the type of skills you need as a manager. Being diplomatic and putting things across in a way that helps you find solutions, instead of just complaining all the time.' (trainee) The programme had inspired many trainees to pursue higher degrees in health and social care management.
'I was inspired to take this MBA [Master of Business Administration]. My special interest has been management. The MBA supplements the introduction provided by the action learning.' (trainee) However, being a future leader of an NHS organisation was not for everyone. Although it was theoretically important to know about finances, commissioning, etc., it was felt that some people were perhaps not seen as natural leaders.
'I'm never going to be clinical director material. I still get frustrated with juggling targets and clinical need. I prefer to be. I'm okay with the operational type level of management. I could never be involved at a more strategic level.' (consultant) Theme three: understanding the larger organisational context All the respondents felt the programme had given them a good grounding in the structure of the NHS, management and finances. A number of sessions involving people in higher management from within the local trust were particularly valued. This made the people in higher management more accessible to the trainees and gave them personalities. It also challenged many assumptions that many trainees held.

Discussion
The leadership development programme was well received by the trainees and consultants, with the questionnaire scoring highly across most sections. This finding was validated through the qualitative interviews. For a number of respondents the principles of action learning had a Higher specialist trainee (ST6) 4 Higher specialist trainee (ST5) 3 lasting impact. There were numerous examples in the interviews of trainees and consultants using the principles in everyday practice and in special projects. For a number of respondents the action learning had whetted their appetite and generated a genuine interest in management and leadership as a future career. This is important because engagement in medical management and leadership has been shown to be associated with better outcomes. 22 Engaging trainees in action learning appears, therefore, to be an approach that should be developed to help lead to higher quality care. Compared with the findings of Hewson et al 7 and Spurgeon & Vassilas, 8 the mean and median scores in this study were slightly lower. However, this study was made up of a larger sample size and included both trainees and consultants who had been in post ranging from 12 to 36 months. The trainees rated questions 5, 7, 13 and 15 higher than the consultants, and this difference reached statistical significance. This could suggest a number of things: (a) the trainees value the programme more but this tends to reduce once they move into the consultant role; (b) the programme has changed in some way and newer cohorts are rating it higher; or (c) the trainees themselves have changed in some way.
The findings from the qualitative interviews support the view that the trainees have changed and that the nonclinical management role of the consultant is becoming more widely embraced. But many were still finding it difficult to gain hands on management experience. Having management incorporated into formal supervision might be one solution, but the trainees themselves may also have to be more proactive in creating opportunities to take on more managerial responsibilities. For those with a genuine interest in medical management as a future career, having more dedicated time to enhance skills might be another option.
This study is one of only a small number to systematically examine the extent to which initiatives such as this are preparing trainees for future consultant posts. The findings from this evaluation suggest that leadership development programmes can be successful in raising awareness and increasing engagement in medical leadership and management.