Skip to main content Accessibility help

Observing competence in CBT supervision: a systematic review of the available instruments

  • Derek L. Milne (a1) and Robert P. Reiser (a2)


Government policy, like evaluations of clinical practice, indicates the growing importance of supervision in fostering practitioner development and in improving the fidelity of therapies. However, instruments with which to measure competent supervision are often problematic, thereby hampering these key activities (e.g. they are rare, rely on self-ratings by participants, and psychometric data can be limited). To contribute to progress, this paper reviews the current options for measuring competent clinical supervision by means of direct observation, a favoured approach within cognitive behaviour therapy (CBT). We systematically reviewed 10 existing instruments that were designed to observe and quantify competent supervision, focusing on three broad criteria for sound measurement (i.e. an instrument's Design, Implementation, and Yield: DIY). Suggestions for future research on instruments that can fulfil the functions that are provided distinctively through direct observation are outlined.


Corresponding author

*Author for correspondence: D. L. Milne, Ph.D., School of Psychology, Ridley Building, Newcastle University, Newcastle upon Tyne NE1 7RU, UK (email:


Hide All
Alliger, GM, Tannenbaum, SI, Bennett, W, Traver, H, Shotland, A (1997). A meta-analysis of the relations among training criteria. Personnel Psychology 50, 341359.
Anastasi, A, Urbina, S (1997). Psychological Testing. NJ: Prentice-Hall.
APA Presidential Task Force (2006). American Psychological Association Presidential Task Force on evidence-based practice in psychology. American Psychologist 61, 271285.
Baer, JS, Ball, SA, Campbell, BK, Miele, GM, Schoener, EP, Tracy, K (2006). Training and fidelity monitoring of behavioural interventions in multisite addictions research. Drug and Alcohol Dependence 87, 107118.
Bambling, M, King, R, Raue, P, Schweitzer, R, Lambert, W (2006). Clinical supervision: its influence on client-rated working alliance and client symptom reduction in the brief treatment of major depression. Psychotherapy Research 16, 317331.
Barkham, M, Evans, C, Margison, F, McGrath, G, Mellor-Clark, J, Milne, DL, Connell, J (1998). The rationale for developing and implementing core outcome batteries for routine use in service settings and psychotherapy outcome research. Journal of Mental Health 7, 3547.
Barlow, DH, Nock, MK, Hersen, M (2009). Single-Case Experimental Designs: Strategies for Studying Behavior Change. Boston: Allyn & Bacon.
Bellg, AJ, Borrelli, B, Resnick, B, Hecht, J, Minicucci, DS, Ory, M, Ogedegbe, G, Orwig, D, Ernst, D, Czajkowski, S (2004). Enhancing treatment fidelity in health behaviour change studies: Best practises and recommendations from the NIH Behaviour Change Consortium. Health Psychology 23, 443451.
Bernard, JM, Goodyear, RK (1998). Fundamentals of Clinical Supervision (2nd edn). London: Pearson.
Cherniss, C (1986). Instrument for observing supervisor behaviour in educational programs for mentally retarded children. American Journal of Mental Deficiency 91, 1821
Clark, HB, Wood, R, Kuehnel, T, Flanagan, S, Mosk, M, Northup, JT (1985). Preliminary validation and training of supervisory interactional skills. Journal of Organizational Behavior Management 7, 95115.
Copeland, HL, Hewson, MG (2000). Developing and testing an instrument to measure the effectiveness of clinical teaching in an academic medical centre. Academic Medicine 75, 161166
Department of Health (1993). A vision for the future. London: NHS Executive.
Department of Health (2001). Working together – learning together: a framework for lifelong learning for the NHS. London: Department of Health.
Department of Health (2004). Organising and delivering psychological therapies. London: Department of Health.
Dreyfus, HL, Dreyfus, SE (1986). Mind over Machine: The Power of Human Intuition and Expertise in the Era of the Computer. Oxford: Blackwell
Ducharme, JM, Williams, L, Cummings, A, Murray, P, Spencer, T (2001). General case quasi-pyramidal staff training to promote generalization of teaching skills in supervisory and direct-care staff. Behavior Modification 25, 233254.
Ellis, MV, Ladany, N (1997). Inferences concerning supervisees and clients in clinical supervision. An integrative review. In: Handbook of Psychotherapy Supervision (ed. Watkins, C. E.), pp. 447507. New York: Wiley.
Ellis, MV, D'Iuso, N, Ladany, N (2008). State of the art in the assessment, measurement, and evaluation of clinical supervision. In: Psychotherapy Supervision: Theory, Research and Practice (ed. Hess, A. K., Hess, K. D. and Hess, T. H.), pp. 473499. Chichester: Wiley.
Epstein, RM, Hundert, EM (2002). Defining and assessing professional competence. Journal of the American Medical Association 287, 226235.
Falender, CA, Shafranske, E (2004). Clinical Supervision: A Competency-based Approach. Washington, DC: APA.
Falender, CA, Shafranske, EP (2008). Casebook for Clinical Supervision: A Competency-Based Approach. Washington, D.C: American Psychological Association.
Falender, CA, Shafranske, E (2010). Psychotherapy-based supervision models in an emerging competency-based era: a commentary. Psychotherapy Theory, Research, Practice and, Training 47, pp. 4550. Washington, DC: APA.
Fleming, RK, Oliver, JR, Bolton, DM (1996). Training supervisors to train staff: a case study in a human service organization. Journal of Organizational Behavior Management 16, 325.
Gilbody, S, Bower, P, Fletcher, J, Richards, D, Sutton, AJ (2006). Collaborative care for depression: a meta analysis and review of longer-term outcomes. Archives of Internal Medicine 166, 23142320.
Gregoire, TK, Propp, J, Poertner, J (1998). Supervisors’ role in the transfer of training. Administration in Social Work 22, 117.
Heaven, C, Clegg, J, McGuire, P (2005). Transfer of communication skills training from workshop to work place: the impact of clinical supervision. Patient Education and Counselling 60, 313325.
Henggeler, SW, Schoenwald, SK, Liao, JG, Letourneau, EJ, Edwards, DL (2002). Transporting efficacious treatments to field settings: the link between supervisory practices and therapist fidelity in MST programmes. Journal of Clinical Child Psychology 31, 155167.
Iberg, JR (1991). Applying statistical process control theory to bring together clinical supervision and psychotherapy research. Journal of Consulting and Clinical Psychology 59, 575586.
James, IA, Blackburn, IM, Milne, DL, Freeston, M (2005). Supervision training and rating scale for cognitive therapy (stars - CT). Newcastle Cognitive Therapy Centre, England.
Kaslow, NJ, Borden, KA, Collins, FL, Forrest, L, Illfelder-Kaye, J, Nelson, PD (2004). Competencies conference: future directions in education and credentialing in professional psychology. Journal of Clinical Psychology 60, 699712.
Kazdin, AE (1998). Research Design in Clinical Psychology. Boston: Allyn Bacon.
Kenkel, MB, Peterson, R.L. (2010). Competency-Based Education for Professional Psychology. Washington, D.C.: American Psychological Association.
Komaki, JL, Zlotnick, S, Jensen, M (1986). Development of an operant-based taxonomy and observational index of supervisory behaviour. Journal of Applied Psychology 71, 260269.
Kraiger, K, Ford, JK, Salas, E (1993). Application of cognitive, skill based and affective theories of learning outcomes to new methods of training evaluation. Journal of Applied Psychology 78, 311328.
Lambert, NJ, Ogles, BM (1997). The effectiveness of psychotherapy supervision. In: Handbook of Psychotherapy Supervision (ed. Watkins, C. E.), pp. 421446. New York: Wiley.
Liese, BS, Beck, JS (1997). Cognitive therapy supervision. In: Handbook of Psychotherapy Supervision (ed. Watkins, C. E.), pp. 114133. New York: Wiley.
Lucock, MP, Hall, P, Noble, R (2006). A survey of influences on the practice of psychotherapists and clinical psychologists in training in the UK. Clinical Psychology and Psychotherapy 13, 123130.
Milne, DL (2007). Evaluation of staff development: the essential ‘SCOPPE’. Journal of Mental Health 16, 389400.
Milne, DL, James, IA (2002). The observed impact of training on competence in clinical supervision. British Journal of Clinical Psychology 41, 5572.
Milne, DL, James, IA, Keegan, D, Dudley, M (2002). Teachers PETS: a new observational measure of experiential training interactions. Clinical Psychology and Psychotherapy 9, 187199.
Milne, DL, Reiser, R, Aylott, H, Dunkerley, C, Fitzpatrick, H, Wharton, S (2010). The systematic review as an empirical approach to improving CBT supervision. International Journal for Cognitive Psychotherapy 3, 278293.
Olk, ME, Friedlander, ML (1992). Trainees experience of role conflict and role ambiguity in supervisor relationships. Journal of Counselling Psychology 39, 389397.
Parsons, MB, Reid, DH (1995). Training residential supervisors to provide feedback for maintaining staff teaching skills with people who have severe disabilities. Journal of Applied Behaviour Analysis 28, 317322.
Petticrew, M, Roberts, H (2006). Systematic Reviews in the Social Sciences: A Practical Guide. Oxford: Blackwell.
Prescott, LE, Nocini, JJ, McKinlay, P, Rennie, JS (2002). Facing the challenges of competency based assessment of post-graduate dental training: longitudinal evaluation of performance (LEP). Medical Education 36, 9297.
Reid, DH, Rotholz, DA, Parsons, MB, Morris, L, Braswell, BA, Green, C, Schell, RM (2003). Training human service supervisors in aspects of PBS: Evaluation of a state-wide, performance based program. Journal of Positive Behavior Interventions 5, 3546.
Roth, A, Fonagy, P (1996). What Works for Whom? A Critical Review of Psychotherapy Research. New York: Guilford Press.
Roth, AD, Pilling, S (2008). Using an evidence-based methodology to identify the competencies required to deliver effective cognitive and behavioural therapy for depression and anxiety disorders. Behavioural and Cognitive Psychotherapy 36, 129147.
Schacht, AJ, Howe, HE junior, Berman, JJ (1988). A short form of the Barrett–Lennard relationship inventory for supervisor relationships. Psychological Reports 63, 699706.
Shanfield, SB, Mohl, PC, Matthews, KL, Hetherly, V (1989). A reliability assessment of the psychotherapy supervisory inventory American Journal of Psychiatry 146, 14471450.
Shanfield, SB, Mohl, PC, Matthews, KL, Hetherly, V (1992). Quantitative assessment of the behaviour of psychotherapy supervisors. American Journal of Psychiatry 149, 352357.
Sudak, D, Wright, J, Bienenfeld, D, Beck, J (2001). Cognitive behaviour therapy supervision checklist. Philadelphia: Cognitive Therapy Centre.
Vleuten, CPM, Schuwirth, LWT (2005). Assessing professional competence: from methods to programmes. Medical Education 39, 309317.
Wampold, BE, David, B, Good, RH (1990). Hypothesis validity of clinical research. Journal of Consulting and Clinical Psychology 58, 360367.
Watkins, CE (ed.) (1997). Handbook of Psychotherapy Supervision. New York: Wiley.
Watkins, CE (1998). Psychotherapy supervision in the 21st century. Journal of Psychotherapy Practice & Research 7, 93101.
White, JHD, Rudolph, BA (2000). A pilot investigation of the reliability and validity of the Group Supervisory Behavior Scale (GSBS). The Clinical Supervisor 19, 161171.



Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed

Observing competence in CBT supervision: a systematic review of the available instruments

  • Derek L. Milne (a1) and Robert P. Reiser (a2)
Submit a response


No Comments have been published for this article.


Reply to: Submit a response

Your details

Conflicting interests

Do you have any conflicting interests? *