Can internet-based tests add anything useful for the psychotherapist?

Computers have made possible the analysis of complex tests that are more psychotherapy friendly but are prohibitively time-consuming to rate by hand. These include the repertory grid. The ease with which tests can be converted into internet forms has also made tests selected, administered and rated by users widely available. Arguably few of these have any psychotherapeutic significance although some, such as the Myers–Briggs personality indicator (http://www.myersbriggs.org), have proven validity even when self-administered, and may serve to interest some internet surfers in psychological matters and psychotherapy. Expert systems, capitalising on the speed and memory of computers, have been slow to penetrate psychotherapy, but are gradually being adopted for treatment selection (e.g. http://www.systematictreatmentselection.com) and for tracking patient progress during therapy.

Computer adaptation is the main advantage of computer-based testing. Computer-adaptive tests are interactive. Item selection can be based on previous item scores and an algorithm, only the most discriminating questions need be asked and only enough need be asked to achieve a predetermined level of accuracy (Reference WainerWainer, 2000). In a study of 744 participants in depression treatment trials (Reference Gardner, Shear and KelleherGardner et al, 2004), a computer-adapted version of the 21-item Beck Depression Inventory showed both high efficiency (asking an average of only 5.6 items) and high accuracy in identifying major depressive episodes and measuring depression severity, making it a useful instrument for clinical practice.

Response to computer-administered tests can be by means of a keyboard, a touch-sensitive screen or mouse clicks on radio buttons. Since computer-adaptive tests are always likely to be shorter than their extended paper-based versions, it is likely that they will replace paper-and-pencil tests even in face-to-face situations, and they are ideally suited for use over the telephone. Tests can be administered on any computer with a suitable operating system and many are suitable for hand-held computers or personal digital assistants (PDAs).

It seems likely that computer-adapted tests will gradually replace other testing methods because of their savings in time. However, many psychologists remain sceptical although recognising that computers enable fast testing and early feedback of test scores to patients, which can accelerate the early stages of therapy (Reference Butcher, Perry and HahnButcher et al, 2004).

Repeat testing has been used as a means of tracking client progress, of predicting outcome, of detecting increased risk of unwanted events such as premature termination of therapy and of auditing professional performance. Owing to its ability to automate both testing and the analysis and feedback of tests, the computer lends itself particularly to such continual monitoring. One such application, the Outcome Questionnaire Analyst, is routinely used by a US managed-care company, PacifiCare Behavioral Health, and embedded into an outcome monitoring and management procedure called Algorithms for Effective Reporting and Treatment (ALERT). Provider groups with exceptional outcomes as reported by ALERT are given bonuses (Reference Percevic, Lambert and KordyPercevic et al, 2004). Feedback to individual therapists from computer analyses of pre- and post-session questionnaires completed by their clients can improve client adherence and therefore outcome, presumably as a result of altered therapist behaviour (Reference Whipple, Lambert and VermeerschWhipple et al, 2003). Monitoring of this kind has become a regular feature in other industries and evaluations suggest that it is accepted provided that it is considered fair (Reference Alder and AmbroseAlder & Ambrose, 2005).

There are other properties of computers that will probably influence test administration and development in the future. These include the ability to construct virtual environments, which carries with it the possibility that instead of answering test questions, individuals being tested could model aspects of their lives by creating environments or maps that simulate their experience of the world.

Anonymity and testing

One unexpected finding is that computer-based tests are answered more honestly when potentially shameful material is being elicited, including alcohol (Reference Lucas, Mullin and LunaLucas et al, 1977) or drug use, sexual activity, failure to take medication (Reference Millstein and IrwinMillstein & Irwin, 1983), history of sexual abuse (Reference Bagley and GenuisBagley & Genuis, 1991) and the frequency of suicidal thoughts (Reference Greist, Gustafson and StraussGreist et al, 1973). This may be related to the disembodiment of the medium that I have already discussed.

Cognitive–behavioural and other procedural therapies

The therapy that I have been considering here is of the exploratory kind that leads to personal change or to the re-evaluation of meaning or purpose in life. We have seen that more focused tasks, for example, test administration, can be carried out successfully by a computer loaded with the appropriate software. Psychotherapy aimed at the relief of distress falls into this category, since it follows well-tried procedures that work for most people, irrespective of their personal circumstances. At least some of these procedures can be reduced to algorithms, making them suitable for computer administration. So, can psychotherapy aimed at symptom relief be delivered by computer?

There is considerable evidence that it can. Reference Cavanagh and ShapiroCavanagh & Shapiro (2004) performed a meta-analysis of five computer-based treatments for anxiety or depression and found a sizeable effect compared with waiting-list controls. In a detailed analysis of one of the studies included in this meta-analysis, comparing computer-based treatment with standard care, Reference McCrone, Knapp and ProudfootMcCrone et al(2004) concluded that there was an 81% chance that computer-based treatment would be cost-effective.

Following its review of computerised cognitive–behavioural therapy (CCBT) the National Institute for Clinical Excellence (now the National Institute for Health and Clinical Excellence) concluded that ‘research suggests that CCBT may be useful for treating anxiety and depression. But at the moment this evidence on how and where CCBT should be delivered is not strong enough for the Institute to recommend introducing CCBT into general use in the NHS’ (National Institute for Clinical Excellence, 2002).

However, a further review is being conducted and the treatment may be yet recommended. NICE will undoubtedly be considering the assessment report from the School of Health and Related Research at the University of Sheffield (Reference Kaltenthaler, Brazier and De NigrisKaltenthaler et al, 2005). This concludes that ‘There is RCT evidence to support the effectiveness of Beating the Blues and FearFighter’ (p. iv).

Well-being

Broadly speaking, health interventions can be considered to be directed towards prevention, towards the reduction of symptoms and disability, or towards increasing positive health or well-being. The positive psychology movement has recently grown from research that suggests that the factors conducive to the relief of symptoms are not necessarily those that improve well-being. In essence, depression, anxiety, pain and certain kinds of disability reduce well-being, but the absence of these symptoms does not confer it. The creation of well-being is more related to social relationships, to spirituality and to engagement in life. Positive psychologists have begun to develop web-based methods to help people identify reasons why they do not experience well-being and to recommend things they can do to increase it. Some might argue that these are the latest recruits to the very large body of self-improvement sites on the internet, and it remains to be seen whether positive psychologists, or psychotherapists, do have more to say than spiritual leaders, churches, nutritionists, mediators, family counsellors, clinical and counselling psychologists and others who are already well-represented.

Virtual reality: psychotherapy by immersion

Immersive virtual environments offer new possibilities for therapy, for example by providing realistic but not actual exposure. This was first exploited in treating fear of flying, where it was but a step on from the flight simulators that were already in use. Virtual environments for psychotherapy are now being developed for conditions such as back pain (Reference Buhrman, Faltenhag and StromBuhrman et al, 2004), fear and avoidance related to speaking in public (Reference Botella, Hofmann and MoscovitchBotella et al, 2004), increasing the impact of psychosocial care in breast cancer (Reference Owen, Klapow and RothOwen et al, 2004), drug misuse (Reference Copeland and MartinCopeland & Martin, 2004), schizophrenia, autism, post-traumatic stress disorder (Reference Wiederhold and WiederholdWiederhold & Wiederhold, 2004), eating disorders (Reference Castelnuovo, Buselli and De FerrariCastelnuovo, et al, 2004) (see example at http://www.vepsy.com/2vrtsession.asf) and male sexual dysfunction (Reference Optale, Pastore and MarinOptale et al, 2004).

Virtual reality environments seem particularly suited to exposure therapy (Reference Emmelkamp, Krijn and HulsboschEmmelkamp et al, 2002), but they have also been used to provide training, for example social skills training for people with Asperger syndrome (Reference Parsons and MitchellParsons & Mitchell, 2002). Virtual reality sites created for many of these purposes suffer from comparison with the graphical rendering of commercial games that young people now take for granted and it seems likely that virtual reality therapy will have to be developed to the same commercial standards. One example of how this can be done is provided by a virtual reality therapy environment for veterans with combat-induced post-traumatic stress disorder (Reference Rizzo, Pair and McNerneyRizzo et al, 2005). The graphics in this program were imported from Full Spectrum Warrior, a commercial game based on a combat tactical simulation whose development was sponsored by the US Department of the Army.

E-learning: surely that’s dead and buried?

Electronic approaches to theoretical learning (e-learning) have been rapidly expanding in the past decade, but the ambitions of their protagonists have proved over-optimistic. Considerable investment was made in websites that would replace physical colleges and even universities. The demise of expensive learning collaborations such as the government-sponsored UK eUniversity (UKeU), which was wound up after it attracted only 900 students after set-up costs of £62 million (Anonymous, 2006), and Fathom, a partnership between several prestigious universities and the British Library, have created a backlash against the feasibility of e-learning (for information about the history of Fathom and free access to the archived materials go to http://www.fathom.com).

These projects probably failed because they made unnecessarily large initial outlays on web technology and web design, and not because, as might be thought, not enough people have access to the internet. In May 2005 statistics published by Internet World Statistics indicated that 35.5% of the 730 991 138 people in Europe had access to the internet, 46.9% in the European Union and 58.7% in the UK. Thus the UK was the fourth ‘most internet-connected’ country in the European Union, after Sweden, Denmark and The Netherlands, and the sixth in the world (http://www.internetworldstats.com/stats4.htm, accessed 30.5.05).

However, it is not access alone that limits the internet’s use for e-learning. There is also the perceived value of e-learning compared with face-to-face learning. In a survey carried out by the University of Cologne on behalf of the European Centre for the Development of Vocational Training, 80% of the 18 000 European Union citizens interviewed thought that life-long learning was important, but only just over a half were confident in using a computer (65% of male and 52% of female respondents). Thirty-one per cent of respondents had taken part in some form of education or training during the preceding year, but only 17% had even thought of taking a course at a school, university, or college, and only 12% would consider distance learning. Even fewer would consider using the internet for collaborative learning (Reference Chisholm, Larson and MossouxChisholm et al, 2004). This is a reflection of a general phenomenon in internet development. New ideas and new programs are quickly taken up by a minority of ‘early adopters’, but then take much longer to be accepted by the majority of internet users.

People who do try e-learning often express satisfaction with it (Reference Chisholm, Larson and MossouxChisholm et al, 2004). It offers a much richer educational environment than most classroom situations. Lectures can, of course, be delivered by computers, and lectures recorded at one site can be broadcast by satellite to many linked educational centres with suitable receiving facilities or, as bandwidth has increased, broadcast over the internet. However, e-learning environments do not need to be so restricted. Text can be provided with hyperlinks in which words or phrases can be linked to definitions, further material or exercises. Images, video clips, audio clips and hyperlinks to the enormous store of web information can also be provided.

Educational philosophy and e-learning

E-learning materials are rarely nowadays just plonked onto the net. They are embedded in ‘virtual learning environments’ (VLEs). These provide additional facilities such as student tracking, virtual whiteboards (which enable users to write or draw with their mouse or other pointing device on a screen area visible to other users), the setting and marking of assessments, and dedicated chatrooms and discussion groups. Most UK universities have a site license for a virtual learning environment, most often either Blackboard, in which Microsoft has a stake, or webCT. There are many other virtual learning environments, often the left-overs of some single-user application, but few have been developed to high commercial standards. One of the strengths of the internet is the provision of freeware, and there are also many ‘open source’ (i.e. free) virtual learning environments. A comparison of one of these, Moodle, with Blackboard found that students were more satisfied with the open source program (Reference Bos, Munoz and Van DuzerBos et al, 2005). This is consistent with my own experience. My colleagues and I have created a virtual learning environment specifically for delivering psychotherapy training (SEPTIMUS: see http://www.psychotherapytraining.net). We have found that the flexibility it provides (it is written using Dreamweaver and incorporates templates and open source hypertext preprocessor (PHP) add-ons) superior to either Blackboard or webCT.

E-learning materials are available from a limited number of universities worldwide, from collaborations such as those that created SEPTIMUS and as short training packages. An example of the latter is rCalipso, developed by a team representing a group of US psychoanalytic organisations (Reference Williams, Aubin and HarkinWilliams et al, 2001). rCalypso focuses on the teaching of therapeutic techniques (of cognitive–behavioural therapy) rather than on the underlying theory of psychotherapy, which is the topic of SEPTIMUS.

As before in this article, we seem to be arriving at the conclusion that computer-based methods are an alternative to traditional teaching, but are neither clearly superior nor inferior – at least while the computer is being used to emulate the classroom. And, as before, we are at the point where we are confronted with both the risks and the advantages of computer-based learning, which are a direct consequence of the technology.

Is e-learning effective?

Early meta-analyses of e-learning v. traditional learning indicate a considerable advantage of the former over the latter, but when like-for-like methods are compared (i.e. the same courses delivered by the same teachers in either e-learning or traditional formats), the differences disappear (Reference Broudo, White, Rodenburg, Scherpbier, van der Vleuten and RethansBroudo et al, 1997). However, e-learning is no less effective than traditional learning and, for students who are distant from training centres or who have family or other commitments that prevent them from attending, it may be the only available kind of learning. E-learning also has the advantage that there is a permanent record of teachers’ comments and interventions so that learners can refer back to these, if they wish.

Disembodiment, again

Computers, as we have seen, do not support the embodiment, the presence, that classrooms do. I have already considered the implications of this for self-revelation (Reference TantamTantam, 2006b ). Although at the moment the main medium continues to be text, increasing bandwidth is making video-conferencing more feasible. (Even text messaging is increasingly being extended by video, graphics and voice-internet telephony, as is evident from the growing complexity of one of the original free messaging programs, MSN Messenger, recently relaunched as Windows Live Messenger.) Video-conferencing allows a limited kind of embodiment and the look can re-enter the repertoire of e-learning gestures – but it will be a look from which much can easily be concealed.

Even with videoconferencing, the presence or absence of a student is likely to remain more obvious in a real than in a virtual classroom. Absence or lack of participation can, as all psychotherapists know, conceal disengagement, emotional turmoil, anger or incipient failure. Yet for such an important sign it is easy to overlook.

Most teachers who use virtual learning environments find that maintaining the materials (pruning broken hyperlinks, finding fresh ones, and updating questions, assessments and other materials) is much harder to do, even though less time-consuming, than writing the materials in the first place. Maintaining activity and surveillance of discussion forums is also harder once the initial excitement and enthusiasm has worn off. The very fact that the activity is flexible and can be fitted in anywhere increases the chance of it always being at the bottom of the priority list. Finally, knowing that one’s responses in forums or chatrooms are archived and can be retrieved puts extra pressure on those who provide teaching or therapy online.

Asynchronous communication and self-directed learning

Discussion forums, like correspondence chess, offer the possibility of interacting with others at a time, place and pace of one’s choosing, but they also provide some of the communicative pressure of an internet chat (when they are used in real time (synchronously)), or they can be a bit more like an exchange of emails, with a time lag between the posting and the reply (hence, asynchronous communication). This produces a different quality of discussion than the chatroom. Replies can be more considered, hyperlinks can be added to messages, and images or documents attached. There is time for reflection.

In my experience, the discussion forum is the most visited page in an educational site. It is an opportunity for students to comment on what they are being taught and to share their ideas.

The learner’s control over the learning

Vygotsky, in his day a critic and rival of Piaget and still hailed as one of the greats in developmental psychology, proposed that children learn through socialisation and not, as Piaget suggested, by the elaboration of mental structures. Teachers influenced by Vygotsky emphasise collaborative learning in which students are encouraged to learn from each other, and this approach is particularly applicable to discussion forums.

Vygotsky argued that individuals learn by extending the boundary of what they know, their ‘zone of proximal development’, and not by acquiring new blocks of knowledge. Each student will have a different zone of proximal development, and so one of the aims of collaborative teaching is to give the learner (the term often preferred over student, as androgogy is often preferred over paedogogy) greater control over their own learning – and over their learning environment – so as to maximise their exposure to learning opportunities that are proper to this zone.

The discussion forum does give learners a degree of control over the medium of their learning, but this can be increased by allowing students to create their own learning materials. Two key technologies for this are the wiki, which I considered in the first article in this series (Reference TantamTantam, 2006a ), and concept mapping.

Concept mapping

If we were to map the concepts in this article the connections between the word ‘web’ and other common words would be unusual and unusually frequent. If we were to display this graphically, one way would be to show the word ‘web’ in the centre of the screen and around it a cluster of short links to other words, for example embodiment, disinhibition, growing, accessibility and so on. This map could be compared with other maps to demonstrate that the centrality of the word ‘web’, its links and the closeness of its links are all idiosyncratic to the discourse used in this and perhaps a few other deviant texts (such as computing magazines). Our implicit use of such maps is one means of identifying what kind of discourse we are dealing with. So some readers who have scanned this article might have seen many instances of the word ‘web’, noticed how it often came up in connection with many of the ideas presented here and concluded that the article is not for them, but for ‘nerds’.

The familiar London underground map created by Harry Beck in 1931 is an example of a schematic in which it is the connections between tube stations that is displayed, not their geography. A concept map, a term coined by Joseph Novak, is a schematic also based on connections, but conceptual ones. Its origins lie in Vygotsky via the assimilation theory of David Ausubel, who stressed the importance of extending new learning out from what is already known, and the constructivist teaching approach based on similar ideas. A concept map presents words or propositions and the connections between them. Connections might represent ordinality, for example in the map shown in Fig. 1, or associations (Fig. 2).

Fig. 1

A simple map of the concept ‘psychosis’ and its super- and sub-ordinal concepts.

Fig. 2

An invented map of a concept and its associated concepts, based on the most frequent words given by a group asked to say what they thought of when they heard the word ‘transference’.

The World Wide Web provides new methods of concept mapping. For example, a concept may be represented by a web page, a connection by a user making a hyperlink jump from one page to another, and the frequency with which a particular jump is made by the closeness of those two pages, or concepts, on the map. Software (for a list see http://ltsnpsy.york.ac.uk/conceptmapping/conceptmapping/html/software.html) simplifies the development and creation of concept maps. Microsoft’s Visio can also be used for this purpose. Concept mapping has been used in a variety of ways in psychotherapy (Reference Goodyear, Tracey and ClaibornGoodyear et al, 2005), but one of the most interesting is as a tool for learners to examine their presumptions about clients and help them to identify areas of future clinical learning. This is the basis of CoCaCo, discussed in the next section.

Mapping programs

The CoCaCo (from ‘coherence in case conceptualisations’) program allows trainees to develop a case conceptualisation by drawing for them maps showing whether statements about a patient fit together (are connected) or exclude each other (are not connected). Trainees using CoCaCo developed more nuanced (more ‘connected’) formulations and 95% were satisfied or highly satisfied with this feature of their training (Reference BergerBerger, 2004).

An alternative approach to formulation is the Intensive Feedback Tool (Reference Caspar, Berger and HautleCaspar et al, 2004), which enables text to be uploaded and analysed to produce a ‘semantic map’ of words that often occur together and are assumed to be semantically related. For example, ‘anxiety’ and ‘fear’ often occur together and are therefore mapped close together in the semantic space. In one application to psychotherapy training, trainees uploaded textual descriptions of videos of clinical scenarios that they had viewed. The program broke these descriptions into fragments, which were compared with the fragments of other descriptions of the same scenarios stored in a database. A similarity bar chart was produced, with a similarity index for each fragment based on the number of words which were semantically related to the words in comparable fragments in the database. These indices provided a way of seeing how close a trainee’s description was to those of trainers’ descriptions on the database.

Virtual reality

A technically much more complicated approach is to have trainees ‘interview’ a virtual patient. Software has been developed that shows on screen a representation of a head and upper torso which can model facial expressions and reproduce dialogue elements extracted from recorded psychotherapy interactions. A clinician observer rates the trainee therapist’s interventions in real time and tells the simulation to ‘improve’ (produce more positive dialogue elements, look more positive), ‘continue’ (with no change in dialogue elements and no change in facial expression) or ‘decompensate’ (become acutely distressed or disturbed) (Reference Beutler and HarwoodBeutler & Harwood, 2004).