Social cognition is an umbrella term encompassing a multitude of social information processing components, such as social perception, social orientation/attention, mentalising/theory of mind, empathy, and emotion and facial affect recognition (FAR). Reference Fiske and Taylor1 In typical development, social cognition components comprise explicit (conscious, controlled, slow) and implicit (unconscious, automatic, fast) processes. Reference Frith and Frith2 Implicit processes precede explicit processes, and the latter can be described as explicit knowledge of implicit representations. Reference Perner and Dienes3 Social cognition is subserved by frontal, limbic and temporal lobe neural circuits referred to as the ‘social brain’. Reference Brothers4 FAR is a core subcomponent of social cognition, relying on face perception and facial expression recognition skills. The fusiform gyrus, located in the ventral occipitotemporal cortex, and the amygdala, a group of nuclei deep in the medial temporal lobes, are particularly involved in FAR. Other pertinent regions include the superior temporal sulcus (STS) and the medial and orbital prefrontal cortex (PFC). Reference Haxby, Hoffman and Gobbini5,Reference Haxby, Hoffman and Gobbini6 Alterations in explicit FAR are present in a variety of mental disorders Reference Collin, Bindra, Raju, Gillberg and Minnis7 but particularly in autism spectrum disorder (ASD). Reference Uljarevic and Hamilton8,Reference Harms, Martin and Wallace9 Nevertheless, some studies have found limited or no evidence of general FAR deficits in ASD. Reference Jones, Pickles, Falcaro, Marsden, Happé and Scott10,Reference Castelli11 In addition to describing impairments in explicit mechanisms, the literature increasingly reports alterations in implicit FAR Reference Kliemann, Rosenblau, Bölte, Heekeren and Dziobek12 and other social signals in ASD. Reference Yoshida, Dziobek, Kliemann, Heekeren, Friston and Dolan13,Reference Senju, Southgate, White and Frith14 At the brain level, these impairments in implicit and explicit facial affect processing in ASD have been associated with aberrant functioning of the social brain, including hypoactivation of the fusiform gyrus, amygdala and medial PFC and abnormal laterality of responses in the STS compared with typically developing control groups. Reference Kleinhans, Richards, Johnson, Weaver, Greenson and Dawson15–Reference Pelphrey, Morris, McCarthy and Labar18
The ability of the brain to compensate or reorganise as a consequence of experience is known as (brain) plasticity or cortical remapping. Reference Pascual-Leone, Freitas, Oberman, Horvath, Halko and Eldaief19,Reference Grafman20 Training-induced plasticity as measured by functional magnetic resonance imaging (fMRI) has successfully been demonstrated for several mental disorders, such as depression and schizophrenia. Reference Goldapple, Segal, Garson, Lau, Bieling and Kennedy21,Reference Wykes, Brammer, Mellers, Bray, Reeder and Williams22 In ASD, (explicit) FAR skills can be increased by behavioural computer training. Reference Bölte, Hubl, Feineis-Matthews, Prvulovic, Dierks and Poustka23–Reference Golan and Baron-Cohen25 In a previous pilot study, we found increased activation of the superior parietal lobule as an effect of computer-aided FAR training aimed at animating the fusiform gyrus specifically, but we found no changes in fusiform gyrus activity. Reference Bölte, Hubl, Feineis-Matthews, Prvulovic, Dierks and Poustka23 This result indicates compensatory neuroplasticity in the parietal cortex, rather than plasticity of the fusiform gyrus, in response to training of FAR. However, the pilot study had several limitations regarding its design. Crucially, (a) the study investigated only fusiform gyrus plasticity rather than effects on the social brain network, and (b) there was no differentiation between explicit and implicit FAR. The current study thus aimed at extending and cross-validating previous results with a controlled cross-sectional FAR training study in ASD using fMRI. The study goal was to determine whether probable improvements in FAR following computer-aided cognitive training of explicit FAR in ASD are accompanied by changes in explicit and/or implicit FAR task-related activation of the social brain, particularly the fusiform gyrus, amygdala, medial PFC and STS. A case–control study comparing ASD with typically developing control group served as the baseline measure pre-training.
Method
Procedure and design
This study was divided into:
-
(a) a case–control study at baseline (typically developing group v. ASD group); and
-
(b) a controlled FAR training study (ASD with FAR training (ASD training group) v. ASD controls without FAR training (ASD control group)) with pre- and post-training measurements.
It comprised four data-collection lab visits: two for behavioural assessment (visits 1 and 4) and two for scanning (visits 2 and 3). Visits 1 and 2 and visits 3 and 4 were carried out at intervals of a few days. The typically developing group included in the case–control part participated only in visits 1 and 2, and individuals with ASD participated in all four visits. In addition, those individuals with ASD who participated in the FAR training group (ASD training group) received a total of eight training sessions at the clinical department, beginning shortly after visit 2, and ending shortly before visit 3. For both parts of the study (case–control and training), comparisons were carried out for explicit FAR tests outside the scanner, explicit and implicit FAR tasks inside the scanner and brain functions associated with implicit and explicit task performance inside the scanner. First, the case–control typically developing v. ASD study was conducted, followed by the ASD FAR training study. In both ASD subsamples (ASD training group and the ASD control group), fMRI images were recorded during completion of the FAR tasks (Fig. 1), shortly before and after (maximum 2 weeks) the training group had undergone the FAR intervention. The face stimuli used for the fMRI tasks were identical between the pre- and post-training scans. Behavioural FAR measures were collected inside and outside the scanner for both groups.
Experimental block paradigm.
The functional magnetic resonance imaging (fMRI) task consisted of four conditions: (a) Explicit facial affect recognition (FAR): participants viewed angry or fearful faces and had to press ‘yes’ only when angry faces were presented; (b) implicit FAR: participants viewed the same series of faces as in (a) but had to press ‘yes’ only for female faces; (c) neutral FAR: participants viewed neutral faces (same series as in a and b) and had to press ‘yes’ only for female faces; and (d) object recognition: ovals containing squares and circles in either 4:6 or 6:4 ratio were shown, with participants required to press ‘yes’ when the oval contained more squares than circles. We used a block design. Each block (29.5 s long) consisted of an initial instruction (2 s) and eight pictures in succession (each picture was displayed for 3 s with an intermediate fixation cross of 0.5 s). Four blocks per condition were presented (total of 16 blocks), and each block was followed by an inter-trial interval (using a fixation cross) of 10 s.
Participants
Study participants were 32 adolescent and adult volunteers with ASD. The mean age was 19.3 years (range 14–33); mean non-verbal, abstract reasoning IQ was 105.7 (range 79–126), according to the Raven's Standard Progressive Matrices; Reference Raven, Court, Horn, Kratzmeier and Raven26 and the median receptive vocabulary skills were in the 87th percentile (range 8–100), according to the Peabody Vocabulary Test-III. Reference Dunn and Dunn27 Diagnoses were based on clinical consensus ICD-10 research criteria 28 for autism (n = 10), Asperger syndrome (n = 14) and atypical autism/pervasive developmental disorder not otherwise specified (n = 8). Diagnoses were confirmed with the Autism Diagnostic Interview-Revised (ADI-R), Reference Rutter, Le Couteur and Lord29 the Autism Diagnostic Observation Schedule (ADOS), Reference Lord, Spence and Corsello30 or both. The typically developing group consisted of 25 individuals with a mean age of 19.7 years (range 14–27), a mean IQ of 109.0 (range 82–126) and average verbal skills in the 92nd percentile (range 13–100). The ASD and typically developing groups did not differ in age (t = 0.36, P = 0.72), gender (χ2 = 1.4, P = 0.45), IQ (t = 1.2, P = 0.36) or handedness (χ2 = 0.143, P = 0.93; as determined by the Edinburgh Handedness Inventory Reference Oldfield31 ). Verbal skills (vocabulary) were higher in the typically developing compared with the ASD group (Z = 7.6, P = 0.013). One of the ASD group members was treated with atypical antipsychotics (clozapine).
Exclusion criteria were assessed using a semi-structured interview conducted by an expert clinician and included neurological disorder (for example epilepsy, traumatic brain injury, brain tumour, cerebral palsy); several mental disorders (for example bipolar disorder, psychosis); and genetic syndromes (for example fragile-X syndrome, Down syndrome). Sample characteristics are shown in Table 1. The ASD group were current or former out-patients of a child and adolescent psychiatric university hospital. The typically developing group were a mixed community sample. Achenbach scales, Reference Achenbach32,Reference Achenbach33 to assess self-rated and parent-reported psychopathology scores, assured that all the typically developing group scored below the borderline clinical cut-off (T≤68).
Sample characteristics and test performance
| All ASD group (n = 32) |
ASD training group (n = 16) |
ASD control group (n = 16) |
Typically developing group (n = 25) |
|
|---|---|---|---|---|
| Gender, female/male: n | 2/30 | 0/16 | 2/14 | 4/21 |
| Non-verbal Raven's IQ, mean (s.d.) | 105.7 (12.0) | 106.8 (12.4) | 103.7 (12.2) | 109.0 (12.2) |
| Peabody Picture Vocabulary Test-III, median (range) | 84 (3–100) | 83 (3–100) | 87 (8–100) | 92 (13–100) |
| Handedness, right/left: n | 30/2 | 15/1 | 15/1 | 24/1 |
| Autism Diagnostic Observation Schedule social-communication total score, mean (s.d.) |
11.4 (4.0) | 11.5 (3.9) | 11.3 (4.3) | |
| Frankfurt Test of Facial Affect Recognition – faces module, mean (s.d.) | ||||
| Pre-interventionFootnote a | 37.9 (6.6) | 37.4 (7.2) | 38.3 (6.0) | 41.7 (4.3) |
| Post-interventionFootnote b | 43.7 (3.8) | 39.1 (5.5) | ||
| Emotion Recognition Test, mean (s.d.) | ||||
| Pre-interventionFootnote a | 16.3 (4.5) | 15.8 (5.4) | 16.8 (3.3) | 20.3 (2.3) |
| Post-interventionFootnote b | 18.5 (5.6) | 16.7 (5.5) | ||
| CANTAB intra/extradimensional shift, total errors: mean (s.d.) | ||||
| Pre-intervention | 31.3 (19.4) | 27.3 (19.4) | ||
| Post-intervention | 25.3 (14.3) | 21.9 (17.2) | ||
| CANTAB spatial recognition memory, total errors: mean (s.d.) | ||||
| Pre-intervention | 26.8 (23.2) | 24.4 (19.6) | ||
| Post-intervention | 26.4 (17.9) | 27.7 (15.8) | ||
CANTAB, Cambridge Neuropsychological Test Automated Battery.
a. Autism spectrum disorder (ASD) training group=ASD control group (t<0.62, P>0.55).
b. ASD training group>ASD control group (P≤0.03, partial eta (η2)≥16.1, 1 −β≥0.73).
For the (explicit) FAR training study, participants with ASD were divided into two: the ASD training group (n = 16) that received FAR training plus standard care (any ongoing individual treatment) and the ASD (non-training) control group (n = 16) that received standard care only. Groups were matched for ASD diagnosis (χ2 = 1.1, P = 0.72), age (t = 0.07, P = 0.95), gender (χ2 = 2.1, P = 0.46), IQ (t = 0.70, P = 0.48), vocabulary skills (Z = −0.11, P= 0.91), autism severity (ADOS social communication total: t = 0.17, P = 0.86) and two pre-training behavioural explicit FAR skills (see below, (t<0.61, P>0.55)).
The study was approved by the local ethics committee of the Medical Faculty at Frankfurt University, Germany. All participants, their legal guardians or both gave written informed consent.
Explicit FAR training
The ASD training group was trained in FAR skills using the Frankfurt Training for Facial Affect Recognition (FAR training). Reference Bölte, Hubl, Feineis-Matthews, Prvulovic, Dierks and Poustka23 FAR training is a computer-based cognitive intervention to teach explicit recognition of facially expressed basic emotions, which have previously demonstrated large training effects. Reference Bölte, Hubl, Feineis-Matthews, Prvulovic, Dierks and Poustka23 It uses the cross-cultural concept of seven fundamental affective states (happy, sad, angry, surprised, disgusted, fearful and neutral) Reference Ekman, Friesen and Ellsworth34 for verbal labelling of affect in photographs of whole faces and eye regions. The training comprises about 500 facial affect teaching items as well as educational texts and comics on three training levels. At level 1, correct verbal labelling of facial affect stimuli is followed by visual and auditory reinforcement. If the given answer is incorrect, a feedback button appears on the screen. Clicking on the link leads to text explaining the emotion and item solution (level 2). A further in-depth engagement in the specific emotion is provided by the opportunity to look at a comic strip and again choose a corresponding emotion word (level 3). In this study, the FAR training was administered during 60 min sessions weekly over a period of 8 weeks, assisted by a clinical psychologist. A more detailed description of the training is provided elsewhere. Reference Bölte, Feineis-Matthews, Leber, Dierks, Hubl and Poustka35
Pre- and post-training measures
Behavioural: FAR tests
For assessment of FAR changes attributable to training, participants in the ASD training and ASD control groups completed two computer-based FAR tests outside the scanner: (a) the Frankfurt Test for Facial Affect Recognition (FEFA) Reference Bölte, Hubl, Feineis-Matthews, Prvulovic, Dierks and Poustka23,Reference Bölte, Feineis-Matthews, Leber, Dierks, Hubl and Poustka35 and (b) the Emotion Recognition Test (ERT). Reference Bölte, Feineis-Matthews, Leber, Dierks, Hubl and Poustka35 The FEFA assesses explicit FAR skills by verbal labelling of emotions (happy, sad, angry, surprised, disgusted, fearful and neutral) expressed in the eye regions and in whole faces using Ekman's concept of basic emotions. Reference Bölte, Hubl, Feineis-Matthews, Prvulovic, Dierks and Poustka23 For further details on the FEFA, please see the original publication. Reference Ekman, Friesen and Ellsworth34 In this study, the module used for whole face FAR has excellent internal consistency (r tt = 0.95) and retest reliability (r tt = 0.92). The ERT also measures explicit FAR using computer-based verbal emotion labelling of seven emotions (happy, sad, angry, surprised, disgusted, fearful and contempt) displayed in faces, without overlap with FEFA items, and has been extensively validated. Reference Merten36 On both FAR tests, one point is given for each correctly classified emotion (maximum 50 for the FEFA; maximum 28 for the ERT).
Behavioural: neurocognitive control tests
Performance on FAR tests might be related to ASD-relevant cognitive factors other than emotional processing and social cognition. Thus, to control for non-specific FAR training effects, the Intra-Extra Dimensional Set Shift test (IED, cognitive flexibility) and spatial recognition memory (spatial working memory) from the Cambridge Neuropsychological Test Automated Battery 37 were also administered pre- and post-FAR training in both the ASD training and ASD control groups.
Imaging: acquisition and functional imaging FAR tasks
Pre- (ASD v. typically developing group) and post-FAR training (ASD training v. ASD control group) neuroimaging was performed using a 3T Siemens Allegra scanner at the Brain Imaging Center in Frankfurt. Functional images were acquired using an echoplanar imaging (EPI) sequence. We obtained 325 whole brain scans in one session. One volume consisted of 30 axially tilted slices (slice thickness 3 mm+0.75 mm gap; acquisition ascending and interleaved; FOV = 192 mm; repetition time (TR) = 2 s; echo time (TE) = 30 ms, 64×64 matrix; flip angle 80°). The first four volumes were discarded to allow for T 2 equilibration. In addition, anatomical whole brain images were obtained by using a T 1-weighted magnetisation-prepared, 3D gradient-echo pulse sequence with the following parameters: TR = 10.55 ms, TE = 3.06 ms, flip angle 22°, FOV = 256×256 mm, 176 sagittal slices with 1 mm thickness.
Participants performed different FAR tasks using a pseudo-randomised block design containing four experimental conditions: (a) explicit FAR (recognise basic emotions in pictures of facial affect); (b) implicit FAR (recognise gender in pictures of facial affect); (c) ‘neutral FAR’ (or more precisely: facial gender recognition, i.e. recognise gender in pictures of neutral facial affect); and (d) object recognition (estimate if more squares than circles are present in an oval shape). In ‘explicit’ and ‘implicit FAR’, participants viewed angry and fearful faces whereas in neutral FAR, they viewed only neutral faces. Each block was preceded by some visually presented instruction: (a) for explicit FAR, the question ‘anger?’, corresponding to a button press only for displayed angry faces; (b) and (c) for implicit FAR and neutral FAR, the question ‘female?’, corresponding to a button press only for female faces; and (d) for the control condition the question ‘squares?’, corresponding to a button press when an oval form stimuli contained more squares than circles (Fig. 1). Each condition comprised four blocks, and in each block, eight pictures were displayed for 29.5 s. Blocks consisted of an initial instruction (2 s) followed by the eight pictures in succession. Each picture was displayed for 3 s with an intermediate fixation cross of 0.5 s. After each block, there was a break of 10 s using a fixation cross. The order of blocks was pseudo-randomised across participants. For the three face conditions, we used neutral, angry or fearful face stimuli from the Karolinska Directed Emotional Faces series. Reference Lundqvist, Flykt and Öhman38 We used only these two negative emotions because they activate regions of interest in the social brain that we were interested in, and are dysfunctional in ASD. Reference Ashwin, Baron-Cohen, Wheelwright, O'Riordan and Bullmore39–Reference Kleinhans, Richards, Johnson, Weaver, Greenson and Dawson43 Moreover, we wanted all conditions to have a forced choice format, with just two options, including the non-emotional control task (male or female), and therefore intentionally used only two emotions. Stimuli in the scanner tasks showed no overlap with the pre–post or training material. Before scanning, participants received outside-scanner training with trial stimuli for each condition to become familiar with the task.
Statistics and image processing
Behavioural FAR tests (outside scanner)
To examine whether FAR training increases behavioural FAR skills, we conducted the following analyses. For the explicit FAR tests (FEFA, ERT), the ASD v. typically developing group comparisons were made using multivariate analyses of covariance (MANCOVA). The ASD training v. ASD control group comparison was made using MANOVA for repeated measures. Statistics were computed with a general linear model (GLM) in SPSS 19. The ASD subsamples (ASD training v. ASD control group) were carefully matched for major possible confounds, such as age, gender, IQ, verbal skills, ASD severity, and pre-training FAR skills (see ‘participants’). The same was true for the ASD and typically developing groups, except for receptive vocabulary skills, but effects were partialled out because of a significant correlation with the FAR tests (r = 0.28–0.43, P<0.03).
Behavioural FAR tasks (inside scanner)
Participant response accuracy to FAR tasks (main outcome) and reaction times were measured during scanning for FAR tasks. For technical reasons, we collected behavioural data for only 29 ASD participants (n = 14 for the ASD training group, n = 15 for the ASD control group). For the ASD v. typically developing group comparisons (case–control FAR study), repeated-measures 42 ANOVAs were computed with within-subject factor conditions (four levels: explicit FAR, implicit FAR, neutral FAR and object recognition) and between-subject factor group (two levels: typically developing, ASD). For ASD training v. ASD control group comparisons (ASD FAR training study), we calculated a repeated-measures 4×2×2 ANOVA with the factors condition (four levels: explicit FAR, implicit FAR, neutral FAR and object recognition), time (two levels: pre- and post-training) and group (ASD training group and ASD control group (i.e. untrained)). Data were externally analysed using a GLM in SPSS 19.
Neuroimaging data
We used statistical parametric mapping as implemented in SPM5 (www.fil.ion.ucl.ac.uk/spm/software/spm5/) for image processing and statistics. Functional images were acquired in one session for a total of 382 volumes. Individual functional images were slice-timed and re-aligned with the first image of the scan run; spatially normalised into a standard stereotactic space (Montreal Neurological Institute, MNI) with a volume unit (voxels) of 2×2×2 mm and using the EPI template in SPM5; smoothed with 8 mm full-width at half-maximum Gaussian filter; and ratio normalised to the whole brain global mean. For each individual, the regression model consisted of a set of five regressors modelling the length of the respective part (instruction, stimulus presentation during explicit FAR, implicit FAR, neutral FAR and object recognition) convolved with the haemodynamic response function and six regressors describing residual motion. Statistical contrast images were then obtained for explicit FAR, implicit FAR and neutral FAR v. the control (object) condition. At the second level, a whole brain analysis was performed using different full factorial ANOVAs, as follows.
-
(a) typically developing v. ASD group case–control FAR study: factors were condition (four levels for explicit FAR, implicit FAR, neutral FAR and object recognition) and group (two levels, i.e. ASD, typically developing), resulting in a t-statistic for every voxel (Table 2).
-
(b) ASD training v. ASD control group post training (FAR training study): the interaction effect (grouptime) is reported, and for the resulting first-level contrast, we calculated implicit FAR>object recognition and trained>untrained (control) ASD groups. Factors were group (two levels, i.e. trained, untrained (control)) and time (pre- and post-training), resulting in a t-statistic for every voxel (Table 3). We also calculated two additional models using the same model structure for the contrast of explicit FAR>object recognition and neutral FAR>object recognition. All results are reported with a P<0.05 false discovery rate-corrected across the whole brain k>20. For regression analysis, individual peak voxel data were extracted from the respective contrast and region and analysed externally using SPSS 19. The correlation analyses were performed in an exploratory way, and no alpha correction was applied.
Coordinates and anatomical localisation for the typically developing v. autism spectrum disorder (ASD) group analysisFootnote a
| Implicit FAR v. control condition | Implicit FAR v. explicit FAR condition | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Brain region | x | y | z | t | P | x | y | z | t | P |
| Amygdala | ||||||||||
| 20 | −4 | −16 | 3.40 | 0.023 | ||||||
| −30 | −4 | −16 | 3.33 | 0.025 | −22 | −2 | −14 | 3.38 | 0.040 | |
| Fusiform gyrus | ||||||||||
| 54 | −46 | −20 | 3.43 | 0.022 | ||||||
| −36 | −58 | −14 | 4.00 | 0.013 | ||||||
| Superior temporal sulcus | ||||||||||
| 60 | −46 | 0 | 2.92 | 0.037 | ||||||
| −58 | −46 | −6 | 3.93 | 0.013 | −58 | −44 | −12 | 3.88 | 0.029 | |
| Temporal sulcus | ||||||||||
| 60 | −2 | −10 | 3.93 | 0.027 | ||||||
| −58 | −16 | −14 | 3.90 | 0.028 | ||||||
| Dorsolateral prefrontal cortex | ||||||||||
| 44 | 16 | 30 | 3.80 | 0.016 | ||||||
| −46 | 26 | 22 | 3.97 | 0.013 | ||||||
| Orbitomedial prefrontal cortex | ||||||||||
| −10 | 38 | −12 | 4.00 | 0.013 | 8 | 50 | − 18 | 3.48 | 0.038 | |
| −6 | 48 | −2 | 3.45 | 0.038 | ||||||
| Medial prefrontal cortex | −16 | 60 | 22 | 3.57 | 0.0019 | |||||
a. The contrast implicit facial affect recognition (FAR) condition compared with the control (object recognition) and compared with the explicit FAR condition. The contrast explicit FAR compared with the control condition as well as the neutral faces condition compared with control revealed no differences. Activations are reported on Montreal Neurological Institute (MNI) coordinates with a significance level of P<0.05, false discovery rate corrected (k>20).
Coordinates and anatomical localisation for autism spectrum disorder (ASD) trained group v. ASD untrained (control) group analysis for the implicit condition compared with the control condition (pre–post training)Footnote a
| Implicit FAR | |||||
|---|---|---|---|---|---|
| Brain region | x | y | z | t | P |
| Amygdala | |||||
| 12 | −4 | −16 | 3.66 | 0.037 | |
| −22 | 0 | −16 | 3.40 | 0.041 | |
| Fusiform gyrus | |||||
| 32 | −54 | −10 | 4.40 | 0.023 | |
| −46 | −48 | −18 | 3.40 | 0.041 | |
| Medial prefrontal cortex | 6 | 62 | 6 | 3.30 | 0.043 |
| Dorsolateral prefrontal cortex | 40 | 30 | 26 | 3.40 | 0.041 |
| Precuneus | −2 | −34 | 38 | 4.48 | 0.018 |
| Posterior superior temporal sulcus | −56 | −50 | 28 | 3.48 | 0.040 |
| Medial temporal pole | |||||
| 62 | −40 | −12 | 3.54 | 0.039 | |
| −54 | −48 | −18 | 3.75 | 0.036 | |
a. The contrast ASD training group v. ASD control group (pre–post training) for implicit facial affect recognition (FAR) task compared with the control task (object recognition) ((trained ASD group>untrained control group)>(post-FAR>pre-FAR training)). Activations are reported on Montreal Neurological Institute (MNI) coordinates at a significance level of P<0.05 false discovery rate-corrected, (k>20).
Results
Typically developing v. ASD group, pre-training
FAR tests (outside scanner)
For the case–control comparison of ASD and typically developing groups for basic explicit FAR skills, we found that the ASD group performed more poorly than the typically developing group on both explicit FAR tests (MANCOVA, F(1/52)≥5.9, P≤0.02, partial eta Reference Frith and Frith2 (η2)≥0.11, 1−β≥0.67). FEFA scores were on average 37.9 (s.d. = 6.6) in the ASD group and 41.7 (s.d. = 4.3) in the typically developing group. For the ERT, the average was 16.3 (s.d. = 4.5) in the ASD group and 20.3 (s.d. = 2.3) in the typically developing group (Table 1, Fig. 2).
Explicit facial affect recognition (FAR) test results outside the scanner.
Frankfurt Test for Facial Affect Recognition (FEFA) and Emotion Recognition Test (ERT) performance for typically developing (TD) v. autism spectrum disorder (ASD) group (case–control study) and ASD training group (ASD-TG) v. ASD control group (ASD-CG) (pre–post-training study). Pre, pre-training; post, post-training. *P<0.05, **P<0.01.
FAR tasks (inside scanner)
For the typically developing v. ASD group case–control accuracy comparison for inside scanner performance, the ANOVA showed a significant main effect for condition (F(1/52) = 60.7, P<0.0001) and group (F(1/52) = 11.1, P = 0.002), as well as a significant (condition×group) interaction (F(1/52) = 5.38, P= 0.008). Bonferroni-corrected post hoc comparisons revealed that the ASD group performed more poorly for explicit FAR (P = 0.002). Reaction time analyses for the same comparison showed a significant main effect for condition (F(1/52) = 30.6, P<0.0001) but not for group (F(1/52) = 2.4, P = 0.13) or for the (condition×group) interaction (F(1/52) = 1.1, P = 0.34) (see also online Table DS1a and online Fig. DS1).
fMRI
The fMRI analysis for case–control ASD v. typically developing group showed reduced activation for implicit FAR in the fusiform gyrus, amygdala and STS (Table 2) as compared with the typically developing group ((implicit>object task)>(typically developing>ASD group)). In prefrontal regions, blood oxygenation level-dependent (BOLD) signal was reduced in the medial, dorsolateral, and orbitomedial PFC (for details including P and t-statistics, please see Fig. 3, Table 2).
Reduced activation in autism before training.
Participants with autism showed less activation during implicit facial affect recognition (FAR) compared with the control condition in the amygdala, fusiform gyrus, medial prefrontal cortex and orbital medial prefrontal cortex, superior temporal sulcus and inferior frontal gyrus (false discovery rate-corrected P<0.05, k>20; n = 25 typically developing (TD) group, n = 32 autism spectrum disorder (ASD) group). (a) Bar plots indicate size of the effect at the maximum activated voxel in the right amygdala for all conditions. The amount of amygdala activation in the ASD group positively correlated with Frankfurt Test for Facial Affect Recognition (FEFA) score (r = 0.56, P = 0.001, two-tailed). (b) Sagittal (lateral and middle) view of the brain for the contrast TD group>ASD group for implicit FAR. EXP, explicit FAR; IMP, implicit FAR; NE, neutral FAR; OB, Object recognition.
The interaction ((implicit>explicit)>(typically developing>ASD group)) was significant in the left amygdala, medial orbital PFC, left STS, and bilateral temporal pole (Table 2). Explicit and neutral FAR did not reveal differential activation between the typically developing and ASD groups. Right amygdala activation during implicit FAR correlated positively with the outside-scanner explicit FEFA performance in ASD (r = 0.56, P = 0.001, two-tailed) (Fig. 3), even when an obvious outlier was excluded (P = 0.04). No correlation was shown between the ERT test and right amygdala (P = 0.17) (for additional correlation analysis see online Table DS2a.
ASD training v. ASD control group, post-training
FAR tests (outside scanner)]
The FAR training study results showed that compared with untrained ASD controls, the ASD training group improved on both explicit FAR tests after the FAR training (repeated-measures MANOVA (group×tests interaction), univariate: F(1/30)≥5.1, P≤0.03, partial η2≥16.1, 1−β≥0.73); FEFA: 37.4 (s.d. = 7.2) to 43.7 (s.d. = 3.8) and ERT: 15.8 (s.d. = 5.4) to 18.5 (s.d. = 5.6) in the ASD training group compared with FEFA: 38.3 (s.d. = 6.0) to 39.1 (s.d. = 5.5) and ERT: 16.8 (s.d. = 3.3) to 16.7 (s.d. = 5.5) in the ASD control group. No changes in the cognitive control measures for IED or spatial recognition memory were observed (F(1/30)≤1.7, P≥0.20, partial η2≤0.01), suggesting a specific effect of the FAR training on FAR rather than on general cognitive functioning. There was no main effect of group on any of the behavioural measures (F(1/30)<1.04, P≥0.21, partial η2≤0.03) (Table 1, Fig. 2).
FAR tasks (inside scanner)
For the untrained v. trained ASD accuracy comparison, the ANOVA showed a significant main effect for condition (F(1/27) = 37.1, P<0.0001), but not a significant effect for time (F(1/27) = 0.06, P = 0.8) and group (F(1/27) = 0.58, P = 0.45). We observed a trend for the interaction effect of condition×time×group (F(1/27) = 3.1, P = 0.057). Reaction time analyses for the same comparison showed a significant main effect for condition (F(1/27) = 91.17, P<0.0001) and a significant effect for time (F(1/27) = 12.86, P = 0.001), but not for group (F(1/27) = 0.006, P = 0.93) or for the condition×time×group interaction (F(1/27) = 0.45, P= 0.601) (see also online Table DS1b and online Fig. DS2).
fMRI
To investigate changes in specific brain activations in the ASD training group compared with the ASD control group, we calculated activation maps for the interaction of timegroup. For implicit FAR, an increase in the BOLD response was detected in the fusiform gyrus, amygdala and temporal pole bilaterally, as well as in the medial PFC and left posterior STS in the ASD training group after the FAR training (Fig. 4, Table 3). The increased activation in the medial PFC showed a trend to a negative correlation with the ADOS – communication score (r = −0.49, P = 0.053). No correlation was shown between the ADOS – social interaction score and medial PFC (P = 0.72) (for additional correlation analysis, see also online Table DS2b). In the additional analysis for explicit and neutral fMRI FAR tasks, no differences were observed.
Training effects in autism spectrum disorder (ASD) for implicit facial affect recognition (FAR).
The activity within the social brain network increased significantly after training when comparing the trained (ASD training group) with the untrained (ASD control group) autism group (false discovery rate-corrected P<0.05, k>20) during the implicit FAR task. The figure shows activation for the interaction (analysis time (pre-, post-FAR training)×group (ASD training group, ASD control group)) in the (a) right amygdala, (b) left fusiform gyrus (FG), and (c) medial prefrontal cortex (PFC). Furthermore, within the ASD training group, the amount of medial PFC activation tended to correlate negatively with the Autism Diagnostic Observation Schedule – communication (ADOS-com) score (r = −0.49, P = 0.053).
This study did not include a direct measure of eye movements; however, as a crude proxy for the amount of eye movement, we analysed the variance of the eye signal in the scanner directly from magnetic resonance data Reference Beauchamp44 (for technical reasons, we collected eye signal data for 27 ASD participants (n = 13 ASD training group and n = 14 ASD control group)). As for the variance of the eye signal, no differences were found for either main factor: group (ASD training group v. ASD control group) (F(1/25) = 2.52, P = 0.12) or time (F(1/25) = 0.02, P = 0.89).
Discussion
Main findings
This study confirms previous findings of impaired behavioural explicit FAR skills in people with ASD compared with typically developing controls Reference Uljarevic and Hamilton8,Reference Harms, Martin and Wallace9 and of the efficacy of FAR training for increasing FAR skills. Reference Bölte, Hubl, Feineis-Matthews, Prvulovic, Dierks and Poustka23–Reference Golan and Baron-Cohen25,Reference Bölte, Feineis-Matthews, Leber, Dierks, Hubl and Poustka35 Moreover, it highlights for the first time that increasing amygdala and fusiform gyrus activation after training is a neural correlate of the observed cognitive improvement in ASD. Thus, our findings demonstrate a potentially fruitful combination of strategies of evidence-based clinical intervention and experimental neuroscience.
Prior to FAR training, individuals with ASD showed reduced activation in regions of the social brain compared with the typically developing control group. Reduced activation was observed only during the fMRI task for implicit processing, consistent with the notion that social cognition alterations in ASD are pronounced in implicit social information processing. Reference Kliemann, Rosenblau, Bölte, Heekeren and Dziobek12–Reference Senju, Southgate, White and Frith14 During the explicit FAR task, when social cognition was a conscious decision, FAR showed no activation differences between groups. Therefore, neurodevelopment seems unaltered or at least comparable with controls for consciously acquired top–down-driven explicit FAR. Nevertheless, explicit FAR difficulties outside the scanner correlated with hypoactivation of the social brain during implicit FAR in the scanner. These results suggest that explicit FAR training might improve brain activation during implicit FAR, which in turn would lead to improved explicit FAR performance. This expectation was in fact met in the FAR training study. The ASD training group showed improvements in explicit FAR behaviour, but there were no further changes in performance on other neurocognitive measures pre–post explicit FAR training. It is thus unlikely that the FAR training had a general, non-specific effect on cognitive functioning. Instead, these findings indicate a transfer of acquired FAR skills and specific FAR training effects, in turn implying that the FAR performance gain is unlikely to be attributable to non-specifically trained cognitive factors.
Explicit behavioural FAR performance also improved in the scanner, and regions that were previously hypoactive during implicit FAR were more activated post-training. Strikingly, this result suggests functional neuroplasticity of key social brain areas in ASD. According to Grafman's Reference Grafman20 conceptualisation of functional neuroplasticity, the findings in the current study might be classified as ‘compensatory masquerade’: a novel allocation of a particular cognitive process to perform a task. Along the typical developmental trajectory, implicit processes are believed to precede explicit processes. Reference Perner and Dienes3 Nevertheless, implicit and explicit social cognition might be less distinguishable in ASD than in typical development. Reference Kliemann, Rosenblau, Bölte, Heekeren and Dziobek12 Therefore, our explicit FAR training data could indicate that this common sequence (from implicit to explicit cognition) might be partly reversed in ASD. One may speculate that top–down explicit social cognition stimulates bottom–up implicit cognition in ASD, not vice versa.
Unfortunately, as in earlier FAR training studies in ASD, the behavioural outcome measures used in this study can confirm or discard only explicit FAR changes, not effects on implicit processing. Additional implicit behavioural changes are not ruled out, but they were not directly tested here. It has been proposed that high-functioning individuals with ASD fail in implicit mentalising, but through experience and teaching can acquire an explicit ability to read others' emotions from faces and even higher cognitive mental states. Reference Frith45 Our results provide some evidence that the neural basis for implicit affective face processing in ASD is altered comparedwith explicit face processing. The results additionally signify an underlying dysfunction in FAR, a subcomponent of social cognition, which might be malleable with explicit FAR training.
Limitations
This study has several limitations that might compromise the generalisability of our results. First, because of the exclusively high-functioning individuals examined, the applicability of the results to those who fall within the lower functioning part of the ASD spectrum is questionable. This limitation is a major shortcoming in terms of external clinical validity because low to intermediate functioning is found in the majority of people with ASD. Second, measures of eye movement cannot compare with simultaneous eye tracking in the scanner, Reference Dalton, Nacewicz, Johnstone, Schaefer, Gernsbacher and Goldsmith46 which was not part of this study. Therefore, differences in eye movements, attention to face stimuli or increased social motivation Reference Chevallier, Kohls, Troiani, Brodkin and Schultz47 cannot be excluded as confounders and could limit the validity of our results. Third, the small number of females limited the possibility of investigating gender differences. A fourth limitation is the lack of a more comparable active pseudo-training control ASD group, rather than ASD standard care (one that, for instance, uses computer games that include social stimuli of the same intensity). This limitation reduced the probability of crystallising the specific and net cognitive and neural effects of the FAR training. Fifth, although the outside of the scanner FAR tests and the FAR training included all basic emotions, the scanner tasks focused on two negative emotions only (fear, anger). The generalisability of our implicit FAR findings from MRI beyond these basic negative emotions is therefore unknown. In this context, it is worth mentioning that Bird et al, Reference Bird, Catmur, Silani, Frith and Frith48 also reported less attentional modulation in the left fusiform gyrus in ASD for implicit processing for fear. Sixth, the use of static facial emotion stimuli makes it impossible to judge whether the current findings would generalise to real-world social situations with rapid, dynamic and contextual stimuli. Seventh, although the current study is well controlled with regard to confounding factors such as age or intellectual functioning, the findings require confirmation in a random training assignment. Finally, although the question of whether the findings are related to specific emotions rather than to global affective processing was analysed for the behavioural FAR tasks in the scanner, this analysis was not done for the behavioural FAR tests outside of the scanner. Therefore, it is unknown if a specific basic emotion drives the observed FAR changes outside the scanner and the correlations with MRI activation, or which of the emotions are eventually resistant to change. A previous study Reference Albertowski49 found that on one of the FAR tests (FEFA), the ASD group performed more poorly on angry and fearful stimuli compared with the typically developing group. Therefore, one might presume that these two more specific emotions were most open to training change even for the behavioural outcome FAR tests, but this inference is pure speculation.
In conclusion, circumscribed computerised FAR training for people with ASD was effective in specifically ameliorating explicit FAR, an integral part of social cognition, on a behavioural level. This shift was accompanied by evidence of neuroplasticity towards normalising brain activation related to implicit processing of social information. Future studies need to confirm our findings using a more naturalistic FAR training and FAR task and outcome set-up, particularly with regard to implicit FAR intervention effects.
Funding
S.B. was supported by the Swedish Research Council (grant no. 523-2009-7054).
eLetters
No eLetters have been published for this article.