Experimental Results and the Replication Crisis in Neuroscience

Ned Jenkinson (University of Birmingham): an Experimental Results author is a Senior Lecturer in the School of Sport, Exercise and Rehab Sciences and Researcher in the Centre for Human Brain Health.

Imagine doing an experiment where you are told to use a joystick to control a cursor situated at the centre of a computer screen to hit targets dispersed at 45˚ intervals around the perimeter of an invisible circle (see figure one). Suddenly, the relationship between the movement of the joystick and the cursor changes, so that when you move the joystick toward the target, the cursor veers wildly away from the target you intended to hit. After a couple of movements, the friendly experimenter stops you and explains:

“You just made two large errors because we imposed a rotation that pushes you 45° counterclockwise. You can counter the error by aiming for the neighboring clockwise target, which is also at 45°.”

Sure enough, using this simple strategy, you find that once again you can hit the targets. Relieved, you continue the task. However, and to your frustration, trial-by-trial you progressively become more inaccurate and miss the target by bigger and bigger margins. How can this be? How can the simple and obvious strategy that initially worked, slowly fail?

This counterintuitive finding was described in 2006 in a Brief Communication in the Journal of Neuroscience called ‘An Implicit Plan Overrides an Explicit Strategy during Visuomotor Adaptation’ by John Krakauer and Pietro Mazzoni. (1)

The system that maintains the accuracy of movements in the face of systematic errors (such as the visuomotor rotation described above) is known as motor adaptation. These processes are important for all day-to-day movements like eye movements or gait. The processes are also thought to have a large implicit component, that is, the brain adapts to these movements without us knowing about or being aware of it. The authors explained that the reason the strategy failed in their experiment is that implicit adaptation is more powerful than the explicit strategy given by the experimenter, and when the two are put head-to-head implicit motor adaptation prevails.

Over the intervening 15 years this study has become a classic in the literature with over 600 citations according to Google Scholar, and yet has never been replicated. Given the improbable behaviour seen in the study, we had long intended to try to copy the experiment in my lab, to see for ourselves if it could be replicated.

The opportunity came when Sarah Voets came to my lab (then in Oxford) as an intern to carry out an experimental study as part of her MSc at VU University Amsterdam. We decided to copy the study as closely as possible but use TMS to try to inhibit the Cerebellum. (2) We did this as we knew the cerebellum is crucial to motor adaptation and we had previously used TMS inhibition of the cerebellum to suppress motor adaptation of eye movements. Our thoughts were that if cerebellar TMS inhibited the implicit motor adaptation process, then the explicit strategy would be successful for longer.

As it happened the TMS of the cerebellum had no effect on the behaviour – for reasons I could go into in another blog. However, both the control and TMS groups demonstrated the idiosyncratic behaviour seen in the original paper; immediately reducing their errors after being instructed how to combat the joystick/cursor rotation, but slowly become more inaccurate thereafter.

Regardless of the disappointment of a null result we were excited to have replicated this important finding. Though with no obvious place to publish such a simple, but important, replication result the paper sat in the ‘bottom drawer’ that all scientists have of null results. That is probably where it would have remained had I not happened across a new journal called Experimental Results that promised a “forum for experimental findings that disclose the small incremental steps vitally important to experimental research”.

Given the so-called replication crisis in Neuroscience, a place to publish small replication studies is an important step in addressing this problem, and we will be back with other studies. And though the short word count was an (interesting) challenge we think our paper provides crucial evidence that An Implicit Plan Still Overrides an Explicit Strategy during Visuomotor Adaptation!

References:

(1) Mazzoni, P and Krakauer, J. An Implicit Plan Overrides an Explicit Strategy during Visuomotor Adaptation. Journal of Neuroscience  2006 Apr, 26 (14) 3642-3645 DOI: https://doi.org/10.1523/JNEUROSCI.5317-05.2006

(2) Jenkinson N, Miall RC. Disruption of saccadic adaptation with repetitive transcranial magnetic stimulation of the posterior cerebellum in humans. Cerebellum. 2010 Dec;9(4):548-55. doi: 10.1007/s12311-010-0193-6

We are looking forward to the second volume of Experimental Results, and have already published a number of papers which can be viewed here.

Find out more about publishing your research in Experimental Results and follow us on Twitter @ExpResults.


Leave a reply

Your email address will not be published. Required fields are marked *