Weighing in at about three pounds, the human brain is a marvel of evolutionary engineering. The brain transforms signals from millions of sensors located all over the body into appropriate muscle commands to enact a behavior suitable to the task at hand. This closed-loop, real-time control system remains unsurpassed by any artificially created system despite decades of attempts by computer scientists and engineers.

The brain’s unique information processing capabilities arise from its massively parallel and distributed way of computing. The workhorse of the brain is a type of cell known as a neuron, a complex electrochemical device that receives information from hundreds of other neurons, processes this information, and conveys its output to hundreds of other neurons. Furthermore, the connections between neurons are plastic, allowing the brain’s networks to adapt to new inputs and changing circumstances. This adaptive and distributed mode of computation sets the brain apart from traditional computers, which are based on the von Neumann architecture with a separate central processing unit, memory units, fixed connections between components, and a serial mode of computation.

In the previous chapter, we learned about BCIs that required placing electrodes inside the brain. While such an approach provides a high-fidelity window into the spiking activity of neurons, it also comes with significant risks: (1) possible infections due to penetration of the blood-brain barrier, (2) encapsulation of electrodes by immunologically reactive tissue, which can degrade signal quality over time, and (3) the potential for damage to intact brain circuits during implantation.

To counter these risks, researchers have investigated the use of BCIs that do not penetrate the brain surface. Such BCIs can be regarded as semi-invasive BCIs. We will focus on two types of semi-invasive BCIs: electrocorticographic (ECoG) BCIs and BCIs based on recording from nerves outside the brain. As discussed in Chapter 3, ECoG requires surgical placement of electrodes underneath the skull, either under the dura mater (subdural ECoG) or outside the dura mater (epidural ECoG). The procedure is invasive but less so than the methods of the previous chapter. In this chapter, we explore the ability of ECoG BCIs to control cursors and prosthetic devices.

As described in the previous chapter, the brain communicates using spikes, which are all-or-none electrical pulses produced when the neuron receives a sufficient amount of input current from other neurons via synaptic connections. It is therefore not surprising that some of the first technologies for recording brain activity were based on detecting changes in electrical potentials in neurons (invasive techniques based on electrodes) or large populations of neurons (noninvasive techniques such as electroencephalography or EEG). More recent techniques have been based on detecting neural activity indirectly by measuring changes in blood flow that result from increased neural activity in a region (fMRI) or by measuring the minute changes in the magnetic field around the skull caused by neural activity (MEG).

In this chapter, we review some of these technologies that serve as sources of input signals for BCIs. We also briefly describe technologies that can be used to stimulate neurons or brain regions, thereby allowing BCIs to potentially provide direct feedback to the brain based on interactions with the physical world.

The preceding chapters introduced you to the basic concepts in neuroscience, recording and stimulation technologies, signal processing, and machine learning. We are now ready to put it all together to consider the process of building an actual BCI.

Major Types of BCIs

BCIs today can be broadly divided into three major types:

• Invasive BCIs: These involve recording from or stimulating neurons inside the brain.

• Semi-invasive BCIs: These involve recording from or stimulating the brain surface or nerves.

• Noninvasive BCIs: These employ techniques for recording from or stimulating the brain without penetrating the skin or skull.

Within each of these types, we can have BCIs that:

• Only record from the brain (and translate the neural data into control signals for output devices).

• Only stimulate the brain (and cause certain desired patterns of neural activity in the brain).

• Both record and stimulate the brain.

In the next i ve chapters, we will encounter concrete examples of the major typesof BCIs defined above. Before we proceed to these concrete BCI examples, it is usefulto first discuss some of the major types of brain responses that researchers haveexploited for building BCIs.

The field of brain-computer interfacing has witnessed tremendous growth over the past decade. Invasive BCIs based on multielectrode arrays have allowed laboratory animals to precisely control the movement of robotic arms. Implants and semi-invasive BCIs have enabled human subjects to quickly acquire control of computer cursors and simple devices. Noninvasive BCIs, particularly those based on EEG, have allowed humans to control cursors in multiple dimensions and issue commands to semi-autonomous robots. Commercially available BCIs such as cochlear implants and deep brain stimulators have helped improve the quality of life of hundreds of hearing-impaired individuals and patients suffering from debilitating neurological diseases.

The achievements of the field thus far are impressive, but many obstacles remain. As pointed out by Gilja, Shenoy, and colleagues (2011), invasive BCIs have yet to achieve the same levels of performance, multidecade robustness, and naturalistic proprioception and somatosensation as able-bodied people. Furthermore, invasive BCIs remain risky for humans and are used only as a last resort in severely disabled patients. The most popular noninvasive BCIs, based on EEG, suffer from a number of problems:

• Electrode placement is cumbersome and setup time is typically long (up to half an hour depending on the number of electrodes).

• Results of training and learning may not be transferable from one day to the next due to shifts in electrode locations, noisy contacts with scalp, etc.

• Low signal-to-noise ratio and on-line adaptation in subjects necessitate the availability of powerful amplifiers as well as efficient machine-learning and signal processing algorithms.

• Signal attenuation and summation between the brain and the scalp, together with sparse sampling of activity, limits the range of useful control signals that can be extracted.

We have thus far studied BCIs that either record from the brain to control an external device (Chapters 7–9) or stimulate the brain to restore sensory or motor function (Chapter 10). The most general type of BCI is one that can simultaneously record from and stimulate different parts of the brain. Such BCIs are called bidirectional (or recurrent) BCIs. Bidirectional BCIs can provide direct feedback to the brain by stimulating sensory neurons to convey the consequences of operating a prosthetic device using motor signals recorded from the same brain. Furthermore, signals recorded from one part of the brain can be used to modulate the neural activity or induce plasticity in a different part of the brain.

In Chapter 1, we discussed the pioneering work of Delgado (1969) on an implantable BCI called the stimoceiver, which can be regarded as the first example of a bidirectional BCI. In this chapter, we briefly review a few more recent examples to illustrate the possibilities opened up by bidirectional BCIs and conclude by noting that the most flexible BCIs of the future will likely be bidirectional, though this flexibility will likely come at the cost and the associated risk of being invasive.

The field of machine learning has played an important role in the development of brain-computer interfaces by providing techniques that can learn to map neural activity to appropriate control commands. Algorithms for machine learning can be broadly divided into two classes: supervised learning and unsupervised learning. In supervised learning, we are given training data that consists of a set of inputs and corresponding outputs. The goal is to learn the underlying function from the training data such that new test inputs are mapped to the correct outputs. If the outputs are discrete classes, the problem is called classification. If the outputs are continuous, the problem is equivalent to regression. Given the emphasis on discovering an underlying function, supervised learning is sometimes also called function approximation. Unsupervised learning, on the other hand, emphasizes discovery of hidden statistical structure in unlabeled data: the training data consists of inputs, which are typically high-dimensional vectors, and the goal is to learn a statistical model that may be compact or useful for subsequent analysis. We have already discussed two prominent unsupervised learning techniques (PCA and ICA) in the previous chapter.

In this chapter, we focus on the two major types of supervised learning techniques: classification and regression. Classification is the problem of assigning one of N labels to a new input signal, given labeled training data consisting of known inputs and their corresponding output labels. Regression is the problem of mapping input signals to a continuous output signal. Many BCIs based on EEG, ECoG, fMRI, and fNIR have relied on classification to generate discrete control outputs (e.g., move a cursor up or down by a small amount). BCIs based on neuronal recordings, on the other hand, have predominantly utilized regression to generate continuous output signals, such as position or velocity signals for a prosthetic device. In general, the choice of whether to use classification or regression when designing a BCI will depend on both the type of brain signal being recorded and the type of application being controlled.

Some of the most important developments in brain-computer interfacing have come from BCIs based on invasive recordings. As reviewed in Chapter 3, invasive recording techniques allow the activities of single neurons or populations of neurons to be recorded. This chapter describes some of the achievements of invasive BCIs in animals and humans.

Two Major Paradigms in Invasive Brain- Computer Interfacing

BCIs Based on Operant Conditioning

A number of BCIs in animals have been based on operant conditioning, a phenomenon discussed in Section 6.2.1. In operant conditioning, an animal receives a reward upon selection of an appropriate action, e.g., pressing a lever. After repetitive pairing, the animal learns to execute the action in anticipation of the reward. In a BCI paradigm, the animal is rewarded if it selectively activates a neuron or population of neurons to move a cursor or prosthetic device in an appropriate manner.

Early BCI Studies

In the late 1960s, in one of the earliest demonstrations of brain- computer interfacing,Eberhard Fetz at the University of Washington in Seattle utilized the idea of operantconditioning to demonstrate that the activity of a single neuron in a primate’s motorcortex can be conditioned to control the needle of an analog meter (Fetz, 1969). Themovement of the needle was directly coupled to the firing rate of the neuron: whenthe needle crossed a threshold, the monkey was rewarded. After several trials, themonkey learned to consistently move the needle past the threshold by increasingthe firing rate of the recorded neuron (Figure 7.1). In this example of operant conditioning, the action (needle movement) that produces reward is coupled to increasedactivity in the recorded neuron (the conditioned response).