In the previous chapter, we learned about BCIs that required placing electrodes inside the brain. While such an approach provides a high-fidelity window into the spiking activity of neurons, it also comes with significant risks: (1) possible infections due to penetration of the blood-brain barrier, (2) encapsulation of electrodes by immunologically reactive tissue, which can degrade signal quality over time, and (3) the potential for damage to intact brain circuits during implantation.

To counter these risks, researchers have investigated the use of BCIs that do not penetrate the brain surface. Such BCIs can be regarded as semi-invasive BCIs. We will focus on two types of semi-invasive BCIs: electrocorticographic (ECoG) BCIs and BCIs based on recording from nerves outside the brain. As discussed in Chapter 3, ECoG requires surgical placement of electrodes underneath the skull, either under the dura mater (subdural ECoG) or outside the dura mater (epidural ECoG). The procedure is invasive but less so than the methods of the previous chapter. In this chapter, we explore the ability of ECoG BCIs to control cursors and prosthetic devices.

As described in the previous chapter, the brain communicates using spikes, which are all-or-none electrical pulses produced when the neuron receives a sufficient amount of input current from other neurons via synaptic connections. It is therefore not surprising that some of the first technologies for recording brain activity were based on detecting changes in electrical potentials in neurons (invasive techniques based on electrodes) or large populations of neurons (noninvasive techniques such as electroencephalography or EEG). More recent techniques have been based on detecting neural activity indirectly by measuring changes in blood flow that result from increased neural activity in a region (fMRI) or by measuring the minute changes in the magnetic field around the skull caused by neural activity (MEG).

In this chapter, we review some of these technologies that serve as sources of input signals for BCIs. We also briefly describe technologies that can be used to stimulate neurons or brain regions, thereby allowing BCIs to potentially provide direct feedback to the brain based on interactions with the physical world.

The preceding chapters introduced you to the basic concepts in neuroscience, recording and stimulation technologies, signal processing, and machine learning. We are now ready to put it all together to consider the process of building an actual BCI.

Major Types of BCIs

BCIs today can be broadly divided into three major types:

• Invasive BCIs: These involve recording from or stimulating neurons inside the brain.

• Semi-invasive BCIs: These involve recording from or stimulating the brain surface or nerves.

• Noninvasive BCIs: These employ techniques for recording from or stimulating the brain without penetrating the skin or skull.

Within each of these types, we can have BCIs that:

• Only record from the brain (and translate the neural data into control signals for output devices).

• Only stimulate the brain (and cause certain desired patterns of neural activity in the brain).

• Both record and stimulate the brain.

In the next i ve chapters, we will encounter concrete examples of the major types of BCIs defined above. Before we proceed to these concrete BCI examples, it is useful to first discuss some of the major types of brain responses that researchers have exploited for building BCIs.

The field of brain-computer interfacing has witnessed tremendous growth over the past decade. Invasive BCIs based on multielectrode arrays have allowed laboratory animals to precisely control the movement of robotic arms. Implants and semi-invasive BCIs have enabled human subjects to quickly acquire control of computer cursors and simple devices. Noninvasive BCIs, particularly those based on EEG, have allowed humans to control cursors in multiple dimensions and issue commands to semi-autonomous robots. Commercially available BCIs such as cochlear implants and deep brain stimulators have helped improve the quality of life of hundreds of hearing-impaired individuals and patients suffering from debilitating neurological diseases.

The achievements of the field thus far are impressive, but many obstacles remain. As pointed out by Gilja, Shenoy, and colleagues (2011), invasive BCIs have yet to achieve the same levels of performance, multidecade robustness, and naturalistic proprioception and somatosensation as able-bodied people. Furthermore, invasive BCIs remain risky for humans and are used only as a last resort in severely disabled patients. The most popular noninvasive BCIs, based on EEG, suffer from a number of problems:

• Electrode placement is cumbersome and setup time is typically long (up to half an hour depending on the number of electrodes).

• Results of training and learning may not be transferable from one day to the next due to shifts in electrode locations, noisy contacts with scalp, etc.

• Low signal-to-noise ratio and on-line adaptation in subjects necessitate the availability of powerful amplifiers as well as efficient machine-learning and signal processing algorithms.

• Signal attenuation and summation between the brain and the scalp, together with sparse sampling of activity, limits the range of useful control signals that can be extracted.

We have thus far studied BCIs that either record from the brain to control an external device (Chapters 7–9) or stimulate the brain to restore sensory or motor function (Chapter 10). The most general type of BCI is one that can simultaneously record from and stimulate different parts of the brain. Such BCIs are called bidirectional (or recurrent) BCIs. Bidirectional BCIs can provide direct feedback to the brain by stimulating sensory neurons to convey the consequences of operating a prosthetic device using motor signals recorded from the same brain. Furthermore, signals recorded from one part of the brain can be used to modulate the neural activity or induce plasticity in a different part of the brain.

In Chapter 1, we discussed the pioneering work of Delgado (1969) on an implantable BCI called the stimoceiver, which can be regarded as the first example of a bidirectional BCI. In this chapter, we briefly review a few more recent examples to illustrate the possibilities opened up by bidirectional BCIs and conclude by noting that the most flexible BCIs of the future will likely be bidirectional, though this flexibility will likely come at the cost and the associated risk of being invasive.

The field of machine learning has played an important role in the development of brain-computer interfaces by providing techniques that can learn to map neural activity to appropriate control commands. Algorithms for machine learning can be broadly divided into two classes: supervised learning and unsupervised learning. In supervised learning, we are given training data that consists of a set of inputs and corresponding outputs. The goal is to learn the underlying function from the training data such that new test inputs are mapped to the correct outputs. If the outputs are discrete classes, the problem is called classification. If the outputs are continuous, the problem is equivalent to regression. Given the emphasis on discovering an underlying function, supervised learning is sometimes also called function approximation. Unsupervised learning, on the other hand, emphasizes discovery of hidden statistical structure in unlabeled data: the training data consists of inputs, which are typically high-dimensional vectors, and the goal is to learn a statistical model that may be compact or useful for subsequent analysis. We have already discussed two prominent unsupervised learning techniques (PCA and ICA) in the previous chapter.

In this chapter, we focus on the two major types of supervised learning techniques: classification and regression. Classification is the problem of assigning one of N labels to a new input signal, given labeled training data consisting of known inputs and their corresponding output labels. Regression is the problem of mapping input signals to a continuous output signal. Many BCIs based on EEG, ECoG, fMRI, and fNIR have relied on classification to generate discrete control outputs (e.g., move a cursor up or down by a small amount). BCIs based on neuronal recordings, on the other hand, have predominantly utilized regression to generate continuous output signals, such as position or velocity signals for a prosthetic device. In general, the choice of whether to use classification or regression when designing a BCI will depend on both the type of brain signal being recorded and the type of application being controlled.

Some of the most important developments in brain-computer interfacing have come from BCIs based on invasive recordings. As reviewed in Chapter 3, invasive recording techniques allow the activities of single neurons or populations of neurons to be recorded. This chapter describes some of the achievements of invasive BCIs in animals and humans.

Two Major Paradigms in Invasive Brain- Computer Interfacing

BCIs Based on Operant Conditioning

A number of BCIs in animals have been based on operant conditioning, a phenomenon discussed in Section 6.2.1. In operant conditioning, an animal receives a reward upon selection of an appropriate action, e.g., pressing a lever. After repetitive pairing, the animal learns to execute the action in anticipation of the reward. In a BCI paradigm, the animal is rewarded if it selectively activates a neuron or population of neurons to move a cursor or prosthetic device in an appropriate manner.

Early BCI Studies

In the late 1960s, in one of the earliest demonstrations of brain- computer interfacing, Eberhard Fetz at the University of Washington in Seattle utilized the idea of operant conditioning to demonstrate that the activity of a single neuron in a primate’s motor cortex can be conditioned to control the needle of an analog meter (Fetz, 1969). The movement of the needle was directly coupled to the firing rate of the neuron: when the needle crossed a threshold, the monkey was rewarded. After several trials, the monkey learned to consistently move the needle past the threshold by increasing the firing rate of the recorded neuron (Figure 7.1). In this example of operant conditioning, the action (needle movement) that produces reward is coupled to increased activity in the recorded neuron (the conditioned response).

Introduction

The general equations of motion, developed in Chapter 10, are, without question, the complete and proper model of our dynamic system. However, because of their nonlinear character, they are not directly amenable to the use of the many mathematical tools that are available for linear systems. For example, many vibration and automatic control techniques are directly valid only for linear systems. Indeed, even the electronic instrumentation that is available for measurement of the dynamics of multibody systems is often designed to operate in the frequency domain and, thus, inherently assumes that the system treated is linear.

If there is any hope for a general solution technique for these equations of motion, it is probably through numeric integration by digital computer. We will investigate such an approach in Chapter 14. However, before looking at the general case, let us first study the dynamics of our system in the local vicinity of its current posture.

Linearization Assumptions

At its current posture, whatever posture this might be, we assert that the system in question exists in accordance with our general dynamic equations of motion. If it is not in equilibrium in the sense of being stationary or operating at constant velocity, then it is in dynamic equilibrium, meaning that its accelerations are consistent with these same equations of motion.

Mechanical Design: Synthesis versus Analysis

There are two completely different aspects of the study of mechanical systems: design and analysis. The concept embodied in the word design might be more properly termed synthesis, the process of contriving a scheme or a device for accomplishing a given purpose. Design is the process of developing the sizes, shapes, material compositions, types and arrangements of parts, and manufacturing processes so that the final system will perform a prescribed task. Although there are many phases of the design process that can be approached in a well-ordered scientific manner, the process is, by its very nature, as much an art as a science. It calls for imagination, intuition, creativity, judgment, and experience. The role of science in the design process can be viewed as providing tools to be used as the designer practices this art. Computer programs and computations that allow a designer to simulate a system and evaluate its potential performance play an important role in helping the designer practice the art. This is why scientific techniques such as the matrix methods discussed in this text play such an important role in dealing with the design of three-dimensional mechanisms and multibody systems.

Introduction

Through simulation of multibody systems as explained in the preceeding chapters, we can solve a variety of useful problems with no further enhancement. However, with the methods explained so far, we still lack the capability to simulate collisions, either between moving bodies or between a single moving body and its fixed surroundings. Simulation software developed strictly with the formulae presented so far assumes that a moving body may simply pass through others with no interference or impact. Clearly, this can benefit from enhancement.

Collision or contact between bodies cannot be detected unless it is through computations relating the geometries of the bodies’ surfaces. Therefore, we must have accurate geometric shapes for all bodies for which collisions are to be considered, and in as much detail and accuracy as we wish to monitor their possible contact. We need data for vertices, edges, and surfaces, and we need to distinguish the material from the exterior sides of such surfaces. Therefore, we need solid models of the bodies to be considered. Either constructive solid geometry (CSG) or boundary representation (B-Rep) or hybrid combinations may be considered, but wire-frame data are not sufficient.