Perceiving, representing, and reasoning about the human body is an incredibly difficult task; the fact that we traffic in this ability with such ease is no mean feat. After all, bodies are more than complex objects. They are more than a collection of parts that co-articulate, more than things that can be acted on, more than entities that are in the world and of the world. Rather, bodies act on the world. Bodies, by definition, are bodies in action: limbs move, joints articulate, digits bend and curl. Critically, many of these actions convey meaning: they are about the world. And it is precisely this meaningfulness of the human body in relation to the world that makes the task of perceiving, representing and reasoning about the human body both so complex and so critical.

Among other things, understanding the human body is central to our everyday social reasoning and social interactions: we perceive the body to read the mind. Goals and intentions, in particular, while generated by the mind, are instantiated in bodily acts. An event in which an arm moves at a 45-degree trajectory, and a rate of 15 centimeters a second, culminating in contact with a wine glass, is more than the collection of its surface features. It signifies an actor’s goal (obtaining the wine glass), reveals the actor’s underlying intention (getting a drink of wine), and can be a window into the actor’s proclivities and dispositions (liking wine).

Finding the design principle of reward functions is a big challenge in both artificial intelligence and neuroscience. Successful acquisition of a task usually requires rewards to be given not only for goals but also for intermediate states to promote effective exploration. We propose a method to design “intrinsic” rewards for autonomous robots by combining constrained policy gradient reinforcement learning and embodied evolution. To validate the method, we use the Cyber Rodent robots, in which collision avoidance, recharging from battery pack, and “mating” by software reproduction are three major “extrinsic” rewards. We show in hardware experiments that the robots can find appropriate intrinsic rewards for the visual properties of battery packs and potential mating partners to promote approach behaviors.

Introduction

In application of reinforcement learning (Sutton and Barto, 1998) to real-world problems, the design of the reward function is critical for successful achievement of the task. Designing appropriate reward functions is a nontrivial, time-consuming process in practical applications. Although it appears straightforward to assign positive rewards to desired goal states and negative rewards to states to be avoided, finding a good balance between multiple rewards often needs careful tuning. Furthermore, if rewards are given only at isolated goal states, blind exploration of the state space takes a long time. Rewards at intermediate subgoals, or even along the trajectories leading to the goal, promote focused exploration, but appropriate design of such additional rewards usually requires prior knowledge of the task or trial and error by the experimenter.

Neuromorphic and brain-based robots are not encapsulated in a single field with its own journal or conference. Rather, the field crosses many disciplines, and groundbreaking neuromorphic robot research is carried out in computer science, engineering, neuroscience, and many other departments. The field is known by many names: biologically inspired robots, brain-based devices, cognitive robots, neuromorphic engineering, neurobots, neurorobots, and many more. Arguably, the field may have begun with William Grey Walter’s turtles, created in the 1950s, whose simple yet interesting behaviors were guided by an analog electronic nervous system. Another landmark was the fascinating thought experiments in the book by Valentino Braitenberg, Vehicles: Experiments in Synthetic Psychology. Braitenberg’s Vehicles inspired a generation of hobbyists and scientists, present company included, to use synthetic methodology (Braitenberg’s term) to study brain, body, and behavior together. We like to think of synthetic methodology as “understanding through building” and it is certainly an apt mission statement for neuromorphic and brain-based robots.

Introduction

Why build robot models of animals and their nervous systems? One answer is that in building a robot model of a target organism, which mimics sufficiently some aspects of that animal’s body, brain, and behavior, we can expect to learn a good deal about the original creature. Synthesis (engineering) is quite different from analysis (reverse-engineering), is often easier, and teaches fascinating lessons (Braitenberg, 1986). Another answer is that a robot model should allow us to conduct experiments that will help us better understand the biological system, and that would be impossible or at least much more difficult to perform in the original animal (Rosenblueth and Wiener, 1945). In this chapter our target organism is the rat and our specific focus is on the sophisticated tactile sensory system provided by that animal’s facial whiskers (vibrissae). Neurobiology shows us that the brain nuclei and circuits that process vibrissal touch signals, and that control the positioning and movement of the whiskers, form a neural architecture that is a good model of how the mammalian brain, in general, coordinates sensing with action. Thus, by building a robot whisker system we can take a significant step towards building the first robot “mammal.” Following a short review of relevant rat biology, this chapter will describe the design and development of two whiskered robot platforms – Whiskerbot and SCRATCHbot – that we have constructed in order to better understand the rat whisker system, and to test hypotheses about whisker control and vibrissal sensing in a physical brain-based device. We provide a description of each platform, including mechanical, electronic, and software components, discussing, in relation to each component, the design constraints we sought to meet and the trade-offs made between biomimetic ideals and engineering practicalities. Some results obtained using each platform are described together with a brief outline of future development plans. Finally, we discuss the use of biomimetic robots as scientific models and consider, using the example of whiskered robots, what contribution robotics can make to the brain and behavioral sciences.

Introduction

One of the most amazing aspects of brain function is that free will and consciousness emerges from the simple elemental functions of neurons. How do a hundred billion neurons produce global functions, such as intention, mind, and consciousness? As gathering a billion people is not equal to making a civilized society, the brain is not merely a combination of neurons. There would be rules of relation and principles of action. I have been interested for many years in the neurodynamics of situated cognition and contextual decision making, particularly focusing on synchronization mechanisms in the brain. Neural synchronization is well known in spinal motor coordination (e.g. central pattern generators, CPG), circadian rhythms and EEG recordings of human brain activities during mental tasks. Synchronized population activity plays functional roles in memory formation and context-dependent utilization of personal experiences in animal models. However, those experiments and models have dealt with a specific brain circuit in a fixed condition, or at least less attention has been given to an embodied view, where the brain, body, and environment comprise a closed whole loop. The embodied view is the natural setting for a brain functioning in the real world. I have recently become interested in building an online and on-demand experimental platform to link the robotic body with its neurodynamics. This platform is implemented in a remote computer and gives us the advantage of studying brain functions in a dynamic environment, and to offer qualitative analyses of behavioral time, in contradistinction to neuronal time, or mental time. This chapter relates past work to present work in an informal way that might be uncommon in journal papers. By taking advantage of this opportunity, I will use informal speech and explanations, as well as personal anecdotes to guide the reader to understand important trends and perspectives in this topic. Section 12.1 gives an introduction to artificial systems that makes a commitment to biology, and argues a point of biologically inspired robotics in the viewpoint of being life. Section 12.2 overviews the multiple memory systems of the brain in terms of conscious awareness. Section 12.3 describes robotic methodologies by using neural dynamics of oscillatory components to enable the system to provide online decision making in cooperation with involuntary motor controls, and discusses necessities for future work. Section 12.4 summarizes key concepts and future perspectives.

Introduction

After several decades of developmental research on intelligent robotics in our lab, we began to focus on the realization of mammalian adaptability functions for our upper-body humanoid robot ISAC (Intelligent Soft Arm Control) described in Kawamura et al. (2000, 2004). Currently, most engineering solutions used in robot designs do not have this level of learning and adaptation. Mammalian adaptability is highly desirable in a robot, because mammals are singularly adaptable goal-directed agents. Mammals learn from experiences with a distinctive degree of flexibility and richness that assures goal accomplishment by a very high proportion of individuals. Thus, in the future, robot capability will be substantially advanced once robots can actively seek goal-directed experiences and learn about new tasks under dynamic and challenging environments.

Seeking inspiration for how to achieve this goal, we look to the mammalian brain; in particular, to the structural and functional commonalities observed across mammalian species. From rodents to humans, mammals share many neural mechanisms and control processes relevant to adaptability. Mammals typically accomplish goals in a timely fashion, in situations from the familiar to the new and challenging. Moreover, mammals learn how to function effectively, with few innate capabilities and with little or no supervision of their learning. Albeit with many gaps in knowledge of what makes the human brain distinctively capable, enough seems to be known about the whole mammalian brain to inform architectural analysis and embodied modeling of mammalian brains.

Introduction

It has been a huge challenge to program autonomous robots for unstructured and new environments. Various modules are difficult to program and so is the coordination among modules and motors. Existing neuroanatomical studies have suggested that the brain uses similar mechanisms to coordinate the different sensor modalities (e.g. visual and auditory) and the different motor modalities (e.g. arms, legs, and the vocal tract). Via sensorimotor interactions with the robot’s internal and external environments, autonomous mental development (AMD) in this chapter models the brain as not only an information processor (e.g. brain regions and their interconnections), but also the causality for its development (e.g. why each region does what it does). The mechanisms of AMD suggest that the function of each brain region is not preset statically before birth by the genome, but is instead the emergent consequence of its interconnections with other brain regions through the lifetime experience. The experience of interactions not only greatly shapes what each region does, but also how different regions cooperate. The latter seems harder to program than a static function. As a general-purpose model of sensorimotor systems, this chapter describes the developmental program for the visuomotor system of a developmental robot. Based on the brain-inspired mechanisms, the developmental program enables a network to wire itself and to adapt “on the fly” using bottom-up signals from sensors and top-down signals from externally supervised or self-supervised acting activities. These simple mechanisms are sufficient for the neuromorphic Where What Network 1 (WWN-1) to demonstrate small-scale but practical-grade performance for the two highly intertwined problems of vision – attention and recognition – in the presence of complex backgrounds.

Introduction

Rats are superior to the most advanced robots when it comes to creating and exploiting spatial representations. A wild rat can have a foraging range of hundreds of meters, possibly kilometers, and yet the rodent can unerringly return to its home after each foraging mission, and return to profitable foraging locations at a later date (Davis, et al., 1948). The rat runs through undergrowth and pipes with few distal landmarks, along paths where the visual, textural, and olfactory appearance constantly change (Hardy and Taylor, 1980; Recht, 1988). Despite these challenges the rat builds, maintains, and exploits internal representations of large areas of the real world throughout its two to three year lifetime. While algorithms exist that allow robots to build maps, the questions of how to maintain those maps and how to handle change in appearance over time remain open.

The robotic approach to map building has been dominated by algorithms that optimize the geometry of the map based on measurements of distances to features. In a robotic approach, measurements of distance to features are taken with range-measuring devices such as laser range finders or ultrasound sensors, and in some cases estimates of depth from visual information. The features are incorporated into the map based on previous readings of other features in view and estimates of self-motion. The algorithms explicitly model the uncertainty in measurements of range and the measurement of self-motion, and use probability theory to find optimal solutions for the geometric configuration of the map features (Dissanayake, et al., 2001; Thrun and Leonard, 2008). Some of the results from the application of these algorithms have been impressive, ranging from three-dimensional maps of large urban structures (Thrun and Montemerlo, 2006) to natural environments (Montemerlo, et al., 2003).

An increasing number of projects worldwide are investigating the possibility of including robots in assessment and therapy practices for individuals with autism. There are two major reasons for considering this possibility: the special interest of autistic people in robots and electronic tools, and the rapid developments in multidisciplinary studies on the nature of social interaction and on autism as atypical social behavior.

Several branches of the social sciences and neurosciences, which aim to understand the social brain, advocate the perspective that social behaviors (e.g. shared attention, turn taking, and imitation) have evolved as an additional functionality of a general sensorimotor system for action. The basic feature of this system is the existence of a common representation between perception for action and the action itself. An extended social brain system facilitates processing of emotional stimuli, empathy, and perspective taking.

Introduction

We can easily manipulate a variety of objects with our hands. When exploring an object, we gather rich sensory information through both haptics and vision. The haptic and visual information obtained through such exploration is, in turn, key for realizing dexterous manipulation. Reproducing such codevelopment of sensing and adaptive/dexterous manipulation by a robotic hand is one of the ultimate goals of robotics, and further, it would be essential for understanding human object recognition and manipulation.

Although many robotic hands have been developed, their performance is by far inferior to that of human hands. One reason for this performance difference may be due to differences in grasping strategies. Historically, research on robotic hands has mainly focused on pinching manipulation (e.g. Nagai and Yoshikawa, 1993) because the analysis was easy with point-contact conditions. Based on the analysis, roboticists applied control schemes using force/touch sensors at the fingertips (Kaneko et al., 2007; Liu et al., 2008). Since the contact points are restricted to the fingertips, it is easy for the robot to calculate how it grasps an object (e.g. a holding polygon) and how large it should exert force based on friction analysis. However, the resultant grasping is very brittle since a slip of just one of the contacting fingertips may lead to dropping the object.

Introduction

Within the past few decades, the nature of consciousness has become a central issue in neuroscience, and it is increasingly the focus of both theoretical and empirical work. Studying consciousness is vital to developing an understanding of human perception and behavior, of our relationships with one another, and of our relationships with other potentially conscious animals. Although the study of consciousness through the construction of artificial models is a recent innovation, the advantages of such an approach are clear. First, models allow us to investigate consciousness in ways that are currently not feasible using human subjects or other animals. Second, an artifact that exhibits the necessary and sufficient properties of consciousness may conceivably be the forerunner of a new and very useful class of neuromorphic robots.

A model of consciousness must take into account current theories of its biological bases. Although the field of artificial consciousness is a new one, it is striking how little attention has been given to modeling mechanisms. Instead, great – and perhaps undue – emphasis has been placed on purely phenomenological models. Many of these models are strongly reductionist in aim and fail to specify neural mechanisms.