From hardware and software to kernels and envelopes: a concept shift for robotics, developmental psychology, and brain sciences

doi:10.1017/CBO9780511994838.011

10 - From hardware and software to kernels and envelopes: a concept shift for robotics, developmental psychology, and brain sciences

from Part IV - Philosophical and theoretical considerations

Published online by Cambridge University Press: 05 February 2012

Frédéric Kaplan and

Pierre-Yves Oudeyer

Edited by

Jeffrey L. Krichmar and

Hiroaki Wagatsuma

Show author details

Jeffrey L. Krichmar: Affiliation:
University of California, Irvine
Hiroaki Wagatsuma: Affiliation:
Kyushu Institute of Technology (KYUTECH), Japan

Book contents

Get access

Summary

From hardware and software to kernels and envelopes

At the beginning of robotics research, robots were seen as physical platforms on which different behavioral programs could be run, similar to the hardware and software parts of a computer. However, recent advances in developmental robotics have allowed us to consider a reversed paradigm in which a single software, called a kernel, is capable of exploring and controlling many different sensorimotor spaces, called envelopes. In this chapter, we review studies we have previously published about kernels and envelopes to retrace the history of this concept shift and discuss its consequences for robotic designs and also for developmental psychology and brain sciences.

Information

Type: Chapter
Information: Neuromorphic and Brain-Based Robots , pp. 217 - 250

DOI: https://doi.org/10.1017/CBO9780511994838.011 [Opens in a new window]

Publisher: Cambridge University Press

Print publication year: 2011

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Book purchase

Temporarily unavailable

References

Asada, M.Hosoda, K.Kuniyoshi, Y. 2009 Cognitive developmental robotics: a surveyIEEE Transactions on Autonomous Mental Development 1 12CrossRef Google Scholar

Baldassarre, G. 2002 A modular neural-network model of the basal ganglia’s role in learning and selection motor behaviorsJournal of Cognitive Systems Research 3 5CrossRef Google Scholar

Baldwin, J. M. 1925 Mental Development in the Child and the RaceNew YorkMacmillanGoogle Scholar

Barto, A. G. 1995 Houk, J. C.Davis, J. L.Beiser, D. G.Models of Information Processing in the Basal GangliaCambridge, MAMIT Press215Google Scholar

Barto, A. G.Singh, S.Chentanez, N. 2004

Bell, D. S. 1973 The experimental reproduction of amphetamine psychosisArchives in General Psychiatry 29 35CrossRef Google Scholar PubMed

Berlyne, D. E. 1960 Conflict, Arousal and CuriosityMcGraw-HillCrossRef Google Scholar

Bernheimer, H.Birkmayer, W.Hornykiewicz, O., Jellinger, K.Seitelberger, F. 1973 Brain dopamine and the syndromes of Parkinson and Huntington: clinical, morphological and neurochemical correlationsJournal of the Neurological Sciences 20 415CrossRef Google Scholar PubMed

Berridge, K. C. 2007 The debate over dopamine’s role in reward: the case of incentive saliencePsychopharmacology 191 391CrossRef Google Scholar PubMed

Brooks, R. 1999 Cambrian Intelligence: The Early History of the New AICambridge, MAMIT PressGoogle Scholar

Bruner, J. 1962 On Knowing: Essays for the Left HandCambridge, MAHarvard University PressGoogle Scholar

Cameron, J.Pierce, W.D. 2002 Rewards and Intrinsic Motivation: Resolving the ControversySouth Hadley, MABergin and Garvey PressGoogle Scholar

Carboni, E.Imperato, A.Perezzani, L.Di Chiara, G. 1989 Amphetamine, cocaine, phencyclidine and nomifensine increases extra cellullar dopamine concentrations preferentially in the nucleus accumbens of freely moving ratsNeuroscience 28 653CrossRef Google Scholar

Clark, A. 2004 Natural-born Cyborgs: Minds, Technologies and the Future of Human IntelligenceOxford, UKOxford University PressGoogle Scholar

Csikszenthmihalyi, M. 1991 Flow: The Psychology of Optimal ExperienceNew YorkHarper PerennialGoogle Scholar

Daw, N.Kakade, S.Dayan, P. 2002 Opponent interactions between serotonin and dopamineNeural Networks 15 603CrossRef Google Scholar PubMed

Dayan, P.Sejnowski, T. J. 1996 Exploration bonuses and dual controlMachine Learning 25 5CrossRef Google Scholar

Dayan, P.Hinton, G.Giles, C.Hanson, S.Cowan, J. 1993 San Mateo, CAMorgan Kaufmann271

De Charms, R. 1968 Personal Causation: The Internal Affective Determinants of BehaviorNew YorkAcademic PressGoogle Scholar

Deci, E. L.Ryan, R. M. 1985 Intrinsic Motivation and Self-Determination in Human BehaviorNew York:Plenum PressCrossRef Google Scholar

Dember, W. N.Earl, R. W. 1957 Analysis of exploratory, manipulatory and curiosity behaviorsPsychological Review 64 91CrossRef Google Scholar PubMed

Deneve, S.Duhamel, J.-R.Pouget, A. 2007 Optimal sensorimotor integration in recurrent cortical networks: a neural implementation of Kalman filtersJournal of Neuroscience 27 5744CrossRef Google Scholar PubMed

Depue, R. A.Iacono, W. G. 1989 Neurobehavioral aspects of affective disordersAnnual Review of Psychology 40 457CrossRef Google Scholar PubMed

Dommett, E.Coizet, V.Blatha, C. D. 2005 How visual stimuli activate dopaminergic neurons at short latencyScience 307 1476CrossRef Google Scholar PubMed

Douglas, R.Martin, K. 1998 Shepherd, G. M.The Synaptic Organization of the BrainOxford, UKOxford University Press,459Google Scholar

Doya, K. 1999 What are the computations of cerebellum, basal ganglia, and the cerebral cortex?Neural Networks 12 961CrossRef Google Scholar PubMed

Doya, K. 2002 Metalearning and neuromodulationNeural Networks 15CrossRef Google Scholar PubMed

Doya, K.Samejima, K.Katagiri, K.Kawato, M. 2002 Multiple model-based reinforcement learningNeural Computation 14 1347CrossRef Google Scholar PubMed

Fedorov, V. V. 1972 Theory of Optimal ExperimentNew YorkAcademic PressGoogle Scholar

Festinger, L. 1957 A Theory of Cognitive DissonanceEvanston, ILRow, PetersonGoogle Scholar

Fiorillo, C. D. 2004 The uncertain nature of dopamineMolecular Psychiatry 9 122CrossRef Google Scholar PubMed

Gibson, J. 1986 The Ecological Approach to Visual PerceptionHillsdale, NJLawrence Erlbaum AssociatesGoogle Scholar

Grace, A. A. 1991 Phasic versus tonic dopamine release and the modulation of dopamine system responsivity: a hypothesis for the etiology of schizophreniaNeuroscience 41CrossRef Google Scholar PubMed

Gray, J. A. 1982 The Neuropsychology of Anxiety: An Enquiry into the Functions of the Septo-hippocampal SystemOxford, UKClarendon PressGoogle Scholar

Gray, J. A. 1990 Brain systems that mediate both emotion and cognitionCognition and Emotion 4 269CrossRef Google Scholar

Grey Walter, W. 1953 The Living BrainLondonPenguinGoogle Scholar

Harlow, H. F. 1950 Learning and satiation of response in intrinsically motivated complex puzzle performances by monkeysJournal of Comparative and Physiological Psychology 43 289CrossRef Google Scholar

Head, H.Holmes, G. 1911 Sensory disturbances from cerebral lesionsBrain 34CrossRef Google Scholar

Heath, R. G. 1963 Electrical self-stimulation of the brain in ManAmerican Journal of Psychiatry 120 571CrossRef Google Scholar PubMed

Hebb, D. O. 1955 Drives and the C.N.S (conceptual nervous system)Psychological Review 62 243CrossRef Google Scholar

Hooks, M. S.Kalivas, P. W. 1994 Involvement of dopamine and excitatory amino acid transmission in novelty-induced motor activityJournal of Pharmacology and Experimental Therapeutics 269 976Google Scholar PubMed

Horvitz, J.-C. 2000 Mesolimbocortical and nigrostriatal dopamine responses to salient non-reward eventsNeuroscience 96 651CrossRef Google Scholar PubMed

Horvitz, J.-C. 2002 Dopamine gating of glutamatergic sensorimotor and incentive motivational input signals to the striatumBehavioral and Brain Research 137 65CrossRef Google Scholar PubMed

Houk, J. C.Adams, J. L.Barto, A. G. 1995 Houk, J. C.Davis, J. L.Beiser, D. G.Models of Information Processing in the Basal GangliaCambridge, MAMIT Press, pp.Google Scholar

Huang, X.Weng, J. 2002 Prince, C.Demiris, Y.Marom, Y.Kozima, H.Balkenius, C.Proceedings of the 2nd International Workshop on Epigenetic Robotics: Modeling Cognitive Development in Robotic SystemsLund, SwedenLund UniversityGoogle Scholar

Hull, C. L. 1943 Principles of Behavior: An Introduction to Behavior TheoryNew YorkAppleton-Century-CroftGoogle Scholar

Hunt, J. McV. 1965 Nebraska Symposium on MotivationLincoln, NEUniversity of Nebraska Press189

Ikemoto, S.Panksepp, J. 1999 The role of nucleus accumbens dopamine in motivated behavior: a unifying interpretation with special reference to reward-seekingBrain Research Reviews 31 6CrossRef Google Scholar PubMed

Iriki, A.Tanaka, A.Iwamura, Y. 1996 Coding of modified schema during tool use by macaque postcentral neuronesCognitive Neuroscience and Neuropsychology 7 2325Google Scholar PubMed

Jaeger, H. 2001 The Echo State Approach to Analyzing and Training Recurrent Neural NetworksBonn, GermanyGerman National Research Institute for Computer Science (GMD)Google Scholar

Jaeger, H.Haas, H. 2004 Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communicationScience 304 78CrossRef Google Scholar PubMed

Joel, D.Niv, Y.Ruppin, E. 2002 Actor-critic models of the basal ganglia: new anatomical and computational perspectivesNeural Networks 15 535CrossRef Google Scholar PubMed

Jordan, M.Jacobs, R. 1994 Hierarchical mixtures of experts and the EM algorithmNeural Computation 6 181CrossRef Google Scholar

Kagan, J. 1972 Motives and developmentJournal of Personality and Social Psychology 22 51CrossRef Google Scholar PubMed

Kakade, S.Dayan, P. 2002 Dopamine: generalization and bonusesNeural Networks 15 549CrossRef Google Scholar PubMed

Kaplan, F. 2004 Who is afraid of the humanoid? Investigating cultural differences in the acceptance of robotsInternational Journal of Humanoid Robotics 1 465CrossRef Google Scholar

Kaplan, F. 2005 Les machines apprivoisees: comprendre les robots de loisirParisVuibert, Collections Automates IntelligentsGoogle Scholar

Kaplan, F.Oudeyer, P.-Y. 2003 Prince, C. G.Berthouze, L.Kozima, H.Proceedings of the 3rd International Workshop on Epigenetic Robotics: Modeling Cognitive Development in Robotic SystemsLund University Cognitive Studies 101Lund, SwedenLund University73Google Scholar

Kaplan, F.Oudeyer, P.-Y. 2006 Kaplan, F.Oudeyer, P.-Y.Revel, A.Proceedings of the Sixth International Workshop on Epigenetic Robotics: Modeling Cognitive Development in Robotic SystemsLund University Cognitive Studies 128Lund, SwedenLund UniversityGoogle Scholar

Kaplan, F.Oudeyer, P.-Y. 2007 In search of the neural circuits of intrinsic motivationFrontiers in Neuroscience 1 225CrossRef Google Scholar PubMed

Kaplan, F.Oudeyer, P.-Y. 2007 Nehaniv, C.Dautenhahn, K.Models and Mechanisms of Imitation and Social Learning: Behavioural, Social and Communication DimensionsCambridge, UKCambridge University Press, pp.361Google Scholar

Kaplan, F.Oudeyer, P.-Y. 2007 Un robot motivé pour apprendre: le rôle des motivations intrinsèques dans le développement sensorimoteurEnfance 59 46CrossRef Google Scholar

Kaplan, F.Oudeyer, P.-Y. 2008 Le corps comme variable expérimentaleLa Revue Philosophique 2008 287Google Scholar

Kaplan, F.Oudeyer, P.-Y. 2009 Stable kernels and fluid body envelopesSICE Journal of Control, Measurement, and System Integration 48Google Scholar

Kaplan, F.d’Esposito, M.Oudeyer, P.-Y. 2006 http://aibo.playroom.fr

Kaplan, F.Oudeyer, P.-Y.Bergen, B. 2007 Computational models in the debate over language learnabilityInfant and Child Development 17 55CrossRef Google Scholar

Kawato, M. 1999 Internal models for motor control and trajectory planningCurrent Opinion in Neurobiology 9 718CrossRef Google Scholar PubMed

Khamassi, M.Lachèze, L.Girard, B.Berthoz, A.Guillot, A. 2005 Actor-critic models of reinforcement learning in the basal gangliaAdaptive Behavior 13 131CrossRef Google Scholar

Lakoff, G.Johnson, M. 1998 Philosophy in the Flesh: The Embodied Mind and its Challenge to Western ThoughtNew YorkBasic BooksGoogle Scholar

Lakoff, G.Nunez, R. 2001 Where Mathematics Comes From: How the Embodied Mind Brings Mathematics Into BeingNew YorkBasic BooksGoogle Scholar

Lungarella, M.Metta, G.Pfeifer, R.Sandini, G. 2003 Developmental robotics: a surveyConnection Science 15 151CrossRef Google Scholar

Maas, W.Natschlager, T.Markram, H. 2002 Real-time computing without stable states: a new framework for neural computation based on perturbationsNeural Computation 14 2531CrossRef Google Scholar

Marshall, J.Blank, D.Meeden, L. 2004

McClure, S.Daw, N. D.Montague, P. R. 2003 A computational substate for incentive salienceTrends in Neurosciences 26 423CrossRef Google Scholar

Meltzoff, A.Gopnick, A. 1993 Baron-Cohen, Tager-Flusberg, Cohen, D.Understanding Other MindsOxfordUKOxford University Press335Google Scholar

Merleau-Ponty, M. 1942 La structure du comportementParis:Presses Universitaires de FranceGoogle Scholar

Merleau-Ponty, M. 1945 Phénoménologie de la PerceptionParisGallimardGoogle Scholar

Montague, P. R.Dayan, P.Sejnowski, T. J. 1996 A framework for mesencephalic dopamine systems based on predictive Hebbian learningJournal of Neuroscience 16 1936CrossRef Google Scholar PubMed

Montgomery, K. C. 1954 The role of exploratory drive in learningJournal of Comparative and Physiological Psychology 47 60CrossRef Google Scholar

Moore, C.Corkum, V. 1994 Social understanding at the end of the first year of lifeDevelopmental Review 14 349CrossRef Google Scholar

Mountcastle, V. 1978 Edelman, G.Mountcastle, V.The Mindful BrainCambridge, MAMIT Press17Google Scholar

Niv, Y.Daw, N. D.Joel, D.Dayan, P. 2006 Tonic dopamine: opportunity costs and the control of response vigorPsychopharmacology507Google Scholar PubMed

Oades, R. D. 1985 The role of noradrenaline in tuning and dopamine in switching between signals in the CNSNeuroscience Biobehavorial Review 9 261CrossRef Google Scholar PubMed

Oudeyer, P.-Y.Kaplan, F. 2006 Discovering communicationConnection Science 18 189CrossRef Google Scholar

Oudeyer, P.-Y.Kaplan, F. 2007 What is intrinsic motivation? A typology of computational approachesFrontiers in Neurorobotics 1CrossRef Google Scholar PubMed

Oudeyer, P.-Y.Kaplan, F.Hafner, V. V.Whyte, A. 2005 42

Oudeyer, P.-Y.Kaplan, F.Hafner, V. 2007 Intrinsic motivation systems for autonomous mental developmentIEEE Transactions on Evolutionary Computation 11 265CrossRef Google Scholar

Panksepp, J. 1998 Affective Neuroscience: The Foundations of Human and Animal EmotionsNew YorkOxford University PressGoogle Scholar

Pettit, H. O.Justice, J. B. 1989 Dopamine in the nucleus accumbens during cocaine self-administration as studied by in vivo microdialysisPharmacology Biochemistry and Behavior 34 899CrossRef Google Scholar PubMed

Pfeifer, R.Bongard, J. 2007 How the Body Shapes the Way We Think: A New View of IntelligenceCambridge, MAMIT PressGoogle Scholar

Pfeifer, R.Scheier, C. 1999 Understanding IntelligenceBoston, MAMIT PressGoogle Scholar

Piaget, J. 1952 The Origins of Intelligence in ChildrenNew YorkNorton PressCrossRef Google Scholar

Piaget, J. 1962 Play, Dreams and Imitation in ChildhoodNew YorkNorton PressGoogle Scholar

Ploghaus, A.Tracey, I.Clare, S. 2000 Learning about pain: the neural substrate of the prediction error of aversive eventsPNAS 97 9281CrossRef Google Scholar PubMed

Quaade, F.Vaernet, K.Larsson, S. 1974 Stereotaxic stimulation and electrocoagulation of the lateral hypothalamus in obese humansActa 30 111Google Scholar PubMed

Redgrave, P.Prescott, T.Gurney, K. 1999 Is the short latency dopamine response too short to signal reward error?Trends in Neurosciences 22 146CrossRef Google Scholar PubMed

Ring, M. 1994 Continual learning in reinforcement environmentsUniversity of Texas at AustinGoogle Scholar

Rochat, P. 2002 Melzoff, A.Prinz, W.The Imitative Mind: Development, Evolution and Brain BasesCambridge, UKCambridge University PressGoogle Scholar

Rolls, E. T. 1999 The Brain and EmotionOxford, UKOxford University PressGoogle Scholar

Roth, G.Dicke, U. 2005 Evolution of the brain and intelligenceTrends 9 250Google Scholar PubMed

Ryan, R.Deci, E.L. 2000 Intrinsic and extrinsic motivations: classic definitions and new directionsContemporary Educational Psychology 25 54CrossRef Google Scholar PubMed

Schaffer, H. 1977 Proceedings of Loch Lomonds SymposiumNew YorkAcademic Press, pp.3Google Scholar

Schilder, P. 1935 L’image du corpsParisGallimardGoogle Scholar

Schmidhuber, J. 1991

Schmidhuber, J. 1992 Learning complex, extended sequences using the principle of history compressionNeural Computation 4 234CrossRef Google Scholar

Schmidhuber, J. 2006 Optimal artificial curiosity, developmental robotics, creativity, music, and the fine artsConnection Science 18 173CrossRef Google Scholar

Schultz, W. 1998 Predictive reward signal of dopamine neuronsJournal of Neurophysiology 80 1CrossRef Google Scholar PubMed

Schultz, W. 2006 Behavioral theories and the neurophysiology of rewardAnnual Review of Psychology 57 87CrossRef Google Scholar

Schultz, W.Dayan, P.Montague, P. R. 1997 A neural substrate of prediction and rewardScience 275 1593CrossRef Google Scholar PubMed

Shepherd, G. M. 1988 Gazzaniga, M.Perspectives in Memory ResearchCambridge, MA:MIT Press93Google Scholar

Smith, A.Li, M.Becker, S.Kapur, S. 2006 Dopamine, prediction error and associative learning: a model-based accountNetwork: Computation in Neural Systems 17 61CrossRef Google Scholar PubMed

Steels, L. 1994 The artificial life roots of artificial intelligenceArtificial Life Journal 1 89Google Scholar

Steels, L. 2003 Intelligence with representationPhilosophical Transactions of the Royal Society A 361 2381CrossRef Google Scholar PubMed

Steels, L. 2004 Fumiya, I.Pfeifer, R.Steels, L.Kunyoshi, K.Embodied Artificial IntelligenceBerlinSpringer Verlag231CrossRef Google Scholar

Stellar, J. R. 1985 The Neurobiology of Motivation and RewardNew YorkSpringer VerlagCrossRef Google Scholar

Storck, J.Hochreiter, S.Schmidhuber, J. 1995 Proceedings of ICANN 1995: International Conference on Artificial Neural NetworksParisEC2159Google Scholar

Suri, R. E.Schultz, W. 2001 Temporal difference model reproduces anticipatory neural activityNeural Computation 13 841CrossRef Google Scholar PubMed

Sutton, R. S. 1988 Learning to predict by the methods of temporal differencesMachine Learning 3 9CrossRef Google Scholar

Sutton, R. S.Barto, A. G. 1998 Reinforcement Learning: An IntroductionCambridge, MAMIT PressGoogle Scholar

Sutton, R. S.Precup, D.Singh, S. 1999 Between MDPSs and semi-MDPs: a framework for temporal abstraction in reinforcement learningArtificial Intelligence 112 181CrossRef Google Scholar

Tani, J. 2007 On the interactions between top-down anticipation and bottom-up regressionFrontiers in Neurorobotics 1CrossRef Google Scholar PubMed

Tani, J.Nolfi, S. 1999 Learning to perceive the world as articulated: an approach for hierarchical learning in sensory-motor systemsNeural Network 12 1131CrossRef Google Scholar PubMed

Thorndike, E. L. 1911 Animal Intelligence: Experimental StudiesNew YorkMacmillanCrossRef Google Scholar

Turing, A. 1950 Computing machinery and intelligenceMind 59 433CrossRef Google Scholar

Varela, F. J.Thompson, E.Rosch, E. 1991 The Embodied Mind: Cognitive Science And Human ExperienceCambridge, MAMIT PressGoogle Scholar

von Uexküll, J. 1909 Umwelt und Innenwelt der TiereBerlinSpringerGoogle Scholar

Vygotsky, L. 1978 Mind in Society: The Development of Higher Psychological ProcessesCambridge, MAHarvard University PressGoogle Scholar

Walkins, C. J. C. H.Dayan, P. 1992 Q-learningMachine Learning 8 279CrossRef Google Scholar

Warnier, J-P. 1999 Construire la culture materielle: l’homme qui pensait avec les doigtsParisPresses Universitaires de FranceCrossRef Google Scholar

Weinberger, D. R. 1987 Implications of normal brain development for the pathogenesis of schizophreniaArchives in General Psychiatry 44 660CrossRef Google Scholar PubMed

Weiner, I.Joel, D. 2002 Chiara, G. D.Handbook of Experimental Pharmacology. Vol 154/II: Dopamine in the CNS IIBerlinSpringer417Google Scholar

White, N. M. 1989 Reward or reinforcement: what’s the difference?Neuroscience Biobehavorial Review 13 181CrossRef Google Scholar PubMed

White, R. N. 1959 Motivation reconsidered: the concept of competencePsychological Review 66 297CrossRef Google Scholar

Wiering, M.Schmidhuber, J. 1997 HQ-learningAdaptive Behavior 6 219CrossRef Google Scholar

Wise, R. A. 1989 Liebman, J. M.Cooper, S. J.The Neuropharmacological Basis of RewardOxford, UKClarendon Press377Google Scholar

Yoshimoto, K.McBride, W.J.Lumeng, L.Li, T.-K. 1991 Alcohol stimulates the release of dopamine and serotonin in the nucleus accumbensAlcohol 9 17CrossRef Google Scholar

Accessibility standard: Unknown

Accessibility compliance for the PDF of this book is currently unknown and may be updated in the future.