Skip to main content Accessibility help

Self-reproduction for articulated behaviors with dual humanoid robots using on-line decision tree classification

  • Jane Brooks Zurn (a1), Yuichi Motai (a1) (a2) and Scott Vento (a3)

We have proposed a new repetition framework for vision-based behavior imitation by a sequence of multiple humanoid robots, introducing an on-line method for delimiting a time-varying context. This novel approach investigates the ability of a robot “student” to observe and imitate a behavior from a “teacher” robot; the student later changes roles to become the “teacher” for a naïve robot. For the many robots that already use video acquisition systems for their real-world tasks, this method eliminates the need for additional communication capabilities and complicated interfaces. This can reduce human intervention requirements and thus enhance the robots' practical usefulness outside the laboratory. Articulated motions are modeled in a three-layer method and registered as learned behaviors using color-based landmarks. Behaviors were identified on-line after each iteration by inducing a decision tree from the visually acquired data. Error accumulated over time, creating a context drift for behavior identification. In addition, identification and transmission of behaviors can occur between robots with differing, dynamically changing configurations. ITI, an on-line decision tree inducer in the C4.5 family, performed well for data that were similar in time and configuration to the training data but the greedily chosen attributes were not optimized for resistance to accumulating error or configuration changes. Our novel algorithm, OLDEX identified context changes on-line, as well as the amount of drift that could be tolerated before compensation was required. OLDEX can thus identify time and configuration contexts for the behavior data. This improved on previous methods, which either separated contexts off-line, or could not separate the slowly time-varying context into distinct regions at all. The results demonstrated the feasibility, usefulness, and potential of our unique idea for behavioral repetition and a propagating learning scheme.

    • Send article to Kindle

      To send this article to your Kindle, first ensure is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

      Note you can select to send to either the or variations. ‘’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

      Find out more about the Kindle Personal Document Service.

      Self-reproduction for articulated behaviors with dual humanoid robots using on-line decision tree classification
      Available formats
      Send article to Dropbox

      To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

      Self-reproduction for articulated behaviors with dual humanoid robots using on-line decision tree classification
      Available formats
      Send article to Google Drive

      To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

      Self-reproduction for articulated behaviors with dual humanoid robots using on-line decision tree classification
      Available formats
The online version of this article is published within an Open Access environment subject to the conditions of the Creative Commons Attribution-NonCommercial-ShareAlike licence . The written permission of Cambridge University Press must be obtained for commercial re-use.
Corresponding author
*Corresponding author. E-mail:
Hide All
1.Amit, R. and Matarić, M., “Learning Movement Sequences from Demonstration,” Proceedings of the International Conference on Development and Learning (ICDL '02), Cambridge, Massachusetts (2002) pp. 203208.
2.Arsenio, A., “Children, Humanoid Robots and Caregivers,” Proceedings of the 4th International Workshop on Epigenetic Robotics: Modeling Cognitive Development in Robotic Systems Children, Genoa, Italy (2004) vol. 117, pp. 1926.
3.Atkeson, C. G., Moore, A. W. and Schaal, S., “Locally weighted learning,” Artif. Intell. Rev. 11 (1), 1173 (1997).
4.Bentivegna, D. and Atkeson, C. G., “Using Primitives in Learning from Observation,” Proceedings of the 1st IEEE-RAS International Conference on Humanoid Robots, Boston, MA (2000).
5.Bentivegna, D. C. and Atkeson, C. G., “Learning from Observation Using Primitives,” Proceedings of the IEEE International Conference on Robotics and Automation (ICRA'01) (IEEE, Piscataway, NJ, USA, 2001), Seoul, Korea, vol. 2, pp. 19881993.
6.Billard, A. and Hayes, G., “DRAMA, a connectionist architecture for control and learning in autonomous robots,” Adapt. Behav. 7 (1), 3563 (1999).
7.Billard, A. and Matarić, M. J., “Learning human arm movements by imitation: Evaluation of a biologically inspired connectionist architecture,” Robot. Auton. Syst. 37 (2–3), 145160 (2001).
8.Bongard, J. and Pfeifer, R., “Evolving Complete Agents Using Artificial Ontogeny,”. In: Morpho-functional Machines: The New Species (Designing Embodied Intelligence) (Springer-Verlag, Berlin, 2003) pp. 237258.
9.Bongard, J., Zykov, V. and Lipson, H., “Resilient machines through continuous self-modeling,” Science 314 (5802), 11181121 (2006).
10.Boyd, R. S., 2009, “Robots are narrowing the gap with humans,”
11.Breiman, L., Friedman, J. H., Olshen, R. A. and Stone, C. J., Classification and Regression Trees (Wadsworth, Belmont, CA, 1984).
12.Calinon, S. and Billard, A., “Recognition and Reproduction of Gestures using a Probabilistic Framework combining PCA, ICA and HMM,” Proceedings of the International Conference on Machine Learning (ICML), Bonn, Germany, August 2005 (2005) pp. 105–112.
13.Calinon, S., Guenter, F., and Billard, A., “On learning, representing, and generalizing a task in a humanoid robot,” IEEE Trans. Syst. Man Cybern. B, 37 (2), 286298 (2007).
14.Cao, F. and Shepherd, B., “MIMIC: A Robot Planning Environment Integrating Real and Simulated Worlds,” Proceedings, IEEE International Symposium on Intelligent Control (IEEE, Piscataway, NJ, USA, Sep. 25–26, 1989), Albany, NY, USA, pp. 459464.
15.Cole, E., “AMARSi project could see robots learn from co-workers,” Retrieved Mar. 17, 2010,
16.Dey, A. K. and Abowd, G. D., “Towards a Better Understanding of Context and Context-Awareness,” Technical Report, GIT-GVU-99-22. Georgia Institute of Technology (1999).
17.Drumwright, E. and Matarić, M. J., “Generating and Recognizing Free-Space Movements in Humanoid Robots,” Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE, Piscataway, NJ, USA, 2003), Las Vegas, NV, USA, vol. 2, pp. 16721678.
18.Drury, J. L., Scholtz, J. and Yanco, H. A., “Awareness in Human-Robot Interactions,” Proceedings of the IEEE International Conference on Systems, Man and Cybernetics— (IEEE, Piscataway, NJ, USA, 2003), Washington, DC, USA, vol. 1, pp. 912918.
19.Fong, T., Nourbakhsh, I. and Dautenhahn, K., “A Survey of Socially Interactive Robots,” Robotics and Autonomous Systems 42 (3–4), 143166 (2003).
20.Friedman, J. H., “A recursive partitioning decision rule for nonparametric classification,” IEEE Trans. Comput. 26 (4), 404408 (1977).
21.Hamner, E., Gockley, R., Porter, E. and Nourbakhsh, I., “The personal rover project: The comprehensive design of a domestic personal robot,” Robot. Auton. Syst. Special Issue on Socially Interact. Robots 42 (3–4), 245258 (2003).
22.Haritaoglu, I., Harwood, D. and Davis, L. S., “W4: Real-time surveillance of people and their activities,” IEEE Trans. Pattern Anal. Mach. Intell. 22 (8), 809830 (2000).
23.Harries, M. B., Sammut, C. and Horn, K., “Extracting hidden context,” Mach. Learn. 32 (2), 101126 (1998).
24.Hulten, G., Spencer, L. and Domingos, P., “Mining Time-Changing Data Streams,” Paper Presented at the Proceedings of the 7th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA (2001) pp. 97106.
25.Hunt, E. B., Marin, J. and Stone, P. J., Experiments in Induction (Academic Press, New York, NY, USA, 1966).
26.Ijspeert, A. J., Nakanishi, J., Shibata, T. and Schaal, S., “Nonlinear Dynamical Systems for Imitation with Humanoid Robots,” Proceedings of the 2nd IEEE-RAS International Conference on Humanoid Robots, Tokyo, Japan, (IEEE, Piscataway, NJ, USA, 2001) pp. 219226.
27.Inoue, Y., Tohge, T. and Iba, H., “Object Transportation by Two Humanoid Robots Using Cooperative Learning,” Proceedings of the 2004 Congress Evolutionary Computation, Portland, OR, USA (IEEE, Piscataway, NJ, USA, 2004) vol. 1, pp. 12011208.
28.Khalid, O., “A unified approach for motion and force control of robot manipulators: the operational space formulation,” IEEE J. Robot. Autom. 3 (1), 4353 (1987).
29.Kim, B. and Lee, G., “Decision-Tree Based Error Correction for Statistical Phrase Break Prediction in Korean,” Paper presented at the Proceedings of the 18th Conference on Computational linguistics (COLING) (Morgan Kaufmann Publishers, Saarbrücken, Germany, San Francisco, CA, USA, 2000) vol. 2, pp. 10511055.
30.Klingspor, V., Demiris, J., and Kaiser, M., “Human-robot-communication and machine learning,” Applied Artificial Intelligence Journal 11 (7/8), 719746 (1997).
31.Kondo Kagaku Co., Ltd., Jun. 30, 2008, Retrieved Nov. 7, 2008,
32.Kosuge, K. and Oosumi, T., “Decentralized Control of Multiple Robots Handling an Object,” Proceedings of the IEEE Int. Conf. Intelligent Robots and Systems (IROS '96), Osaka, Japan (IEEE, Piscataway, NJ, USA, Nov. 4–8, 1996).
33.Kozima, H. and Yano, H., “A Robot that Learns to Communicate with Human Caregivers,” Proceedings of the 1st International Workshop on Epigenetic Robotics (Lund University Cognitive Studies, Lund, Sweden, Lund, Sweden, 2001).
34.Kruger, V., Herzog, D., Baby, S., Ude, A. and Kragic, D., “Learning actions from observations,” IEEE Robot. Autom. Mag. 17 (2), 3043 (2010).
35.Liu, J.-S., Liang, T.-C. and Lin, Y.-A., “Realization of a ball passing strategy for a robot soccer game: A case study of integrated planning and control,” Robotica 22 (3), 329338 (2004).
36.Loh, W.-Y. and Shih, Y.-S., “Split selection methods for classification trees,” Statistica Sinica 7, 815840 (1997).
37.Martinoli, A., Ijspeert, A. J. and Gambardella, L. M., “A Probabilistic Model for Understanding and Comparing Collective Aggregation Mechanisms,” Proceedings of the 5th European Conference on Advances in Artificial Life (Springer-Verlag, Berlin/Heidelberg, 1999), Lausanne, Switzerland, vol. 1674, pp. 575584.
38.Matarić, M. J., “Reinforcement learning in the multi-robot domain,” Auton. Robots 4 (1), 7383 (1997).
39.Matarić, M. J., “Sensory-Motor Primitives as a Basis for Imitation: Linking Perception to Action and Biology to Robotics,” In: Imitation in Animals and Artifacts (Dautenhahn, K. and Nehaniv, C. L., eds.), (MIT Press, Cambridge, MA, 2002) pp. 391422.
40.McCallum, R. A., “Hidden state and reinforcement learning with instance-based state identification,” IEEE Trans. Syst. Man Cybern. 26 (3), 464473 (1996).
41.Motion Analysis, Inc., Retrieved Nov. 7, 2008,
42.Nicolescu, M. N. and Matarić, M. J., “Natural Methods for Robot Task Learning: Instructive Demonstrations, Generalization and Practice,” in Proceedings Second International Joint Conference on Autonomous Agents and Multi-Agent Systems pages 241–248, Melbourne, Australia, July 14–18, 2003.
43.A.P.A.S, Ariel Dynamics, Jun. 30, 2008, Retrieved Nov. 7, 2008,
44.Pereira, G. A. S., Kumar, V., Spletzer, J. R., Taylor, C. J. and Campos, M. F. M., “Cooperative Transport of Planar Objects by Multiple Mobile Robots Using Object Closure,” In: Experimental Robotics VIII (Springer, Berlin/Heidelberg, 2003) vol. 5, pp. 287296.
45.Peters, J. and Schaal, S., “Policy Gradient Methods for Robotics,” Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Beijing, China, (2006) pp. 22192225.
46.Perzanowski, D., Schultz, A. C., Adams, W., Marsh, E. and Bugajska, M., “Building a multimodal human-robot interface,” Intell. Syst. 16 (1), 1621 (2001).
48.Piaget, J., Play, Dreams, and Imitation in Childhood (Gattegno, C. and Hodgson, F. M., Trans.) (Norton, New York, NY, USA, 1962 (translation), 1945 (French)).
49.Pollard, N. S., Hodgins, J. K., Riley, M. J. and Atkeson, C. G., “Adapting Human Motion for the Control of a Humanoid Robot,” Proceedings of the IEEE International Conference on Robotics and Automation (IEEE, Piscataway, NJ, USA, 2002) Washington, DC, USA, vol. 2, pp. 13901397.
50.Quinlan, J. R., “Discovering Rules by Induction from Large Collections of Examples,” In: Expert Systems in the Micro-electronic Age (Michie, D., ed.) (Edinburgh University Press, Edinburgh, UK, 1979).
51.Quinlan, J. R., “Induction of decision trees,” Mach. Learn. 1 (1), 81106 (1986).
52.Quinlan, J. R., C4.5: Programs for Machine Learning (Morgan Kaufmann Publishers, Inc., San Francisco, CA, USA, 1993).
53.Rescorla, R. A., “Probability of shock in the presence and absence of CS in fear conditioning,” J. Comp. Physiol. Psychol. 66, 15 (1968).
54.Schaal, S., “Is imitation learning the route to humanoid robots?,” Trends Cogn. Sci. 3 (6), 233242 (1999).
55.Schlimmer, J. C. and Fisher, D., “A Case Study of Incremental Concept Induction,” Proceedings of the 5th National Conference on Artificial Intelligence (Morgan Kaufmann, Philadelphia, PA, USA, 1986) Philadelphia, PA, USA, Vol. 1, pp. 495501.
56.Schlimmer, J. C. and Granger, R. H., “Incremental learning from noisy data,” Mach. Learn. 1 (3), 317354 (1986).
57.Shah Hamzei, G. H., Mulvaney, D. J. and Sillitoe, I. P. W., “Batch-Mode Decision Tree Learning Applied to Intelligent Reactive Robot Control,” Proceedings of the 6th International Conference on Emerging Technologies and Factory Automation (ETFA '97) Los Angeles, CA, USA, (IEEE, Piscataway, NJ, USA, 1997) pp. 416420.
58.Stauffer, C. and Grimson, W. E. L., “Learning patterns of activity using real-time tracking,” IEEE Trans. Pattern Anal. Mach. Intell. 22 (8), 747757 (2000).
59.Steels, L. and Vogt, P., “Grounding Adaptive Language Games in Robotic Agents,” In Proceedings of the 4th European Conference on Artificial Life Brighton, UK, (MIT Press, Cambridge, MA, USA/London, 1997) pp. 474482.
60.Tani, J., Nishimoto, R., Namikawa, J., and Ito, M., “Codevelopmental learning between human and humanoid robot using a dynamic neural-network model”, IEEE Trans. on Syst. Man and Cybern. Part B-Cybernetics 38 (1), pp. 4359, 2008.
61.Tan, K. C., Chen, Y. J., Tan, K. K. and Lee, T. H., “Task-oriented developmental learning for humanoid robots,” IEEE Trans. Ind. Electron. 52 (3), 906914 (2005).
62.Utgoff, P., Mar. 23, 2001, “Incremental tree induction,” Retrieved Nov. 7, 2008,
63.Utgoff, P. E., “Incremental induction of decision trees,” Mach. Learn. 4 (2), 161186 (1989).
64.Utgoff, P. E., Berkman, N. C. and Clouse, J. A., “Decision tree induction based on efficient tree restructuring,” Mach. Learn. 29 (1), 544 (1997).
65.Vicon Motion Systems, Oxford Metrics Ltd., Retrieved Nov. 7, 2008,
66.Vijayakumar, S., D'Souza, A., Shibata, T., Conradt, J. and Schaal, S., “Statistical learning for humanoid robots,” Auton. Robots 12 (1), 5569 (2002).
67.Widmer, G. and Kubat, M., “Learning in the presence of concept drift and hidden contexts,” Mach. Learn. 23 (1), 69101 (1996).
68.Wren, C. R., Azarbayejani, A., Darrell, T. and Pentland, A. P., “Pfinder: real-time tracking of the human body,” IEEE Trans. Pattern Anal. Mach. Intell. 19 (7), 780785 (1997).
69.Yanco, H. A., “Synthetic Robot Language Development,” In Proceedings of the Twelfth National Conference on Artificial Intelligence. Seattle, Washington, USA. AAAI Press/The MIT Press, 1994. p. 1500.
70.Yokokohji, Y., Kitaoka, Y. and Yoshikawa, T., “Motion capture from demonstrator's viewpoint and its application to robot teaching,” J. Robot. Syst. 22 (2), 8797 (2005).
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

  • ISSN: 0263-5747
  • EISSN: 1469-8668
  • URL: /core/journals/robotica
Please enter your name
Please enter a valid email address
Who would you like to send this to? *



Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed