Hostname: page-component-76fb5796d-qxdb6 Total loading time: 0 Render date: 2024-04-29T22:59:33.460Z Has data issue: false hasContentIssue false
  • English
  • Français

Dynamic bipedal walking assisted by learning

Published online by Cambridge University Press:  06 September 2002

Chee-Meng Chew  and
Gill A. Pratt
Chee-Meng Chew
Affiliation:
Department of Mechanical Engineering, National University of Singapore, 10 Kent Ridge Crescent, Singapore 119260
Gill A. Pratt
Affiliation:
F.W. Olin College of Engineering, 1735 Great Plain Ave., Needham, MA 02492–1245 (USA)
Get access Rights & Permissions [Opens in a new window]

Summary

This paper presents a general control architecture for bipedal walking which is based on a divide-and-conquer approach. Based on the architecture, the sagittal-plane motion-control algorithm is formulated using a control approach known as Virtual Model Control. A reinforcment learning algorithm is used to learn the key parameter of the swing leg control task so that speed control can be achieved. The control algorithm is applied to two simulated bipedal robots. The simulation analyses demonstrate that the local speed control mechanism based on the stance ankle is effective in reducing the learning time. The algorithm is also demonstrated to be general in that it is applicable across bipedal robots that have different length and mass parameters.

Keywords

Bipedal walkingVirtual model controlReinforcement learningQ-learningCMAC
Type
Research Article
Information
Robotica , Volume 20 , Issue 5 , September 2002 , pp. 477 - 491
Copyright
Copyright © Cambridge University Press 2002

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

1. Gubina, F. and Hemami, H. McGhee, R. B., “On the dynamic stability of biped locomotion”, IEEE Transactions on Biomedical Engineering BME-21, No. 2, 102–108 (Mar., 1974).CrossRefGoogle Scholar
2. Arimoto, S. and Miyazaki, F., “Biped locomotion robots”, Japan Annual Review in Electronics, Computers and Telecommunications 12, 194–205 (1984).Google Scholar
3. Miyazaki, F. and Arimoto, S., “A control theoretic study on dynamical biped locomotion”, ASME Journal of Dynamic Systems, Measurement, and Control 102, 233–239 (1980).CrossRefGoogle Scholar
4. Miura, H. and Shimoyama, I., “Dynamic walk of a biped”, Int. J. Robotic Research 3, No. 2, 60–74 (1984).CrossRefGoogle Scholar
5. Bay, J. S. and Hemami, H., “Modeling of a neural pattern generator with coupled nonlinear oscillators”, IEEE Transactions on Biomedical Engineering BME-34, No. 4, 297–306 (April, 1987).CrossRefGoogle Scholar
6. Hirai, K., Hirose, M., Haikawa, Y. and Takenaka, T., “The development of honda humanoid robot”, IEEE International Conference on Robotics and Automation (1998) pp. 1321–1326.Google Scholar
7. Hurmuzlu, Y., “Dynamics of bipedal gait: Part i – objective functions and the contact event of a planar five-link biped”, Journal of Applied Mechanics 60, 331–336 (1993).CrossRefGoogle Scholar
8. Kajita, Shuuji, Yamaura, Tomio and Kobayashi, Akira, “Dynamic walking control of a biped robot along a potential energy conserving orbit”, IEEE Transactions on Robotics and Automation 6, No. 1, 431–438 (1992).CrossRefGoogle Scholar
9. Golden, J. A. and Zheng, Y. F., “Gait synthesis for the sd-2 biped robot to climb stairs”, Int. J. Robotics and Automation 5 No. 4, 149–159 (1990).Google Scholar
10. Pratt, Gill A., “Legged robots at mit – what's new since raibert”, 2nd International Conference on Climbing and Walking Robots (1999) pp 29–33.Google Scholar
11. Kawamura, S., Kawamura, T., Fujino, D. and Arimoto, S., “Realization of biped locomotion by motion pattern learning”, Journal of the Robotics Society of Japan 3, No. 3, 177–187 (1985).CrossRefGoogle Scholar
12. Yamaguchi, Jin’ichi, Takanish, Atsuo and Kato, Ichiro, “Development of a biped walking robot compensating for three-axis moment by trunk motion”, IEE International Conference on Intelligent Robots and Systems (1993) pp 561–566.Google Scholar
13. Wang, H., Lee, T. T. and Grover, W. A., “A neuromorphic controller for a three-link biped robot”, IEEE Transactions on Systems, Man, and Cybernetics 22, No. 1, 164–169 (Jan./Feb., 1992).CrossRefGoogle Scholar
14. Miller, Thomas W., “Real time neural network control of a biped walking robot”, IEEE Control Systems Magazine 41–48 (Feb., 1994).CrossRefGoogle Scholar
15. Benbrahim, H. and Franklin, J. A., “Biped dynamic walking using reinforcement learning”, Robotics and Autonomous Systems 22 283–302 (1997).CrossRefGoogle Scholar
16. Pratt, Gill A. and Williamson, Matthew M., “Series elastic actuators”, IEEE International Conference on Intelligent Robots and Systems (1995) Vol. 1, pp. 399–406.Google Scholar
17. Rosenthal, D. E. and Sherman, M. A., “High performance multibody simulations via symbolic equation manipulation and kane's method”, Journal of Astronautical Sciences 34, No. 3, 223–239 (1986).Google Scholar
18. Pratt, J., Chew, C.-M., Torres, A., Dilworth, P. and Pratt, G., “Virtual model control: An intuitive approach for bipedal locomotion”, Int. J. Robotics Research 20, No. 2, 129–143 (2001).CrossRefGoogle Scholar
19. Vukobratovic, M., Borovac, B., Surla, D. and Stokic, D., Biped Locomotion: Dynamics, Stability, Control, and Applications (Springer-Verlag, Berlin, 1990).CrossRefGoogle Scholar
20. C. J. C. H., Watkins and Dayan, P., “Q-learning”, Machine Learning 8, 279–292 (1992).Google Scholar
21. Tsitsiklis, John N., “Asynchronous stochastic approximation and q-learning”, Machine Learning 16, No. 3, 185–202 (1994).Google Scholar
22. Jaakkola, T., Singh, S. P. and Jordan, M. I., “On the convergence of stochastic iterative dynamics programming algorithms”, Neural Computation 6, 1185–1201 (1994).CrossRefGoogle Scholar
23. Sutton, R. S. and Barto, A. G., Reinforcement Learning An Introduction (MIT Press, Cambridge, MA, 1998).CrossRefGoogle Scholar
24. Bertsekas, D. and Tsitsiklis, J., Neuro-Dynamic Programming (Athena Scientific, Belmont, MA, 1996).Google Scholar
25. Albus, J. S., Brain, Behavior and Robotics (BYTE Books, McGraw-Hill, Peterborough, NH, 1981). Chapter 6, pp. 139–179.Google Scholar
26. Miller, W. T., Glanz, F. H. and Carter, M. J., “Cmac: An associative neural network alternative to backpropagation”, IEEE Proceedings No. 78, 1561–1567 (1990).Google Scholar
27. Miller, Thomas W. and Glanz, Filson H., “The university of new hampshire implementation of the cerebellar model arithmetic computer – cmac”, Unpublished reference guide to a CMAC program (August, l994).Google Scholar
28. Flash, T. and Hogan, N., “The coordination of arm movements: An experimentally confirmed mathematical model”, Journal of Neuroscience 5, No. 7, 1688–1703 (1985).CrossRefGoogle Scholar
29. Chew, Chee-Meng and Pratt, Gill A., “Adaptation to load variations of a planar biped: Height control using robust adaptive control”, Robotics and Autonomous Systems 35, No. 1, 1–22 (2001).CrossRefGoogle Scholar
30. Slotine, Jean-Jacques E. and Li, Weiping, Appiled Nonlinear Control (Prentice-Hall, Cambridge, MA, 1991).Google Scholar
31. Chew, Chee-Meng, “Dynamic Bipedal Walking Assisted by Learning”, PhD dissertation (Massachusetts Institute of Technology, Sept 2000).Google Scholar