Hostname: page-component-89b8bd64d-sd5qd Total loading time: 0 Render date: 2026-05-10T13:39:07.127Z Has data issue: false hasContentIssue false
  • English
  • Français

Command-based voice teleoperation of a mobile robot via a human-robot interface

Published online by Cambridge University Press:  28 January 2014

Alberto Poncela  and
Leticia Gallardo-Estrella
Alberto Poncela*
Affiliation:
Departamento Tecnología Electrónica, ETSI Telecomunicación, Universidad de Málaga, Campus de Teatinos, 29071 Málaga, Spain
Leticia Gallardo-Estrella
Affiliation:
Department of Radiology, Radboud University Nijmegen Medical Centre, Geert Grooteplein 10, 6525 GA Nijmegen, The Netherlands
*
*Corresponding author. E-mail: apg@dte.uma.es
Get access Rights & Permissions [Opens in a new window]

Summary

Verbal communication is the most natural way of human–robot interaction. Such an interaction is usually achieved by means of a human-robot interface (HRI). In this paper, a HRI is presented to teleoperate a robotic platform via the user's voice. Hence, a speech recognition system is necessary. In this work, a user-dependent acoustic model for Spanish speakers has been developed to teleoperate a robot with a set of commands. Experimental results have been successful, both in terms of a high recognition rate and the navigation of the robot under the control of the user's voice.

Keywords

TeleoperationHuman-robot interfaceVoice interfaceSpeech recognitionAcoustic model

Information

Type
Articles
Information
Robotica , Volume 33 , Issue 1 , January 2015 , pp. 1 - 18
Copyright
Copyright © Cambridge University Press 2014 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

1.Hexmoor, H., Castelfranchi, C. and Falcone, R., “Multiagent Systems, Artificial Societies, and Simulated organizations,” In: Agent Autonomy, vol. 7 (Springer, Germany, 2003).Google Scholar
2.Sheridan, T. H. and Parasuraman, R., “Human-automation interaction,” Rev. Hum. Factors Ergon. 1, 89–129 (2006).Google Scholar
3.Courreges, F., Edkie, A., Poisson, G. and Vieyres, P., “Ergonomic Mouse Based Interface for 3D Orientation Control of a Tele-Sonography Robot,” Proceedings of the 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems, St. Louis, USA (2009) pp. 61–66.Google Scholar
4.Zhang, L., Huang, Q., Liu, Q., Liu, T., Li, D. and Lu, Y., “A Teleoperation System for a Humanoid Robot with Multiple Information Feedback and Operational Modes,” Proceedings of the 2005 IEEE International Conference on Robotics and Biomimetics, Hong Kong, China (2005) pp. 290–294.Google Scholar
5.Mahony, R., Schill, F., Corke, P. and Oh, Y. S., “A New Framework for Force Feedback Teleoperation of Robotic Vehicles Based on Optical Flow,” Proceedings of the 2009 IEEE International Conference on Robotics and Automation, Kobe, Japan (2009) pp. 1079–1085.Google Scholar
6.Harada, T., Sato, T. and Mori, T., “Human Motion Tracking System Based on Skeleton and Surface Integration Model Using Pressure Sensors Distribution Bed,” Proceedings of the Workshop of Human Motion, Austin, USA (2000) pp. 99–106.Google Scholar
7.Fukuda, O., Tsuji, T., Kaneko, M. and Otsuka, A., “A human-assisting manipulator teleoperated by EMG signals and arm motions,” IEEE Trans. Robot. Autom. 19 (2), 210–222 (2003).CrossRefGoogle Scholar
8.Ferre, M., Macias-Guarasa, J., Aracil, R. and Barrientos, A., “Voice Command Generation for Teleoperated Robot Systems,” Proceedings of the IEEE ROMAN 1998, Takamatsu, Japan (1998) pp. 679–685.Google Scholar
9.Drygajlo, A., Prodanov, P. J., Ramel, G., Meisser, M. and Siegwart, R., “On developing a voice-enabled interface for interactive tour-guide robots,” Adv. Robot. 17 (7), 599–616 (2003).Google Scholar
10.Nishimori, M., Saitoh, T. and Konishi, R., “Voice Controlled Intelligent Wheelchair,” Proceedings of the 2007 SICE Annual Conference, Takamatsu, Japan (2007) pp. 336–340.Google Scholar
11.Galindo, C., González, J. and Fernández-Madrigal, J. A., “Control architecture for human-robot integration application to a robotic wheelchair,” IEEE Trans. Syst. Man Cybern. 36 (5), 1053–1067 (2006).Google Scholar
12.Barkana, D. E., Das, J., Wang, F., T. E. Groomes and Sarkar, N., “Incorporating verbal feedback into a robot-assisted rehabilitation system,” Robotica 29 (3), 433–443 (2011).Google Scholar
13.Pérez-Vidal, C., Carpintero, E., García-Aracil, N., Sabater-Navarro, J. M., Azorín, J. M., Candela, A. and Fernández, E., “Steps in the development of a robotic scrub nurse,” Robot. Auton. Syst. 60 (6), 901–911 (2012).Google Scholar
14.Lee, H. and Ko, H., “Competing models-based text-prompted speaker independent verification algorithm,” Speech commun. 48 (1), 28–44 (2006).Google Scholar
15.Glas, D. F., Kanda, T., Ishiguro, H. and Hagita, N., “Teleoperation of multiple social robots,” IEEE Trans. Syst. Man Cybern. Part A: Systems and Humans 42 (3), 530–544 (2012).Google Scholar
16.Chatterjee, A., Pulasinghe, K., Watanabe, K. and Izumi, K., “A particle swarm optimized fuzzy neural network for voice controlled robot systems,” IEEE Trans. Ind. Electron. 52 (6), 1478–1489 (2005).CrossRefGoogle Scholar
17.Medicherla, H. and Sekmen, A., “Human-robot interaction via voice-controllable intelligent user interface,” Robotica 25 (5), 521–527 (2007).Google Scholar
18.Urdiales, C., Bandera, A., Pérez, E. J., Poncela, A. and Sandoval, F., “Hierarchical Planning in a Mobile Robot for Map Learning and Navigation,” In: Autonomous Robotic Systems (Physica-Verlag, Heidelberg, Germany, 2003) pp. 2165–2188.Google Scholar
19.Elfes, A., “Sonar-based real-world mapping and navigation,” IEEE J. Robot. Autom. 3, 249–265 (1987).Google Scholar
20.Spolsky, J., User Interface Design for Programmers (Apress Springer-Verlag, USA, 2001).CrossRefGoogle Scholar
21.Tognazzini, B., Tog on Interface (Addison-Wesley, USA, 1992).Google Scholar
22.Nielsen, J., Designing Web Usability. The Practice of Simplicity (New Riders Publishing, Indiana, 2000).Google Scholar
23.Nielsen, J., Usability Engineering (Morgan-Kaufmann Publishers, San Francisco, 1993).Google Scholar
24.Norman, D., Emotional Design: Why We Love (or Hate) Everyday Things (Basic Books, New York, 2004).Google Scholar
25.Johnson, J., GUI bloopers: Don'ts and Do's for Software Developers and Web Designers (Morgan-Kaufmann Publishers, San Francisco, 2000).Google Scholar
26.Schneiderman, B., Designing the User Interface (Addison-Wesley, USA, 1997).Google Scholar
27.Nielsen, J., “Heuristic Evaluation,” In: Usability Inspection Methods (J. Nielsen and Mack, R., eds.) (John Wiley & Sons, New York, 1994).CrossRefGoogle Scholar
28.Severinson-Eklundh, K., Green, A. and Hüttenrauch, H., “Social and collaborative aspects of interaction with a service robot,” Robot. Auton. Syst. 42 (3–4), 223–234 (2003).Google Scholar
29.Hura, S., “Voice User Interfaces,” In: HCI Beyond the GUI (Elsevier, UK, 2003) pp. 197–228.Google Scholar
30.Lee, K. F., Hon, H. W., Hwang, M. Y., Mahajan, S. and Reddy, R., “The Sphinx Speech Recognition System,” Proceedings of the 1989 International Conference on Acoustic, Speech and Signal Processing, Glasgow, UK (1989) pp. 445–448.Google Scholar
31.Lee, A., Kawahara, T. and Shikano, K., “Julius - An Open-Source Real-Time Large Vocabulary Recognition System,” Proceedings of the Interspeech 2001, Aalborg, Denmank (2001) pp. 1691–1694.Google Scholar
32.Lee, A. and Kawahara, T., “Recent Development of Open-Source Speech Recognition Engine Julius,” Proceedings of the 2009 APSIPA Annual Summit and Conference, Sapporo, Japan (2009) pp. 131–137.Google Scholar
33.Fohr, D., Mella, O., Cerisara, C. and Illina, I., “The automatic news transcription system: ANTS, some real time experiments,” Proceedings of Interspeech 2004, Jeju Island, Korea (2004) pp. 377–380.Google Scholar
34.Yang, D., Iwano, K. and Furui, S., “Accent Analysis for Mandarin Large Vocabulary Continuous Speech Recognition,” Asian Workshop on Speech Science and Technology (Tokyo, Japan, 2008) pp. 87–91.Google Scholar
35.Jongtaveesataporn, M., Wutiwiwatchai, C., Iwano, K. and Furui, S., “Development of a Thai broadcast news corpus and an LVCSR system,” ASJ Annual Meeting (Okayama, Japan, 2008).Google Scholar
36.Alumäe, T., “Large Vocabulary Continuous Speech Recognition for Estonian using Morphemes and Classes,” Proceedings of Interspeech 2006, Pittsburgh, USA (2004) pp. 389–392.Google Scholar
37.Rotovnik, T., Maucec, M. S., Horvat, B. and Kacic, Z., “A Comparison of HTK, ISIP and Julius in Slovenian Large Vocabulary Continuous Speech Recognition,” Proceedings of Interspeech 2002, Denver, USA (2002) pp. 671–684.Google Scholar
38.Kim, J. G., Jung, H. Y. and Chung, H. Y., “A Keyword Spotting Approach Based on Pseudo N-gram Language Model,” Proceedings of SPECOM, St. Petersburg, Russia (2004) pp. 256–259.Google Scholar
39.Young, S., Evermann, G., Gales, M., Hain, T., Kershaw, D., Liu, X., Moore, G., Odell, J., Ollason, D. and Povey, D., The HTK Book, version 3.4 (Engineering Department, Cambridge University, Cambridge, UK, 2006).Google Scholar
40.Yutai, W., Bo, L., Xiaoqing, J., Feng, L. and Lihao, W., “Speaker Recognition Based on Dynamic MFCC Parameters,” Proceedings of the Conference on Image Analysis and Signal Processing IASP, Ningbo, China (2009) pp. 406–409.Google Scholar
41.Jeong, Y., “Robust speaker adaptation based on parallel factor analysis of training models,” Electron. Lett. 47 (7), 465–467 (2011).Google Scholar
42.Nasersharif, B. and Akbari, A., “Sub-band weighted projection measure for sub-band speech recognition in noise,” Electron. Lett. 42 (14), 829–831 (2006).CrossRefGoogle Scholar
43.Cerisara, C., “Automatic discovery of topics and acoustic morphemes from speech,” Comput. Speech Lang. 23 (2), 220–239 (2009).Google Scholar
44.Lee, L. M., “Adaptation of hidden Markov models for half frame rate observations,” Electron. Lett. 46 (10), 723–724 (2010).Google Scholar
45.Marín, R., Vila, P., Sanz, P. J. and Marzal, A., “Automatic Speech Recognition for Teleoperate a Robot Via Web,” Proceedings of the 2002 IEEE/RSJ International Conference on Intelligent Robots and Systems EPFL, Laussane, Switzerland (2002) pp. 1278–1283.Google Scholar
46.Mayorga, P., Martín, J., Hernández, A. M. and Flores, J., “Gaussian Components Optimization for a Robot Controlled by Speech Commands in Mexican Spanish,” 4th Congress of Electronics, Robotics and Automotive Mechanics (CERMA 2007), Cuernavaca, Mexico (2007) pp. 124–129.Google Scholar