Hostname: page-component-89b8bd64d-46n74 Total loading time: 0 Render date: 2026-05-07T21:45:48.150Z Has data issue: false hasContentIssue false
  • English
  • Français

Human–robot interaction via voice-controllable intelligent user interface

Published online by Cambridge University Press:  01 September 2007

Harsha Medicherla  and
Ali Sekmen
Harsha Medicherla*
Affiliation:
Department of Electrical and Computer Engineering, Tennessee State University, 3500 John A. Merritt Blvd. Nashville, TN 37209, USA
Ali Sekmen*
Affiliation:
Department of Computer Science, Tennessee State University, 3500 John A. Merritt Blvd. Nashville, TN 37209, USA
*
*Corresponding author: E-mail: asekmen@tnstate.edu
Get access Rights & Permissions [Opens in a new window]

Summary

An understanding of how humans and robots can successfully interact to accomplish specific tasks is crucial in creating more sophisticated robots that may eventually become an integral part of human societies. A social robot needs to be able to learn the preferences and capabilities of the people with whom it interacts so that it can adapt its behaviors for more efficient and friendly interaction. Advances in human– computer interaction technologies have been widely used in improving human–robot interaction (HRI). It is now possible to interact with robots via natural communication means such as speech. In this paper, an innovative approach for HRI via voice-controllable intelligent user interfaces is described. The design and implementation of such interfaces are described. The traditional approaches for human–robot user interface design are explained and the advantages of the proposed approach are presented. The designed intelligent user interface, which learns user preferences and capabilities in time, can be controlled with voice. The system was successfully implemented and tested on a Pioneer 3-AT mobile robot. 20 participants, who were assessed on spatial reasoning ability, directed the robot in spatial navigation tasks to evaluate the effectiveness of the voice control in HRI. Time to complete the task, number of steps, and errors were collected. Results indicated that spatial reasoning ability and voice-control were reliable predictors of efficiency of robot teleoperation. 75% of the subjects with high spatial reasoning ability preferred using voice-control over manual control. The effect of spatial reasoning ability in teleoperation with voice-control was lower compared to that of manual control.

Keywords

Human–robot interactionMobile robotsSpeech recognitionIntelligent user interfaces

Information

Type
Article
Information
Robotica , Volume 25 , Issue 5 , September 2007 , pp. 521 - 527
Copyright
Copyright © Cambridge University Press 2007

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

1.Bartneck, C. and Forlizzi, J., “A Design-Centered Framework for Social Human-Robot Interaction,” Proceedings of the 13th IEEE International Workshop on Robot and Human Interactive Communication Sept. 2004 pp. 591–594.Google Scholar
2.Breazeal, C., Designing Sociable Robots (MIT Press, Cambridge, MA, 2002).Google Scholar
3.Breazeal, C., Sociable machines: expressive social exchange between humans and robots, Dissertation (Department of Electrical Engineering and Computer Science, MIT, 2000).Google Scholar
4.Fujita, M., “AIBO: towards the era of digital creatures,” Int. J. Robot. Res. 20 (10), 781–794 2001.CrossRefGoogle Scholar
5.Pollack, M. et al. , “Pearl: A Mobile Robotic Assistant for the Elderly,” Proceedings of AAAI Workshop on Automation as Eldercare 2002 pp. 85–92.Google Scholar
6.Simmons, R. et al. , “GRACE and GEORGE: Autonomous Robots for the AAAI Robot Challenge,” Proceedings of AAAI 2004 Mobile Robot Competition Workshop 2004 pp. 15–20.CrossRefGoogle Scholar
7.Kawamura, K., Rogers, T. and Ao, X., “Development of a Cognitive Model of Humans in a Multi-Agent Framework for Human–Robot Interaction,” Proceedings of the 1st International Joint Conference on Autonomous Agents and Multi-Agent System, Bologna, Italy 2002 pp. 1379–1386.CrossRefGoogle Scholar
8.Scassellati, B., Theory of mind for a humanoid robot, Ph.D. dissertation (Department of Electrical Engineering and Computer Science, MIT, 2001).CrossRefGoogle Scholar
9.Kiesler, S. and Goetz, J., “Mental Models and Cooperation With Robotic Assistants” In: CHI 2002 Extended Abstracts (ACM Press, Minneapolis, MN, 2002) pp. 576–577.Google Scholar
10.Reeves, B. and Nass, C., The Media Equation (Cambridge University Press, Cambridge, UK, 1996).Google Scholar
11.Sekmen, A., Wilkes, M., Goldman, S. and Sabatto, S., “Exploring importance of location and prior knowledge in mobile robot control,” Int. J. Human Comput. Stud. 58 (1), 5–20 2003.CrossRefGoogle Scholar
12.Bruemmer, D. J., Few, D. A. and Nielson, C. W., “Spatial Reasoning for Human–Robot teams,” In: Emerging Spatial Information Systems and Applications (Hilton, Brian, ed.) (Idea Group Inc., 2006) pp. 350–372.Google Scholar
13.Sekmen, A., Human–robot interaction methodology, Ph.D. dissertation (Vanderbilt University, 2000).Google Scholar
14.Granic, A. and Glavinic, V., “Automatic Adaptation of User Interfaces for Computerized Educational Systems,” Proceedings of 10th IEEE International Conference on Electronics, Circuits and Systems, NJ, USA 2003, pp. 1232–1235.Google Scholar
15.Benyon, D. and Murray, D., “Adaptive systems: from intelligent tutoring to autonomous systems,” Knowl. Based Syst. 6 (4), 197–219 1993.CrossRefGoogle Scholar
16.Chignell, M. H. and Hancock, P. A., Intelligent Interfaces, Handbook of Human-Computer Interaction (Elsevier, Amsterdam, The Netherlands, 1988).Google Scholar
17.Hook, K., “Steps to take before IUI becomes real,” Journal of Interacting with Computers 12 (4), 2000, pp. 409–426.CrossRefGoogle Scholar
18.Szekely, P., “Structuring Programs to Support Intelligent Interfaces,” In: Intelligent User Interfaces (Sullivan, J. and Tyler, S., eds.) (ACM Press, 1991) pp. 445–464.CrossRefGoogle Scholar
19.Karagiannidis, C., Koumpis, A. and Stephanidis, C., “Decision Making in Intelligent User Interfaces,” Proceedings of the ACM International Conference on Intelligent User Interfaces 1997 pp. 195–202.Google Scholar
20.Hutchins, E. L., Hollan, J. D. and Norman, D., “Direct manipulation interfaces,” Human-Comput. Interact. 1, 311–338 1985.CrossRefGoogle Scholar
21.Cook, R. and Kay, J., “The Justified User Model: A Viewable, Explained User Model,” Proceedings of the 4th International Conference on User Modeling, Hyannis, Massachusetts 1994 pp. 145–150.Google Scholar
22.Gorniak, P. and Poole, D., “Predicting Future User Actions by Observing Unmodified Applications,” Proceedings of the 17th National Conference on Artificial Intelligence Aug. 2000 pp. 217–222.Google Scholar
23.Davison, B. D. and Hirsh, H., “Predicting sequences of user actions,” Predicting the Future: AI Approaches to Time Series Problems, Technical Report 1998 pp. 5–12.Google Scholar
24.Gajos, K. Z., Czerwinski, M., Tanb, D. S. and Weld, D. S., “Exploring the Design Space for Adaptive Graphical User Interfaces,” Proceedings of the Working Conference on Advanced Visual Interfaces 2006 pp. 201–208.CrossRefGoogle Scholar
25.Oviatt, S., Darves, C. and Coulston, R., “Toward adaptive conversational interfaces: modeling speech convergence with animated personas,” ACM Trans. Comput.–Human Interact., 3 (11), 300–328 2004.CrossRefGoogle Scholar
26.Heckermen, D., Geiger, D. and Chickering, D., “Learning Bayesian networks: the combination of knowledge and statistical data,” Technical Report, Microsoft Research 1994.CrossRefGoogle Scholar
27.Cheng, J., Bell, D. and Liu, W., “Learning Bayesian Networks From Data: An Efficient Approach Based on Information Theory,” Proceedings of the 6th ACM International Conference on Information and Knowledge Management 1997 pp. 325–331.CrossRefGoogle Scholar
28.Pearl, J., Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference (Morgan Kaufmann, San Mateo, CA, 1988).Google Scholar
29.Pearl, J. and Verda, T. M., “A Theory of Inferred Causation,” Proceedings of the 2nd Conference on Principles of Knowledge Representation and Reasoning 1991 pp. 441–452.Google Scholar
30.Rebane, T. and Pearl, J., “The Recovery of Causal Poly-Trees From Statistical Data,” In: Uncertainty in Artificial Intelligence (Kanal, L.N., Levitt, T.S. and Lemmer, J.F., eds.) (Amsterdam, The Netherlands, 1989).Google Scholar