Skip to main content

Development and analysis of an operator steering model for teleoperated mobile robots under constant and variable latencies

  • Steve Vozar (a1), Justin Storms (a2) and D. M. Tilbury (a2)

Latency hinders a mobile robot teleoperator's ability to perform remote tasks. However, this effect is not well modeled. This paper develops a model for teleoperator steering behavior as a PD controller based on projected lateral displacement, which was tuned to reflect user performance determined by a 31-subject user study under constant and variable latency (having mean latencies between 0 and 750 ms). Additionally, we determined that operator performance under variable latency could be mapped to the expected performance of an equivalent constant latency. We then tested additional latency distributions in simulation and demonstrated equivalent steering performance among several different latency distributions.

Corresponding author
*Corresponding author. E-mail:
Hide All
1. Luck J. P., McDermott P. L., Allender L. and Russell D. C., “An Investigation of Real World Control of Robotic Assets Under Communication Latency,” Proceedings of the 1st ACM SIGCHI/SIGART Conference on Human-Robot Interaction, HRI'06, New York, NY, USA, ACM (2006) pp. 202209.
2. Davis J., Smyth C. and McDowell K., “The effects of time lag on driving performance and a possible mitigation,” IEEE Trans. Robot. 26 (3), 590593 (Jun. 2010).
3. Harriott C. E. and Adams J. A., “Modeling human performance for human-robot systems,” Rev. Human Factors Ergon. 9, 94130 (Nov. 2013).
4. Yip M. C., Tavakoli M. and Howe R. D., “Performance analysis of a haptic telemanipulation task under time delay,” Adv. Robot. 25 (5), 651673 (2011).
5. Yang T., Fu Y. and Tavakoli M., “Digital versus analog control of bilateral teleoperation systems: A task performance comparison,” Control Eng. Pract. 38, 4656 (2015).
6. MacAdam C. C., “Understanding and modeling the human driver,” Veh. Syst. Dyn. 40 (1–3), 101134 (2003).
7. Chen J., Haas E. and Barnes M., “Human performance issues and user interface design for teleoperated robots,” IEEE Trans. Syst. Man Cybern. Part C: Appl. Rev. 37 (6), 12311245 (2007).
8. Vozar S., A Framework for Improving the Speed and Performance of Teleoperated Mobile Manipulators Ph.D. Thesis (Ann Arbor: University of Michigan, Aug. 2013).
9. Sheridan T. B. and Ferrell W. R., “Remote manipulative control with transmission delay,” IEEE Trans. Human Factors Electron. HFE-4 (1), 2529 (Sep. 1963).
10. Slawiñski E. and Mut V., “Control scheme including prediction and augmented reality for teleoperation of mobile robots,” Robotica 28 (01), 1122 (2010).
11. Xiong Y., Li S. and Xie M., “Predictive display and interaction of telerobots based on augmented reality,” Robotica 24 (04), 447453 (2006).
12. Sheik-Nainar M. A., Kaber D. B. and Chow M.-Y., “Control gain adaptation in virtual reality mediated human–telerobot interaction,” Human Factors Ergon. Manuf. Service Ind. 15 (3), 259274 (2005).
13. Goodrich M. A., Olsen D. R., Crandall J. and Palmer T. J., “Experiments in Adjustable Autonomy,” Proceedings of IJCAI Workshop on Autonomy, Delegation and Control: Interacting with Intelligent Agents (2001) pp. 1624–1629.
14. Marge M., Powers A., Brookshire J., Jay T., Jenkins O. C. and Geyer C., “Comparing heads-up, hands-free operation of ground robots to teleoperation,” Proceedings of Robotics: Science and Systems VII (2012) p. 193.
15. Wang B., Li Z. and Ding N., “Speech Control of a Teleoperated Mobile Humanoid Robot,” Proceedings of 2011 IEEE International Conference on Automation and Logistics ICAL (2011) pp. 339–344.
16. Lane J. Corde Carignan C., Sullivan B., Akin D., Hunt T. and Cohen R., “Effects of Time Delay on Telerobotic Control of Neutral Buoyancy Vehicles,” Proceedings of the IEEE International Conference on Robotics and Automation, ICRA '02, vol. 3 (2002) pp. 2874–2879.
17. Lee D., Martinez-Palafox O. and Spong M. W., “Bilateral Teleoperation of a Wheeled Mobile Robot Over Delayed Communication Network,” Proceedings of the IEEE Internation Robotics and Automation ICRA, (2006) pp. 3298–3303.
18. Hashemzadeh F. and Tavakoli M., “Position and force tracking in nonlinear teleoperation systems under varying delays,” Robotica 33 (04), 10031016 (2015).
19. Janabi-Sharifi F. and Hassanzadeh I., “Experimental analysis of mobile-robot teleoperation via shared impedance control,” IEEE Trans. Syst. Man Cybern. Part B: Cybern. 41 (2), 591606 (2011).
20. Sanders D., “Analysis of the effects of time delays on the teleoperation of a mobile robot in various modes of operation,” Ind. Robot: Int. J. 36 (6), 570584 (2009).
21. McCracken H., I Drove Ford's Golf Cart in Atlanta (Note: I Was in Silicon Valley at the Time), Fast Company (Jan. 2015).
22. Fitts P. M., “The information capacity of the human motor system in controlling the amplitude of movement,” J. Exp. Psychol. 47 (6), 381 (1954).
23. Accot J. and Zhai S., “Beyond Fitts' Law: Models for Trajectory-Based HCI Tasks,” Proceedings of the ACM SIGCHI Conference on Human Factors in Computing SystemsCHI'97, New York, NY, USA, ACM (1997) pp. 295–302.
24. Zhai S., Accot J. and Woltjer R., “Human action laws in electronic virtual worlds: An empirical study of path steering performance in VR,” Presence: Teleoperators Virtual Environ. 13 (2), 113127 (Apr. 2004).
25. Pavlovych A. and Stuerzlinger W., “Target Following Performance in the Presence of Latency, Jitter and Signal Dropouts,” Proceedings of Graphics InterfaceGI'11, School of Computer Science, University of Waterloo, Waterloo, Ontario, Canada, Canadian Human-Computer Communications Society (2011), pp. 33–40.
26. Kaber D., Li Y., Clamann M. and Lee Y.-S., “Investigating human performance in a virtual reality haptic simulator as influenced by fidelity and system latency,” IEEE Trans. Syst. Man Cybern. Part A: Syst. Humans 42 (6), 15621566 (2012).
27. MacKenzie I. S. and Ware C., “Lag as a Determinant of Human Performance in Interactive Systems,” Proceedings of the INTERACT'93 and CHI'93 Conference on Human Factors in Computing System, New York, NY, USA, ACM (1993) pp. 488–493.
28. Tipsuwan Y. and Chow M.-Y., “Control methodologies in networked control systems,” Control Eng. Pract. 11 (10), 10991111 (2003).
29. Xie X., Yin S., Gao H. and Kaynak O., “Asymptotic stability and stabilisation of uncertain delta operator systems with time-varying delays,” Control Theory Appl. IET 7 (8), 10711078 (2013).
30. McRuer D. T., “Human pilot dynamics in compensatory systems,” Technical report, DTIC Document (1965).
31. MacAdam C. C., “An optimal preview control for linear systems,” J. Dyn. Syst. Meas. Control 102 (3), 188190 (1980).
32. Van De Vegte J. M., Milgram P. and Kwong R. H., “Teleoperator control models: Effects of time delay and imperfect system knowledge,” IEEE Trans. Syst. Man Cybern. 20 (6), 12581272 (1990).
33. Delice I. and Ertugrul S., “Intelligent Modeling of Human Driver: A Survey,” Proceedings of the 2007 IEEE Intelligent Vehicles Symposium (2007) pp. 648–651.
34. Palma L. Brito, Coito F. Vieira and Gil P. Sousa, “Low Order Models for Human Controller–Mouse Interface,” Proceedings of the 2012 IEEE 16th International Conference on Intelligent Engineering Systems INES (2012) pp. 515–520.
35. Vozar S. and Tilbury D. M., “Driver Modeling for Teleoperation with Time Delay,” Proceedings of the 19th IFAC World Congress (2014) pp. 3551–3556.
36. Ritter F. E., Kukreja U. and Amant R. S., “Including a model of visual processing with a cognitive architecture to model a simple teleoperation task,” J. Cogn. Eng. Decis. Making 1 (2), 121147 (Jun. 2007).
37. ACT-r, (Aug. 2014).
38. APRIL Robotics Laboratory, APRIL Laboratory: Autonomy * Perception * Robotics * Interfaces * Learning, (Mar. 2012).
39. Huang A., Olson E. and Moore D., “LCM: Lightweight Communications and Marshalling,” Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems IROS (Oct. 2010).
40. Ögren P., Svenmarck P., Lif P., Norberg M. and Söderbäck N. E., “Design and Implementation of a New Teleoperation Control Mode for Differential Drive UGVs,” Auton. Robots (Nov. 2013), pp. 19.
41. iRobot Corporation, iRobot 510 PackBot – specifications (2012). Accessed online 2014.
42. Hollands J. G. and Lamb M., “Viewpoint tethering for remotely operated vehicles effects on complex terrain navigation and spatial awareness,” Human Factors: J. Human Factors Ergon. Soc. 53 (2), 154167 (Apr. 2011).
43. Anand D., Bhatia M., Moyne J., Shahid W. and Tilbury D., “Wireless test results booklet,” Technical report, University of Michigan ERC/RMS (2010).
44. E54 Committee, “Test method for evaluating emergency response robot capabilities: Mobility: Maneuvering tasks: Sustained speed,” Technical report, ASTM International (2011).
45. Son H. I., Chuang L., Franchi A., Kim J., Lee D., Lee S.-W., Bulthoff H. and Giordano P., “Measuring an Operator's Maneuverability Performance in the Haptic Teleoperation of Multiple Robots,” Proceedings of the 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems IROS) (2011) pp. 3039–3046.
46. Grabe V., Pretto P., Giordano P. R. and Bülthoff H. H., “Influence of Display Type on Drivers Performance in a Motion-Based Driving Simulator,” Proceedings of the Driving Simulation Conference (2010).
47. AUVSI, “Unmanned ground vehicles: Core capabilities & market background,” Technical report, The Association for Unmanned Vehicle Systems International (Aug. 2013).
48. Frigge M., Hoaglin D. C. and Iglewicz B., “Some implementations of the boxplot,” Am. Statistician 43 (1), 5054 (Feb. 1989).
49. Rouse R. III,, “What's your perspective? SIGGRAPH Comput. Graph. 33 (3), 912 (Aug. 1999).
50. Pazuchanics S. L., “The Effects of Camera Perspective and Field of View on Performance in Teleoperated Navigation,” Proc. Human Factors Ergon. Soc. Annu. Meet. 50 (16):15281532 (Oct. 2006).
51. Ulsoy A. G., Peng H. and Çakmakci M., Automotive Control Systems (New York, NY, Cambridge University Press, 2012).
52. Nise N. S., Control Systems Engineering, 4th ed. (Hoboken, NJ, John Wiley & Sons, 2004).
53. Ungoren A. and Peng H., “An adaptive lateral preview driver model,” Veh. Syst. Dyn. 43 (4), 245259 (2005).
54. Toffin D., Reymond G., Kemeny A. and Droulez J., “Role of steering wheel feedback on driver performance: Driving simulator and modeling analysis,” Veh. Syst. Dyn. 45 (4), 375388 (2007).
55. Jagacinski R. J., “A qualitative look at feedback control theory as a style of describing behavior,” Human Factors: J. Human Factors Ergon. Soc. 19 (4), 331347 (1977).
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

  • ISSN: 0263-5747
  • EISSN: 1469-8668
  • URL: /core/journals/robotica
Please enter your name
Please enter a valid email address
Who would you like to send this to? *



Full text views

Total number of HTML views: 0
Total number of PDF views: 60 *
Loading metrics...

Abstract views

Total abstract views: 303 *
Loading metrics...

* Views captured on Cambridge Core between 5th October 2016 - 21st January 2018. This data will be updated every 24 hours.