Hostname: page-component-8448b6f56d-xtgtn Total loading time: 0 Render date: 2024-04-19T00:00:18.742Z Has data issue: false hasContentIssue false
  • English
  • Français

A modular bilateral haptic control framework for teleoperation of robots

Published online by Cambridge University Press:  30 October 2018

Zeki Y. Bayraktaroglu ,
Omer F. Argin  and
Sinan Haliyo Open the ORCID record for Sinan Haliyo [Opens in a new window]
Zeki Y. Bayraktaroglu*
Affiliation:
Mechanical Engineering Department, Istanbul Technical University, Istanbul, Turkey
Omer F. Argin
Affiliation:
Mechatronics Engineering Department, Istanbul Technical University, Istanbul, Turkey. E-mail: oargin@itu.edu.tr
Sinan Haliyo
Affiliation:
Sorbonne Université, CNRS, Institut des Systèmes Intelligents et de Robotique, ISIR, Paris, France. E-mail: sinan.haliyo@sorbonne-universite.fr
*
*Corresponding author. E-mail: zeki.bayraktaroglu@itu.edu.tr
Get access Rights & Permissions [Opens in a new window]

Summary

This paper presents a novel approach to implement bilateral control loops between local haptic devices and remote industrial manipulators using a layer of simulation and virtual reality. The remote scene of manipulation has been visualized in an open-source software environment, where forward and inverse kinematics of the manipulators can be computed. Therefore, the explicit knowledge of mathematical models of the robots is not required for the implementation of the proposed bilateral control schemes. A haptic coupling has been designed between the human operator and the task in the remote environment. Virtually introduced force feedback has contributed to the performance of the proposed bilateral loop by facilitating the adaptation of unexperienced human operators. Teleoperation of one remote manipulator has been experimentally demonstrated with the proposed controllers. Structural modularity of the bilateral haptic control schemes makes them directly extendable for the teleoperation of multiple collaborative robots. Stability and transparency of the proposed bilateral haptic controllers have been theoretically and experimentally investigated.

Keywords

Control of robotic systemsHaptic interfacesMan-machine systemsTeleoperationBilateral control systems
Type
Articles
Information
Robotica , Volume 37 , Issue 2 , February 2019 , pp. 338 - 357
Copyright
Copyright © Cambridge University Press 2018 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

1. Bolopion, A., Cagneau, B., Haliyo, D. and Régnier, S., “Analysis of stability and transparency for nanoscale force feedback in bilateral coupling,” J. Micro-Nano Mechatron. 4 (4), 145–158 (2008).Google Scholar
2. Mohand Ousaid, A., Bolopion, A., Haliyo, S., Régnier, S. and Hayward, V., “Stability and Transparency Analysis of a Teleoperation Chain for Microscale Interaction,” Proceedings of the IEEE International Conference on Robotics and Automation (2014) pp. 5946–5951.Google Scholar
3. Anderson, R. J. and Spong, M. W., “Asymptotic Stability for Force Reflecting Teleoperators with Time Delay,” Proceedings of the IEEE International Conference on Robotics and Automation (1989) pp. 1618–1625.Google Scholar
4. Anderson, R. J. and Spong, M. W., “Bilateral control of teleoperators with time delay,” IEEE Trans. Automat. Control 34 (5), 494–501 (1989).Google Scholar
5. Niemeyer, G. and Slotine, J. J. E., “Stable adaptive teleoperation,” IEEE J. Ocean. Eng. 16 (1), 152–162 (1991).Google Scholar
6. Niemeyer, G. and Slotine, J. J. E., “Transient Shaping in Force Reflecting Teleoperation,” Proceedings of the International Conference on Advanced Robotics (1991) pp. 261–266.Google Scholar
7. Zhu, W. H. and Salcudean, S. E., “Teleoperation with Adaptive Motion/Force Control,” Proceedings of the IEEE International Conference on Robotics and Automation (1999) pp. 231–237.Google Scholar
8. Hashemzadeh, F., Hassanzadeh, T., Tavakoli, M. and Alizadeh, G., “Adaptive control for state synchronization of nonlinear haptic telerobotic systems with asymmetric varying time delays,” J. Intell. Robot. Syst. 68 (3), 245–259 (2012).Google Scholar
9. Sarras, I., Nuño, E. and Basañez, L., “An adaptive controller for nonlinear teleoperators with variable time-delays,” J. Franklin Inst. 351 (10), 4817–4837 (2014).Google Scholar
10. Hou, X. and Sourina, O., “Real-time adaptive prediction method for smooth haptic rendering,” Cornell University Library, preprint arXiv 1603.06674 (2016).Google Scholar
11. Kim, B.-Y. and Ahn, H.-S., “A design of bilateral teleoperation systems using composite adaptive controller,” Control Eng. Pract. 21 (12), 1641–1652 (2013).Google Scholar
12. Alfi, A. and Farrokhi, M., “A simple structure for bilateral transparent teleoperation systems with time delay,” J. Dyn. Syst., Meas. Control 130 (4), 044502 (2008).Google Scholar
13. Alfi, A. and Farrokhi, M., “Force reflecting bilateral control of master–slave systems in teleoperation,” J. Intell. Robot. Syst. 52 (2), 209–232 (2008).Google Scholar
14. Mohammadi, L., Alfi, A. and Xu, B., “Robust bilateral control for state convergence in uncertain teleoperation systems with time-varying delay: A guaranteed cost control design,” Nonlinear Dyn. 88 (2), 1413–1426 (2017).Google Scholar
15. Aracil, R. et al., “Bilateral control by state convergence based on transparency for systems with time delay,” Robotics Auton. Syst. 61 (2), 86–94 (2013).Google Scholar
16. Alfi, A. et al. “Design and implementation of robust-fixed structure controller for telerobotic systems,” J. Intell. Robot. Syst. 83 (2), 253–269 (2016).Google Scholar
17. Chopra, N., Spong, M. W., Ortega, R. and Barabanov, N. E., “On tracking performance in bilateral teleoperation,” IEEE Trans. Robot. 22 (4), 861–866 (2006).Google Scholar
18. Chopra, N., Berestesky, P. and Spong, M. W., “Bilateral teleoperation over unreliable communication networks,” IEEE Trans. Control Syst. Technol. 16 (2), 304–313 (2008).Google Scholar
19. Lee, D. and Spong, M. W., “Passive bilateral teleoperation with constant time delay,” IEEE Trans. Robot. 22 (2), 269–281 (2006).Google Scholar
20. Nuño, E., “Haptic Guidance with Force Feedback to Assist Teleoperation Systems Via High Speed Networks,” Proceedings of the IEEE International Symposium on Robotics (2006) pp. 1–14.Google Scholar
21. Nuño, E., Ortega, R., Barabanov, N. and Basañez, L., “A globally stable PD controller for bilateral teleoperators,” IEEE Trans. Robot. 24 (3), 753–758 (2008).Google Scholar
22. Nuño, E., Basañez, L. and Ortega, R., “Passivity-based control for bilateral teleoperation: A tutorial,” Automatica 47, 485–495 (2011).Google Scholar
23. Li, Z., Ding, L., Gao, H., Duan, G. and Su, C. Y., “Trilateral teleoperation of adaptive fuzzy force/motion control for nonlinear teleoperators with communication random delays,” IEEE Trans. Fuzzy Syst. 21 (4), 610–624 (2013).Google Scholar
24. Li, Z. and Su, C. Y., “Neural-adaptive control of single-master–multiple-slaves teleoperation for coordinated multiple mobile manipulators with time-varying communication delays and input uncertainties,” IEEE Trans. Neural Netw. Learn. Syst. 24 (9), 1400–1413 (2013).Google Scholar
25. Li, Z., Cao, X. and Ding, N., “Adaptive fuzzy control for synchronization of nonlinear teleoperators with stochastic time-varying communication delays,” IEEE Trans. Fuzzy Syst. 19 (4), 745–757 (2011).Google Scholar
26. Lu, Z., Huang, P. and Liu, Z., “Predictive approach for sensorless bimanual teleoperation under random time delays with adaptive fuzzy control,” IEEE Trans. Ind. Electron. 65 (3), 2439–2448 (2018).Google Scholar
27. Zhao, Z., Huang, P., Lu, Z. and Liu, Z., “Augmented reality for enhancing tele-robotic system with force feedback,” Robot. Auton. Syst. 96, 93–101 (2017).Google Scholar
28. Desbats, P., Geffard, F., Piolain, G. and Coudray, A., “Force-feedback teleoperation of an industrial robot in a nuclear spent fuel reprocessing plant,” Ind. Robot 33 (3), 178–186 (2006).Google Scholar
29. Soyguder, S. and Abut, T., “Haptic industrial robot control with variable time delayed bilateral teleoperation,” Ind. Robot 43 (4), 390–402 (2016).Google Scholar
30. Khademian, B. and Hashtrudi-Zaad, K., “A framework for unconditional stability analysis of multimaster/multislave teleoperation systems,” IEEE Trans. Robot. 29 (3), 684–694 (2013).Google Scholar
31. Razi, K. and Hashtrudi-Zaad, K., “Analysis of coupled stability in multilateral dual-user teleoperation systems,” IEEE Trans. Robot. 30 (3), 631–641 (2014).Google Scholar
32. Li, J., Tavakoli, M. and Huang, Q., “Absolute stability of multi-dof multilateral haptic systems,” IEEE Trans. Control Syst. Technol. 22 (6), 2319–2328 (2014).Google Scholar
33. Lu, Z. et al., “Enhanced transparency dual-user shared control teleoperation architecture with multiple adaptive dominance factors,” Int. J. Control. Autom. Syst. 15 (5), 2301–2312 (2017).Google Scholar
34. Colgate, J. E., Stanley, M. C. and Brown, J. M., “Issues in the Haptic Display of Tool Use,” Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (1995) pp. 140–145.Google Scholar
35. Zilles, C. B. and Salisbury, J. K., “A constraint-based god-object method for haptic display,” Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (1995) pp. 146–151.Google Scholar
36. Howard, B. M. and Vance, J. M., “Desktop haptic virtual assembly using physically based modelling,” Virtual Reality 11 (4), 207–215 (2007).Google Scholar
37. Hou, X. and Sourina, O., “Stable adaptive algorithm for six degrees-of-freedom haptic rendering in a dynamic environment,” Vis. Comput. 29 (10), 1063–1075 (2013).Google Scholar
38. Blender, “https://www.blender.org/” (2017).Google Scholar
39. Arimoto, S. and Miyazaki, F., “Stability and Robustness of PID Feedback Control for Robot Manipulators of Sensory Capability,” Proceedings of the International Symposium on Robotics Research (1984) pp. 783–799.Google Scholar
40. Khalil, W. and Dombre, E., “Modeling, identification and control of robots,” Kogan Page Science Paper Edition, Butterworth-Heinemann (2004). ISBN-10: 190399666XGoogle Scholar
41. Nuño, E., Basañez, L. and Prada, M., “Asymptotic Stability of Teleoperators with Variable Time-Delays,” Proceedings of the IEEE International Conference on Robotics and Automation (2009) pp. 4332–4337.Google Scholar
42. Nuño, E., Basañez, L., Ortega, R. and Spong, M. W., “Position tracking for non-linear teleoperators with variable time Delay,” Int. J. Robot. Res. 28 (7), 895–910 (2009).Google Scholar
43. Hua, C. C. and Liu, X. P., “Delay-dependent stability criteria of teleoperation systems with asymmetric time-varying delays,” IEEE Trans. Robot. 26 (5), 895–910 (2010).Google Scholar
44. Slotine, J. J. E. and Li, W., Applied Nonlinear Control (Prentice Hall, 1991).Google Scholar
45. Lawrence, D., “Stability and transparency in bilateral teleoperation,” IEEE Trans. Robot. Autom. 9 (5), 624–637 (1993).Google Scholar
46. Hokayem, P. F. and Spong, M. W., “Bilateral teleoperation: An historical survey,” Automatica 42 (12), 2035–2057 (2006).Google Scholar
47. Stäubli Technical Documentation Interactive CD-ROM, “Arm–RX series 160 family Instruction Manual” (2008).Google Scholar

Bayraktaroglu et al. supplementary material

Bayraktaroglu et al. supplementary material 1

Download Bayraktaroglu et al. supplementary material(Video)
Video 15 MB