Hostname: page-component-77c78cf97d-d2fvj Total loading time: 0 Render date: 2026-04-24T22:29:27.279Z Has data issue: false hasContentIssue false
  • English
  • Français

3D-point-cloud registration and real-world dynamic modelling-based virtual environment building method for teleoperation

Published online by Cambridge University Press:  09 September 2016

Dejing Ni ,
Aiguo Song ,
Xiaonong Xu ,
Huijun Li ,
Chengcheng Zhu  and
Hong Zeng
Dejing Ni
Affiliation:
School of Instrument Science and Engineering, Southeast University, Nanjing 210096, P.R. China. E-mails: ndj_seu@163.com, seuxxn@gmail.com, lihuijun@seu.edu.cn, zccins_seu@seu.edu.cn, hzeng@seu.edu.cn
Aiguo Song*
Affiliation:
School of Instrument Science and Engineering, Southeast University, Nanjing 210096, P.R. China. E-mails: ndj_seu@163.com, seuxxn@gmail.com, lihuijun@seu.edu.cn, zccins_seu@seu.edu.cn, hzeng@seu.edu.cn
Xiaonong Xu
Affiliation:
School of Instrument Science and Engineering, Southeast University, Nanjing 210096, P.R. China. E-mails: ndj_seu@163.com, seuxxn@gmail.com, lihuijun@seu.edu.cn, zccins_seu@seu.edu.cn, hzeng@seu.edu.cn
Huijun Li
Affiliation:
School of Instrument Science and Engineering, Southeast University, Nanjing 210096, P.R. China. E-mails: ndj_seu@163.com, seuxxn@gmail.com, lihuijun@seu.edu.cn, zccins_seu@seu.edu.cn, hzeng@seu.edu.cn
Chengcheng Zhu
Affiliation:
School of Instrument Science and Engineering, Southeast University, Nanjing 210096, P.R. China. E-mails: ndj_seu@163.com, seuxxn@gmail.com, lihuijun@seu.edu.cn, zccins_seu@seu.edu.cn, hzeng@seu.edu.cn
Hong Zeng
Affiliation:
School of Instrument Science and Engineering, Southeast University, Nanjing 210096, P.R. China. E-mails: ndj_seu@163.com, seuxxn@gmail.com, lihuijun@seu.edu.cn, zccins_seu@seu.edu.cn, hzeng@seu.edu.cn
*
*Corresponding author. E-mail: a.g.song@seu.edu.cn
Get access Rights & Permissions [Opens in a new window]

Summary

It is a challenging task for a human operator to manipulate a robot from a remote distance, especially in an unknown environment. Excellent teleoperation provides the human operator with a sense of telepresence, mainly including real-world vision, haptic perception, etc. This paper presents a novel virtual environment building method using the red–green–blue (RGB) colour information, the surface normal feature-based 3D-point-cloud registration method and the weighted sliding-average least-square-method-based real-world dynamic modelling for teleoperation. The experiments prove the method to be an accurate and effective means of teleoperation.

Keywords

TeleoperationVirtual environment3D point cloudDynamic modelling

Information

Type
Articles
Information
Robotica , Volume 35 , Issue 10 , October 2017 , pp. 1958 - 1974
Copyright
Copyright © Cambridge University Press 2016 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

1. Kohlbrecher, S., Romay, A., Stumpf, A., Gupta, A., von Stryk, O., Bacim, F., Bowman, D. A., Goins, A., Balasubramanian, R. and Conner, D. C., “Human-robot teaming for rescue missions: Team ViGIR's approach to the 2013 DARPA robotics challenge trials,” J. Field Robot. 32 (3), 352–377 (2015).Google Scholar
2. Sheridan, T. B., “Space teleoperation through time delay: Review and prognosis,” IEEE Trans. Robot. Autom. 9 (5), 592–606 (1993).Google Scholar
3. Chan, L., Naghdy, F. and Stirling, D., “Application of adaptive controllers in teleoperation systems: A survey,” IEEE Trans. Human-Mach. Syst. 44 (3), 337–352 (2014).Google Scholar
4. Passenberg, C., Peer, A. and Buss, M., “A survey of environment-, operator-, and task-adapted controllers for teleoperation systems,” Mechatronics 20 (7), 787–801 (2010).CrossRefGoogle Scholar
5. Arcara, P. and Melchiorri, C., “Control schemes for teleoperation with time delay: A comparative study,” Robot. Auton. Syst. 38 (1), 49–64 (2002).CrossRefGoogle Scholar
6. Bejczy, A. K., Kim, W. S. and Venema, S. C., “The Phantom Robot: Predictive Displays for Teleoperation with Time Delay,” Proceedings of IEEE Int. Conf. on Robotics and Automation, Cincinnati, America (May 13–18, 1990) pp. 546–551.Google Scholar
7. Kim, W. S. and Bejczy, A. K., “Graphics Displays for Operator Aid in Telemanipulation,” In: Title of Decision Aiding for Complex Systems, Proceedings of IEEE International Conference on Systems, Man, and Cybernetics, Charlottesville, America (Oct. 13–16, 1991) pp. 1059–1067.Google Scholar
8. Park, J. H. and Sheridan, T. B., “Supervisory Teleoperation Control using Computer Graphics,” Proceedings of IEEE International Conference on Robotics and Automation, Sacramento, America (Apr. 9–11, 1991) pp. 493–498.Google Scholar
9. Barth, M., Burkert, T., Eberst, C., Stoffler, N. O. and Farber, G., “Photo-Realistic Scene Prediction of Partially Unknown Environments for the Compensation of Time Delays in Telepresence Applications,” Proceedings of IEEE International Conference on Robotics and Automation, San Francisco, America (Apr. 24–28, 2000) pp. 3132–3137.Google Scholar
10. Kawabata, K., Sekine, T., Suzuki, T., Fujii, T. and Endo, H. A. I., “Mobile robot teleoperation system utilizing a virtual world,” Adv. Robot. 15 (1), 1–16 (2012).CrossRefGoogle Scholar
11. Jiang, Z., Liu, H., Wang, J. and Huang, J., “Virtual reality-based teleoperation with robustness against modelling errors,” Chinese J. Aeronaut. 22 (3), 325–333 (2009).Google Scholar
12. Mitra, P. and Niemeyer, G., “Model-mediated telemanipulation,” Int. J. Robot. Res. 27 (2), 253–262 (2008).Google Scholar
13. Xu, X., Cizmeci, B., Al-Nuaimi, A. and Steinbach, E., “Point cloud-based model-mediated teleoperation with dynamic and perception-based model updating,” IEEE Trans. Instrum. Meas. 63 (11), 2558–2569 (2014).Google Scholar
14. Yanco, H. A., Norton, A., Ober, W., Shane, D., Skinner, A. and Vice, J., “Analysis of human robot interaction at the DARPA robotics challenge trials,” J. Field Robot. 32 (3), 420–444 (2015).CrossRefGoogle Scholar
15. Kadavasal, M. S., “Sensor augmented virtual reality based teleoperation using mixed autonomy,” J. Comput. Inf. Sci. Eng. 9 (1), 185–194 (2009).Google Scholar
16. Ni, T., Zhang, H., Xu, P. and Yamada, H., “Vision-based virtual force guidance for tele-robotic system,” Comput. Electr. Eng. 39 (7), 2135–2144 (2013).Google Scholar
17. Kaul, L., Zlot, R. and Bosse, M., “Continuous-time three-dimensional mapping for micro aerial vehicles with a passively actuated rotating laser scanner,” J. Field Robot. 33 (1), 103–132 (2016).CrossRefGoogle Scholar
18. Tukey, J. W., Exploratory Data Analysis (Addison-Wesley: Cambridge, England, 1977) pp. 210–214.Google Scholar
19. Salvi, J., Matabosch, C., Fofi, D. and Forest, J., “A review of recent range image registration methods with accuracy evaluation,” Image Vis. Comput. 25 (5), 578–596 (2007).Google Scholar
20. Besl, P. J. and Mckay, H. D., “A method for registration of 3-d shapes,” IEEE Trans. Pattern Anal. Mach. Intell. 14, 239–256 (1992).Google Scholar
21. Bay, H., Ess, A., Tuytelaars, T. and Gool, L. V., “Speeded-up robust features (SURF),” Comput. Vis. Image Understanding 110 (3), 346–359 (2008).CrossRefGoogle Scholar
22. Rusu, R. B., Marton, Z. C., Blodow, N., Dolha, M. and Beetz, M., “Towards 3d point cloud based object maps for household environments,” Robot. Auton. Syst. 56 (11), 927–941 (2008).Google Scholar
23. Chen, C. S., Hung, Y. P. and Cheng, J. B., “RANSAC-based DARCES: A new approach to fast automatic registration of partially overlapping range images,” IEEE Trans. Pattern Anal. Mach. Intell. 21 (11), 1229–1234 (1999).Google Scholar
24. Segal, A., Hähnel, D. and Thrun, S., “Generalized-ICP,” Robot. Sci. Syst. 2 (4), (2009).Google Scholar
25. Ryden, F. and Chizeck, H. J., “A proxy method for real-time 3-dof haptic rendering of streaming point cloud data,” IEEE Trans. Haptics 6 (3), 257–267 (2013).CrossRefGoogle ScholarPubMed
26. Leeper, A., Chan, S. and Salisbury, K., “Point Clouds can be Represented as Implicit Surfaces for Constraint-Based Haptic Rendering,” Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Saint Paul, America (May, 14–18 2012) pp. 5000–5005.CrossRefGoogle Scholar
27. Leeper, A., Chan, S., Hsiao, K., Ciocarlie, M. and Salisbury, K., “Constraint-Based Haptic Rendering of Point Data for Teleoperated Robot Grasping,” Proceedings of the IEEE Haptics Symposium, Vancouver, Canada (Mar. 4–7, 2012) pp. 377–383.Google Scholar
28. Song, A., Wu, J., Qin, G. and Huang, W., “A novel self-decoupled four degree-of-freedom wrist force/torque sensor,” Measurement 40 (9), 883–891 (2007).Google Scholar
29. Doebelin, E., System Dynamics: Modelling, Analysis, Simulation, Design. (CRC Press, United States, 1998).Google Scholar
30. Song, A., Morris, D. and Colgate, J. E., “Haptic Telemanipulation of Soft Environment without Direct Force Feedback,” Proceedings of the IEEE International Conference on Information Acquisition, (Jun. 27–Jul. 3 2005) pp. 21–25.Google Scholar
31. Gilardi, G. and Sharf, I., “Literature survey of contact dynamics modelling,” Mech. Mach. Theory 37 (10), 1213–1239 (2002).CrossRefGoogle Scholar
32. Li, H. and Song, A., “Virtual-environment modelling and correction for force-reflecting teleoperation with time delay,” IEEE Trans. Ind. Electron. 54 (2), 1227–1233 (2007).Google Scholar