Hostname: page-component-8448b6f56d-cfpbc Total loading time: 0 Render date: 2024-04-17T02:52:07.863Z Has data issue: false hasContentIssue false
  • English
  • Français

Real-time dense map fusion for stereo SLAM

Published online by Cambridge University Press:  20 June 2018

Taihú Pire ,
Rodrigo Baravalle ,
Ariel D'Alessandro  and
Javier Civera
Taihú Pire*
Affiliation:
CIFASIS, French Argentine International Center for Information and Systems Sciences (CONICET-UNR), Argentina. E-mails: baravalle@cifasis-conicet.gov.ar, ariel.dalessandro@gmail.com
Rodrigo Baravalle
Affiliation:
CIFASIS, French Argentine International Center for Information and Systems Sciences (CONICET-UNR), Argentina. E-mails: baravalle@cifasis-conicet.gov.ar, ariel.dalessandro@gmail.com
Ariel D'Alessandro
Affiliation:
CIFASIS, French Argentine International Center for Information and Systems Sciences (CONICET-UNR), Argentina. E-mails: baravalle@cifasis-conicet.gov.ar, ariel.dalessandro@gmail.com
Javier Civera
Affiliation:
I3A, University of Zaragoza, Spain. E-mail: jcivera@unizar.es
*
*Corresponding author. E-mail: pire@cifasis-conicet.gov.ar
Get access Rights & Permissions [Opens in a new window]

Summary

A robot should be able to estimate an accurate and dense 3D model of its environment (a map), along with its pose relative to it, all of it in real time, in order to be able to navigate autonomously without collisions.

As the robot moves from its starting position and the estimated map grows, the computational and memory footprint of a dense 3D map increases and might exceed the robot capabilities in a short time. However, a global map is still needed to maintain its consistency and plan for distant goals, possibly out of the robot field of view.

In this work, we address such problem by proposing a real-time stereo mapping pipeline, feasible for standard CPUs, which is locally dense and globally sparse and accurate. Our algorithm is based on a graph relating poses and salient visual points, in order to maintain a long-term accuracy with a small cost. Within such framework, we propose an efficient dense fusion of several stereo depths in the locality of the current robot pose.

We evaluate the performance and the accuracy of our algorithm in the public datasets of Tsukuba and KITTI, and demonstrate that it outperforms single-view stereo depth. We release the code as open-source, in order to facilitate the system use and comparisons.

Keywords

Dense MappingVisual SLAMStereo Vision
Type
Articles
Information
Robotica , Volume 36 , Issue 10 , October 2018 , pp. 1510 - 1526
Copyright
Copyright © Cambridge University Press 2018 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

1. Alcantarilla, P., Beall, C. and Dellaert, F., “Large-Scale dense 3D Reconstruction from Stereo Imagery,” Proceedings of the 5th Workshop on Planning, Perception and Navigation for Intelligent Vehicles PPNV2013. Georgia Institute of Technology (Nov. 2013).Google Scholar
2. Bailey, T. and Durrant-Whyte, H., “Simultaneous localization and mapping (SLAM): Part II,” IEEE Robot. Autom. Mag. 13 (3), 108–117 (Sep. 2006).Google Scholar
3. Bao, S. Y., Chandraker, M., Lin, Y. and Savarese, S., “Dense Object Reconstruction with Semantic Priors,” Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition CVPR2013, Washington, DC, USA, IEEE Computer Society (2013) pp. 1264–1271.Google Scholar
4. Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I. and Leonard, J. J., “Past, present, and future of simultaneous localization and mapping: Toward the robust-perception age,” IEEE Trans. Robot. 32 (6), 1309–1332 (Dec. 2016).Google Scholar
5. Cole, D. M. and Newman, P. M., “Using Laser Range Data for 3D SLAM in Outdoor Environments,” Proceedings of the IEEE International Conference on Robotics and Automation ICRA2006 (May 2006) pp. 1556–1563.Google Scholar
6. Concha, A., Hussain, W., Montano, L. and Civera, J., “Manhattan and Piecewise-Planar Constraints for Dense Monocular Mapping,” Proceedings of Robotics: Science and Systems, Berkeley, USA (Jul. 2014).Google Scholar
7. Concha, A., Loianno, G., Kumar, V. and Civera, J., “Visual-Inertial Direct SLAM,” Proceedings of the IEEE International Conference on Robotics and Automation ICRA2016 (May 2016) pp. 1331–1338.Google Scholar
8. Davison, A. J., Reid, I. D., Molton, N. D. and Stasse, O., “MonoSLAM: Real-time single camera SLAM,” IEEE Trans. Pattern Anal. Mach. Intell. 29 (6), 1052–1067 (Jun. 2007).Google Scholar
9. Durrant-Whyte, H. and Bailey, T., “Simultaneous localization and mapping: Part I,” IEEE Robot. Autom. Mag. 13 (2), 99–110 (Jun. 2006).Google Scholar
10. Engel, J., Koltun, V. and Cremers, D., “Direct sparse odometry,” IEEE Trans. Pattern Anal. Mach. Intell. (2017).Google Scholar
11. Engel, J., Stückler, J. and Cremers, D., “Large-Scale Direct Slam with Stereo Cameras,” Proceedings of the IEEE International Conference on Intelligent Robots and Systems IROS2015, IEEE (2015) pp. 1935–1942.Google Scholar
12. Geiger, A., Ziegler, J. and Stiller, C., “StereoScan: Dense 3D Reconstruction in Real-Time,” Proceedings of the IEEE Intelligent Vehicles Symposium (IV) (Jun. 2011) pp. 963–968.Google Scholar
13. Geiger, A., Lenz, P., Stiller, C. and Urtasun, R., “Vision meets robotics: The KITTI dataset,” Int. J. Robot. Res. 32 (11), 1231–1237 (Sep. 2013).Google Scholar
14. Geiger, A., Roser, M. and Urtasun, R., Efficient Large-Scale Stereo Matching (Springer, Berlin, Heidelberg, 2011) pp. 25–38.Google Scholar
15. Graber, G., Pock, T. and Bischof, H., “Online 3D Reconstruction using Convex Optimization,” Proceedings of the IEEE International Conference on Computer Vision Workshops (ICCV Workshops) (Nov. 2011) pp. 708–711.Google Scholar
16. Klingensmith, M., Dryanovski, I., Srinivasa, S. and Xiao, J., “Chisel: Real Time Large Scale 3D Reconstruction Onboard a Mobile Device using Spatially Hashed Signed Distance Fields,” Proceedings of Robotics: Science and Systems, volume 4, Rome, Italy (Jul. 2015).Google Scholar
17. Kuschk, G., Božič, A. and Cremers, D., “Real-Time Variational Stereo Reconstruction with Applications to Large-Scale Dense SLAM,” Proceedings of the 2017 IEEE Intelligent Vehicles Symposium (IV) (Jun. 2017) pp. 1348–1355.Google Scholar
18. Ladický, L., Sturgess, P., Russell, C., Sengupta, S., Bastanlar, Y., Clocksin, W. and Torr, P., “Joint optimization for object class segmentation and dense stereo reconstruction,” Int. J. Comput. Vis. 100 (2), 122–133 (2012).Google Scholar
19. Maddern, W. and Newman, P., “Real-Time Probabilistic Fusion of Sparse 3D LIDAR and Dense Stereo,” Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), IEEE (2016) pp. 2181–2188.Google Scholar
20. Miksik, O., Amar, Y., Vineet, V., Prez, P. and Torr, P., “Incremental Dense Multi-Modal 3D Scene Reconstruction,” Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (Sep. 2015) pp. 908–915.Google Scholar
21. Mur-Artal, R. and Tardós, J. D., “ORB-SLAM2: An open-source SLAM system for monocular, stereo and RGB-D cameras,” IEEE Trans. Robot. 33 (5), 1255–1262 (Oct. 2017).Google Scholar
22. Newcombe, R. A., Lovegrove, S. J. and Davison, A. J., “DTAM: Dense Tracking and Mapping in Real-time,” Proceedings of the IEEE International Conference on Computer Vision ICCV2011, Washington, DC, USA, IEEE Computer Society (2011) pp. 2320–2327.Google Scholar
23. Oleynikova, H., Taylor, Z., Fehr, M., Siegwart, R. and Nieto, J., “Voxblox: Incremental 3D Euclidean Signed Distance Fields for on-board MAV planning,” Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems IROS2017 (Sep. 2017) pp. 1366–1373.Google Scholar
24. Peris, M., Martull, S., Maki, A., Ohkawa, Y. and Fukui, K., “Towards a Simulation Driven Stereo Vision System,” Proceedings of the 21st International Conference on Pattern Recognition ICPR2012 (Nov. 2012) pp. 1038–1042.Google Scholar
25. Pire, T., Fischer, T., Castro, G., De~Cristóforis, P., Civera, J. and Jacobo~Berlles, J., “S-PTAM: Stereo parallel tracking and mapping,” Robot. Autom. Syst. 93, 27–42 (2017).Google Scholar
26. Pire, T., Fischer, T., Civera, J., De~Cristóforis, P. and Berlles, J. J., “Stereo Parallel Tracking and Mapping for Robot Localization,” Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems IROS2015 (Sep. 2015) pp. 1373–1378.Google Scholar
27. Pizzoli, M., Forster, C. and Scaramuzza, D., “REMODE: Probabilistic, Monocular Dense Reconstruction in Real Time,” Proceedings of the IEEE International Conference on Robotics and Automation ICRA2014, IEEE (2014) pp. 2609–2616.Google Scholar
28. Schöps, T., Sattler, T., Häne, C. and Pollefeys, M., “Large-scale outdoor 3D reconstruction on a mobile device,” Comput. Vis. Image Understanding 157(C), 151–166 (Apr. 2017).Google Scholar
29. Sengupta, S., Greveson, E., Shahrokni, A. and Torr, P., “Urban 3D Semantic Modelling Using Stereo Vision,” Proceedings of the IEEE International Conference on Robotics and Automation (May 2013) pp. 580–585.Google Scholar
30. Stühmer, J., Gumhold, S. and Cremers, D., Real-Time Dense Geometry from a Handheld Camera (Springer, Berlin, Heidelberg, 2010) pp. 11–20.Google Scholar
31. Tanner, M., Pinies, P., Paz, L. M. and Newman, P., “DENSER cities: A system for dense efficient reconstructions of cities,” arXiv:1604.03734, 2016.Google Scholar
32. Tippetts, B., Lee, Dah~Jye, Lillywhite, K., and Archibald, James, “Review of stereo vision algorithms and their suitability for resource-limited systems,” J. Real-Time Image Process. 11 (1), 5–25 (Jan. 2016).Google Scholar
33. Vineet, V., Miksik, O., Lidegaard, M., Nießner, M., Golodetz, S., Prisacariu, V. A., Kähler, O., Murray, D. W., Izadi, S., Pérez, P. and Torr, P. H. S., “Incremental Dense Semantic Stereo Fusion for Large-Scale Semantic Scene Reconstruction,” Proceedings of the 2015 IEEE International Conference on Robotics and Automation ICRA2015 (May 2015) pp. 75–82.Google Scholar
34. Wang, R., Schwörer, M. and Cremers, D., Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV) “Stereo DSO: Large-scale direct sparse visual odometry with stereo cameras,” doi: 10.1109/ICCV.2017.421 (2017) pp. 3923–3931.Google Scholar