Skip to main content
×
×
Home

Real-time dense map fusion for stereo SLAM

  • Taihú Pire (a1), Rodrigo Baravalle (a1), Ariel D'Alessandro (a1) and Javier Civera (a2)
Summary

A robot should be able to estimate an accurate and dense 3D model of its environment (a map), along with its pose relative to it, all of it in real time, in order to be able to navigate autonomously without collisions.

As the robot moves from its starting position and the estimated map grows, the computational and memory footprint of a dense 3D map increases and might exceed the robot capabilities in a short time. However, a global map is still needed to maintain its consistency and plan for distant goals, possibly out of the robot field of view.

In this work, we address such problem by proposing a real-time stereo mapping pipeline, feasible for standard CPUs, which is locally dense and globally sparse and accurate. Our algorithm is based on a graph relating poses and salient visual points, in order to maintain a long-term accuracy with a small cost. Within such framework, we propose an efficient dense fusion of several stereo depths in the locality of the current robot pose.

We evaluate the performance and the accuracy of our algorithm in the public datasets of Tsukuba and KITTI, and demonstrate that it outperforms single-view stereo depth. We release the code as open-source, in order to facilitate the system use and comparisons.

Copyright
Corresponding author
*Corresponding author. E-mail: pire@cifasis-conicet.gov.ar
References
Hide All
1. Alcantarilla, P., Beall, C. and Dellaert, F., “Large-Scale dense 3D Reconstruction from Stereo Imagery,” Proceedings of the 5th Workshop on Planning, Perception and Navigation for Intelligent Vehicles PPNV2013. Georgia Institute of Technology (Nov. 2013).
2. Bailey, T. and Durrant-Whyte, H., “Simultaneous localization and mapping (SLAM): Part II,” IEEE Robot. Autom. Mag. 13 (3), 108117 (Sep. 2006).
3. Bao, S. Y., Chandraker, M., Lin, Y. and Savarese, S., “Dense Object Reconstruction with Semantic Priors,” Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition CVPR2013, Washington, DC, USA, IEEE Computer Society (2013) pp. 1264–1271.
4. Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I. and Leonard, J. J., “Past, present, and future of simultaneous localization and mapping: Toward the robust-perception age,” IEEE Trans. Robot. 32 (6), 13091332 (Dec. 2016).
5. Cole, D. M. and Newman, P. M., “Using Laser Range Data for 3D SLAM in Outdoor Environments,” Proceedings of the IEEE International Conference on Robotics and Automation ICRA2006 (May 2006) pp. 1556–1563.
6. Concha, A., Hussain, W., Montano, L. and Civera, J., “Manhattan and Piecewise-Planar Constraints for Dense Monocular Mapping,” Proceedings of Robotics: Science and Systems, Berkeley, USA (Jul. 2014).
7. Concha, A., Loianno, G., Kumar, V. and Civera, J., “Visual-Inertial Direct SLAM,” Proceedings of the IEEE International Conference on Robotics and Automation ICRA2016 (May 2016) pp. 1331–1338.
8. Davison, A. J., Reid, I. D., Molton, N. D. and Stasse, O., “MonoSLAM: Real-time single camera SLAM,” IEEE Trans. Pattern Anal. Mach. Intell. 29 (6), 10521067 (Jun. 2007).
9. Durrant-Whyte, H. and Bailey, T., “Simultaneous localization and mapping: Part I,” IEEE Robot. Autom. Mag. 13 (2), 99110 (Jun. 2006).
10. Engel, J., Koltun, V. and Cremers, D., “Direct sparse odometry,” IEEE Trans. Pattern Anal. Mach. Intell. (2017).
11. Engel, J., Stückler, J. and Cremers, D., “Large-Scale Direct Slam with Stereo Cameras,” Proceedings of the IEEE International Conference on Intelligent Robots and Systems IROS2015, IEEE (2015) pp. 1935–1942.
12. Geiger, A., Ziegler, J. and Stiller, C., “StereoScan: Dense 3D Reconstruction in Real-Time,” Proceedings of the IEEE Intelligent Vehicles Symposium (IV) (Jun. 2011) pp. 963–968.
13. Geiger, A., Lenz, P., Stiller, C. and Urtasun, R., “Vision meets robotics: The KITTI dataset,” Int. J. Robot. Res. 32 (11), 12311237 (Sep. 2013).
14. Geiger, A., Roser, M. and Urtasun, R., Efficient Large-Scale Stereo Matching (Springer, Berlin, Heidelberg, 2011) pp. 2538.
15. Graber, G., Pock, T. and Bischof, H., “Online 3D Reconstruction using Convex Optimization,” Proceedings of the IEEE International Conference on Computer Vision Workshops (ICCV Workshops) (Nov. 2011) pp. 708–711.
16. Klingensmith, M., Dryanovski, I., Srinivasa, S. and Xiao, J., “Chisel: Real Time Large Scale 3D Reconstruction Onboard a Mobile Device using Spatially Hashed Signed Distance Fields,” Proceedings of Robotics: Science and Systems, volume 4, Rome, Italy (Jul. 2015).
17. Kuschk, G., Božič, A. and Cremers, D., “Real-Time Variational Stereo Reconstruction with Applications to Large-Scale Dense SLAM,” Proceedings of the 2017 IEEE Intelligent Vehicles Symposium (IV) (Jun. 2017) pp. 1348–1355.
18. Ladický, L., Sturgess, P., Russell, C., Sengupta, S., Bastanlar, Y., Clocksin, W. and Torr, P., “Joint optimization for object class segmentation and dense stereo reconstruction,” Int. J. Comput. Vis. 100 (2), 122133 (2012).
19. Maddern, W. and Newman, P., “Real-Time Probabilistic Fusion of Sparse 3D LIDAR and Dense Stereo,” Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), IEEE (2016) pp. 21812188.
20. Miksik, O., Amar, Y., Vineet, V., Prez, P. and Torr, P., “Incremental Dense Multi-Modal 3D Scene Reconstruction,” Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (Sep. 2015) pp. 908–915.
21. Mur-Artal, R. and Tardós, J. D., “ORB-SLAM2: An open-source SLAM system for monocular, stereo and RGB-D cameras,” IEEE Trans. Robot. 33 (5), 12551262 (Oct. 2017).
22. Newcombe, R. A., Lovegrove, S. J. and Davison, A. J., “DTAM: Dense Tracking and Mapping in Real-time,” Proceedings of the IEEE International Conference on Computer Vision ICCV2011, Washington, DC, USA, IEEE Computer Society (2011) pp. 2320–2327.
23. Oleynikova, H., Taylor, Z., Fehr, M., Siegwart, R. and Nieto, J., “Voxblox: Incremental 3D Euclidean Signed Distance Fields for on-board MAV planning,” Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems IROS2017 (Sep. 2017) pp. 1366–1373.
24. Peris, M., Martull, S., Maki, A., Ohkawa, Y. and Fukui, K., “Towards a Simulation Driven Stereo Vision System,” Proceedings of the 21st International Conference on Pattern Recognition ICPR2012 (Nov. 2012) pp. 1038–1042.
25. Pire, T., Fischer, T., Castro, G., De~Cristóforis, P., Civera, J. and Jacobo~Berlles, J., “S-PTAM: Stereo parallel tracking and mapping,” Robot. Autom. Syst. 93, 2742 (2017).
26. Pire, T., Fischer, T., Civera, J., De~Cristóforis, P. and Berlles, J. J., “Stereo Parallel Tracking and Mapping for Robot Localization,” Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems IROS2015 (Sep. 2015) pp. 1373–1378.
27. Pizzoli, M., Forster, C. and Scaramuzza, D., “REMODE: Probabilistic, Monocular Dense Reconstruction in Real Time,” Proceedings of the IEEE International Conference on Robotics and Automation ICRA2014, IEEE (2014) pp. 2609–2616.
28. Schöps, T., Sattler, T., Häne, C. and Pollefeys, M., “Large-scale outdoor 3D reconstruction on a mobile device,” Comput. Vis. Image Understanding 157(C), 151166 (Apr. 2017).
29. Sengupta, S., Greveson, E., Shahrokni, A. and Torr, P., “Urban 3D Semantic Modelling Using Stereo Vision,” Proceedings of the IEEE International Conference on Robotics and Automation (May 2013) pp. 580–585.
30. Stühmer, J., Gumhold, S. and Cremers, D., Real-Time Dense Geometry from a Handheld Camera (Springer, Berlin, Heidelberg, 2010) pp. 1120.
31. Tanner, M., Pinies, P., Paz, L. M. and Newman, P., “DENSER cities: A system for dense efficient reconstructions of cities,” arXiv:1604.03734, 2016.
32. Tippetts, B., Lee, Dah~Jye, Lillywhite, K., and Archibald, James, “Review of stereo vision algorithms and their suitability for resource-limited systems,” J. Real-Time Image Process. 11 (1), 525 (Jan. 2016).
33. Vineet, V., Miksik, O., Lidegaard, M., Nießner, M., Golodetz, S., Prisacariu, V. A., Kähler, O., Murray, D. W., Izadi, S., Pérez, P. and Torr, P. H. S., “Incremental Dense Semantic Stereo Fusion for Large-Scale Semantic Scene Reconstruction,” Proceedings of the 2015 IEEE International Conference on Robotics and Automation ICRA2015 (May 2015) pp. 75–82.
34. Wang, R., Schwörer, M. and Cremers, D., Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV) “Stereo DSO: Large-scale direct sparse visual odometry with stereo cameras,” doi: 10.1109/ICCV.2017.421 (2017) pp. 3923–3931.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Robotica
  • ISSN: 0263-5747
  • EISSN: 1469-8668
  • URL: /core/journals/robotica
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×

Keywords

Metrics

Full text views

Total number of HTML views: 0
Total number of PDF views: 61 *
Loading metrics...

Abstract views

Total abstract views: 187 *
Loading metrics...

* Views captured on Cambridge Core between 20th June 2018 - 21st July 2018. This data will be updated every 24 hours.