Skip to main content

LIDAR and stereo combination for traversability assessment of off-road robotic vehicles

  • Giulio Reina (a1), Annalisa Milella (a2) and Rainer Worst (a3)

Reliable assessment of terrain traversability using multi-sensory input is a key issue for driving automation, particularly when the domain is unstructured or semi-structured, as in natural environments. In this paper, LIDAR-stereo combination is proposed to detect traversable ground in outdoor applications. The system integrates two self-learning classifiers, one based on LIDAR data and one based on stereo data, to detect the broad class of drivable ground. Each single-sensor classifier features two main stages: an adaptive training stage and a classification stage. During the training stage, the classifier automatically learns to associate geometric appearance of 3D data with class labels. Then, it makes predictions based on past observations. The output obtained from the single-sensor classifiers are statistically combined in order to exploit their individual strengths and reach an overall better performance than could be achieved by using each of them separately. Experimental results, obtained with a test bed platform operating in rural environments, are presented to validate and assess the performance of this approach, showing its effectiveness and potential applicability to autonomous navigation in outdoor contexts.

Corresponding author
*Corresponding author. E-mail:
Hide All
1. Aycard, O., Baig, Q., Bota, S., Nashashibi, F., Nedevschi, S., Pantilie, C. D., Parent, M., Resende, P. and Vu, Trung-Dung, “Intersection Safety using Lidar and Stereo Vision Sensors,” IEEE Intelligent Vehicles Symposium, Baden-Baden, Germany, (2011), pp. 863–869.
2. Badino, H., Huber, D. and Kanade, T., “Integrating Lidar into Stereo for Fast and Improved Disparity Computation,” 3D Imaging, Modeling, Processing, Visualization, Transmission (3DIMPVT), International Conference on, Hangzhou, China, (2011) pp. 1–8.
3. Bajracharya, M., Maimone, M. W. and Helmick, D., “Autonomy for mars rovers: Past, present, and future,” Computer 41 (12), 4450 (2008).
4. Bradski, G. and Kaehler, A., “Learning OpenCV: Computer Vision with the OpenCV Library (O'Reilly Media, 2008).
5. Broggi, A., Cappalunga, A., Caraffi, C., Cattani, S., Ghidoni, S., Grisleri, P., Porta, P., Posterli, M. and Zani, P., “Terramax vision at the urban challenge 2007,” IEEE Trans. Intell. Transp. Syst. 11 (1), 194205 (2010).
6. Broggi, A., Cardarelli, E., Cattani, S. and Sabbatelli, M., “Terrain Mapping for Off-Road Autonomous Ground Vehicles using Rational B-Spline Surfaces and Stereo Vision,” IEEE Intelligent Vehicles Symposium (2013), Gold Coast City, Australia, pp. 648–653.
7. Broggi, A., Cattani, S., Patander, M., Sabbatelli, M. and Zani, P., A Full-3D Voxel-Based Dynamic Obstacle Detection for Urban Scenario using Stereo Vision,” International IEEE Annual Conference on Intelligent Transportation Systems (ITSC 2013) (2013) pp. 71–76.
8. Dahlkamp, H. A., Kaehler, D. S., Thrun, S. and Bradski, G., “Self-Supervised Monocular road Detection in Desert Terrain,” Robotics Science and Systems Conference (2006), Philadelphia, USA, pp. 1–6.
9. Dima, C., Vandapel, N. and Hebert, M., “Classifier fusion for outdoor obstacle detection,” IEEE International Conference on Robotics and Automation (2004), New Orleans, LA, USA, pp. 665–671.
10. Foo, P. and NG, G., “High-level information fusion: An overview,” J. Adv. Inform. Fusion 8 (1), 3372 (2013).
11. Hadsell, R., Sermanet, P., Ben, J., Erkan, A., Scoffie, M. and Kavukcuoglu, K., “Learning long-range vision for autonomous off-road driving,” J. Field Robot. 26 (2), 120144 (2009).
12. Hague, T., Marchant, J. and Tillett, N., “Ground-based sensing systems for autonomous agricultural vehicles,” Comput. Electron. Agric. 25 (1–2), 1128 (2000).
13. Hastie, T., Tibshirani, R. and Friedman, J., “The Elements of Statistical Learning (Springer-Verlag, New York, 2003).
14. Konolige, K., Agrawal, M., Blas, M. R., Bolles, R. C., Gerkey, B. P., Solà, J. and Sundaresan, A., “Mapping, navigation, and learning for off-road traversal,” J. Field Robot. 26 (1), 88113 (2009).
15. Konolige, K., Bowman, J., Chen, J., Mihelick, P., Lepetit, M. C. V. and Fua, P., “View-based maps,” Int. J. Robot. Res. 29 (8), 941957 (2010).
16. Lalonde, J., Vandapel, N., Huber, D. and Hebert, M., “Natural terrain classification using three-dimensional ladar data for ground robot mobility,” J. Field Robot. 23 (10), 839861 (2006).
17. Manduchi, R., Castano, A., Talukder, A. and Matthies, L., “Obstacle detection and terrain classification for autonomous off-road navigation,” Auton. Robot 18, 81102 (2004).
18. Milella, A. and Reina, G., “3D reconstruction and classification of natural environments by an autonomous vehicle using multi-baseline stereo,” Intell. Serv. Robot. 7, 7992 (2014).
19. Milella, A., Reina, G. and Underwood, J., “A self-learning framework for statistical ground classification using radar and monocular vision,” J. Field Robot. 32 (1), 2041 (2015).
20. Mousazadeh, H., “A technical review on navigation systems of agricultural autonomous off-road vehicles,” J. Terramech. 50 (3), 211232 (2013).
21. Nüchter, A., Lingemann, K., Hertzberg, J. and Surmann, H., “6D SLAM–3D mapping outdoor environments,” J. Field Robot. 24, 699722 (2007).
22. Nedevschi, S., Danescu, R., Frentiu, D., Marita, T., Oniga, F., Pocol, C., Graf, T. and Schmidt, R., “High Accuracy Stereovision Approach for Obstacle Detection on Non-Planar Roads,” Proceedings of IEEE INES, (2004) Cluji-Napoca, Romania (2004) pp. 211–216.
23. Nickels, K., Castano, A. and Cianci, C. M., “Fusion of Lidar and Stereo Range for Mobile Robots,” International Conference on Advanced Robotics (2003), Coimbra, Portugal, pp. 1–6.
24. Perrollaz, M., Yoder, J. D. and Laugier, C., “Using Obstacles and Road Pixels in the Disparity-Space Computation of Stereo-Vision Based Occupancy Grids,” IEEE Conference on Intelligent Transportation Systems (ITSC) (2010) pp. 1147–1152.
25. Point Grey, “Triclops software development kit,” Available at: [accessed on June 1st, 2015].
26. Poppinga, J., Birk, A. and Pathak, K., “Hough-based terrain classification for realtime detection of drivable ground,” J. Field Robot. 25 (1–2), 6788 (2008).
27. Reina, G., Ishigami, G., Nagatani, K. and Yoshida, K., “Vision-Based Estimation of Slip Angle for Mobile Robots and Planetary Rovers,” Proceedings of IEEE International Conference on Robotics and Automation, Pasadena, CA, USA (2008) pp. 486–491.
28. Reina, G. and Milella, A., “Toward autonomous agriculture: Automatic ground detection using trinocular stereovision,” Sensors 60 (11), 1240512423 (2012).
29. Reina, G., Milella, A., Halft, W. and Worst, R., “LIDAR and Stereo Imagery Integration for Safe Navigation in Outdoor Settings,” IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR) (2013) pp. 1–6.
30. Reina, G., Milella, A. and Underwood, J., “Self-learning classification of radar features for scene understanding,” Robot. Auton. Syst. 60 (11), 13771388 (2012).
31. Rohmer, E., Reina, G. and Yoshida, K., “Dynamic simulation-based action planner for a reconfigurable hybrid legwheel planetary exploration rover,” Adv. Robot. 24 (8–9), 12191238 (2010).
32. Santana, P., Guedes, M., Correia, L. and Barata, J., “A Saliency-Based Solution for Robust Off-Road Obstacle Detection,” IEEE International Conference on Robotics and Automation, Anchorage, Alaska, USA (2010), pp. 3096–3101.
33. Tax, D., One-Class Classification. Concept Learning in the Absence of Counter Examples Ph.D. Thesis (Delft University of Technology, Delft, Netherlands, 2001).
34. Wedel, A., Badino, H., Rabe, C., Loose, H., Franke, U. and Cremers, D., “B-spline modeling of road surfaces with an application to free-space estimation,” IEEE Trans. Intell. Transp. Syst. 10 (4), 572583 (2009).
35. Weiss, U. and Biber, P., “Plant detection and mapping for agricultural robots using a 3D LIDAR sensor,” Robot. Auton. Syst. 59 (5), 265273 (2011).
36. Wellington, C. and Stentz, A., “Online Adaptive Rough-Terrain Navigation in Vegetation,” Proceedings of International Conference on Robotics and Automation (2004), New Orleans, LA, USA, pp. 96–101.
37. Wurm, K. M., Hornung, A., Bennewitz, M., Stachniss, C. and Burgard, W., “Octomap: A Probabilistic, Flexible, and Compact 3D Map Representation for Robotic Systems,” ICRA Workshop on Best Practice in 3D Perception and Modeling for Mobile Manipulation, Anchorage, Alaska, USA (2010), pp. 1–8.
38. Zhang, Q. and Pless, R., “Extrinsic calibration of a camera and laser range finder (improves camera calibration),” Proceedings of IEEE/RSJ International Conference onIntelligent Robots and Systems, Sendai, Japan, vol. 3 (2004) pp. 2301–2306.
39. Zhao, J., Katupitiya, J. and Ward, J., “Global Correlation Based Ground Plane Estimation using v-Disparity Image,” International Conference on Robotics and Automation, Rome, Italy (2007), pp. 529–534.
40. Zhou, S., Xi, J., McDaniel, M. W., Nishihata, T., Salesses, P. and Iagnemma, K., “Self-supervised learning to visually detect terrain surfaces for autonomous robots operating in forested terrain,” J. Field Robot. 29 (2), 277297 (2012).
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

  • ISSN: 0263-5747
  • EISSN: 1469-8668
  • URL: /core/journals/robotica
Please enter your name
Please enter a valid email address
Who would you like to send this to? *



Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed