Skip to main content
×
×
Home

Automated robotic monitoring and inspection of steel structures and bridges

  • Hung Manh La (a1), Tran Hiep Dinh (a2), Nhan Huu Pham (a1), Quang Phuc Ha (a2) and Anh Quyen Pham (a1)...
Summary

This paper presents visual and 3D structure inspection for steel structures and bridges using a developed climbing robot. The robot can move freely on a steel surface, carry sensors, collect data and then send to the ground station in real-time for monitoring as well as further processing. Steel surface image stitching and 3D map building are conducted to provide a current condition of the structure. Also, a computer vision-based method is implemented to detect surface defects on stitched images. The effectiveness of the climbing robot's inspection is tested in multiple circumstances to ensure strong steel adhesion and successful data collection. The detection method was also successfully evaluated on various test images, where steel cracks could be automatically identified, without the requirement of some heuristic reasoning.

Copyright
Corresponding author
*Corresponding author. E-mail: hla@unr.edu
References
Hide All
1. U.S. Department of Transportation Federal Highway Administration, “National bridge inventory data,” Available at: http://www.fhwa.dot.gov/bridge/nbi.cfm, accessed January 30, 2016.
2. Minnesota Department of Transportation, “I-35W St. Anthony Falls Bridge collapse,” Available at: http://www.dot.state.mn.us/i35wbridge/collapse.html, accessed January 30, 2016.
3. La, H. M., Lim, R. S., Basily, B. B., Gucunski, N., Yi, J., Maher, A., Romero, F. A. and Parvardeh, H., “Mechatronic systems design for an autonomous robotic system for high-efficiency bridge deck inspection and evaluation,” IEEE/ASME Trans. Mech. 18 (6), 16551664 (2013).
4. La, H. M., Gucunski, N., Kee, S. H. and Nguyen, L. V., “Data analysis and visualization for the bridge deck inspection and evaluation robotic system,” Vis. Eng. 3 (1), 116 (2015).
5. La, H. M., Gucunski, N., Kee, S. H. and Nguyen, L. V., “Visual and Acoustic Data Analysis for the Bridge Deck Inspection Robotic System,” Proceedings of the 31st International Symposium on Automation and Robotics in Construction and Mining (ISARC) (2014) pp. 50–57.
6. La, H. M., Gucunski, N., Kee, S. H. Yi, J., Senlet, T. and Nguyen, L. V., “Autonomous Robotic System for Bridge Deck Data Collection and Analysis” Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Chicago, USA (Sep. 14–18, 2014) pp. 1950–1955.
7. La, H. M., Gucunski, N., Dana, K., and Kee, S. H.. “Development of an Autonomous Bridge Deck Inspection Robotic System,” J. Field Robot. 34 (8), 14891504 (2017).
8. Lim, R. S., La, H. M., Shan, Z. and Sheng, W., “Developing a Crack Inspection Robot for Bridge Maintenance,” Proceedings of the 2011 IEEE International Conference on Robotics and Automation (ICRA), (2011) pp. 6288–6293.
9. Lim, R. S., La, H. M. and Sheng, W., “A robotic crack inspection and mapping system for bridge deck maintenance,” IEEE Trans. Autom. Sci. Eng. 11 (2), 367378 (2014).
10. La, H. M., Lim, R. S., Basily, B. B., Gucunski, N., Yi, J., Maher, A., Romero, F. A. and Parvardeh, H., “Autonomous Robotic System for High-Efficiency Non-Destructive Bridge Deck Inspection and Evaluation,” Proceedings of the IEEE International Conference on Automation Science and Engineering (CASE), Madison, WI, USA (August 17–21, 2013) pp. 1053–1058.
11. Li, B., Cao, J., Xiao, J., Zhang, X. and Wang, H., “Robotic Impact-Echo Non-Destructive Evaluation Based On FFT and SVM,” Proceedings of the 11th World Congress on Intelligent Control and Automation (WCICA) (2014) pp. 2854–2859.
12. Gucunski, N., Basily, B., Kee, S-H., La, H. M., Parvardeh, H., Maher, A. and Ghasemi, H., “Multi NDE Technology Condition Assessment of Concrete Bridge Decks by RABITTM Platform,” Proceedings of the NDE/NDT for Structural Materials Technology for Highway & Bridges (2014) pp. 161–168.
13. Gucunski, N., Kee, S.-H., La, H. M., Basily, B., Maher, A. and Ghasemi, H., “Implementation of a Fully Autonomous Platform for Assessment of Concrete Bridge Decks RABIT,” Proceedings of Structures Congress (2015) pp. 367–378.
14. Gucunski, N., Kee, S.-H., La, H. M., Basily, B. and Maher, A., “Delamination and concrete quality assessment of concrete bridge decks using a fully autonomous RABIT platform,” Int. J. Struct. Monit. Maint. 2 (1), 1934 (2015).
15. Gibb, S., Le, T. D., La, H. M., Schmid, R. and Berendsen, T., “A Multi-Functional Inspection Robot for Civil Infrastructure Evaluation and Maintenance,” Proceedings of the 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, Canada (Sep. 24–28, 2017).
16. Le, T. D., Gibb, S., Pham, N. H., La, H. M., Falk, L. and Berendsen, T., “Autonomous Robotic System using Non-Destructive Evaluation methods for Bridge Deck Inspection,” Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA), May 29–June 3, 2017, Singapore.
17. Gibb, S. and La, H. M., “Automated Rebar Detection for Ground-Penetrating Radar,” Proceedings of the 12th International Symposium on Visual Computing, Las Vegas, Nevada, USA (Dec. 12–14, 2016).
18. Dinh, K., Gucunski, N., Kim, J. Y., Duong, T. and La, H. M., “Attenuation-based Methodology for Condition Assessment of Concrete Bridge Decks using GPR,” Proceedings of the 32nd International Symposium on Automation and Robotics in Construction and Mining (ISARC), Oulu, Finland, Jun. 15–18, 2015.
19. Gucunski, N., Maher, A., Basily, B. B., La, H. M., Lim, R. S., Parvardeh, H. and Kee, S. H., “Robotic Platform RABIT for Condition Assessment of Concrete Bridge Decks Using Multiple NDE Technologies,” J. Croatian Soc. Non Destr. Testing 12, 512 (2013).
20. Prasanna, P., Dana, K. J., Gucunski, N., Basily, B. B., La, H. M., Lim, R. S. and Parvardeh, H., “Automated crack detection on concrete bridges,” IEEE Trans. Autom. Sci. Eng. 13 (2), 591599 (Apr. 2016).
21. Xu, F. and Wang, X., “Design and Experiments on A New Wheel-Based Cable Climbing Robot,” Proceedings of the IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM) (2008) pp. 418–423.
22. Cho, K. H., Kim, H. M., Jin, Y. H., Liu, F., Moon, H., Koo, J. C. and Choi, H. R., “Inspection robot for hanger cable of suspension bridge,” IEEE/ASME Trans. Mech. 18 (6), 16651674 (2013).
23. Mazumdar, A. and Asada, H. H., “Mag-Foot: A Steel Bridge Inspection Robot,” Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2009) pp. 1691–1696.
24. Wang, R. and Kawamura, Y., “A Magnetic Climbing Robot for Steel Bridge Inspection,” Proceedings of the 11th World Congress on Intelligent Control and Automation (WCICA) (2014) pp. 3303–3308.
25. Liu, Q. and Liu, Y., “An Approach for Auto Bridge Inspection Based On Climbing Robot,” Proceedings of the IEEE International Conference on Robotics and Biomimetics (ROBIO) (2013) pp. 2581–2586.
26. Liu, Y., Dai, Q. and Liu, Q., “Adhesion-Adaptive Control of a Novel Bridge-Climbing Robot,” Proceedings of the IEEE 3rd Annual International Conference on Cyber Technology in Automation, Control and Intelligent Systems (CYBER) (2013) pp. 102–107.
27. Leibbrandt, A., Caprari, G., Angst, U., Siegwart, R. Y., Flatt, R. J. and Elsener, B., “Climbing Robot for Corrosion Monitoring of Reinforced Concrete Structures,” Proceedings of the 2nd International Conference on Applied Robotics for the Power Industry (CARPI) (2012) pp. 10–15.
28. Leon-Rodriguez, H., Hussain, S. and Sattar, T., “A Compact Wall-Climbing and Surface Adaptation Robot for Non-Destructive Testing,” Proceedings of the 12th International Conference on Control, Automation and Systems (ICCAS) (2012) pp. 404–409.
29. San-Millan, A., “Design of a Teleoperated Wall Climbing Robot for Oil Tank Inspection,” Proceedings of the 23th Mediterranean Conference on Control and Automation (MED) (2015) pp. 255–261.
30. Zhu, D., Guo, J., Cho, C., Wang, Y. and Lee, K.M., “Wireless Mobile Sensor Network for the System Identification of a Space Frame Bridge,” IEEE/ASME Trans. Mech. 17 (3), 499507 (2012).
31. Koch, C., Georgieva, K., Kasireddy, V., Akinci, B. and Fieguth, P., “A review on computer vision based defect detection and condition assessment of concrete and asphalt civil infrastructure,” Adv. Eng. Inform. 29 (2), 196210 (2015).
32. Oh, J. K. et al., “Bridge inspection robot system with machine vision,” Autom. Constr. 18, 929941 (2009).
33. Adhikari, R. S., Moselhi, O. and Bagchi, A., “Image-based retrieval of concrete crack properties for bridge inspection,” Autom. Constr. 39, 180194 (2014).
34. Lins, R. G. and Givigi, S. N., “Automatic Crack Detection and Measurement Based on Image Analysis,” IEEE Trans. Instrum. Meas. 65 (3), 583590, (2016).
35. K&J Magnetics Inc. Orignial magnet calculator, Available a: https://www.kjmagnetics.com/calculator.asp, access November 10, 2016.
36. DS8911HV Servo, Available at: http://www.jramericas.com/116293/JRPS8911HV/, access November 10, 2016.
37. Forsyth, D. A. and Ponce, J., Computer Vision: A Modern Approach, 2nd Edition, Nov 5, 2011 (Prentice Hall, Upper Saddle River, NJ).
38. Brown, M. and Lowe, D. G., “Automatic panoramic image stitching using invariant features,” Int. J. Comput. Vis. 74 (1), 5973 (2007).
39. Besl, P. and McKay, N., “A method for registration of 3-D shapes,” IEEE Trans. Pattern Anal. Mach. Intell. 14 (2), 239256 (1992).
40. Rusu, R. B., Blodow, N. and Beetz, M., “Fast Point Feature Histograms (FPFH) for 3D registration,” Proceedings of the IEEE International Conference on Robotics and Automation, ICRA '09, (2009) pp. 3212–3217.
41. Chen, Y. and Medioni, G., “Object Modelling by Registration of Multiple Range Images,” Proceedings of the IEEE International Conference on Robotics and Automation, vol. 3 (2015) pp. 2724–2729.
42. Holz, D., Ichim, A. E., Tombari, F., Rusu, R. B. and Behnke, S., “Registration with the point cloud library: A modular framework for aligning in 3-D,” IEEE Robot. Autom. Mag. 2 (4), 110124 (2015).
43. Yu, Y.-H., Kwok, N. M. and Ha, Q. P., “Color tracking for multiple robot control using a system-on-programmable-chip,” Autom. Constr. 20 (6), 669676 (2011).
44. Dinh, T. H., Pham, M. T., Phung, M. D., Nguyen, D. M. and Tran, Q. V., “Image Segmentation Based on Histogram of Depth and an Application in Driver Distraction Detection,” Proceedings of the 13th International Conference on Control Automation Robotics & Vision (ICARCV) (2014) pp. 969–974.
45. Otsu, N., “A threshold selection method from gray-level histograms,” IEEE Trans. Syst. Man Cybern. 9 (1), 6266 (1979).
46. Dirami, A., Hammouche, K., Diaf, M. and Siarry, P., “Fast multilevel thresholding for image segmentation through a multiphase level set method,” Signal Proces., 93 (1), 139153 (2013).
47. Yuan, X. C., Wu, L. S. and Peng, Q., “An improved Otsu method using the weighted object variance for defect detection,” Appl. Surface Sci. 349, 472484 (2015).
48. Sato, Y., Nakajima, S., Shiraga, N., Atsumi, H., Yoshida, S., Koller, T., Gerig, G. and Kikinis, R., “Three-dimensional multi-scale line filter for segmentation and visualization of curvilinear structures in medical images,” Medical Image Anal. 2 (2), 143168 (1998).
49. Fujita, Y. and Hamamoto, Y., “A robust automatic crack detection method from noisy concrete surfaces,” Mach. Vis. Appl. 22 (2), 245254 (2011).
50. Zhao, Y. Q., Wang, X. H., Wang, X. F. and Shih, F. Y., “Retinal vessels segmentation based on level set and region growing,” Pattern Recognit. 47 (7), 24372446 (2014).
51. Sezan, M. I., “A peak detection algorithm and its application to histogram-based image data reduction.” Comput. Vis. Graph. Image Process. 49 (1), 3651 (1990).
52. Yuan, B. and Liu, M., “Power histogram for circle detection on images,” Pattern Recognit. 48 (10) 32683280 (2015).
53. Sauvola, J. and Pietikäinen, M., “Adaptive document image binarization,” Pattern Recognit. 33 (2), 225236 (2000).
54. Pham, N. H. and La, H. M., “Design and Implementation of an Autonomous Robot for Steel Bridge Inspection,” Proceedings of the 54th Annual Allerton Conference on Communication, Control, and Computing, Urbana-Champaign, Illinois, USA (Sep. 27–30, 2016) pp. 556–562.
55. Nguyen, L. V., La, H. M., Sanchez, J. and Vu, T., “A Smart Shoe for Building a Real-Time 3D Map,” Elsevier J. Autom. Constr. 71, 212 (Sep. 2016).
56. Pham, N. H., La, H. M., Ha, Q. P., Dang, S. N., Vo, A. H. and Dinh, Q. H., “Visual and 3D mapping for steel bridge inspection using a climbing robot,” Proceedings of the 33rd International Symposium on Automation and Robotics in Construction and Mining (ISARC), Auburn, Alabama, USA (July 18–21, 2016).
57. Dinh, T. H., Ha, Q. P. and La, H. M., “Computer vision-based method for concrete crack detection,” Proceedings of the 14th International Conference on Control, Automation, Robotics and Vision (ICARCV), Phuket, Thailand (Nov. 13–15, 2016).
58. La, H. M., Sheng, W. and Chen, J., “Cooperative and active sensing in mobile sensor networks for scalar field mapping,” IEEE Trans. Syst. Man Cybern.: Syst. 45 (1), 112 (Jan. 2015).
59. La, H. M., Lim, R. and Sheng, W., “Multi-robot cooperative learning for predator avoidance,” IEEE Trans. Control Syst. Technol. 23 (1), 5263 (Jan. 2015).
60. La, H. M. and Sheng, W., “Multi-agent motion control in cluttered and noisy environments,” J. Commun. 8 (1), 3246 (Jan. 2013).
61. La, H. M. and Sheng, W., “Dynamic targets tracking and observing in a mobile sensor network,” Elsevier J. Robot. Autom. Syst. 60 (7), 9961009 (Jul. 2012).
62. La, H. M. and Sheng, W., “Distributed sensor fusion for scalar field mapping using mobile sensor networks,” IEEE Trans. Cybern. 43 (2), 766778 (April, 2013).
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Robotica
  • ISSN: 0263-5747
  • EISSN: 1469-8668
  • URL: /core/journals/robotica
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×

Keywords

Metrics

Full text views

Total number of HTML views: 0
Total number of PDF views: 51 *
Loading metrics...

Abstract views

Total abstract views: 441 *
Loading metrics...

* Views captured on Cambridge Core between 11th January 2018 - 21st July 2018. This data will be updated every 24 hours.