Hostname: page-component-78c5997874-4rdpn Total loading time: 0 Render date: 2024-10-30T00:53:30.917Z Has data issue: false hasContentIssue false
  • English
  • Français

Identity recognition on waterways: a novel ship information tracking method based on multimodal data

Published online by Cambridge University Press:  25 June 2021

Zishuo Huang ,
Qinyou Hu ,
Qiang Mei ,
Chun Yang  and
Zheng Wu
Zishuo Huang
Affiliation:
Merchant Marine College, Shanghai Maritime University, Shanghai, China.
Qinyou Hu*
Affiliation:
Merchant Marine College, Shanghai Maritime University, Shanghai, China.
Qiang Mei
Affiliation:
Merchant Marine College, Shanghai Maritime University, Shanghai, China. Navigation College, Jimei University, Xiamen, China.
Chun Yang
Affiliation:
Merchant Marine College, Shanghai Maritime University, Shanghai, China.
Zheng Wu
Affiliation:
Department of Mathematics and Computer Science, Information Engineering University, Zhengzhou, China
*
*Corresponding author. E-mail: qyhu@shmtu.edu.cn
Get access Rights & Permissions [Opens in a new window]

Abstract

Video monitoring is an important means of ship traffic supervision. In practice, regulators often need to use an electronic chart platform to determine basic information concerning ships passing through a video feed. To enrich the information in the surveillance video and to effectively use multimodal maritime data, this paper proposes a novel ship multi-object tracking technology based on improved single shot multibox detector (SSD) and DeepSORT algorithms. In addition, a night contrast enhancement algorithm is used to enhance the ship identification performance in night scenes and a multimodal data fusion algorithm is used to incorporate the ship automatic identification system (AIS) information into the video display. The experimental results indicate that the ship information tracking accuracies in the day and night scenes are 78⋅2% and 70⋅4%, respectively. Our method can effectively help regulators to quickly obtain ship information from a video feed and improve the supervision of a waterway.

Keywords

ship information trackingmultimodal dataobject detectionobject tracking
Type
Research Article
Information
The Journal of Navigation , Volume 74 , Issue 6 , November 2021 , pp. 1336 - 1352
Check for updates
Copyright
Copyright © The Author(s), 2021. Published by Cambridge University Press on behalf of The Royal Institute of Navigation

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

An, J., Qiao, T., Yang, X., Hong, H. and Bai, X. (2019). Design of a Visual Analysis Platform for Sea Route Based on AIS Data. 2nd International Conference on Artificial Intelligence and Big Data (ICAIBD). Chengdu: IEEE.Google Scholar
Betti, A., Michelozzi, B., Bracci, A. and Masini, A. (2020). Real-Time Object Detection in Maritime Scenarios Based on YOLOv3 Model. the 9th International Symposium on Optronics in Defence and Security. Paris: 3AF.Google Scholar
Bewley, A., Ge, Z., Ott, L., Ramos, F. and Upcroft, B. (2016) Simple Online and Realtime Tracking. IEEE International Conference on Image Processing (ICIP), Phoenix: IEEE.CrossRefGoogle Scholar
Can, X. (2017). Research and simulation of information fusion technology for inland river AIS and VTS. Ship Science and Technology, 39(22), 49–51.Google Scholar
Cao, J., Chen, Q., Guo, J. and Shi, R. (2020). Attention-guided Context Feature Pyramid Network for Object Detection. Computer Vision and Pattern Recognition. Online: IEEE.Google Scholar
Chen, D., Yuan, Z., Wu, Y., Zhang, G. and Zheng, N. (2014). Constructing Adaptive Complex Cells for Robust Visual Tracking. IEEE International Conference on Computer Vision, Sydney: IEEE.Google Scholar
Chen, Z., Li, B., Tian, L. F. and Chao, D. (2017). Automatic Detection and Tracking of Ship Based on Mean Shift in Corrected Video Sequences. The 2nd International Conference on Image, Vision and Computing (ICIVC), Chengdu: IEEEGoogle Scholar
Chen, X., Xu, X., Yang, Y., Wu, H., Tang, J. and Zhao, J. (2020). Augmented ship tracking under occlusion conditions from maritime surveillance videos. IEEE Access, 8, 42884–42897.CrossRefGoogle Scholar
Cheng, Z., Zhilin, L. and University, H. E. (2019). Application of improved kernel correlation filtering algorithm in small ship dynamic object tracking. Applied Science and Technology, 46(01), 36–42.Google Scholar
Dong, C., Zheng, B., Li, B., Tian, L. F. and Liu, W. (2019). Shiptarget tracking with improved kernelized correlation filters. Optics and Precision Engineering, 27(4), 911–921.CrossRefGoogle Scholar
Dorai, Y., Chausse, F., Gazzah, S. and Amara, N. E. B. (2017). Multi Object Tracking by Linking Tracklets with aConvolutional Neural Network. International Conference on Computer Vision Theory and Applications, Porto: IEEE.Google Scholar
Frydenberg, S., Nordby, K. and Eikenes, J. O. (2018). Exploring designs of augmented reality systems for ship bridges in Arctic waters. Human Factors. London: RINA.Google Scholar
Girshick, R. (2015) Fast R-CNN. International Conference on Computer Vision (ICCV). Santiago: IEEE.CrossRefGoogle Scholar
Girshick, R., Donahue, J., Darrell, T. and Malik, J. (2014). Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation. IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Columbus: IEEE.Google Scholar
Grimaldi, M., Bechar, I., Lelore, T., Guis, V. and Bouchara, F. (2015). An Unsupervised Approach to Automatic Object Extraction From A Maritime Video Scene. 4th International Conference on Image Processing Theory, Tools and Applications (IPTA). Paris: IEEE.Google Scholar
Guang, Y., Qichao, L. and Feng, G. (2011). A Novel Ship Detection Method Based on Sea State Analysis From Optical Imagery. Sixth International Conference on Image and Graphics. Hefei: IEEE.Google Scholar
Guo, H., Yang, X., Wang, N., Song, B. and Gao, X. (2020). A rotational libra R-CNN method for ship detection. IEEE Transactions on Geoence and Remote Sensing, 58(8), 5772–5781.CrossRefGoogle Scholar
He, L., Yi, S., Mu, X. and Zhang, L. (2019). Ship Detection Method Based on Gabor Filter and Fast RCNN Model in Satellite Images of Sea. the 3rd International Conference on Computer Science and Application Engineering(CSAE), 111, 1–7.CrossRefGoogle Scholar
Henriques, J. F., Rui, C., Martins, P. and Batista, J. (2012). Exploiting the Circulant Structure of Tracking-by-Detection with Kernels. The 12th European Conference on Computer Vision (ECCV), Berlin: Springer.Google Scholar
Henriques, J. F., Caseiro, R., Martins, P. and Batista, J. (2015). High-Speed tracking with kernelized correlation filters. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(3), 583–596.CrossRefGoogle ScholarPubMed
Huang, Y., Li, Y., Zhang, Z. and Liu, W. (2020). GPU-Accelerated Compression and visualization of large-scale vessel trajectories in maritime IoT industries. IEEE Internet of Things Journal, 7(11), 10794–10812.CrossRefGoogle Scholar
Hugues, O., Cieutat, J. M. and Guitton, P. (2014). Real-time infinite horizon tracking with data fusion for augmented reality in a maritime operations context. Virtual Reality, 18(2), 129–138.CrossRefGoogle Scholar
Kartika, I. V., Siswandari, N. A and Puspitorini, O. (2018). Application of Genetic Algorithm for Placement of AIS (Automatic Identification System) Base Station. Electrical Power, Electronics, Communications, Controls and Informatics Seminar (EECCIS). Batu: IEEE.Google Scholar
Kim, H. T., Park, J. S. and Yu, Y. S. (2010). Ship detection using background estimation of video and AIS informations. Journal of the Korea Institute of Information and Communication Engineering, 14(12), 2636–2641.CrossRefGoogle Scholar
Lee, J. M., Lee, K. H. and Nam, B. (2016). Study on Image-Based Ship Detection for AR Navigation. 6th International Conference on IT Convergence and Security (ICITCS). Prague: IEEE.CrossRefGoogle Scholar
Li, Y. and Zhu, J. (2014). A scale adaptive kernel correlation filter tracker with feature integration. Lecture Notes in Computer Science, 8926, 254–265.CrossRefGoogle Scholar
Li, F., Tian, C., Zuo, W., Zhang, L. and Yang, M. H. (2018). Learning Spatial-Temporal Regularized Correlation Filters for Visual Tracking. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City: IEEE.Google Scholar
Liu, J. (2010). Moving ship detection and tracking from infrared image for collision avoidance of ships. Opto-Electronic Engineering, 37(9), 8–13.Google Scholar
Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C. Y. and Berg, A. C. (2016). SSD: Single Shot MultiBox Detector. European Conference on Computer Vision (ECCV). Amsterdam: Springer.Google Scholar
Liu, W., Nie, J., Garg, S., Xiong, Z. and Hossain, M. S. (2020). Data-Driven trajectory quality improvement for promoting intelligent vessel traffic services in 6G-enabled maritime IoT systems. IEEE Internet of Things Journal, 8(7), 5374–5385.CrossRefGoogle Scholar
Lukas, U., Vahl, M. and Mesing, B. (2014). Maritime Applications of Augmented Reality- Experiences and Challenges. Virtual, Augmented and Mixed Reality. Applications of Virtual and Augmented Reality - 6th International Conference. Berlin: Springer.Google Scholar
Oh, J., Park, S. and Kwon, O. S. (2016). Advanced navigation aids system based on augmented reality. International Journal of E Navigation & Maritime Economy, 5(C), 21–31.CrossRefGoogle Scholar
Pang, S., Coz, J. J. D., Yu, Z., Luaces, O. and Diez, J. (2017). Deep learning to frame objects for visual target tracking. Engineering Applications of Artificial Intelligence, 65(oct.), 406–420.CrossRefGoogle Scholar
Rao, A., Wang, H., Hu, Z. C. and Mullane, J. (2014). A Gaussian Particle Filter Based Factorised Solution to the Simultaneous Localization and Mapping Problem. Advanced Robotics and Its Social Impacts. Tokyo: IEEE.Google Scholar
Redmon, J., Divvala, S., Girshick, R. and Farhadi, A. (2016). You Only Look Once: Unified, Real-Time Object Detection. IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas: IEEECrossRefGoogle Scholar
Ren, S., He, K., Girshick, R. and Sun, J. (2017). Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6), 1137–1149.CrossRefGoogle ScholarPubMed
Shan, Y., Zhou, X., Liu, S., Zhang, Y. and Huang, K. (2020). Siamfpn: A deep learning method for accurate and real-time maritime ship tracking. IEEE Transactions on Circuits and Systems for Video Technology, doi:10.1109/TCSVT.2020.2978194Google Scholar
Vivone, G., Braca, P. and Horstmann, J. (2015). Knowledge-Based multiobject ship tracking for HF surface wave radar systems. IEEE Transactions on Geoence and Remote Sensing, 53(7), 3931–3949.CrossRefGoogle Scholar
Wang, Y., Wang, C., Hong, Z., Cheng, Z. and Fu, Q. (2017). Combing Single Shot Multibox Detector with Transfer Learning for Ship Detection Using Chinese Gaofen-3 Images. Progress in Electromagnetics Research Symposium - Fall (PIERS - FALL). Singapore: IEEE.Google Scholar
Wojke, N., Bewley, A. and Paulus, D. (2017). Simple Online and Realtime Tracking with A Deep Association Metric. IEEE International Conference on Image Processing (ICIP), Beijing: IEEE.Google Scholar
Xiao, Y. and Gang, X. (2011). Camshift ship tracking algorithm based on multi-feature adaptive fusion. Opto- Electronic Engineering, 38(5), 52–58.Google Scholar
Yang, M., Nie, X. and Liu, R. W. (2019). Coarse-to-Fine Luminance Estimation for Low-Light Image Enhancement in Maritime Video Surveillance. IEEE Intelligent Transportation Systems Conference (ITSC). Auckland: IEEE.Google Scholar