Hostname: page-component-848d4c4894-75dct Total loading time: 0 Render date: 2024-06-07T12:52:11.634Z Has data issue: false hasContentIssue false
  • English
  • Français

Hybrid methodology based on computational vision and sensor fusion for assisting autonomous UAV on offshore messenger cable transfer operation

Published online by Cambridge University Press:  31 January 2022

Gabryel S. Ramos Open the ORCID record for Gabryel S. Ramos [Opens in a new window] ,
Milena F. Pinto Open the ORCID record for Milena F. Pinto [Opens in a new window] ,
Fabricio O. Coelho ,
Leonardo M. Honório  and
Diego B. Haddad
Gabryel S. Ramos*
Affiliation:
Federal Center for Technological Education Celso Suckow da Fonseca (CEFET-RJ), Rio de Janeiro, Brazil
Milena F. Pinto
Affiliation:
Federal Center for Technological Education Celso Suckow da Fonseca (CEFET-RJ), Rio de Janeiro, Brazil
Fabricio O. Coelho
Affiliation:
Federal University of Juiz de Fora (UFJF), Juiz de Fora, Brazil
Leonardo M. Honório
Affiliation:
Federal University of Juiz de Fora (UFJF), Juiz de Fora, Brazil
Diego B. Haddad
Affiliation:
Federal Center for Technological Education Celso Suckow da Fonseca (CEFET-RJ), Rio de Janeiro, Brazil
*
*Corresponding author. E-mail: gabryelsr@gmail.com
Get access Rights & Permissions [Opens in a new window]

Abstract

The recent development of new offshore projects in pre-salt deepwater fields has placed offshore loading operations as the main production outflow alternative, increasing the operational complexity and risks. Numerous dangerous situations are associated with oil offloading, such as the messenger line transfer during the mooring stage. Nowadays, this critical task is realized by launching a thin messenger cable using the pneumatic line throwing apparatus. This is a complex and slow process since the operation usually occurs with the ship opposite to the wind. This work proposes a hybrid flight methodology based on computer vision and sensor fusion techniques for autonomous unmanned aerial vehicles (UAVs). The UAV takes off from an oil rig and precisely reaches a specific point in the shuttle tanker without using expensive positioning devices and augmenting UAV’s orientation (yaw) precision since the compass can suffer from severe interference due to naval metallic structures near the vehicle. The proposed framework was tested in a realistic simulated environment considering several practical operational constraints. The results demonstrated both the robustness and efficiency of the methodology.

Keywords

shuttle tanker vessel operationscable transfer operationautonomous offshore technologiescomputer visionUAV
Type
Research Article
Information
Robotica , Volume 40 , Issue 8 , August 2022 , pp. 2786 - 2814
Copyright
© The Author(s), 2022. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Bogue, R., “Robots in the offshore oil an d gas industries: A review of recent developments,” Ind. Robot Int. J. Rob. Res. Appl. 47(1), 1–6 (2019). CrossRefGoogle Scholar
Tan, J. H., Roberts, B., Sundararaju, P., Sintive, C., Facheris, L., Vanden Bosch, J., Lehning, V. and Pegg, M., “Transforming Offshore Oil and Gas Production Platforms into Smart Unmanned Installations,” Offshore Technology Conference Asia. Offshore Technology Conference (2020).CrossRefGoogle Scholar
Petillot, Y. R., Antonelli, G., Casalino, G. and Ferreira, F., “Underwater robots: From remotely operated vehicles to intervention-autonomous underwater vehicles,” IEEE Rob. Autom. Mag. 26(2), 94–101 (2019).CrossRefGoogle Scholar
Durdevic, P., Ortiz-Arroyo, D., Li, S. and Yang, Z., “Vision aided navigation of a quad-rotor for autonomous wind-farm inspection,” IFAC-PapersOnLine 52(8), 61–66 (2019a).CrossRefGoogle Scholar
Yu, L., Yang, E., Ren, P., Luo, C., Dobie, G., Gu, D. and Yan, X., “Inspection Robots in Oil and Gas Industry: A Review of Current Solutions and Future Trends,” 2019 25th International Conference on Automation and Computing (ICAC) (IEEE, 2019) pp. 1–6.CrossRefGoogle Scholar
Stavinoha, S., Chen, H., Walker, M., Zhang, B. and Fuhlbrigge, T., “Challenges of Robotics and Automation in Offshore Oil&Gas Industry,” The 4th Annual IEEE International Conference on Cyber Technology in Automation, Control and Intelligent (IEEE, 2014) pp. 557–562.CrossRefGoogle Scholar
Menegaldo, L. L., Ferreira, G. A. N., Santos, M. F. and Guerato, R. S., “Development and navigation of a mobile robot for floating production storage and offloading ship hull inspection,” IEEE Trans. Ind. Electron. 56(9), 3717–3722 (2009). doi: 10.1109/TIE.2009.2025716.CrossRefGoogle Scholar
Oil Companies International Marine Forum OCIMF, Guidelines for Offshore Tanker Operations (Oil Companies lnternational Marine Forum, London, UK, 2018).Google Scholar
Sphaier, S. H., Amaral Netto, J. D. do, Oliveira, L. F., Junior, J. S. S., Henrique Sá Correa da Silva, S. and Henaut, M. P. G., “On the safety of offloading operation in the oil industry,” Int. J. Comput. Appl. Technol. 43(3), 207 (2012). doi: 10.1504/ijcat.2012.046307.CrossRefGoogle Scholar
Rodriguez, C. E. P., de Souza, G. F. M., Gilberto, F. M. and Martins, M. R., “Risk-based Analysis of Offloading Operations with FPSO Production Units,” Proceedings of the 20th International Congress of Mechanical Engineering. Gramado, Brazil (2009).Google Scholar
Faiz Ahmed, S., Kushsairy, K., Bakar, M. I. A., Hazry, D. and Kamran Joyo, M., “Attitude Stabilization of Quad-Rotor (UAV) System Using Fuzzy PID Controller (an Experimental Test),” 2015 Second International Conference on Computing Technology and Information Management (ICCTIM) (IEEE, 2015) pp. 99–104.CrossRefGoogle Scholar
Pinto, M. F., Honório, L. M., Marcato, A. L. M., Dantas, M. A. R., Melo, A. G., Capretz, M. and Urdiales, C., “Arcog: An aerial robotics cognitive architecture,” Robotica 39(3), 1–20 (2020a).Google Scholar
Pinto, M. F., Honorio, L. M., Melo, A. and Marcato, A. L. M., “A robotic cognitive architecture for slope and dam inspections,” Sensors 20(16), 4579 (2020b).CrossRefGoogle Scholar
Rodin, C. D., de Lima, L. N., de Alcantara Andrade, F. A., Haddad, D. B., Johansen, T. A. and Storvold, R., “Object Classification in Thermal Images Using Convolutional Neural Networks for Search and Rescue Missions with Unmanned Aerial Systems,” 2018 International Joint Conference on Neural Networks (IJCNN) (2018) pp. 1–8. doi: 10.1109/IJCNN.2018.8489465.Google Scholar
Andrade, F. A. A., Hovenburg, A., L. N. de de Lima, C. D. Rodin, T. A. Johansen, R. Storvold, C. A. M. Correia and D. B. Haddad, “Autonomous unmanned aerial vehicles in search and rescue missions using real-time cooperative model predictive control,” Sensors 19(19), 4067 (2019). ISSN 1424-8220. doi: 10.3390/s19194067.CrossRefGoogle Scholar
Silva, M. F., Ribeiro, A. C., Santos, M. F., Carmo, M. J., Honório, L. M., Oliveira, E. J. and Vidal, V. F., “Design of Angular Pid Controllers for Quadcopters Built with Low Cost Equipment,” 2016 20th International Conference on System Theory, Control and Computing (ICSTCC) (IEEE, 2016) pp. 216–221.Google Scholar
Mullenders, V., Design of a motion monitoring system for unmanned offshore topside installation based on real-time visual object tracking using drones and fixed cameras on an SSCV (2019).Google Scholar
Selecký, M., Rollo, M., Losiewicz, P., Reade, J. and Maida, N., “Framework for Incremental Development of Complex Unmanned Aircraft Systems,” 2015 Integrated Communication, Navigation and Surveillance Conference (ICNS) (2015) pp. J3–1–J3–9.Google Scholar
Faria Pinto, M., Melo, A., Marcato, A. and Urdiales, C., “Case-based Reasoning Approach Applied to Surveillance System using an Autonomous Unmanned Aerial Vehicle,” (2017) pp. 1324–1329. doi: 10.1109/ISIE.2017.8001437.CrossRefGoogle Scholar
Pinto, M. F., Marcato, A. L. M., Melo, A. G., Honório, L. M. and Urdiales, C., “A framework for analyzing fog-cloud computing cooperation applied to information processing of UAVs,” Wireless Commun. Mobile Comput. 2019, 14 (2019).Google Scholar
Coelho, F. O., Carvalho, J. P., Pinto, M. F. and Marcato, A. L., “EKF and Computer Vision for Mobile Robot Localization,” 2018 13th APCA International Conference on Automatic Control and Soft Computing (CONTROLO) (IEEE, 2018)pp. 148–153.Google Scholar
Guo, Y., Miao, L. and Zhang, X., “Quantitative Research on GPS Positioning in an East-North-up Coordinate System,” 2019 Chinese Control Conference (CCC) (2019) pp. 3958–3963. doi: 10.23919/ChiCC.2019.8866190.CrossRefGoogle Scholar
SPE Society of Petroleum Engineers, Tugboat company takes its business flying with drones (2018). https://jpt.spe.org/tugboat-company-takes-its-business-flying-drones. Accessed: 2020-06-15.Google Scholar
Kotug, Drone technology for safer tug operations (2018). https://www.kotug.com/newsmedia/drone-technology-safer-tug-operations. Accessed: 2020-06-15.Google Scholar
Frederiksen, M. H. and Knudsen, M. P., Drones for Offshore and Maritime Missions: Opportunities and Barriers. Center for Integrative Innovation Management, SDU, April 2018.Google Scholar
Kim, G., Park, H. and Zhang, X., “A Method on Using Drone for Connecting Messenger Lines in Power Transmission Line Construction,” 2019 International Conference on Information and Communication Technology Convergence (ICTC) (2019) pp. 731–736. doi: 10.1109/ICTC46691.2019.8939933.CrossRefGoogle Scholar
Tomaštík, J., Mokroš, M., Surový, P., Grznárová, A. and Merganič, J., “UAV RTK/PPK method—an optimal solution for mapping inaccessible forested areas?,” Remote Sens. 11(6) (2019). ISSN 2072-4292. doi: 10.3390/rs11060721.CrossRefGoogle Scholar
Gryte, K., Hansen, J. M., Johansen, T. and Fossen, T. I., Robust Navigation of UAV using Inertial Sensors Aided by UWB and RTK GPS. doi: 10.2514/6.2017-1035.CrossRefGoogle Scholar
Casella, E., Drechsel, J., Winter, C., Benninghoff, M. and Rovere, A., “Accuracy of sand beach topography surveying by drones and photogrammetry,” Geo-Marine Lett. 40(2), 255–268 (2020). ISSN 1432-1157. doi: 10.1007/s00367-020-00638-8.CrossRefGoogle Scholar
Zimmermann, F., Eling, C., Klingbeil, L. and Kuhlmann, H., Precise positioning of UAVs - dealing with challenging RTK-GPS measurement conditions during automated UAV flights,” ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci. 42W3, 95–102 (2017). doi: 10.5194/isprs-annals-IV-2-W3-95-2017.CrossRefGoogle Scholar
Durdevic, P., Ortiz-Arroyo, D., Li, S. and Yang, Z., “Vision aided navigation of a quad-rotor for autonomous wind-farm inspection,” IFAC-PapersOnLine 52(8), 61–66 (2019 b). ISSN 2405-8963. doi: 10.1016/j.ifacol.2019.08.049. https://www.sciencedirect.com/science/article/pii/S2405896319303696. 10th IFAC Symposium on Intelligent Autonomous Vehicles IAV 2019.CrossRefGoogle Scholar
Qin, X. and Wang, T., “Visual-based Tracking and Control Algorithm Design for Quadcopter UAV,” 2019 Chinese Control and Decision Conference (CCDC) (2019) pp. 3564–3569. doi: 10.1109/CCDC.2019.8832545.CrossRefGoogle Scholar
Chuang, H.-M., He, D. and Namiki, A., “Autonomous target tracking of UAV using high-speed visual feedback,” Appl. Sci. 9(21) (2019). ISSN 2076-3417. doi: 10.3390/app9214552.CrossRefGoogle Scholar
Zhang, Y., Li, Q.-Z. and Zang, F.-N., “Ship detection for visual maritime surveillance from non-stationary platforms,” Ocean Eng. 141, 53–63 (2017). ISSN 0029-8018. doi: 10.1016/j.oceaneng.2017.06.022. https://www.sciencedirect.com/science/article/pii/S0029801817303190.CrossRefGoogle Scholar
Pitsikalis, M., Do, T.-T., Lisitsa, A. and Luo, S., “Logic Rules Meet Deep Learning: A Novel Approach for Ship Type Classification,” International Joint Conference on Rules and Reasoning (Springer, 2021) pp. 203–217.CrossRefGoogle Scholar
Zhao, H., Zhang, W., Sun, H. and Xue, B., “Embedded deep learning for ship detection and recognition,” Future Internet 11(2), 53 (2019).CrossRefGoogle Scholar
Koenig, N. and Howard, A., “Design and Use Paradigms for Gazebo, an Open-Source Multi-robot Simulator,” 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)(IEEE Cat. No. 04CH37566), vol. 3 (IEEE, 2004)pp. 2149–2154.Google Scholar
Silva Ramos, G. and Faria Pinto, M., Shuttle tanker, FPSO and FSO image dataset for ship recognition (2021).doi: 10.21227/9xzy-fd09.CrossRefGoogle Scholar
Karney, C. F. F. and Deakin, R. E., “F.w. bessel (1825): The calculation of longitude and latitude from geodesic measurements,” Astronomical Notes 331(8), 852–861 (2010).Google Scholar
Veritas, D. N., Recommended practice dnv-rp-h103: Modelling and analysis of marine operations. https://home.hvl.no/ansatte/tct/FTP/H2020%20Marinteknisk%20Analyse/Regelverk%20og%20standarder/DnV_documents/RP-H103.pdf (2020).Google Scholar
Chang, P. and Padır, T., “Model-based manipulation of linear flexible objects: Task automation in simulation and real world,” Machines 8(3) (2020). ISSN 2075-1702. doi: 10.3390/machines8030046.CrossRefGoogle Scholar
Hiemann, A., Kautz, T., Zottmann, T. and Hlawitschka, M., “Enhancement of speed and accuracy trade-off for sports ball detection in videos—finding fast moving, small objects in real time,” Sensors 21(9), 2021. ISSN 1424-8220. doi: 10.3390/s21093214.CrossRefGoogle ScholarPubMed
Choi, J., Chun, D., Kim, H. and Lee, H.-J., “Gaussian Yolov3: An Accurate and Fast Object Detector Using Localization Uncertainty for Autonomous Driving,” Proceedings of the IEEE International Conference on Computer Vision (2019)pp. 502–511.Google Scholar
Gonzalez, M., Kacete, A., Murienne, A. and Marchand, E., Yoloff: You only learn offsets for robust 6DOF object pose estimation. arXiv preprint arXiv:2002.00911 (2020).CrossRefGoogle Scholar
Airbus, Airbus ship detection challenge. https://www.kaggle.com/c/airbus-ship-detection/data (2021). Accessed in 08/03/2020.Google Scholar
Redmon, J., Divvala, S., Girshick, R. and Farhadi, A., “You Only Look Once: Unified, Real-Time Object Detection,” In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016) pp. 779–788. doi: 10.1109/CVPR.2016.91.CrossRefGoogle Scholar
Padilla, R., Netto, S. L. and da Silva, E. A. B., “A Survey on Performance Metrics for Object-Detection Algorithms,” 2020 International Conference on Systems, Signals and Image Processing (IWSSIP) (2020) pp. 237–242.Google Scholar
Kalman, R. E., “A new approach to linear filtering and prediction problems”, ASME. J. Basic Eng. March 82(1), 35–45 (1960).CrossRefGoogle Scholar
Mao, G., Drake, S. and Anderson, B. D. O., “Design of an Extended Kalman Filter for UAV Localization,” In: 2007 Information, Decision and Control (2007) pp. 224–229. doi: 10.1109/IDC.2007.374554.CrossRefGoogle Scholar
Zhang, T. and Liao, Y., “Attitude measure system based on extended kalman filter for multi-rotors,” Comput. Electron. Agric. 134, 19–26 (2017). ISSN 0168-1699. doi: 10.1016/j.compag.2016.12.021. https://www.sciencedirect.com/science/article/pii/S0168169916312418.CrossRefGoogle Scholar
Brescianini, D., Hehn, M. and D’Andrea, R., Nonlinear quadrocopter attitude control: Technical report. Technical report (2013). http://hdl.handle.net/20.500.11850/154099.Google Scholar
PixHawk, Controller diagrams. https://docs.px4.io/master/en/flight_stack/controller_diagrams.html (2021). Accessed in 05/26/2021.Google Scholar
Garrido-Jurado, S., Muñoz-Salinas, R., Madrid-Cuevas, F. J. and Marn-Jiménez, M. J., “Automatic generation and detection of highly reliable fiducial markers under occlusion,” Pattern Recognit. 47(6), 2280–2292 (2014).CrossRefGoogle Scholar
Blanger, L. and Hirata, N. S. T., “An Evaluation of Deep Learning Techniques for QR Code Detection,” 2019 IEEE International Conference on Image Processing (ICIP) (2019) pp. 1625–1629. doi: 10.1109/ICIP.2019.8803075.CrossRefGoogle Scholar
Rezatofighi, H., Tsoi, N., Gwak, J., Sadeghian, A., Reid, I. and Savarese, S., “Generalized intersection over union: A Metric and a Loss for Bounding Box Regression”, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 658–666 (2019).Google Scholar
CVAT, Computer vision annotation tool. https://cvat.org/ (2021). Accessed in 01/09/2020.Google Scholar
VulcanUAV, Aircraft series. http://vulcanuav.com/aircraft/ (2021). Accessed in 02/28/2021.Google Scholar
Rikimaru, Rp9 professional pe. http://www.fishingc.com/productinfo/99929.html (2020). Accessed in 10/16/2020.Google Scholar
Yuneec, Typhoon h user manual v1.1. https://static.bhphotovideo.com/lit_files/319649.pdf (2020). Accessed in 10/16/2020.Google Scholar