Hostname: page-component-77f85d65b8-jkvpf Total loading time: 0 Render date: 2026-04-14T06:38:57.909Z Has data issue: false hasContentIssue false
  • English
  • Français

A 2-stage vision-based localization methodology for efficient automatic charging of electric vehicles in uncertain environments

Published online by Cambridge University Press:  04 August 2025

Qi Chen Open the ORCID record for Qi Chen [Opens in a new window] ,
Kai Wu ,
Yong Zhong ,
Weihua Li  and
Mingfeng Wang Open the ORCID record for Mingfeng Wang [Opens in a new window]
Qi Chen
Affiliation:
South China University of Technology, SHIEN-MING WU School of Intelligent Engineering, Guangzhou 510000, China
Kai Wu*
Affiliation:
South China University of Technology, SHIEN-MING WU School of Intelligent Engineering, Guangzhou 510000, China
Yong Zhong
Affiliation:
South China University of Technology, SHIEN-MING WU School of Intelligent Engineering, Guangzhou 510000, China
Weihua Li
Affiliation:
South China University of Technology, School of Mechanical and Automotive Engineering, Guangzhou 510000, China
Mingfeng Wang
Affiliation:
Department of Mechanical and Aerospace Engineering, Brunel University London, London, UB83PH UK
*
Corresponding author: Kai Wu; Email: whphwk@scut.edu.cn
Get access Rights & Permissions [Opens in a new window]

Abstract

Automatic visual localization of electric vehicle (EV) charging ports presents significant challenges in uncertain environments, such as varying surface textures, reflections, lighting and observation distance. Existing methods require extensive real-world training data and well-focused images to achieve robust and accurate localization. However, both requirements are difficult to meet under variable and unpredictable conditions. This paper proposes a 2-stage vision-based localization approach. Firstly, the image synthesis technique is used to reduce the cost of real-world data collection. A task-oriented parameterization protocol (TOPP) is proposed to optimize the quality of the synthetic images. Secondly, an autofocus and servoing strategy is proposed. A hybrid detector is employed to enhance sharpness assessment performance, while a visual servoing method based on single exponential smoothing (SES) is developed to enhance stability and efficiency during the search process. Experiments were conducted to evaluate image synthesis efficiency, detection accuracy, and servoing performance. The proposed method achieved 99% detection accuracy on the real-world port images, and guided the robot to the optimal imaging position within 16 s, outperforming comparable approaches. These results highlight its potential for robust automated charging in real-world scenarios.

Keywords

automatic charging robotsynthetic imagesSim2real transfer learningvision-based servoing

Information

Type
Research Article
Information
Robotica , Volume 43 , Issue 8 , August 2025 , pp. 2992 - 3010
Copyright
© The Author(s), 2025. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

Ismail, A., Mehri, M., Sahbani, A. and Amara, N. E. B., “A dual-stage system for real-time license plate detection and recognition on mobile security robots,” Robotica, pp. 1–22 (2025). Available at https://www.cambridge.org/core/journals/robotica/article/dualstage-system-for-realtime-license-plate-detection-and-recognition-on-mobile-security-robots/C32711F3F95E1BA4496233500801F474 Google Scholar
Pan, M., Sun, C., Liu, J. and Wang, Y., “Automatic recognition and location system for electric vehicle charging port in complex environment,” IET Image Process 14(10), 2263–2272 (2020).CrossRefGoogle Scholar
Lv, X., Chen, G., Hu, H. and Lou, Y., “A Robotic Charging Scheme for Electric Vehicles Based on Monocular Vision and Force Perception,” In: IEEE International Conference on Robotics and Biomimetics (ROBIO) (2019) pp. 2958–2963.Google Scholar
Tadic, V., “Study on automatic electric vehicle charging socket detection using ZED 2i depth sensor,” Electronics 12(4), 4 (2023).CrossRefGoogle Scholar
Quan, P., Lou, Y., Lin, H., Liang, Z. and Di, S., “Research on fast identification and location of contour features of electric vehicle charging port in complex scenes,” IEEE Access 10, 26702–26714 (2022).CrossRefGoogle Scholar
Quan, P., Lou, Y., Lin, H., Liang, Z., Wei, D. and Di, S., “Research on fast recognition and localization of an electric vehicle charging port based on a cluster template matching algorithm,” Sensors-Basel 22(9), 3599 (2022).CrossRefGoogle ScholarPubMed
A.R.G., S. and L.P.P., S., “Design of Automatic Charging System for Electric Vehicles using Rigid-Flexible Manipulator,” In: 2nd International Conference on Power, Control and Computing Technologies (ICPC2T) (2022) pp. 1–6.Google Scholar
Zhang, J., Geng, T., Xu, J., Li, Y. and Zhang, C., “Electric Vehicle Charging Robot Charging Port Identification Method Based On Multi-Algorithm Fusion,” In: Intelligent Robotics and Applications, Liu, X.-J., Nie, Z. Yu, J. and Song, R., eds.) Springer International Publishing, Cham, 2021) pp. 680–693.CrossRefGoogle Scholar
Guo, D., Xie, L., Yu, H., Wang, Y. and Xiong, R.. Electric Vehicle Automatic Charging System Based on Vision-force Fusion. In: IEEE International Conference on Robotics and Biomimetics (ROBIO) (2021) pp. 405–410.Google Scholar
Ding, W., Pei, Z., Yang, T. and Chen, T., “Dynamic simultaneous localization and mapping based on object tracking in occluded environment,” Robotica 42(7), 2209–2225 (2024).CrossRefGoogle Scholar
Csurka, G., “Domain adaptation for visual applications: A comprehensive survey (2017). arXiv:1702.05374.Google Scholar
Tobin, J., Fong, R., Ray, A., Schneider, J., Zaremba, W. and Abbeel, P.. Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2017) pp. 23–30.Google Scholar
Tsirikoglou, A., Kronander, J., Wrenninge, M. and Unger, J., “Procedural modeling and physically based rendering for synthetic data generation in automotive applications (2017). arXiv:1710.06270.Google Scholar
Niu, S., Li, B., Wang, X. and Lin, H., “Defect image sample generation with GAN for improving defect recognition,” IEEE Trans. Autom. Sci. Eng. 17(3), 1–12 (2020).CrossRefGoogle Scholar
Bang, Y., Lee, Y. and Kang, B., “Image-to-image translation-based data augmentation for robust EV charging inlet detection,” IEEE Robot. Autom. Lett. 7(2), 3726–3733 (2022).CrossRefGoogle Scholar
Jing, X., Qian, K., Jianu, T. and Luo, S., “Unsupervised adversarial domain adaptation for sim-to-real transfer of tactile images,” IEEE Trans. Instrument. Meas. 72, 1–11 (2023).CrossRefGoogle Scholar
Chen, H., Wan, W., Matsushita, M., Kotaka, T. and Harada, K., “Automatically prepare training data for YOLO using robotic in-hand observation and synthesis,” IEEE Trans. Autom. Sci. Eng. 21(3), 1–17 (2024).CrossRefGoogle Scholar
Dirr, J., Bauer, J. C., Gebauer, D. and Daub, R., “Cut-paste image generation for instance segmentation for robotic picking of industrial parts,” Int. J. Adv. Manuf. Technol. 130(1), 191–201 (2024).CrossRefGoogle Scholar
Zhang, Y., Yi, J., Chen, Y., Dai, Z., Han, F. and Cao, S., “Pose estimation for workpieces in complex stacking industrial scene based on RGB images,” Appl. Intell. 52(8), 8757–8769 (2022).CrossRefGoogle Scholar
Zeng, Y., Teoh, Y. S., Zhu, G., Toh, E. and Chen, I.-M., “Vision-based food handling system for high-resemblance random food items,” Robotica 42(7), 2108–2124 (2024).CrossRefGoogle Scholar
Gong, Y., Liu, M., Wang, X., Liu, C. and Hu, J., “Few-shot defect detection using feature enhancement and image generation for manufacturing quality inspection,” Appl. Intell. 54(1), 375–397 (2024).CrossRefGoogle Scholar
Todorov, E., Erez, T. and Tassa, Y.. MuJoCo: A Physics Engine for Model-Based Control. In: 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems (2012) pp. 5026–5033.Google Scholar
Triyonoputro, J. C., Wan, W. and Harada, K., “Quickly Inserting Pegs into Uncertain Holes using Multi-view Images and Deep Network Trained on Synthetic Data,” In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2019) pp. 5792–5799.Google Scholar
Xiao, R., Yang, C., Jiang, Y. and Zhang, H., “One-shot sim-to-real transfer policy for robotic assembly via reinforcement learning with visual demonstration,” Robotica 42(4), 1074–1093 (2024).CrossRefGoogle Scholar
Barisic, A., Petric, F. and Bogdan, S., “Sim2Air – synthetic aerial dataset for UAV monitoring,” IEEE Robot. Autom. Lett. 7(2), 3757–3764 (2022).CrossRefGoogle Scholar
Horváth, D., Erdős, G., Istenes, Z., Horváth, T. and Földi, S., “Object detection using Sim2Real domain randomization for robotic applications,” IEEE Trans. Robot. 39(2), 1–19 (2022).Google Scholar
Singh, R., Seneviratne, L. and Hussain, I., “Robust pollination for tomato farming using deep learning and visual servoing,” Robotica 43(1), 1–24 (2024).Google Scholar
Kollar, T., Laskey, M., Stone, K., Thananjeyan, B. and Tjersland, M., “SimNet: Enabling robust unknown object manipulation from pure synthetic data via stereo (2021). arXiv:2106.16118.Google Scholar
Prakash, A., Boochoon, S., Brophy, M., Acuna, D., Cameracci, E., State, G., Shapira, O. and Birchfield, S., “Structured Domain Randomization: Bridging the Reality Gap by Context-Aware Synthetic Data,” In: International Conference on Robotics and Automation (ICRA) (2019) pp. 7249–7255.Google Scholar
Qin-zhang, W., “An adaptive autofocus method using no-reference structural sharpness,” Opto-Electron. Eng. 38(2), 84–89 (2011).Google Scholar
Li, S., Chen, J., Wan, J., Li, Z. and Lin, L., “Image Definition Evaluation Function Based On Improved Maximum Local Variation and Focusing Window Selection,” In: Data Mining and Big Data (Tan, Y., Shi, Y., Zomaya, A., Yan, H. and Cai, J., eds.) (Springer, Singapore, 2021) pp. 80–89,CrossRefGoogle ScholarPubMed
Jia, D., Zhang, C., Wu, N., Zhou, J. and Guo, Z., “Autofocus algorithm using optimized Laplace evaluation function and enhanced mountain climbing search algorithm,” Multimed. Tools Appl. 81(7), 10299–10311 (2022).CrossRefGoogle Scholar
Tremblay, J., Prakash, A., Acuna, D., Brophy, M., Jampani, V., Anil, C., To, T., Cameracci, E., Boochoon, S. and Birchfield, S.. Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (2018) pp. 969–977. Accessed Jan 2023.Google Scholar
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D. and Batra, D., “Grad-CAM: Visual explanations from deep networks via gradient-based localization,” Int. J. Comput. Vis. 128(2), 336–359 (2020).CrossRefGoogle Scholar
Waqas Zamir, S., Arora, A., Gupta, A., Khan, S., Sun, G., Shahbaz Khan, F., Zhu, F., Shao, L., Xia, G. S. and Bai, X., “iSAID: A Large-scale Dataset for Instance Segmentation in Aerial Images,” In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (2019) pp. 28–37.Google Scholar