Hostname: page-component-76d6cb85b7-jhrpq Total loading time: 0 Render date: 2026-07-22T05:20:28.442Z Has data issue: false hasContentIssue false
  • English
  • Français

Addressing the deadlock problem of the optimal reciprocal collision avoidance algorithm in distributed multi-robot navigation

Published online by Cambridge University Press:  11 June 2026

Zhihao Liu Open the ORCID record for Zhihao Liu [Opens in a new window] ,
Wenjie Na ,
Chenpeng Yao Open the ORCID record for Chenpeng Yao [Opens in a new window] ,
Chengju Liu  and
Qijun Chen
Zhihao Liu
Affiliation:
College of Electronic and Information Engineering, Tongji University , China
Wenjie Na
Affiliation:
College of Electronic and Information Engineering, Tongji University , China
Chenpeng Yao*
Affiliation:
College of Electronic and Information Engineering, Tongji University , China
Chengju Liu
Affiliation:
College of Electronic and Information Engineering, Tongji University , China Shanghai Institute of Intelligent Science and Technology, China
Qijun Chen
Affiliation:
College of Electronic and Information Engineering, Tongji University , China
*
Corresponding author: Chenpeng Yao; Email: 1710823@tongji.edu.cn
Get access Rights & Permissions [Opens in a new window]

Abstract

In multi-robot collision avoidance navigation problems, it is common to encounter situations where robots get stuck and cannot avoid collisions or complete navigation tasks, known as deadlock problems. There are various reasons for the occurrence of deadlock problems, such as issues with the navigation method itself. Optimal reciprocal collision avoidance (ORCA) is widely used in multi-robot navigation. However, in some scenes, such as where robots distribute symmetrically, ORCA usually fails to plan reasonable collision avoidance velocities, resulting in the robot getting stuck. In this paper, we aim to address the deadlock problem caused by ORCA. We first analyze and simplify the planning process of ORCA from the principal horizon to break the deadlock situation. Then, deep reinforcement learning is used to optimize the escape velocity and collision avoidance responsibility of ORCA to guarantee collision-free navigation. We compare our method with the original ORCA and other learning-based methods. The results show that the proposed method is beneficial in reducing the deadlock probability and the collision probability in multi-robot navigation.

Keywords

robot navigationdeep reinforcement learningmulti-agent systems

Information

Type
Research Article
Information
Robotica , First View , pp. 1 - 19
Copyright
© The Author(s), 2026. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

Han, R., Chen, S., Wang, S., Zhang, Z., Gao, R., Hao, Q. and Pan, J., “Reinforcement learned distributed multi-robot navigation with reciprocal velocity obstacle shaped rewards,” IEEE Robot. Autom. Lett. 7(3), 5896–5903 (2022).10.1109/LRA.2022.3161699CrossRefGoogle Scholar
Berg, J. V. D., Guy, S. J., Lin, M. C. and Manocha, D., “Reciprocal n-Body Collision Avoidance,” 14th International Symposium of Robotic Research, ISRR 2009 (2011).Google Scholar
Fraichard, T. and Levesy, V., “From crowd simulation to robot navigation in crowds,” IEEE Rob. Autom. Lett. 5(2), 729–735 (2020).10.1109/LRA.2020.2965032CrossRefGoogle Scholar
Guo, K., Wang, D., Fan, T. and Pan, J., “VR-ORCA: Variable responsibility optimal reciprocal collision avoidance,” IEEE Rob. Autom. Lett. 6(3), 4520–4527 (2021).10.1109/LRA.2021.3067851CrossRefGoogle Scholar
Chen, L., Wang, Y., Miao, Z., Feng, M., Zhou, Z., Wang, H. and Wang, D., “Toward safe distributed multi-robot navigation coupled with variational Bayesian model,” IEEE Trans. Autom. Sci. Eng. 21(4), 7583–7598 (2023).10.1109/TASE.2023.3346049CrossRefGoogle Scholar
Zhai, Y., Ding, B., Liu, X., Jia, H., Zhao, Y. and Luo, J., “Decentralized multi-robot collision avoidance in complex scenarios with selective communication,” IEEE Rob. Autom. Lett. 6(4), 8379–8386 (2021).10.1109/LRA.2021.3102636CrossRefGoogle Scholar
Everett, M., Chen, Y. F. and How, J. P., “Motion Planning Among Dynamic, Decision-Making Agents with Deep Reinforcement Learning,” 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2018) pp. 3052–3059.Google Scholar
van den Berg, J., Lin, M. and Manocha, D., “Reciprocal Velocity Obstacles for Real-Time Multi-Agent Navigation,” 2008 IEEE International Conference on Robotics and Automation (2008) pp. 1928–1935.Google Scholar
Li, H., Weng, B., Gupta, A., Pan, J. and Zhang, W., “Reciprocal collision avoidance for general nonlinear agents using reinforcement learning. ArXiv, abs/1910.10887 (2019).Google Scholar
Feng, Z., Xue, B., Wang, C. and Zhou, F., “Safe and socially compliant robot navigation in crowds with fast-moving pedestrians via deep reinforcement learning,” Robotica 42(4), 1212–1230 (2024).10.1017/S0263574724000183CrossRefGoogle Scholar
Tang, W., Zhou, Y., Zhang, T., Liu, Y., Liu, J. and Ding, Z., “Cooperative collision avoidance in multirobot systems using fuzzy rules and velocity obstacles,” Robotica 41(2), 668–689 (2023).10.1017/S0263574722001515CrossRefGoogle Scholar
Wu, F., Tang, W., Zhou, Y., Lin, S. W., Ding, Z. and Liu, Y., “Robust motion planning for mobile robots under attacks against obstacle localization,” Robotica 42(8), 2781–2800 (2024).10.1017/S0263574724001115CrossRefGoogle Scholar
Luo, Y., Cai, P., Bera, A., Hsu, D., Lee, W. S. and Manocha, D., “Porca: Modeling and planning for autonomous driving among many pedestrians,” IEEE Rob. Autom. Lett. 3(4), 3418–3425 (2018).10.1109/LRA.2018.2852793CrossRefGoogle Scholar
Snape, J. and Manocha, D., “Navigating Multiple Simple-Airplanes in 3D Workspace,“ 2010 IEEE International Conference on Robotics and Automation (IEEE, 2010) pp. 3974–3980.10.1109/ROBOT.2010.5509580CrossRefGoogle Scholar
Yin, Z., Liu, J. and Wang, L., “Less-Effort Collision Avoidance in Virtual Pedestrian Simulation,” Proceedings of the 2019 International Conference on Artificial Intelligence and Computer Science 2019 (Association for Computing Machinery, New York, NY, USA, 2019) pp. 488–493.10.1145/3349341.3349456CrossRefGoogle Scholar
Wei, B. and Zhang, X., “Frvo: A Filter Enhanced Interaction Model for Pedestrian Path Prediction in Crowded Scenarios,” 2019 Chinese Automation Congress (CAC) (IEEE, 2019) pp. 538–543.10.1109/CAC48633.2019.8997371CrossRefGoogle Scholar
Xie, Z. and Dames, P., “DRL-VO: Learning to navigate through crowded dynamic scenes using velocity obstacles,” IEEE Trans. Rob. 39(4), 2700–2719 (2023).10.1109/TRO.2023.3257549CrossRefGoogle Scholar
Zhou, Z., Zeng, Z., Lang, L., Yao, W., Lu, H., Zheng, Z. and Zhou, Z., “Navigating robots in dynamic environment with deep reinforcement learning,” IEEE Trans. Intell. Transp. 23(12), 25201–25211 (2022).10.1109/TITS.2022.3213604CrossRefGoogle Scholar
Qin, J., Qin, J., Qiu, J., Liu, Q., Li, M. and Ma, Q., “SRL-ORCA: A socially aware multi-agent mapless navigation algorithm in complex dynamic scenes,” IEEE Rob. Autom. Lett. 9(1), 143–150 (2024).10.1109/LRA.2023.3331621CrossRefGoogle Scholar
Schulman, J., Wolski, F., Dhariwal, P., Radford, A. and Klimov, O., “Proximal policy optimization algorithms. CoRR, abs/1707.06347 (2017).Google Scholar
Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., Lin, Z., Desmaison, A., Antiga, L. and Lerer, A., Automatic differentiation in PyTorch, NIPS Workshop (2017).Google Scholar
Chen, C., Liu, Y., Kreiss, S. and Alahi, A., “Crowd-Robot Interaction: Crowd-Aware Robot Navigation with Attention-based Deep Reinforcement Learning,“ International Conference on Robotics and Automation (ICRA) (2019) pp. 6015–6022.Google Scholar
Quigley, M., Conley, K., Gerkey, B., Faust, J., Foote, T., Leibs, J., Berger, E., Wheeler, R., Ng, A. Y., Ros: An open-source robot operating system, ICRA workshop on open source software, 3(3.2), 5, (2009).Google Scholar
Koenig, N. and Howard, A., “Design and Use Paradigms for Gazebo, An Open-Source Multi-Robot Simulator,” 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566), vol. 3 (2004) pp. 2149–2154.Google Scholar
Chen, L., Wang, Y., Miao, Z., Feng, M., Zhou, Z., Wang, H. and Wang, D., “Reciprocal velocity obstacle spatial–temporal network for distributed multirobot navigation,” IEEE Trans. Ind. Electron. 71(11), 14470–14480 (2024).10.1109/TIE.2024.3379630CrossRefGoogle Scholar
Supplementary material: File

Liu et al. supplementary material

Liu et al. supplementary material
Download Liu et al. supplementary material(File)
File 46.3 MB