Hostname: page-component-848d4c4894-ttngx Total loading time: 0 Render date: 2024-06-09T13:22:02.642Z Has data issue: false hasContentIssue false
  • English
  • Français

Navigation for multi-humanoid using MFO-aided reinforcement learning approach

Published online by Cambridge University Press:  30 September 2022

Abhishek Kumar Kashyap Open the ORCID record for Abhishek Kumar Kashyap [Opens in a new window] ,
Dayal R. Parhi  and
Vikas Kumar
Abhishek Kumar Kashyap*
Affiliation:
Robotics Laboratory, Mechanical Engineering Department, National Institute of Technology, Rourkela, Odisha 769008, India Robotics Laboratory, Mechanical Engineering Department, MIT Art Design & Technology University, Pune, Maharashtra 412201, India
Dayal R. Parhi
Affiliation:
Robotics Laboratory, Mechanical Engineering Department, National Institute of Technology, Rourkela, Odisha 769008, India
Vikas Kumar
Affiliation:
Robotics Laboratory, Mechanical Engineering Department, National Institute of Technology, Rourkela, Odisha 769008, India
*
*Corresponding author. E-mail: Akkashyapmech@gmail.com
Get access Rights & Permissions [Opens in a new window]

Abstract

The given article emphasizes the development and modeling of a hybrid navigational controller to optimize the path length and time taken. The proposed navigational controller is developed by hybridizing the metaheuristic moth–flame optimization (MFO) approach and the reinforcement learning (RL) approach. Input parameters like obstacle and target locations are fed to the MFO controller that implements a proper navigational direction selection. It forwards to the RL controller, which exercises further refinement of the output turning angle around obstacles. The collaboration of the global MFO approach with the local-based RL approach helps to optimize the path traversed by the humanoid robot in an unknown environment. The major breakthrough in this article is the utilization of humanoid robots for navigation purposes between various checkpoints. The humanoid robots are placed in a cluttered environment and assigned specific target positions to complete the assigned tasks. In the case of a multi-humanoid robot system, to avoid self-collision, it requires a Petri-Net controller to be configured in the navigation system to prevent deadlock situations and enhance the smooth completion of tasks without inter-collision among the humanoid robots. Simulations and real-time experiments are undertaken using different controllers involving single- and multi-humanoid robot systems. The robustness of the proposed controller is also validated in dynamic environment. Comparisons are carried with an established navigational controller in a similar environmental setup, which proves the proposed hybrid controller to be robust and efficient.

Keywords

moth–flame optimizationreinforcement learning approachhumanoid robotpath planningPetri-Net controller
Type
Research Article
Information
Robotica , Volume 41 , Issue 1 , January 2023 , pp. 346 - 369
Copyright
© The Author(s), 2022. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Berquin, Y. and Zell, A., “A physics perspective on lidar data assimilation for mobile robots,” Robotica 40(4), 862–887 (2022).CrossRefGoogle Scholar
Wu, S., Du, Y. and Zhang, Y., “Mobile robot path planning based on a generalized wavefront algorithm,” Math. Probl. Eng. 1(2), 2020–2012 (2020).Google Scholar
Liang, Y. and Xu, L., “Global Path Planning for Mobile Robot Based Genetic Algorithm and Modified Simulated Annealing Algorithm,” In: Proceedings of the First ACM/SIGEVO Summit on Genetic and Evolutionary Computation (2009) pp. 303–308.Google Scholar
Zhu, Z., Wang, F., He, S. and Sun, Y., “Global path planning of mobile robots using a memetic algorithm,” Int. J. Syst. Sci. 46(11), 1982–1993 (2015).CrossRefGoogle Scholar
Botzheim, J., Toda, Y. and Kubota, N., “Bacterial memetic algorithm for offline path planning of mobile robots,” Memetic Comput. 4(1), 73–86 (2012).Google Scholar
Liang, X. D., Li, L. Y., Wu, J. G. and Chen, H. N., “Mobile robot path planning based on adaptive bacterial foraging algorithm,” J. Cent. South Univ. 20(12), 3391–3400 (2013).CrossRefGoogle Scholar
Chen, M.-Y., Wu, Y.-J. and He, H., “A novel navigation system for an autonomous mobile robot in an uncertain environment,” Robotica 40(3), 421–446 (2022).CrossRefGoogle Scholar
Gao, M. and Tian, J., “Path Planning for Mobile Robot Based on Improved Simulated Annealing Artificial Neural Network,” In: Third International Conference on Natural Computation (ICNC) , vol. 3 (IEEE, 2007) pp. 8–12.CrossRefGoogle Scholar
Jun, H. and Qingbao, Z., “Multi-objective Mobile Robot Path Planning Based on Improved Genetic Algorithm,” In: International Conference on Intelligent Computation Technology and Automation , vol. 2 (IEEE, 2010) pp. 752–756.CrossRefGoogle Scholar
Yue, H. and Wang, Z.-M., “Path Planning of Mobile Robot Based on Compound Shape and Simulated Annealing Hybrid Algorithm,” In: IEEE International Conference on Robotics and Biomimetics-ROBIO (IEEE, 2005) pp. 186–189.Google Scholar
Janabi-Sharifi, F. and Vinke, D., “Integration of the Artificial Potential Field Approach with Simulated Annealing for Robot Path Planning,” In: Proceedings of 8th IEEE International Symposium on Intelligent Control (IEEE, 1993) pp. 536–541.Google Scholar
Martınez-Alfaro, H. and Gomez-Garcıa, S., “Mobile robot path planning and tracking using simulated annealing and fuzzy logic control,” Expert Syst. Appl. 15(3-4), 421–429 (1998).CrossRefGoogle Scholar
Pandey, K. K. and Parhi, D. R., “Trajectory planning and the target search by the mobile robot in an environment using a behavior-based neural network approach,” Robotica 38(9), 1627–1641 (2020).CrossRefGoogle Scholar
Ganganath, N. and Cheng, C.-T., “A 2-dimensional ACO-based Path Planner for Off-Line Robot Path Planning,” In: International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery (IEEE, 2013) pp. 302–307.CrossRefGoogle Scholar
Chang, L., Shan, L., Jiang, C. and Dai, Y., “Reinforcement based mobile robot path planning with improved dynamic window approach in unknown environment,” Auton. Robots 45(1), 51–76 (2021).CrossRefGoogle Scholar
Wei, Y. and Zhao, J., “Designing Human-like behaviors for anthropomorphic arm in humanoid robot NAO,” Robotica 38(7), 1205–1226 (2020).CrossRefGoogle Scholar
Kusuma, M., Riyanto, , Machbub, C., “Humanoid Robot Path Planning and Rerouting Using A-Star Search Algorithm,” In: Proceedings - 2019 IEEE International Conference on Signals and Systems, ICSigSys 2019 (IEEE, 2019) pp. 110–115.Google Scholar
Sabe, K., Fukuchi, M., Gutmann, J.-S., Ohashi, T., Kawamoto, K. and Yoshigahara, T., “Obstacle Avoidance and Path Planning for Humanoid Robots Using Stereo Vision,” In: IEEE International Conference on Robotics and Automation, Proceedings. ICRA 2004 (IEEE, 2004) pp. 592–597.CrossRefGoogle Scholar
Huang, W., Kim, J. and Atkeson, C. G., “Energy-based Optimal Step Planning for Humanoids,” In: IEEE International Conference on Robotics and Automation (IEEE, 2013) pp. 3124–3129.Google Scholar
Lee, M., Heo, Y., Park, J., Yang, H. D., Jang, H. D., Benz, P., Park, H., Kweon, I. S. and Oh, J. H., “Fast Perception, Planning, and Execution for a Robotic Butler: Wheeled Humanoid M-Hubo,” In: IEEE International Conference on Intelligent Robots and Systems (2019) pp. 5444–5451.Google Scholar
Lagaza, K. P., Kashyap, A. K. and Pandey, A., “Spider Monkey Optimization Algorithm Based Collision-Free Navigation and Path Optimization for A Mobile Robot in the Static Environment,” In: Advances in Mechanical Engineering (2020) pp. 1459–1473.Google Scholar
Jalali, S. M. J., Hedjam, R., Khosravi, A., Heidari, A. A., Mirjalili, S. and Nahavandi, S., “Autonomous robot navigation using Moth-Flame-Based neuroevolution,” In: Evolutionary Machine Learning Techniques (Springer 2020) pp. 67–83.Google Scholar
Abdullah, A., Rashid, M. F. F. A., Ponnambalam, S. G. and Ghazalli, Z., “Energy efficient modeling and optimization for assembly sequence planning using moth flame optimization,” Assem. Autom 39(2), 356–368 (2019).CrossRefGoogle Scholar
Mehne, S. H. H. and Mirjalili, S., “Moth-Flame optimization algorithm: theory, literature review, and application in optimal nonlinear feedback control design,” Nat.-Inspir. Optim. 811, 143–166 (2020).Google Scholar
Elaziz, M. A., Ewees, A. A., Ibrahim, R. A. and Lu, S., “Opposition-based moth-flame optimization improved by differential evolution for feature selection,” Math. Comput. Simul. 168(4), 48–75 (2020).CrossRefGoogle Scholar
Gao, P., Liu, Z., Wu, Z. and Wang, D., “A Global Path Planning Algorithm for Robots Using Reinforcement Learning,” In: IEEE International Conference on Robotics and Biomimetics (ROBIO) (IEEE, 2019) pp. 1693–1698.CrossRefGoogle Scholar
Fakoor, M., Kosari, A. and Jafarzadeh, M., “Humanoid robot path planning with fuzzy Markov decision processes,” J. Appl. Res. Technol 14(5), 300–310 (2016).CrossRefGoogle Scholar
Trinh, L. A., Ekström, M. and Cürüklü, B., “Petri Net Based Navigation Planning with Dipole Field and Dynamic Window Approach for Collision Avoidance,” In: 6th International Conference on Control, Decision and Information Technologies (CoDIT) (IEEE, 2019) pp. 1013–1018.CrossRefGoogle Scholar
Parhi, D. R. and Mohanta, J. C., “Navigational control of several mobile robotic agents using Petri-potential-fuzzy hybrid controller,” Appl. Soft Comput. J 11(4), 3546–3557 (2011).CrossRefGoogle Scholar
Kumar, P. B., Muni, M. K. and Parhi, D. R., “Navigational analysis of multiple humanoids using a hybrid regression-fuzzy logic control approach in complex terrains,” Appl. Soft Comput. 89(3), 106088 (2020).CrossRefGoogle Scholar
Ajeil, F. H., Ibraheem, I. K., Sahib, M. A. and Humaidi, A. J., “Multi-objective path planning of an autonomous mobile robot using hybrid PSO-MFB optimization algorithm,” Appl. Soft Comput. 89(4), 106076 (2020).CrossRefGoogle Scholar
Hosseinzadeh, A. and Izadkhah, H., “Evolutionary approach for mobile robot path planning in complex environment,” IJCSI Int. J. Comput. Sci. Issues 7(8), 1–9 (2010).Google Scholar
Alfaverh, F., Denaï, M. and Sun, Y., “Demand response strategy based on reinforcement learning and fuzzy reasoning for home energy management,” IEEE Access 8, 39310–39321 (2020).CrossRefGoogle Scholar
Peterson, J. L., “Petri nets,” ACM Comput. Surv. 9(3), 223–252 (1977).CrossRefGoogle Scholar
Lee, K. B., Myung, H. and Kim, J. H., “Online multiobjective evolutionary approach for navigation of humanoid robots,” IEEE Trans. Ind. Electron. 62(9), 5586–5597 (2015).CrossRefGoogle Scholar
Ali, H., Gong, D., Wang, M. and Dai, X., “Path planning of mobile robot with improved ant colony algorithm and MDP to produce smooth trajectory in Grid-Based environment,” Front. Neurorobot. 14(July), 1–13 (2020).CrossRefGoogle ScholarPubMed