Hostname: page-component-848d4c4894-wzw2p Total loading time: 0 Render date: 2024-06-03T04:46:51.552Z Has data issue: false hasContentIssue false
  • English
  • Français

A reinforcement learning fuzzy system for continuous control in robotic odor plume tracking

Published online by Cambridge University Press:  19 September 2022

Xinxing Chen Open the ORCID record for Xinxing Chen [Opens in a new window] ,
Bo Yang Open the ORCID record for Bo Yang [Opens in a new window] ,
Jian Huang Open the ORCID record for Jian Huang [Opens in a new window] ,
Yuquan Leng  and
Chenglong Fu Open the ORCID record for Chenglong Fu [Opens in a new window]
Xinxing Chen
Affiliation:
Shenzhen Key Laboratory of Biomimetic Robotics and Intelligent Systems, Shenzhen, 518055, China Guangdong Provincial Key Laboratory of Human-Augmentation and Rehabilitation Robotics in Universities, Southern University of Science and Technology, Shenzhen, 518055, China
Bo Yang
Affiliation:
Key Laboratory of Image Processing and Intelligent Control, School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan, 430074, China
Jian Huang
Affiliation:
Key Laboratory of Image Processing and Intelligent Control, School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan, 430074, China
Yuquan Leng
Affiliation:
Shenzhen Key Laboratory of Biomimetic Robotics and Intelligent Systems, Shenzhen, 518055, China Guangdong Provincial Key Laboratory of Human-Augmentation and Rehabilitation Robotics in Universities, Southern University of Science and Technology, Shenzhen, 518055, China
Chenglong Fu*
Affiliation:
Shenzhen Key Laboratory of Biomimetic Robotics and Intelligent Systems, Shenzhen, 518055, China Guangdong Provincial Key Laboratory of Human-Augmentation and Rehabilitation Robotics in Universities, Southern University of Science and Technology, Shenzhen, 518055, China
*
*Corresponding author. E-mail: fucl@sustech.edu.cn
Get access Rights & Permissions [Opens in a new window]

Abstract

In dynamic outdoor environments characterized by turbulent airflow and intermittent odor plumes, robotic odor plume tracking remains challenging, because existing algorithms heavily rely on manually tuning or learning from expert experience, which are hard to implement in an unknown environment. In this paper, a multi-continuous-output Takagi–Sugeno–Kang fuzzy system was designed and tuned with reinforcement learning to solve the robotic odor source localization problem in dynamic odor plumes. Based on the Lévy Taxis plume tracking controller, the proposed fuzzy system determined the parameters of the controller based on the robot’s observation and guided the robot to turn and move towards the odor source at each searching step. The trained fuzzy system was tested in simulated filament-based odor plumes dispersed by a changing wind field. The results showed that the performance of the proposed fuzzy system-based controller trained with reinforcement learning can achieve a similar success rate and higher efficiency compared with a manually tuned and well-designed fuzzy system-based controller. The fuzzy system-based plume tracking controller was also validated through real robotic experiments.

Keywords

reinforcement learningfuzzy inference systemodor plume trackingrobotic olfactiondynamic airflow
Type
Research Article
Information
Robotica , Volume 41 , Issue 3 , March 2023 , pp. 1039 - 1054
Copyright
© The Author(s), 2022. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Chen, X. and Huang, J., “Odor source localization algorithms on mobile robots: A review and future outlook,” Robot. Auton. Syst. 112(1), 123–136 (2019).CrossRefGoogle Scholar
Li, Z., Su, C. Y., Wang, L., Chen, Z. and Chai, T., “Nonlinear disturbance observer-based control design for a robotic exoskeleton incorporating fuzzy approximation,” IEEE Trans. Ind. Electron. 62(9), 5763–5775 (2015).CrossRefGoogle Scholar
Ma, D., Mao, W., Tan, W., Gao, J., Zhang, Z. and Xie, Y., “Emission source tracing based on bionic algorithm mobile sensors with artificial olfactory system,” Robotica 40(4), 976–996 (2022).CrossRefGoogle Scholar
Larsch, J., Flavell, S. W., Liu, Q., Gordus, A., Albrecht, D. R. and Bargmann, C. I., “A circuit for gradient climbing in C. elegans chemotaxis,” Cell Rep. 12(11), 1748–1760 (2015).CrossRefGoogle ScholarPubMed
Chen, X. and Huang, J., “Combining particle filter algorithm with bio-inspired anemotaxis behavior: A smoke plume tracking method and its robotic experiment validation,” Measurement 154, 107482 (2020).CrossRefGoogle Scholar
Vergassola, M., Villermaux, E. and Shraiman, B. I., “infotaxis’ as a strategy for searching without gradients,” Nature 445(7126), 406–409 (2007).CrossRefGoogle ScholarPubMed
Chen, X., Marjovi, A., Huang, J. and Martinoli, A., “Particle source localization with a low-cost robotic sensor system: Algorithmic design and performance evaluation,” IEEE Sens. J. 20(21), 13074–13085 (2020).CrossRefGoogle Scholar
Arya, S. P., Air Pollution Meteorology and Dispersion, vol. 310 (Oxford University Press, New York, 1999).Google Scholar
Chen, X. and Huang, J., “Towards Environmentally Adaptive Odor Source Localization: Fuzzy Lévy Taxis Algorithm and Its Validation in Dynamic Odor Plumes,” In: 2020 5th International Conference on Advanced Robotics and Mechatronics (ICARM) (2020) pp. 282–287.Google Scholar
Wang, L. and Pang, S., “An Implementation of the Adaptive Neuro-Fuzzy Inference System (ANFIS) for Odor Source Localization,” In: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2020) pp. 4551–4558.Google Scholar
Wang, L., Pang, S. and Li, J., “Olfactory-based navigation via model-based reinforcement learning and fuzzy inference methods,” IEEE Trans. Fuzzy Syst. 29(10), 3014–3027 (2021).CrossRefGoogle Scholar
Wang, L., Pang, S. and Li, J., “Learn to Trace Odors: Autonomous Odor Source Localization via Deep Learning Methods,” In: 2021 20th IEEE International Conference on Machine Learning and Applications (ICMLA) (IEEE, 2021) pp. 1429–1436.CrossRefGoogle Scholar
Chen, X., Fu, C. and Huang, J., “A Deep Q-Network for robotic odor/gas source localization: Modeling, measurement and comparative study,” Measurement 183, 109725 (2021).CrossRefGoogle Scholar
Hu, H., Song, S. and Chen, C. P., “Plume tracing via model-free reinforcement learning method,” IEEE Trans. Neural Netw. Learn. Syst. 30(8), 2515–2527 (2019).CrossRefGoogle ScholarPubMed
Chen, C., Zhang, K., Leng, Y., Chen, X. and Fu, C., “Unsupervised sim-to-real adaptation for environmental recognition in assistive walking,” IEEE Trans. Neural Syst. Rehabil. Eng. 30, 1350–1360 (2022).CrossRefGoogle ScholarPubMed
Cao, Y. and Huang, J., “Neural-network-based nonlinear model predictive tracking control of a pneumatic muscle actuator-driven exoskeleton,” IEEE/CAA J. Autom. Sin. 7(6), 1478–1488 (2020).CrossRefGoogle Scholar
Su, H., Hu, Y., Karimi, H. R., Knoll, A., Ferrigno, G. and De Momi, E., “Improved recurrent neural network-based manipulator control with remote center of motion constraints: Experimental results,” Neural Netw. 131(12), 291–299 (2020).CrossRefGoogle ScholarPubMed
Fang, B., Long, X., Sun, F., Liu, H., Zhang, S. and Fang, C., “Tactile-based fabric defect detection using convolutional neural network with attention mechanism,” IEEE Trans. Instrum. Meas. 71, 1–9 (2022).Google Scholar
Li, Z., Cao, X. and Ding, N., “Adaptive fuzzy control for synchronization of nonlinear teleoperators with stochastic time-varying communication delays,” IEEE Trans. Fuzzy Syst. 19(4), 745–757 (2011).CrossRefGoogle Scholar
Yang, C., Jiang, Y., Na, J., Li, Z., Cheng, L. and Su, C.-Y., “Finite-time convergence adaptive fuzzy control for dual-arm robot with unknown kinematics and dynamics,” IEEE Trans. Fuzzy Syst. 27(3), 574–588 (2018).CrossRefGoogle Scholar
Chen, X., Leng, Y. and Fu, C., “A supervised-reinforced successive training framework for a fuzzy inference system and its application in robotic odor source searching,” Front Neurorobot. 16, 5962 (2022).CrossRefGoogle ScholarPubMed
Salehi, M., Pishkenari, H. N. and Zohoor, H., “Position control of a wheel-based miniature magnetic robot using neuro-fuzzy network,” Robotica, 1–16 (2022).Google Scholar
Li, Z., Ren, Z., Zhao, K., Deng, C. and Feng, Y., “Human-cooperative control design of a walking exoskeleton for body weight support,” IEEE Trans. Ind. Inform. 16(5), 2985–2996 (2019).CrossRefGoogle Scholar
Su, H., Qi, W., Chen, J. and Zhang, D., “Fuzzy approximation-based task-space control of robot manipulators with remote center of motion constraint,” IEEE Trans. Fuzzy Syst. 30(6), 1564–1573 (2022).CrossRefGoogle Scholar
Li, F., Zhang, Z., Wu, Y., Chen, Y., Liu, K. and Yao, J., “Improved fuzzy sliding mode control in flexible manipulator actuated by pmas,” Robotica 40(8), 1–14 (2022).CrossRefGoogle Scholar
Veysi, M., Soltanpour, M. R. and Khooban, M. H., “A novel self-adaptive modified bat fuzzy sliding mode control of robot manipulator in presence of uncertainties in task space,” Robotica 33(10), 2045–2064 (2015).CrossRefGoogle Scholar
Wu, D., Yuan, Y., Huang, J. and Tan, Y., “Optimize TSK fuzzy systems for regression problems: Minibatch gradient descent with regularization, DropRule and AdaBound (MBGD-RDA),” IEEE Trans. Fuzzy Syst. 28(5), 1003–1015 (2019).CrossRefGoogle Scholar
Su, H., Qi, W., Schmirander, Y., Ovur, S. E., Cai, S. and Xiong, X., “A human activity-aware shared control solution for medical human–robot interaction,” Assem. Autom. 42(3), 388–394 (2022).CrossRefGoogle Scholar
Yang, B., Huang, J., Chen, X., Xiong, C. and Hasegawa, Y., “Supernumerary robotic limbs: A review and future outlook,” IEEE Trans. Med. Robot. Bionics 3(3), 623–639 (2021).CrossRefGoogle Scholar
Chen, X., Zhang, K., Liu, H., Leng, Y. and Fu, C., “A probability distribution model-based approach for foot placement prediction in the early swing phase with a wearable imu sensor,” IEEE Trans. Neural Syst. Rehabil. Eng. 29, 2595–2604 (2021).CrossRefGoogle ScholarPubMed
Fang, B., Ding, W., Sun, F., Shan, J., Wang, X., Wang, C. and Zhang, X., “Brain-computer interface integrated with augmented reality for human-robot interaction,” IEEE Trans. Cogn. Dev. Syst., 1–1 (2022).Google Scholar
Zhang, K., Luo, J., Xiao, W., Zhang, W., Liu, H., Zhu, J., Lu, Z., Rong, Y., de Silva, C. W. and Fu, C., “A subvision system for enhancing the environmental adaptability of the powered transfemoral prosthesis,” IEEE Trans. Cybern. 51(6), 3285–3297 (2021).CrossRefGoogle ScholarPubMed
Chen, X., Chen, C., Wang, Y., Yang, B., Ma, T., Leng, Y. and Fu, C., “A piecewise monotonic gait phase estimation model for controlling a powered transfemoral prosthesis in various locomotion modes,” IEEE Robot. Autom. Lett. 7(4), 9549–9556 (2022).CrossRefGoogle Scholar
Guo, Y., Song, B., Tang, X., Zhou, X. and Jiang, Z., “A calibration method of non-contact r-test for error measurement of industrial robots,” Measurement 173, 108365 (2021).CrossRefGoogle Scholar
Guo, Y., Song, B., Tang, X., Zhou, X. and Jiang, Z., “A measurement method for calibrating kinematic parameters of industrial robots with point constraint by a laser displacement sensor,” Meas. Sci. Technol. 31(7), 075004 (2020).CrossRefGoogle Scholar
Guo, Y., Tang, X., Zhou, X., Song, B., Jiang, Z., Xie, Y. and Ye, B., “Continuous measurements with single setup for position-dependent geometric errors of rotary axes on five-axis machine tools by a laser displacement sensor,” Int. J. Adv. Manuf. Technol. 99(5), 1589–1602 (2018).CrossRefGoogle Scholar
Cao, Y., Huang, J., Xiong, C.-H., Wu, D., Zhang, M., Li, Z. and Hasegawa, Y., “Adaptive proxy-based robust control integrated with nonlinear disturbance observer for pneumatic muscle actuators,” IEEE/ASME Trans. Mechatron. 25(4), 1756–1764 (2020).CrossRefGoogle Scholar
Huang, J., Guan, Z.-H., Matsuno, T., Fukuda, T. and Sekiyama, K., “Sliding-mode velocity control of mobile-wheeled inverted-pendulum systems,” IEEE Trans. Robot. 26(4), 750–758 (2010).CrossRefGoogle Scholar
Zhang, F., Xia, R. and Chen, X., “An optimal trajectory planning algorithm for autonomous trucks: Architecture, algorithm, and experiment,” Int. J. Adv. Robot. Syst. 17(2), 1–12 (2020).CrossRefGoogle Scholar
Fang, B., Sun, F., Wu, L., Liu, F., Wang, X., Huang, H., Huang, W., Liu, H. and Wen, L., “Multimode grasping soft gripper achieved by layer jamming structure and tendon-driven mechanism,” Soft Robot. 9(2), 233–249 (2022).CrossRefGoogle ScholarPubMed
Nguyen, A.-T., Taniguchi, T., Eciolaza, L., Campos, V., Palhares, R. and Sugeno, M., “Fuzzy control systems: Past, present and future,” IEEE Comput. Intell. Mag. 14(1), 56–68 (2019).CrossRefGoogle Scholar
Wu, D. and Tan, W. W., “Genetic learning and performance evaluation of interval type-2 fuzzy logic controllers,” Eng. Appl. Artif. Intell. 19(8), 829–841 (2006).CrossRefGoogle Scholar
Wang, L.-X. and Mendel, J. M., “Back-Propagation Fuzzy System as Nonlinear Dynamic System Identifiers,” In: [1992 Proceedings] IEEE International Conference on Fuzzy Systems (IEEE, 1992) pp. 1409–1418.Google Scholar
Cui, Y., “PyTSK,” (2022). https://github.com/YuqiCui/PyTSK Google Scholar
Farrell, J. A., Murlis, J., Long, X., Li, W. and Cardé, R. T., “Filament-based atmospheric dispersion model to achieve short time-scale structure of odor plumes,” Environ. Fluid Mech. 2(1-2), 143–169 (2002).CrossRefGoogle Scholar
Pasternak, Z., Bartumeus, F. and Grasso, F. W., “Lévy-taxis: A novel search strategy for finding odor plumes in turbulent flow-dominated environments,” J. Phys. A Math. Theor. 42(43), 434010 (2009).CrossRefGoogle Scholar
Emery, R., Rahbar, F., Marjovi, A. and Martinoli, A., “Adaptive lévy Taxis for Odor Source Localization in Realistic Environmental Conditions,” In: 2017 IEEE International Conference on Robotics and Automation (ICRA) (2017) pp. 3552–3559.Google Scholar
Lochmatter, T., Roduit, P., Cianci, C., Correll, N., Jacot, J. and Martinoli, A., “Swistrack-A Flexible Open Source Tracking Software for Multi-Agent Systems,” In: 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems (IEEE, 2008) pp. 4004–4010.CrossRefGoogle Scholar
Li, Z., Zhao, K., Zhang, L., Wu, X., Zhang, T., Li, Q., Li, X. and Su, C-Y., “Human-in-the-loop control of a wearable lower limb exoskeleton for stable dynamic walking,” IEEE/ASME Trans. Mechatron. 26(5), 2700–2711 (2020).CrossRefGoogle Scholar
Li, Z., Deng, C. and Zhao, K., “Human-cooperative control of a wearable walking exoskeleton for enhancing climbing stair activities,” IEEE Trans. Ind. Electron. 67(4), 3086–3095 (2019).CrossRefGoogle Scholar
Wu, X. and Li, Z., “Cooperative manipulation of wearable dual-arm exoskeletons using force communication between partners,” IEEE Trans. Ind. Electron. 67(8), 6629–6638 (2019).CrossRefGoogle Scholar
Li, G., Li, Z. and Kan, Z., “Assimilation control of a robotic exoskeleton for physical human-robot interaction,” IEEE Robot. Autom. Lett. 7(2), 2977–2984 (2022).CrossRefGoogle Scholar
Supplementary material: File

Chen et al. supplementary material

Chen et al. supplementary material 1

Download Chen et al. supplementary material(File)
File 60 Bytes

Chen et al. supplementary material

Chen et al. supplementary material 2

Download Chen et al. supplementary material(Video)
Video 36 MB