Hostname: page-component-6766d58669-nqrmd Total loading time: 0 Render date: 2026-05-20T18:27:35.800Z Has data issue: false hasContentIssue false
  • English
  • Français

Adaptive reinforcement learning control for a class of missiles with aerodynamic uncertainties and unmodeled dynamics

Published online by Cambridge University Press:  06 July 2023

X. Ning ,
S. Cao ,
B. Han ,
Z. Wang Open the ORCID record for Z. Wang [Opens in a new window]  and
Y. Yin Open the ORCID record for Y. Yin [Opens in a new window]
X. Ning
Affiliation:
National Key Laboratory of Aerospace Flight Dynamics, Northwestern Polytechnical University, Xi’an, China Science and Technology on Electromechanical Dynamic Control Laboratory, Xi’an, China School of Astronautics, Northwestern Polytechnical University, Xi’an, China
S. Cao
Affiliation:
National Key Laboratory of Aerospace Flight Dynamics, Northwestern Polytechnical University, Xi’an, China Science and Technology on Electromechanical Dynamic Control Laboratory, Xi’an, China Xi’an Institute of Electromechanical Information Technology, Xi’an, China
B. Han
Affiliation:
Xi’an Aeronautics Computing Technique Research Institute, Xi’an, China
Z. Wang*
Affiliation:
National Key Laboratory of Aerospace Flight Dynamics, Northwestern Polytechnical University, Xi’an, China Research Center for Unmanned System Strategy Development, Northwestern Polytechnical University, Xi’an, China Northwest Institute of Mechanical and Electrical Engineering, Xianyang, China Unmanned System Research Institute, Northwestern Polytechnical University, Xi’an, China
Y. Yin
Affiliation:
National Key Laboratory of Aerospace Flight Dynamics, Northwestern Polytechnical University, Xi’an, China School of Astronautics, Northwestern Polytechnical University, Xi’an, China
*
Corresponding author: Z. Wang; Email: wz_nwpu@126.com
Get access Rights & Permissions [Opens in a new window]

Abstract

In this paper, a super-twisting disturbance observer (STDO)-based adaptive reinforcement learning control scheme is proposed for the straight air compound missile system with aerodynamic uncertainties and unmodeled dynamics. Firstly, neural network (NN)-based adaptive reinforcement learning control scheme with actor-critic design is investigated to deal with the tracking problems for the straight gas compound system. The actor NN and the critic NN are utilised to cope with the unmodeled dynamics and approximate the cost function that are related to control input and tracking error, respectively. In other words, the actor NN is used to perform the tracking control behaviours, and the critic NN aims to evaluate the tracking performance and give feedback to actor NN. Moreover, with the aid of the STDO disturbance observer, the problem of the control signal fluctuation caused by the mismatched disturbance can be solved well. Based on the proposed adaptive law and the Lyapunov direct method, the eventually consistent boundedness of the straight gas compound system is proved. Finally, numerical simulations are carried out to demonstrate the feasibility and superiority of the proposed reinforcement learning-based STDO control algorithm.

Keywords

straight air compound missile systemunmodeled dynamicsreinforcement learningsuper-twisting disturbance observer

Information

Type
Research Article
Information
The Aeronautical Journal , Volume 128 , Issue 1320 , February 2024 , pp. 292 - 308
Check for updates
Copyright
© The Author(s), 2023. Published by Cambridge University Press on behalf of Royal Aeronautical Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

Antonios, T. and Brian, A. Modern missile flight control design: an overview, IFAC Proc. Vol., 2001, 34, (15), pp 425–430.Google Scholar
Song, C., Kim, S.J., Kim, S.H. and Nam, H.S. Robust control of the missile attitude based on quaternion feedback, Control Eng. Pract., July 2006, 14, (7), pp 811–818.CrossRefGoogle Scholar
Yang, J., Chen, W.H. and Li, S. Non-linear disturbance observer-based robust control for systems with mismatched disturbances/uncertainties, IET Control Theory Appl., December 2011, 5, (18), pp 2053–2062.CrossRefGoogle Scholar
Lee, Y., Kim, Y. and Moon, G. Sliding-mode-based missile-integrated attitude control schemes considering velocity change, J. Guid. Control Dynam., 2016, 39, (3), pp 423–436.CrossRefGoogle Scholar
Zhou, J. and Yang, J. Smooth sliding mode control for missile interception with finite-time convergence, J. Guid. Control Dynam., 2015, 38, (7), pp 1311–1318.CrossRefGoogle Scholar
Shao, X. and Wang, H. Back-stepping active disturbance rejection control design for integrated missile guidance and control system via reduced-order ESO, ISA Trans., 2015, 57, pp 10–22.Google Scholar
Guo, P., Yang, S. and Zhao, L. Second order sliding mode control with back stepping approach for moving mass spinning missiles, J. Beijing Inst. Technol., 2016, 1, pp 17–22.Google Scholar
Wang, L., Zhang, W., Wang, D., Peng, K. and Yang, H. Command filtered back-stepping missile integrated guidance and autopilot based on extended state observer, Adv. Mech. Eng., 2017, 9, (11), pp 1–13.CrossRefGoogle Scholar
Fan, Y., Li, X., Yang, J. and Zhang, Y. Design of autopilot for aerodynamic/reaction-jet multiple control missile using variable structure control, 2008 27th Chinese Control Conference, 2008, pp 642–645.CrossRefGoogle Scholar
Shao, L., Zhang, J. and Cao, Y. Blended robust control method with lateral thrust and aerodynamic force based on robust trail tracking, Aero Weapon, 2016, 291, (1), pp 35–39.Google Scholar
Liu, X., Li, A., Guo, Y., Wang, S. and Wang, C. Fixed-time convergence blended control for air-to-air missile with lateral thrusters and aerodynamic force, J. Harbin Inst. Technol., 2019, 51, (09), pp 29–34+42.Google Scholar
Zhao, Y., Liao, Z., Duan, C. and Zhang, G. Design of blended lateral thrust and aerodynamic control system based on terminal sliding mode, Navig. Position. Timing, 2015, 2, (03), pp 49–54.Google Scholar
Xu, B. and Zhou, D. Backstepping and control allocation for dual aero/propulsive missile control, Syst. Eng. Electron., 2014, 36, (03), pp 527–531.Google Scholar
Zhang, X. Design of compound control system with direct lateral thrust and aerodynamics adopting backstepping method, Modern Defence Technol., 2009, 37, (04), pp 43–46.Google Scholar
Shi, Z., Ma, W., Zhang, Y. and Lin, Q. Fuzzy control algorithm and realization of compound control missile, Harbin Gongcheng Daxue Xuebao/J. Harbin Eng. Univ., 2014, 35, (02), pp 195–201.Google Scholar
Luo, X. and Zhang, T. The application of fuzzy control in combined-guidance, J. Project. Rockets, Miss. Guid., 2001, 02, pp 1–4.Google Scholar
Liu, S., Qu, X. and Liu, Y. Design of missile autopilot based on fuzzy control, 2016 IEEE International Conference on Information and Automation (ICIA), 2016, pp 1339–1343.CrossRefGoogle Scholar
Fan, Y. and Yang, J. The design of aerodynamic/reaction-jet compound controller of missile actuator using neural network model reference control, Fire Control Command Control, 2008, 163, (10), pp 85–87.Google Scholar
Zhou, X., Peng, M. and Li, Y. Autopilot design for dual aero/propulsive missile using genetic algorithm LQR control, Comput. Meas. Control, 2014, 22, (04), pp 1157–1159+1162.Google Scholar
Dong, Z., Chen, J., Song, C. and Cao, H. Design of longitudinal control system for target missiles based on fuzzy adaptive PID control, 2017 29th Chinese Control and Decision Conference (CCDC), 2017, pp 398–402.Google Scholar
Chwa, D. Fuzzy adaptive disturbance observer-based robust adaptive control for skid-to-turn missiles, IEEE Trans. Aerosp. Electron. Syst., 2015, 51, (01), pp 468–478.CrossRefGoogle Scholar
Yang, P., Fang, Y., Chai, D. and Wu, Y. Fuzzy control strategy for hypersonic missile autopilot with blended aero-fin and lateral thrust, Proc. Inst. Mech. Eng. I: J. Syst. Control Eng., 2016, 230, (01), pp 72–81.Google Scholar
Cai, J., Xing, L., Zhang, M. and Shen, L. Adaptive neural network control for missile systems with unknown hysteresis input, IEEE Access, 2017, 05, pp 15839–15847.CrossRefGoogle Scholar
Chen, K. Full state constrained stochastic adaptive integrated guidance and control for STT missiles with non-affine aerodynamic characteristics, Inform. Sci., 2020, 529, pp 42–58.CrossRefGoogle Scholar
Zhang, H., Chen, Z. and Hao, L. Optimization of missile allocation based on adaptive genetic algorithm, Tactical Miss. Technol., 2007, 124, (04), pp 28–30+36.Google Scholar
Shi, Z., Ma, W. and Wang, F. Intelligent control algorithm for missile with lateral jets and aerodynamic surfaces, J. Nanjing Univ. Sci. Technol., 2014, 38, (04), pp 481–489.Google Scholar
Polycarpou, M.M. and Ioannou, P.A. A robust adaptive nonlinear control design, 1993 American Control Conference, 1993, pp 1365–1369.CrossRefGoogle Scholar