Hostname: page-component-f554764f5-nt87m Total loading time: 0 Render date: 2025-04-20T23:34:48.773Z Has data issue: false hasContentIssue false
  • English
  • Français

Reinforcement learning-adaptive fault-tolerant IGC method for a class of aircraft with non-affine characteristics and multiple uncertainties

Published online by Cambridge University Press:  05 November 2024

Z. Wang ,
Y. T. Hao Open the ORCID record for Y. T. Hao [Opens in a new window] ,
J. L. Liu ,
Y. F. Bai  and
D. X. Yu
Z. Wang
Affiliation:
Research Center for Unmanned System Strategy Development, Northwestern Polytechnical University, Xi’an, Shaanxi, China National Key Laboratory of Aerospace Flight Dynamics, Northwestern Polytechnical University, Xi’an, Shaanxi, China Northwest Institute of Mechanical and Electrical Engineering, Xianyang, China Unmanned System Research Institute, Northwestern Polytechnical University, Xi’an, Shaanxi, China
Y. T. Hao*
Affiliation:
Unmanned System Research Institute, Northwestern Polytechnical University, Xi’an, Shaanxi, China
J. L. Liu
Affiliation:
Unmanned System Research Institute, Northwestern Polytechnical University, Xi’an, Shaanxi, China
Y. F. Bai
Affiliation:
China Academy of Launch Vehicle Technology, Beijing, China
D. X. Yu
Affiliation:
School of Artificial Intelligence, OPtics and ElectroNlics (iOPEN), Northwestern Polytechnical University, Xi’an, Shaanxi, China
*
Corresponding author: Y. T. Hao; Email: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

In this paper, a brand-new adaptive fault-tolerant non-affine integrated guidance and control method based on reinforcement learning is proposed for a class of skid-to-turn (STT) missile. Firstly, considering the non-affine characteristics of the missile, a new non-affine integrated guidance and control (NAIGC) design model is constructed. For the NAIGC system, an adaptive expansion integral system is introduced to address the issue of challenging control brought on by the non-affine form of the control signal. Subsequently, the hyperbolic tangent function and adaptive boundary estimation are utilised to lessen the jitter due to disturbances in the control system and the deviation caused by actuator failures while taking into account the uncertainty in the NAIGC system. Importantly, actor-critic is introduced into the control framework, where the actor network aims to deal with the multiple uncertainties of the subsystem and generate the control input based on the critic results. Eventually, not only is the stability of the NAIGC closed-loop system demonstrated using Lyapunov theory, but also the validity and superiority of the method are verified by numerical simulations.

Keywords

Non-affine integrated guidance and controlMultiple uncertaintiesActor-criticAdaptive fault-tolerant control
Type
Research Article
Information
The Aeronautical Journal , Volume 129 , Issue 1331 , January 2025 , pp. 84 - 106
Check for updates
Copyright
© The Author(s), 2024. Published by Cambridge University Press on behalf of Royal Aeronautical Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

Guo, Z., Wang, J., Hu, G. and Guo, J. Research review on uncertainty observation techniques and control methods for aerospace vehicles, Aerospace Technol., 2022, (5), pp 31–44. doi: 10.16338/j.issn.2097-0714.20220091 Google Scholar
Wu, Y., Lu, X. and Wang, Z. Research on integrated design of aircraft spiral maneuver, guidance and control based on sliding mode control, Beijing Ligong Daxue Xuebao/Trans. Beijing Inst. Technol., 2022, 42, (5), pp 523–529. doi: 10.15918/j.tbit1001-0645.2021.089 Google Scholar
Wang, X., Zhang, X., Lin, P. and Li, W. Integrated strategy of penetration and attack based on optimal control, Flight Dyn., 2022, 40, (6), 51–60+71. doi: 10.13645/j.cnki.f.d.20220716.001 Google Scholar
Xu, M., Chen, G. and Wang, W. Aero-control integrated design for reusable launch vehicle based on feedback linearization, Meas. Control Technol., 2018, 37, (9), pp 88–91. doi: 10.19708/j.ckjs.2018.09.021 Google Scholar
Hu, C., Wei, Y. and Wang, X. Fixed-time integrated guidance and control for impact angle constrained interception with multiple uncertainties, J. Projectiles Rockets Missiles Guidance, 2023, 43, (4), pp 98–104. doi: 10.15892/j.cnki.djzdxb.2023.04.015 Google Scholar
Jiang, S., qing Tian, F., yan Sun, S. and ge Liang, W. Integrated guidance and control of guided projectile with multiple constraints based on fuzzy adaptive and dynamic surface, Def. Technol., 2020 16, (6), pp 1130–1141. doi: 10.1016/j.dt.2019.12.003 CrossRefGoogle Scholar
Zhao, K., Cao, D.Q. and Huang, W.H. Integrated guidance and control design for reentry warhead based on adrc, Yuhang Xuebao/J. Astronaut., 2017, 38, (10), pp 1068–1078. doi: 10.3873/j.issn.1000-1328.2017.10.007 Google Scholar
Wang, Z., Yuan, J., Pan, Y. and Wei, J. Neural network-based adaptive fault tolerant consensus control for a class of high order multiagent systems with input quantization and time-varying parameters, Neurocomputing, 2017, 266, pp 315–324. https://doi.org/10.1016/j.neucom.2017.05.043 CrossRefGoogle Scholar
Wang, Z., Zhang, B. and Yuan, J. Decentralized adaptive fault tolerant control for a class of interconnected systems with nonlinear multisource disturbances, J. Franklin Inst., 2018, 355, (11), pp 4493–4514. doi: 10.1016/j.jfranklin.2017.10.038 CrossRefGoogle Scholar
Wang, Z. and Yuan, J. Fuzzy adaptive fault tolerant igc method for stt missiles with time-varying actuator faults and multisource uncertainties, J. Franklin Inst., 2020, 357, (1), pp 59–81. doi: 10.1016/j.jfranklin.2019.09.032 CrossRefGoogle Scholar
Najafi, A., Vu, M.T., Mobayen, S., Asad, J.H. and Fekih, A. Adaptive barrier fast terminal sliding mode actuator fault tolerant control approach for quadrotor uavs, Mathematics, 2022, 10, (16), pp 1–22. doi: 10.3390/math10163009 CrossRefGoogle Scholar
Zhao, L., Zhao, F. and Che, W.W. Distributed adaptive fuzzy fault-tolerant control for multi-agent systems with node faults and denial-of-service attacks, Inf. Sci., 2023, 631, pp 385–395. doi: 10.1016/j.ins.2023.02.059 CrossRefGoogle Scholar
Ashrafifar, A. and Jegarkandi, M.F. Adaptive fin failures tolerant integrated guidance and control based on backstepping sliding mode, Trans. Inst. Meas. Control, 2020, 42, (10), pp 1823–1833. doi: 10.1177/0142331219897430 CrossRefGoogle Scholar
Zhao, D., Research on Integrated Guidance and Control Design of Hypersonic Flight Vehicle, Master’s Thesis, Beijing Jiaotong University, No.3 Shangyuan Village, Haidian District, Beijing, China 2020.Google Scholar
Chen, K. Full state constrained stochastic adaptive integrated guidance and control for stt missiles with non-affine aerodynamic characteristics, Inf. Sci., 2020, 529, pp 42–58. doi: 10.1016/j.ins.2020.03.061 CrossRefGoogle Scholar
Liu, Y.J., Li, S., Tong, S. and Chen, C.L., Adaptive reinforcement learning control based on neural approximation for nonlinear discrete-time systems with unknown nonaffine dead-zone input, IEEE Trans. Neural Networks Learn. Syst., 2019, 30, (1), pp 295–305. doi: 10.1109/TNNLS.2018.2844165 CrossRefGoogle ScholarPubMed
Lopez, V.G. and Lewis, F.L. Dynamic multiobjective control for continuous-time systems using reinforcement learning, IEEE Trans. Autom. Control, 2019, 64, (7), pp 2869–2874. doi: 10.1109/TAC.2018.2869462 CrossRefGoogle Scholar
Ruelens, F., Claessens, B.J., Quaiyum, S., De Schutter, B., Babuška, R. and Belmans, R. Reinforcement learning applied to an electric water heater: From theory to practice, IEEE Trans. Smart Grid, 2018, 9, (4), pp 3792–3800. doi: 10.1109/TSG.2016.2640184.CrossRefGoogle Scholar
Xue, L., Sun, C., Wunsch, D., Zhou, Y. and Yu, F. An adaptive strategy via reinforcement learning for the prisoner’s dilemma game, IEEE/CAA J. Autom. Sinica, 2018, 5, (1), pp 301–310. doi: 10.1109/JAS.2017.7510466 CrossRefGoogle Scholar
Peng, Z., Hu, J., Shi, K., Luo, R., Huang, R., Ghosh, B.K. and Huang, J. A novel optimal bipartite consensus control scheme for unknown multi-agent systems via model-free reinforcement learning, Appl. Math. Comput., 2020, 369, p 124821. doi: 10.1016/j.amc.2019.124821 Google Scholar
Yang, Y., Modares, H., Wunsch, D.C. and Yin, Y. Leader-follower output synchronization of linear heterogeneous systems with active leader using reinforcement learning, IEEE Trans. Neural Networks and Learn. Syst., 2018, 29, (6), pp 2139–2153. doi: 10.1109/TNNLS.2018.2803059 CrossRefGoogle ScholarPubMed
Fan, Q.Y., Yang, G.H. and Ye, D. Quantization-based adaptive actor-critic tracking control with tracking error constraints, IEEE Trans. Neural Networks Learn. Syst., 2018, 29, (4), pp 970–980. doi: 10.1109/TNNLS.2017.2651104 CrossRefGoogle ScholarPubMed
Hu, L., Li, R., Xue, T. and Liu, Y. Neuro-adaptive tracking control of a hypersonic flight vehicle with uncertainties using reinforcement synthesis, Neurocomputing, 2018, 285, pp 141–153. doi: 10.1016/j.neucom.2018.01.031 CrossRefGoogle Scholar
Ouyang, Y., Dong, L., Wei, Y. and Sun, C. Neural network based tracking control for an elastic joint robot with input constraint via actor-critic design, Neurocomputing, 2020, 409, pp 286–295. doi: 10.1016/j.neucom.2020.05.067 CrossRefGoogle Scholar
Liu, J., Shan, J., Rong, J. and Zheng, X. Incremental reinforcement learning flight control with adaptive learning rate, J. Astronaut., 2022, 43, (1), pp 111–121. doi: 10.3873/j.issn.1000-1328.2022.01.013 Google Scholar
Bohao, L., Xuman, A., Xiaofei, Y., Yunjie, W. and Guofei, L. A distributed reinforcement learning guidance method under impact angle constraints, J. Astronaut., 2022, 43, (8), pp 1061–1069. doi: 10.3873/j.issn.1000-1328.2022.08.008 Google Scholar
Pei, P., Shao-ming, H., Jiang, W. and De-fu, L. Integrated guidance and control for missile using deep reinforcement learning, J. Astronaut., 2021, 42, (10), pp 1293–1304. doi: 10.3873/j.issn.1000-1328.2021.10.010 Google Scholar
Song, J., Luo, Y., Zhao, M., Hu, Y. and Zhang, Y. Fault-tolerant integrated guidance and control design for hypersonic vehicle based on ppo, Mathematics, 2022, 10, (18), pp 1–13. doi: 10.3390/math10183401 CrossRefGoogle Scholar
Wang, W., Xiong, S., Wang, S., Song, S. and Lai, C. Three dimensional impact angle constrained integrated guidance and control for missiles with input saturation and actuator failure, Aerospace Sci. Technol., 2016, 53, pp 169–187. doi: 10.1016/j.ast.2016.03.015 CrossRefGoogle Scholar
Wang, Z. and Yuan, J. Full state constrained adaptive fuzzy control for stochastic nonlinear switched systems with input quantization, IEEE Trans. Fuzzy Syst., 2020, 28, (4), pp 645–657. doi: 10.1109/TFUZZ.2019.2912150 CrossRefGoogle Scholar
Huang, J. Research on Command Filter Based Adaptive Control Algorithm For Nonlinear Systems with Full State Constraints, Master’s Thesis, Yangzhou University, No. 88, University South Road, Yangzhou City, Jiangsu Province, China, 2023.Google Scholar
Xia, J., Lian, Y., Su, S.-F., Shen, H. and Chen, G. Observer-based event-triggered adaptive fuzzy control for unmeasured stochastic nonlinear systems with unknown control directions, IEEE Trans. Cybern., 2022, 52, (10), pp 10655–10666. doi: 10.1109/TCYB.2021.3069853 CrossRefGoogle ScholarPubMed
He, W. and Dong, Y. Adaptive fuzzy neural network control for a constrained robot using impedance learning, IEEE Trans. Neural Networks Learn. Syst., 2018, 29, (4), pp 1174–1186. doi: 10.1109/TNNLS.2017.2665581 CrossRefGoogle ScholarPubMed
Wang, Z., Yuan, J., Pan, Y. and Che, D. Adaptive neural control for high order markovian jump nonlinear systems with unmodeled dynamics and dead zone inputs, Neurocomputing, 2017, 247, pp 62–72. doi: 10.1016/j.neucom.2017.03.041 CrossRefGoogle Scholar
Yu, D., Long, J., Chen, C.L.P. and Wang, Z. Adaptive swarm control within saturated input based on nonlinear coupling degree, IEEE Trans. Syst. Man Cybern. Syst., 2022, 52, (8), pp 4900–4911. doi: 10.1109/TSMC.2021.3102587 CrossRefGoogle Scholar