Hostname: page-component-669899f699-tpknm Total loading time: 0 Render date: 2025-04-25T20:36:54.603Z Has data issue: false hasContentIssue false
  • English
  • Français

An improved fuzzy inference strategy using reinforcement learning for trajectory-tracking of a mobile robot under a varying slip ratio

Published online by Cambridge University Press:  25 January 2024

Muhammad Qomaruz Zaman  and
Hsiu-Ming Wu Open the ORCID record for Hsiu-Ming Wu [Opens in a new window]
Muhammad Qomaruz Zaman
Affiliation:
Graduate Institute of Manufacturing Technology, National Taipei University of Technology, Taipei, Taiwan
Hsiu-Ming Wu*
Affiliation:
Department of Intelligent Automation Engineering, National Taipei University of Technology, Taipei, Taiwan
*
Corresponding author: Hsiu-Ming Wu; Email: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

In this study, a fuzzy reinforcement learning control (FRLC) is proposed to achieve trajectory tracking of a differential drive mobile robot (DDMR). The proposed FRLC approach designs fuzzy membership functions to fuzzify the relative position and heading between the current position and a prescribed trajectory. Instead of fuzzy inference rules, the relationship between the fuzzy inputs and actuator voltage outputs is built using a reinforcement learning (RL) agent. Herein, the deep deterministic policy gradient (DDPG) methodology consisted of actor and critic neural networks is employed in the RL agent. Simulations are conducted with considering varying slip ratio disturbances, different initial positions, and two different trajectories in the testing environment. In the meantime, a comparison with the classical DDPG model is presented. The results show that the proposed FRLC is capable of successfully tracking different trajectories under varying slip ratio disturbances as well as having performance superiority to the classical DDPG model. Moreover, experimental results validate that the proposed FRLC is also applicable to real mobile robots.

Keywords

fuzzy reinforcement learning controltrajectory-trackingdifferential drive mobile robotdeep deterministic policy gradientactor-critic neural networks
Type
Research Article
Information
Robotica , Volume 42 , Issue 4 , April 2024 , pp. 1134 - 1152
Copyright
© The Author(s), 2024. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

Dekle, R., “Robots and industrial labor: Evidence from Japan,” J. Jpn. Int. Econ. 58, 101108 (2020).CrossRefGoogle Scholar
Fan, H., Hu, Y. and Tang, L., “Labor costs and the adoption of robots in China,” J. Econ. Behav. Organ. 186, 608–631 (2021).CrossRefGoogle Scholar
Zhao, W., Wang, X., Qi, B. and Runge, T., “Ground-level mapping and navigating for agriculture based on iot and computer vision,” IEEE Access 8, 221975–221985 (2020).CrossRefGoogle Scholar
Fernandez, B., Herrera, P. J. and Cerrada, J. A., “A simplified optimal path following controller for an agricultural skid-steering robot,” IEEE Access 7, 95932–95940 (2019).CrossRefGoogle Scholar
Dutta, A., Roy, S., Kreidl, O. P. and Bölöni, L., “Multi-robot information gathering for precision agriculture: Current state, scope, and challenges,” IEEE Access 9, 161416–161430 (2021).CrossRefGoogle Scholar
Gao, X., Li, J., Fan, L., Zhou, Q., Yin, K., Wang, J., Song, C., Huang, L. and Wang, Z., “Review of wheeled mobile robots’ navigation problems and application prospects in agriculture,” IEEE Access 6, 49248–49268 (2018).CrossRefGoogle Scholar
Pak, J., Kim, J., Park, Y. and Son, H. I., “Field evaluation of path-planning algorithms for autonomous mobile robot in smart farms,” IEEE Access 10, 60253–60266 (2022).CrossRefGoogle Scholar
Jung, Y., Kim, Y., Lee, W. H., Bang, M. S., Kim, Y. and Kim, S., “Path planning algorithm for an autonomous electric wheelchair in hospitals,” IEEE Access 8, 208199–208213 (2020).CrossRefGoogle Scholar
Niloy, M. A. K., Shama, A., Chakrabortty, R. K., Ryan, M. J., Badal, F. R., Tasneem, Z., Ahamed, M. H., Moyeen, S. I., Das, S. K., Ali, M. F., Islam, M. R. and Saha, D. K., “Critical design and control issues of indoor autonomous mobile robots: A review,” IEEE Access 9, 35338–35370 (2021).CrossRefGoogle Scholar
Chen, C. S., Lin, C. J. and Lai, C. C., “Non-contact service robot development in fast-food restaurants,” IEEE Access 10, 31466–31479 (2022).CrossRefGoogle Scholar
Zheng, Y., Chen, S. and Cheng, H., “Real-time cloud visual simultaneous localization and mapping for indoor service robots,” IEEE Access 8, 16816–16829 (2020).CrossRefGoogle Scholar
de Jesus, J. C., Bottega, J. A., de Souza Leite Cuadros, M. A. and Gamarra, D. F. T., “Deep deterministic policy gradient for navigation of mobile robots,” J. Intell. Fuzzy Syst. 40(1), 349–361 (2021).CrossRefGoogle Scholar
Quan, H., Li, Y. and Zhang, Y., “A novel mobile robot navigation method based on deep reinforcement learning,” Int. J. Adv. Rob. Syst. 17(3), 5 (2020).Google Scholar
Gao, J., Ye, W., Guo, J. and Li, Z., “Deep reinforcement learning for indoor mobile robot path planning,” Ah S Sens. 20(19), 5493 (2020).CrossRefGoogle ScholarPubMed
Khlif, N., Nahla, K. and Safya, B., “Reinforcement learning with modified exploration strategy for mobile robot path planning,” Robotica 41(9), 1–15 (2023).CrossRefGoogle Scholar
Gao, X., Gao, R., Liang, P., Zhang, Q., Deng, R. and Zhu, W., “A hybrid tracking control strategy for nonholonomic wheeled mobile robot incorporating deep reinforcement learning approach,” IEEE Access 9, 15592–15602 (2021).CrossRefGoogle Scholar
Bai, C., Yan, P., Pan, W. and Guo, J., “Learning-based multi-robot formation control with obstacle avoidance,” IEEE Trans. Intell. Transp. 23(8), 11811–11822 (2021).CrossRefGoogle Scholar
Wu, K., Wang, H., Esfahani, M. A. and Yuan, S., “Bnd*-ddqn: Learn to steer autonomously through deep reinforcement learning,” IEEE Trans. Cognit. Dev. Syst. 13(2), 249–261 (2021).CrossRefGoogle Scholar
Song, H., Li, A., Wang, T. and Wang, M., “Multimodal deep reinforcement learning with auxiliary task for obstacle avoidance of indoor mobile robot,” Ah S Sens. 21(4), 2 (2021).Google ScholarPubMed
Shi, H., Shi, L., Xu, M. and Hwang, K.-S., “End-to-end navigation strategy with deep reinforcement learning for mobile robots,” IEEE Trans. Ind. Inf. 16(4), 2393–2402 (2020).CrossRefGoogle Scholar
Wang, Y., He, H. and Sun, C., “Learning to navigate through complex dynamic environment with modular deep reinforcement learning,” IEEE Trans. Games 10(4), 400–412 (2018).CrossRefGoogle Scholar
Luong, M. and Pham, C., “Incremental learning for autonomous navigation of mobile robots based on deep reinforcement learning,” J. Intell. Rob. Syst. 101(1), 1 (2021).CrossRefGoogle Scholar
Peng, X., Chen, R., Zhang, J., Chen, B., Tseng, H.-W., Wu, T.-L. and Meen, T.-H., “Enhanced autonomous navigation of robots by deep reinforcement learning algorithm with multistep method,” Sens. Mater. 33(2), SI, 825–842 (2021).Google Scholar
Xie, L., Miao, Y., Wang, S., Blunsom, P., Wang, Z., Chen, C., Markham, A. and Trigoni, N., “Learning with stochastic guidance for robot navigation,” IEEE Trans. Neural Network Learn. Syst. 32(1), 166–176 (2021).CrossRefGoogle ScholarPubMed
Ye, C., Shao, J., Liu, Y. and Yu, S., “Fuzzy active disturbance rejection control method for an omnidirectional mobile robot with my3 wheel,” Ind. Robot Int. J. Rob. Res. Appl. 50(4), 706–716 (2023).CrossRefGoogle Scholar
Boo, J. and Chwa, D., “Fuzzy integral sliding mode observer-based formation control of mobile robots with kinematic disturbance and unknown leader and follower velocities,” IEEE Access 10, 76926–76938 (2022).CrossRefGoogle Scholar
Van, M. and Ge, S. S., “Adaptive fuzzy integral sliding-mode control for robust fault-tolerant control of robot manipulators with disturbance observer,” IEEE Trans. Fuzzy Syst. 29(5), 1284–1296 (2021).CrossRefGoogle Scholar
Chwa, D. and Boo, J., “Adaptive fuzzy output feedback simultaneous posture stabilization and tracking control of wheeled mobile robots with kinematic and dynamic disturbances,” IEEE Access 8, 228863–228878 (2020).CrossRefGoogle Scholar
Cherroun, L., Boumehraz, M. and Kouzou, A., Mobile Robot Path Planning Based on Optimized Fuzzy Logic Controllers (Springer Singapore, Singapore, 2019) pp. 255–283.Google Scholar
Campos, J., Jaramillo, S., Morales, L., Camacho, O., Chávez, D. and Pozo, D., “Pso Tuning for Fuzzy pd + i Controller Applied to a Mobile Robot Trajectory Control,” 2018 International Conference on Information Systems and Computer Science (INCISCOS) (2018) pp. 62–68.Google Scholar
Subbash, P. and Chong, K. T., “Adaptive network fuzzy inference system based navigation controller for mobile robot,” Front. Inf. Tech. Electron. Eng. 20(2), 141–151 (2019).CrossRefGoogle Scholar
Saidi, Y., Nemra, A. and Tadjine, M., “Robust mobile robot navigation using fuzzy type 2 with wheel slip dynamic modeling and parameters uncertainties,” Int. J. Modell. Simul. 40(6), 397–420 (2020).CrossRefGoogle Scholar
Amador-Angulo, L., Castillo, O., Melin, P. and Castro, J. R., “Interval type-3 fuzzy adaptation of the bee colony optimization algorithm for optimal fuzzy control of an autonomous mobile robot,” Micromachines-BASEL 13(9), 1490 (2022).CrossRefGoogle ScholarPubMed
Ha, V. Q., Pham, S. H.-T. and Vu, N. T.-T., “Adaptive fuzzy type-ii controller for wheeled mobile robot with disturbances and wheelslips,” J. Rob. 2021, 1–11 (2021).CrossRefGoogle Scholar
Hua, G., Wang, F., Zhang, J., Alattas, K. A., Mohammadzadeh, A. and The Vu, M., “A new type-3 fuzzy predictive approach for mobile robots,” Mathematics 10(17), 3186 (2022).CrossRefGoogle Scholar
Prieto, P. J., Aguilar, L. T., Cardenas-Maciel, S. L., Lopez-Renteria, J. A. and Cazarez-Castro, N. R., “Stability analysis for mamdani-type integral fuzzy-based sliding-mode control of systems under persistent disturbances,” IEEE Trans. Fuzzy Syst. 30(6), 1640–1647 (2022).CrossRefGoogle Scholar
Selvachandran, G., Quek, S. G., Lan, L. T. H., Son, L. H., Giang, N. L., Ding, W., Abdel-Basset, M. and de Albuquerque, V. H. C., “A new design of mamdani complex fuzzy inference system for multiattribute decision making problems,” IEEE Trans. Fuzzy Syst. 29(4), 716–730 (2021).CrossRefGoogle Scholar
Xia, H., Tang, J., Yu, W., Cui, C. and Qiao, J., “Takagi–sugeno fuzzy regression trees with application to complex industrial modeling,” IEEE Trans. Fuzzy Syst. 31(7), 2210–2224 (2023).CrossRefGoogle Scholar
Tognetti, E. S. and Linhares, T. M., “Dynamic output feedback controller design for uncertain takagi–sugeno fuzzy systems: A premise variable selection approach,” IEEE Trans. Fuzzy Syst. 29(6), 1590–1600 (2021).CrossRefGoogle Scholar
Ruiz-García, G., Hagras, H., Pomares, H. and Ruiz, I. R., “Toward a fuzzy logic system based on general forms of interval type-2 fuzzy sets,” IEEE Trans. Fuzzy Syst. 27(12), 2381–2395 (2019).CrossRefGoogle Scholar
Singh, D. J., Verma, N. K., Ghosh, A. K. and Malagaudanavar, A., “An approach towards the design of interval type-3 t–s fuzzy system,” IEEE Trans. Fuzzy Syst. 30(9), 3880–3893 (2022).CrossRefGoogle Scholar
Mendel, J. M., “Comparing the performance potentials of interval and general type-2 rule-based fuzzy systems in terms of sculpting the state space,” IEEE Trans. Fuzzy Syst. 27(1), 58–71 (2019).CrossRefGoogle Scholar
Hu, Q. and Zheng, B., “An efficient takagi–sugeno fuzzy zeroing neural network for solving time-varying sylvester equation,” IEEE Trans. Fuzzy Syst. 31(7), 2401–2411 (2023).CrossRefGoogle Scholar
Zhang, L., Shi, Y., Chang, Y.-C. and Lin, C.-T., “Federated fuzzy neural network with evolutionary rule learning,” IEEE Trans. Fuzzy Syst. 31(5), 1653–1664 (2023).CrossRefGoogle Scholar
Islam, M. A., Anderson, D. T., Pinar, A. J., Havens, T. C., Scott, G. and Keller, J. M., “Enabling explainable fusion in deep learning with fuzzy integral neural networks,” IEEE Trans. Fuzzy Syst. 28(7), 1291–1300 (2020).CrossRefGoogle Scholar
Dombi, J. and Hussain, A., “Data-driven interval type-2 fuzzy inference system based on the interval type-2 distending function,” IEEE Trans. Fuzzy Syst. 31(7), 2345–2359 (2023).CrossRefGoogle Scholar
Velagic, J., Lacevic, B. and Osmic, N.. Nonlinear Motion Control of Mobile Robot Dynamic Model (IntechOpen, Rijeka, 2008) ch. 27.CrossRefGoogle Scholar
Hatab, R., “Dynamic modelling of differential-drive mobile robots using lagrange and newton-euler methodologies: A unified framework,” Adv. Rob. Autom. 02, 1000107 (2013).Google Scholar
Ali, M. A. and Mailah, M., “A simulation and experimental study on wheeled mobile robot path control in road roundabout environment,” Int. J. Adv. Rob. Syst. 16(2), 1729881419834778 (2019).Google Scholar
Stefek, A., Pham, T. V., Krivanek, V. and Pham, K. L., “Energy comparison of controllers used for a differential drive wheeled mobile robot,” IEEE Access 8, 170915–170927 (2020).CrossRefGoogle Scholar
Ma, Y., Zhao, J., Zhao, H., Lu, C. and Chen, H., “Mpc-based slip ratio control for electric vehicle considering road roughness,” IEEE Access 7, 52405–52413 (2019).CrossRefGoogle Scholar
Cao, K., Hu, M., Wang, D., Qiao, S., Guo, C., Fu, C. and Zhou, A., “All-wheel-drive torque distribution strategy for electric vehicle optimal efficiency considering tire slip,” IEEE Access 9, 25245–25257 (2021).CrossRefGoogle Scholar
Zhang, N., Han, Z., Zhang, Z., Guo, K. and Lu, X., “MAS-based slip ratio fault-tolerant control in finite time for EV,” IEEE Access 9, 45642–45654 (2021).CrossRefGoogle Scholar
Tzafestas, S. G.. Introduction to Mobile Robot Control (Elsevier, Oxford, 2013).Google Scholar
Nurmaini, S. and Chusniah, “Differential Drive Mobile Robot Control using Variable Fuzzy Universe of Discourse,” 2017 International Conference on Electrical Engineering and Computer Science (ICECOS) (2017) pp. 50–55.Google Scholar
Chapman, S. J., Fortran 90/95 for Scientists and Engineers, 1st ed. (WCB/McGraw-Hill, Boston, 1998).Google Scholar
MathWorks, MATLAB The Language of Technical Computing: Function Reference Volume 1: A - E Version 7 (The MathWorks, Inc, Natick, 2004).Google Scholar
Torres, V. and Valls, J., “A fast and low-complexity operator for the computation of the arctangent of a complex number,” IEEE Trans. Very Large Scale Integr. (VLSI) Syst. 25(9), 2663–2667 (2017).CrossRefGoogle Scholar
Zhou, J., Dietrich, M., Walden, P., Kolb, J. and Doppelbauer, M., “The Resolution of atan2-Function,” 2020 IEEE Sensors (2020) pp. 1–4.Google Scholar
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N. M. O., Erez, T., Tassa, Y., Silver, D. and Wierstra, D., Continuous control with deep reinforcement learning, CoRR, vol. abs/1509.02971 (2016).Google Scholar
Zhang, M., Zhang, Y., Gao, Z. and He, X., “An improved ddpg and its application based on the double-layer bp neural network,” IEEE Access 8, 177734–177744 (2020).CrossRefGoogle Scholar
Liu, D., Wang, W., Wang, L., Jia, H. and Shi, M., “Dynamic pricing strategy of electric vehicle aggregators based on ddpg reinforcement learning algorithm,” IEEE Access 9, 21556–21566 (2021).CrossRefGoogle Scholar
Watkins, C. and Dayan, P., “Q-learning,” Mach. Learn. 8(3-4), 279–292 (5 1992).CrossRefGoogle Scholar