Hostname: page-component-7bb8b95d7b-nptnm Total loading time: 0 Render date: 2024-09-19T16:56:13.488Z Has data issue: false hasContentIssue false
  • English
  • Français

Prediction of abnormal gait behavior of lower limbs based on depth vision

Published online by Cambridge University Press:  18 September 2024

Tie Liu Open the ORCID record for Tie Liu [Opens in a new window] ,
Dianchun Bai ,
Hongyu Yi  and
Hiroshi Yokoi
Tie Liu
Affiliation:
School of Electrical Engineering, Shenyang University of Technology, Shenyang 110870, China
Dianchun Bai*
Affiliation:
School of Electrical Engineering, Shenyang University of Technology, Shenyang 110870, China
Hongyu Yi
Affiliation:
Shenyang Fire Research Institute of Emergency Management Ministry, Shenyang 110034, China
Hiroshi Yokoi
Affiliation:
Department of Mechanical Engineering and Intelligent Systems, The University of Electro-Communications, Chofu 182-8585, Japan
*
Corresponding author: Dianchun Bai; Email: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

As a kind of lower-limb motor assistance device, the intelligent walking aid robot plays an essential role in helping people with lower-limb diseases to carry out rehabilitation walking training. In order to enhance the safety performance of the lower-limb walking aid robot, this study proposes a deep vision-based abnormal lower-limb gait prediction model construction method for the problem of abnormal gait prediction of patients’ lower limbs. The point cloud depth vision technique is used to acquire lower-limb motion data, and a multi-posture angular prediction model is trained using long and short-term memory networks to build a model of the user’s lower-limb posture characteristics during normal walking as well as a real-time lower-limb motion prediction model. The experimental results indicate that the proposed lower-limb abnormal behavior prediction model is able to achieve a 97.4% prediction rate of abnormal lower-limb movements within 150 ms. Additionally, the model demonstrates strong generalization ability in practical applications. This paper proposes further ideas to enhance the safety performance of lower-limb rehabilitation robot use for patients with lower-limb disabilities.

Keywords

electronic walkermachine visiondepth learningbehavior prediction
Type
Research Article
Information
Robotica , First View , pp. 1 - 22
Copyright
© The Author(s), 2024. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Chen, B., Ma, H., Qin, L.-Y., Gao, F., Chan, K.-M., Law, S.-W., Qin, L. and Liao, W.-H., “Recent developments and challenges of lower extremity exoskeletons,” J Orthop Transl 5, 26–37 (2016).Google ScholarPubMed
Wang, Y.-L., Wang, K.-Y., Zhao, W.-Y., Wang, W.-L., Han, Z., & Zhang, Z.-X., “Effects of single crouch walking gaits on fatigue damages of lower extremity main muscles, ”J Mech Med Biol 19(07), 1940046 (2019).CrossRefGoogle Scholar
Godecke, E., Armstrong, E., Rai, T., Ciccone, N., Rose, M. L., Middleton, S., Whitworth, A., Holland, A., Ellery, F., Hankey, G. J., Cadilhac, D. A. and Bernhardt, J., “A randomized control trial of intensive aphasia therapy after acute stroke: The very early rehabilitation for spEech (VERSE) study,” Int J Stroke 16(5), 556–572 (2020).CrossRefGoogle ScholarPubMed
Yen, H.-C., Jeng, J.-S., Chen, W.-S., Pan, G.-S., Chuang, W.-Y., Lee, Y.-Y. and Teng, T., “Early mobilization of mild-moderate intracerebral hemorrhage patients in a stroke center: A randomized controlled trial,” Neurorehabil Neural Repair 34(1), 72–81 (2019).CrossRefGoogle Scholar
Yan, Q., Huang, J., Tao, C., Chen, X. and Xu, W., “Intelligent mobile walking-aids: Perception, control and safety,” Adv Robotics 34(1), 2–18 (2020).CrossRefGoogle Scholar
Ominato, K. and Murakami, T., “A Stabilization Control in Two-Wheeled Walker with Passive Mechanism for Walking Support, IECON,” In: 2020 The 46th Annual Conference of the IEEE Industrial Electronics Society, (2020) pp. 65–70.Google Scholar
Oyman, E. L., Korkut, M. Y., Yilmaz, C., Bayraktaroglu, Z. Y. and Arslan, M. S., “Design and control of a cable-driven rehabilitation robot for upper and lower limbs,” Robot 40(1), 1–37 (2022).CrossRefGoogle Scholar
Ye, Y., Zhu, M.-X., Ou, C.-W., Wang, B.-Z., Wang, L. and Xie, N.-G., “Online pattern recognition of lower limb movements based on sEMG signals and its application in real-time rehabilitation training,” Robotica 42(2), 389–414 (2024).CrossRefGoogle Scholar
Wang, Y.-L., Wang, K.-Y., Chai, Y.-J., Mo, Z.-J. and Wang, K.-C., “Research on mechanical optimization methods of cable-driven lower limb rehabilitation robot,” Robotica 40(1), 154–169 (2022).CrossRefGoogle Scholar
Wang, J.-H. and Kim, J.-Y., “Development of a whole-body walking rehabilitation robot and power assistive method using EMG signals,” Intel Serv Robot 16(2), 139–153 (2023).CrossRefGoogle Scholar
Gonçalves, R. S. and Rodrigues, L. A. O., “Development of nonmotorized mechanisms for lower limb rehabilitation,” Robotica 40(1), 102–119 (2022).CrossRefGoogle Scholar
Hwang, S. H., Sun, D. I., Han, J. and Kim, W.-S., “Gait pattern generation algorithm for lower-extremity rehabilitation-exoskeleton robot considering wearer’s condition,” Intel Serv Robot 14(3), 345–355 (2021).CrossRefGoogle Scholar
Ophaswongse, C., Murray, R. C., Santamaria, V., Wang, Q. and Agrawal, S. K., “Human evaluation of wheelchair robot for active postural support (WRAPS),” Robotica 37(12), 2132–2146 (2019).CrossRefGoogle Scholar
Li, Y., Wang, Y., Yuan, S. and Fei, Y., “Design, modeling, and control of a novel soft-rigid knee joint robot for assisting motion,” Robotica 42(3), 817–832 (2024).CrossRefGoogle Scholar
Semwal, V. B., Jain, R., Maheshwari, P. and Khatwani, S., “Gait reference trajectory generation at different walking speeds using LSTM and CNN,” Multimed Tools Appl 82(21), 33401–33419 (2023).CrossRefGoogle Scholar
Kumar, S., Yadav, P. and Semwal, V. B., “A Comprehensive Analysis of Lower Extremity Based Gait Cycle Disorders and Muscle Analysis,” In: Machine Learning, Image Processing, Network Security and Data Sciences (Khare, N., Tomar, D. S., Ahirwal, M. K., Semwal, V. B. and Soni, V., eds.) (Springer Nature Switzerland, Cham, 2022) pp. 325–336.CrossRefGoogle Scholar
Zhang, P., Zhang, J. and Elsabbagh, A., “Fuzzy radial-based impedance controller design for lower limb exoskeleton robot,” Robotica 41(1), 326–345 (2023).CrossRefGoogle Scholar
Yuqi, W., Jinjiang, C., Ranran, G., Lei, Z. and Lei, W., “Study on the design and control method of a wire-driven waist rehabilitation training parallel robot,” Robotica 40(10), 3499–3513 (2022).CrossRefGoogle Scholar
Ercolano, G. and Rossi, S., “Combining CNN and LSTM for activity of daily living recognition with a 3D matrix skeleton representation,” Intel Serv Robot 14(2), 175–185 (2021).CrossRefGoogle Scholar
Qin, T., Yang, Y., Wen, B., Chen, Z., Bao, Z., Dong, H., Dou, K. and Yang, C., “Research on human gait prediction and recognition algorithm of lower limb-assisted exoskeleton robot,” Intel Serv Robot 14(3), 445–457 (2021).CrossRefGoogle Scholar
Challa, S. K., Kumar, A., Semwal, V. B. and Dua, N., “An optimized-LSTM and RGB-D sensor-based human gait trajectory generator for bipedal robot walking,” IEEE Sens J 22(24), 24352–24363 (2022).CrossRefGoogle Scholar
Lou, Y., Wang, R., Mai, J., Wang, N. and Wang, Q., “IMU-based gait phase recognition for stroke survivors,” Robotica 37(12), 2195–2208 (2019).CrossRefGoogle Scholar
Semwal, V. B., Mazumdar, A., Jha, A., Gaud, N. and Bijalwan, V., Speed, Cloth and Pose Invariant Gait Recognition-Based Person Identification,” In: Machine Learning: Theoretical Foundations and Practical Applications (Pandey, M. and Rautaray, S. S., eds.) (Springer Singapore, Singapore, 2021) pp. 39–56.CrossRefGoogle Scholar
Semwal, V. B., Kim, Y., Bijalwan, V., Verma, A., Singh, G., Gaud, N., Baek, H. and Khan, A. M., “Development of the LSTM model and universal polynomial equation for all the sub-phases of human gait,” IEEE Sens J 23(14), 15892–15900 (2023).CrossRefGoogle Scholar
Gaglio, S., Re, G. L. and Morana, M., “Human activity recognition process using 3-D posture data,” IEEE Trans Hum-Mach Syst 45(5), 586–597 (2015).CrossRefGoogle Scholar
Xu, W., Huang, J. and Cheng, L., “A novel coordinated motion fusion-based walking-aid robot system,” Sensors 18(9), 2761 (2018).CrossRefGoogle ScholarPubMed
Xu, W., Xiang, D., Wang, G., Liao, R., Shao, M. and Li, K., “Multiview video-based 3-D pose estimation of patients in computer-assisted rehabilitation environment (CAREN),” IEEE Trans Hum-Mach Syst 52(2), 196–206 (2022).CrossRefGoogle Scholar
White, J., Kameneva, T. and McCarthy, C., “Vision processing for assistive vision: A deep reinforcement learning approach,” IEEE Trans Hum-Mach Syst 52(1), 123–133 (2022).CrossRefGoogle Scholar
Yang, L., Ren, Y. and Zhang, W., “3D depth image analysis for indoor fall detection of elderly people,” Digit Commun Netw 2(1), 24–34 (2016).CrossRefGoogle Scholar
Lee, C. K. and Lee, V. Y., “Fall Detection System Based on Kinect Sensor Using Novel Detection and Posture Recognition Algorithm,” In: Inclusive Society: Health and Wellbeing in the Community, and Care at Home (Biswas, J., Kobayashi, H., Wong, L., Abdulrazak, B. and Mokhtari, M., eds.) (Springer Berlin Heidelberg,, Berlin, Heidelberg, 2013) pp. 238–244.CrossRefGoogle Scholar
Gupta, A. and Semwal, V. B., “Occluded gait reconstruction in multi person gait environment using different numerical methods,” Multimed Tools Appl 81(16), 23421–23448 (2022).CrossRefGoogle Scholar
Stübinger, J. and Walter, D., “Using multi-dimensional dynamic time warping to identify time-varying lead-lag relationships,” Sesnors 22(18), 6884 (2022).Google ScholarPubMed
Tax, D. M. J. and Duin, R. P. W., “Support vector data description,” Mach Learn 54(1), 45–66 (2004).CrossRefGoogle Scholar