Hostname: page-component-586b7cd67f-l7hp2 Total loading time: 0 Render date: 2024-11-22T00:06:36.090Z Has data issue: false hasContentIssue false
  • English
  • Français

Shared control methodology based on head positioning and vector fields for people with quadriplegia

Published online by Cambridge University Press:  20 May 2021

Guilherme M. Maciel ,
Milena F. Pinto Open the ORCID record for Milena F. Pinto [Opens in a new window] ,
Ivo C. da S. Júnior ,
Fabricio O. Coelho ,
Andre L. M. Marcato Open the ORCID record for Andre L. M. Marcato [Opens in a new window]  and
Marcelo M. Cruzeiro
Guilherme M. Maciel*
Affiliation:
Department of Electrical Engineering, Federal University of Juiz de Fora (UFJF), Juiz de Fora, Brazil
Milena F. Pinto
Affiliation:
Department of Electronics, Federal Center for Technological Education of Rio de Janeiro (CEFET-RJ), Rio de Janeiro, Brazil
Ivo C. da S. Júnior
Affiliation:
Department of Electrical Engineering, Federal University of Juiz de Fora (UFJF), Juiz de Fora, Brazil
Fabricio O. Coelho
Affiliation:
Department of Electrical Engineering, Federal University of Juiz de Fora (UFJF), Juiz de Fora, Brazil
Andre L. M. Marcato
Affiliation:
Department of Electrical Engineering, Federal University of Juiz de Fora (UFJF), Juiz de Fora, Brazil
Marcelo M. Cruzeiro
Affiliation:
Department of Clinical Medicine, Federal University of Juiz de Fora (UFJF), Juiz de Fora, Brazil
*
*Corresponding author. Email: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

Mobile robotic systems are used in a wide range of applications. Especially in the assistive field, they can enhance the mobility of the elderly and disable people. Modern robotic technologies have been implemented in wheelchairs to give them intelligence. Thus, by equipping wheelchairs with intelligent algorithms, controllers, and sensors, it is possible to share the wheelchair control between the user and the autonomous system. The present research proposes a methodology for intelligent wheelchairs based on head movements and vector fields. In this work, the user indicates where to go, and the system performs obstacle avoidance and planning. The focus is developing an assistive technology for people with quadriplegia that presents partial movements, such as the shoulder and neck musculature. The developed system uses shared control of velocity. It employs a depth camera to recognize obstacles in the environment and an inertial measurement unit (IMU) sensor to recognize the desired movement pattern measuring the user’s head inclination. The proposed methodology computes a repulsive vector field and works to increase maneuverability and safety. Thus, global localization and mapping are unnecessary. The results were evaluated by simulated models and practical tests using a Pioneer-P3DX differential robot to show the system’s applicability.

Keywords

shared controlvector fieldsmobile robotshead positioningquadriplegia
Type
Article
Information
Robotica , Volume 40 , Issue 2 , February 2022 , pp. 348 - 364
Copyright
© The Author(s), 2021. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Fattal, C., Leynaert, V., Laffont, I., Baillet, A., Enjalbert, M. and Leroux, C., “Sam, an assistive robotic device dedicated to helping persons with quadriplegia: Usability study,” Int. J. Soc. Rob. 11, 1–15 (2018).Google Scholar
Jiang, H., Duerstock, B. S. and Wachs, J. P., “Variability analysis on gestures for people with quadriplegia,” IEEE Trans. Cybern. 48(1), 346–356 (2016).CrossRefGoogle ScholarPubMed
Kader, M. A., Alam, M. E., Jahan, N., Bhuiyan, M. A. B., Alam, M. S. and Sultana, Z., “Design and Implementation of a Head Motion-Controlled Semi-Autonomous Wheelchair for Quadriplegic Patients Based on 3-Axis Accelerometer,” 2019 22nd International Conference on Computer and Information Technology (ICCIT) (2019) pp. 1–6.Google Scholar
Wolpert, D. M., Doya, K. and Kawato, M., “A unifying computational framework for motor control and social interaction,” Philos. Trans. R. Soc. London Ser. B Biol. Sci. 358(1431), 593–602 (2003).CrossRefGoogle ScholarPubMed
Hirata, Y. and Kosuge, K., “Distributed Robot Helpers Handling a Single Object in Cooperation with a Human,” Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No. 00CH37065), vol. 1 (2000) pp. 458–463.Google Scholar
Neto, W. A., Pinto, M. F., Marcato, A. L., da Silva, I. C. and d. A. Fernandes, D., “Mobile robot localization based on the novel leader-based bat algorithm,” J. Control Autom. Electr. Syst. 30(3), 1–10 (2019).CrossRefGoogle Scholar
Saxena, A., Driemeyer, J. and Ng, A. Y., “Robotic grasping of novel objects using vision,” Int. J. Rob. Res. 27(2), 157–173 (2008).CrossRefGoogle Scholar
Gelin, R., Detriche, J., Lambert, J. and Malblanc, P., “The sprint of coach,” Proceedings of IEEE Systems Man and Cybernetics Conference-SMC, vol. 3 (IEEE, 1993) pp. 547–552.Google Scholar
Simpson, R., LoPresti, E., Hayashi, S., Guo, S., Ding, D., Ammer, W., Sharma, V. and Cooper, R., “A prototype power assist wheelchair that provides for obstacle detection and avoidance for those with visual impairments,” J. NeuroEng. Rehabil. 2(1), 30 (2005).CrossRefGoogle ScholarPubMed
Levine, S. P., Bell, D. A., Jaros, L. A., Simpson, R. C., Koren, Y. and Borenstein, J., “The navchair assistive wheelchair navigation system,” IEEE Trans. Rehabil. Eng. 7(4), 443–451 (1999).CrossRefGoogle ScholarPubMed
Lauretti, C., Cordella, F., Guglielmelli, E. and Zollo, L., “Learning by demonstration for planning activities of daily living in rehabilitation and assistive robotics,” IEEE Rob. Autom. Lett. 2(3), 1375–1382 (2017).CrossRefGoogle Scholar
Schwesinger, D., Shariati, A., Montella, C. and Spletzer, J., “A smart wheelchair ecosystem for autonomous navigation in urban environments,” Auto. Rob. 41(3), 519–538 (2017).CrossRefGoogle Scholar
Mandel, C., Laue, T. and Autexier, S., “Smart-Wheelchairs,” In: Smart Wheelchairs and Brain-Computer Interfaces (2018) pp. 291–322.Google Scholar
Leaman, J. and La, H. M., “A comprehensive review of smart wheelchairs: past, present, and future,” IEEE Trans. Hum. Mach. Syst. 47(4), 486–499 (2017).CrossRefGoogle Scholar
Pereira, G., “Comparison of Human Machine Interfaces to Control a Robotized Wheelchair,” XIII Simpósio Brasileiro de Automação Inteligente - Porto Alegre, RS (2017) pp. 2301–2306.Google Scholar
Long, J., Li, Y., Wang, H., Yu, T., Pan, J. and Li, F., “A hybrid brain computer interface to control the direction and speed of a simulated or real wheelchair,” IEEE Trans. Neural Syst. Rehabil. Eng. 20(5), 720–729 (2012).CrossRefGoogle ScholarPubMed
Ren, Y., Wu, Y.-N., Yang, C.-Y., Xu, T., Harvey, R. L. and Zhang, L.-Q., “Developing a wearable ankle rehabilitation robotic device for in-bed acute stroke rehabilitation,” IEEE Trans. Neural Syst. Rehabil. Eng. 25(6), 589–596 (2016).CrossRefGoogle ScholarPubMed
Schettino, V. and Demiris, Y., “Inference of User-Intention in Remote Robot Wheelchair Assistance Using Multimodal Interfaces,” Proceedings of iROS2019 (to appear).CrossRefGoogle Scholar
Escobedo, A., Spalanzani, A. and Laugier, C., “Multimodal Control of a Robotic Wheelchair: Using Contextual Information for Usability Improvement,” International Conference on Intelligent Robots and Systems (IROS) (IEEE, 2013) pp. 4262–4267.CrossRefGoogle Scholar
Kim, J., Park, H., Bruce, J., Sutton, E., Rowles, D. M., Pucci, D., Holbrook, J., Minocha, J., Nardone, B., West, D. P., Laumann, A. E., Roth, E. J., Jones, M., Veledar, E. and Ghovanloo, M., “The tongue enables computer and wheelchair control for people with spinal cord injury,” Sci. Trans. Med. 5(213), 166–213 (2013).CrossRefGoogle ScholarPubMed
Naeem, A., Qadar, A. and Safdar, W., “Voice controlled intelligent wheelchair using raspberry pi,” Int. J. Technol. Res. 2(2), 65 (2014).Google Scholar
Liu, Y., Li, Z., Zhang, T. and Zhao, S., “Brain-robot interface-based navigation control of a mobile robot in corridor environments,” IEEE Trans. Syst. Man Cybern. Syst. 50(8), 3047–3058 (2018).CrossRefGoogle Scholar
Ruiz-Gómez, S., Gómez, C., Poza, J., Tobal, G. C. G., Arribas, M. A. T., Cano, M. and Hornero, R., “Automated multiclass classification of spontaneous EEG activity in Alzheimer’s disease and mild cognitive impairment,” Entropy, Multidisciplinary Digital Publishing Institute, vol. 20 (Elsevier, 2018) pp. 22–30.CrossRefGoogle Scholar
Rohmer, E., Pinheiro, P., Cardozo, E., Bellone, M. and Reina, G., “Laser Based Driving Assistance for Smart Robotic Wheelchairs,” 2015 IEEE 20th Conference on Emerging Technologies & Factory Automation (ETFA) (IEEE, 2015) pp. 1–4.CrossRefGoogle Scholar
Shinde, K. D., Tarannum, S., Veerabhadrappa, T., Gagan, E. and Kumar, P. V., “Implementation of Low Cost, Reliable, and Advanced Control with Head Movement, Wheelchair for Physically Challenged People,” In: Progress in Advanced Computing and Intelligent Engineering (Springer, 2018) pp. 313–328.CrossRefGoogle Scholar
Marins, G., Carvalho, D., Marcato, A. and Junior, I., “Development of a Control System for Electric Wheelchairs Based on Head Movements,” Intelligent Systems Conference (IntelliSys) (IEEE, 2017) pp. 996–1001.Google Scholar
Torkia, C., Reid, D., Korner-Bitensky, N., Kairy, D., Rushton, P. W., Demers, L. and Archambault, P. S., “Power wheelchair driving challenges in the community: A users’ perspective,” Disability Rehabil. Assistive Technol. 10(3), 211–215 (2015).CrossRefGoogle ScholarPubMed
Chen, S.-H., Chen, Y.-L., Chiou, Y.-H., Tsai, J.-C. and Kuo, T.-S., “Head-Controlled Device with M3S-Based for People with Disabilities,” Proceedings of the 25th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (IEEE Cat. No. 03CH37439), vol. 2 (IEEE, 2003) pp. 1587–1589.Google Scholar
Chauhan, R., Jain, Y., Agarwal, H. and Patil, A., “Study of Implementation of Voice Controlled Wheelchair,” 2016 3rd International Conference on Advanced Computing and Communication Systems (ICACCS), vol. 1 (2016) pp. 1–4. doi: 10.1109/ICACCS.2016.7586329.CrossRefGoogle Scholar
Huo, X. and Ghovanloo, M., “Evaluation of a wireless wearable tongue–computer interface by individuals with high-level spinal cord injuries,” J. Neural Eng. 7(2), 26008 (2010).CrossRefGoogle ScholarPubMed
Lin, C.-S., Ho, C.-W., Chen, W.-C., Chiu, C.-C. and Yeh, M.-S., “Powered wheelchair controlled by eye-tracking system.,” Optica Applicata 36 (2006).Google Scholar
Kucukyilmaz, A. and Demiris, Y., “Learning shared control by demonstration for personalized wheelchair assistance,” IEEE Trans. Haptics 11(3), 431–442 (2018).CrossRefGoogle Scholar
Kucukyilmaz, A. and Demiris, Y., “One-Shot Assistance Estimation from Expert Demonstrations for a Shared Control Wheelchair System,” 2015 24th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN) (IEEE, 2015) pp. 438–443.CrossRefGoogle Scholar
Ruzaij, M. F., Neubert, S., Stoll, N. and Thurow, K., “Multi-Sensor Robotic-Wheelchair Controller for Handicap and Quadriplegia Patients Using Embedded Technologies,” 2016 9th International Conference on Human System Interactions (HSI) (IEEE, 2016) pp. 103–109.CrossRefGoogle Scholar
Dey, P., Hasan, M. M., Mostofa, S. and Rana, A. I., “Smart Wheelchair Integrating Head Gesture Navigation,” 2019 International Conference on Robotics, Electrical and Signal Processing Techniques (ICREST) (IEEE, 2019) pp. 329–334.CrossRefGoogle Scholar
Jia, P. and Hu, H., “Head Gesture Based Control of an Intelligent Wheelchair,” Proceedings of the 11th Annual Conference of the Chinese Automation and Computing Society in the UK [CACSUK05] (2005) pp. 85–90.Google Scholar
Teodorescu, C. S., Zhang, B. and Carlson, T., “Probabilistic Shared Control for a Smart Wheelchair: A Stochastic Model-Based Framework,” 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC) (2019) pp. 3136–3141.Google Scholar
Triharminto, H. H., Wahyunggoro, O., Adji, T. B. and Cahyadi, A. I., “An integrated artificial potential field path planning with kinematic control for nonholonomic mobile robot,” Int. J. Adv. Sci. Eng. Inf. Technol. 6(4), 410–418 (2016).CrossRefGoogle Scholar
Seki, H., Shibayama, S., Kamiya, Y. and Hikizu, M., “Practical Obstacle Avoidance Using Potential Field for a Nonholonmic Mobile Robot with Rectangular Body,” International Conference on Emerging Technologies and Factory Automation (2008) pp. 326–332.Google Scholar
Diao, C., Jia, S., Zhang, G., Sun, Y., Zhang, X., Xue, Y. and Li, X., “Design and Realization of a Novel Obstacle Avoidance Algorithm for Intelligent Wheelchair bed Using Ultrasonic Sensors,” Chinese Automation Congress (CAC), 2017 (2017) pp. 4153–4158.Google Scholar
Olivi, L., Souza, R., Rohmer, E. and Cardozo, E., “Shared Control for Assistive Mobile Robots Based on Vector Fields,” International Conference on Ubiquitous Robots and Ambient Intelligence (URAI) (2013) pp. 96–101.Google Scholar
Wang, J., Chen, W. and Liao, W., “An improved localization and navigation method for intelligent wheelchair in narrow and crowded environments,” IFAC Proc. Volumes 46(13), 389–394 (2013).CrossRefGoogle Scholar
Tomari, M. R. M., Kobayashi, Y. and Kuno, Y., “Development of smart wheelchair system for a user with severe motor impairment,” Procedia Eng. 41, 538–546 (2012). doi: 10.1016/j.proeng.2012.07.209.CrossRefGoogle Scholar
Burhanpurkar, M., Labbé, M., Guan, C., Michaud, F. and Kelly, J., “Cheap or Robust? the Practical Realization of Self-Driving Wheelchair Technology,” International Conference on Rehabilitation Robotics (ICORR) (2017) pp. 1079–1086.Google Scholar
Huang, Q., Chen, Y., Zhang, Z., He, S., Zhang, R., Liu, J., Zhang, Y., Shao, M. and Li, Y., “An EOG-based wheelchair robotic arm system for assisting patients with severe spinal cord injuries,” J. Neural Eng. 16(2), 21–26 (2019).CrossRefGoogle ScholarPubMed
Qassim, H. M., Eesee, A. K., Osman, O. T. and Jarjees, M. S., “Controlling a motorized electric wheelchair based on face tilting,” Bio-Algorithms Med-Syst. 15(4), 1–7 (2019).Google Scholar
Pinto, M. F, Coelho, F. O, Souza, J. P., Melo, A. G., Marcato, A. L. M. and Urdiales, C., “EKF Design for Online Trajectory Prediction of a Moving Object Detected Onboard of a UAV,” 2018 13th APCA International Conference on Automatic Control and Soft Computing (CONTROLO) (IEEE, 2018) pp. 407–412.CrossRefGoogle Scholar
Almasri, M. M., Alajlan, A. M. and Elleithy, K. M., “Trajectory planning and collision avoidance algorithm for mobile robotics system,” IEEE Sens. J. 16(12), 5021–5028 (2016).CrossRefGoogle Scholar
Warren, C. W., “Global Path Planning Using Artificial Potential Fields,” Proceedings, 1989 International Conference on Robotics and Automation (1989) pp. 316–321.Google Scholar
Pinto, M. F., Mendonça, T. R., Olivi, L. R., Costa, E. B. and Marcato, A. L., “Modified Approach Using Variable Charges to Solve Inherent Limitations of Potential Fields Method,” 2014 11th IEEE/IAS International Conference on Industry Applications (INDUSCON) (2014) pp. 1–6.Google Scholar
Woods, A. C. and La, H. M., “A novel potential field controller for use on aerial robots,” IEEE Trans. Syst. Man Cybern. Syst. 49(4), 665–676 (2017).CrossRefGoogle Scholar
Coelho, F. O., Pinto, M. F., Souza, J. P. C. and Marcato, A. L., “Hybrid methodology for path planning and computational vision applied to autonomous mission: A new approach,” Robotica, 38(6), 1000–1018 (2020).CrossRefGoogle Scholar
“Smart wheelchair.” https://github.com/patilnabhi/nuric_wheelchair_model_02. Accessed 19th August 2019 (2016).Google Scholar
Hart, S. G., “Nasa-Task Load Index (NASA-TLX); 20 Years Later,” Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 50 (2006) pp. 904–908.Google Scholar
Maciel, G. M., Pinto, M. F., da S. Júnior, I. C. and Marcato, A. L. M., “Methodology for autonomous crossing narrow passages applied on assistive mobile robots,” J. Control Autom. Electr. Syst. 30(6), 943–953 (2019).CrossRefGoogle Scholar