1. Introduction
Surface electromyography (sEMG) signals are a kind of bioelectrical signals generated with muscle contraction, which contains a large amount of information about various human movements. The difference between different limb movements produces different characteristics of sEMG signals. Therefore, sEMG signals can directly reflect the state of muscle contractions [Reference Kusche and Ryschka1]. sEMG signals are usually generated 30–150 ms ahead of limb movements [Reference Bi, Feleke and Guan2] and can predict human behaviors. Researches show that the human brain has plasticity. Through a certain amount of repeated training, patients can remember muscle contractions and achieve a certain rehabilitation effect [Reference Hara3–Reference Govil, Noohu and Krebs5]. Therefore, it is of a great application value to recognize the patient’s movement intention based on the sEMG signals, and then the recognition results are used to control the lower limb rehabilitation robot to perform the corresponding movement and drive the lower limb disabled patients to carry out the rehabilitation training. However, the main key problems that have not been popularized and applied in clinical practice are as follows:
(1) sEMG signals have nonstationary and weak characteristics. To improve the recognition rate of multiple lower limb motion patterns, it is usually necessary to collect sEMG signals from multiple muscles of the lower limb. However, the sEMG disposable electrode is glued to the skin surface. When we collect data, the electrode piece is loose due to sweat or large amplitude of motion, and the collected data are inaccurate [Reference Scheme, Englehart and Hudgins6, Reference Ding, Zhao and Han7]. At the same time, multichannel acquisition will multiply the data dimension and increase the calculation amount of the host computer and the time cost [Reference Veer, Agarwal and Kumar8, Reference Phukpattaranont, Thongpanja, Anam, Al-Jumaily and Limsakul9]. More importantly, it is not convenient for some muscles of special disabled people to attach EMG electrode, and it is difficult to complete the multichannel acquisition. Therefore, both reducing the number of acquisition channels and improving the accuracy of pattern recognition need to be considered at the same time. Zheng et al. [Reference Zheng and Li10] collected the original sEMG signals of flexor carpi ulnaris and extensor carpi ulnaris of five subjects (four men and one woman) by using the self-developed EMG acquisition system with two channels, including fist clenching, palm extension, wrist extension, and wrist flexion. Further, the original sEMG signals were processed with 277 ms sliding window and active end detection technology, and the absolute mean, root mean square (RMS), and variance were extracted for the online pattern recognition of gesture. Through comparative analysis, the highest recognition rates of random tree, k-nearest neighbor (KNN), and support vector machine (SVM) could reach 95.19%, 97.12%, and 97.12%, but the time of test set based on SVM was shorter. Therefore, the classifier based on SVM had the best effect. Gupta et al. [Reference Gupta and Agarwal11] used two channels to collect the sEMG signals of 25 subjects from 2 groups of experiments for 5 lower limb movements, namely walking on the ordinary sidewalk, going up and down stairs and going up and down ramps, respectively. In addition, an algorithm based on an iterative feature selection was proposed to optimize the principal component analysis classifier for the offline pattern recognition. Time-domain and frequency-domain characteristic values were selected as feature vectors, and a 10-cross-validation technology was used to improve the stability of data. Through the comparative analysis, the accuracy of the pattern recognition of the proposed algorithm (97.87%) was better than that of the traditional SVM (96.83%), LDA (97.45%), and NN (97.61%). Especially in dealing with the untrained data, it also showed some advantages. Tavakoli et al. [Reference Tavakoli, Benussi and Lourenco12] proposed a method combining the high-dimensional feature space and SVM by using a single channel to collect four gesture movements: hand closing, hand opening, wrist bending, and double wrist bending. Through pattern recognition, the accuracy of different actions could reach 100% and 98%, and the recognition time was less than 10 ms. The manipulator was successfully controlled by sEMG signals, which verified the effectiveness of the active control. Aiming at the current situation of weak, unstable and difficult identification of single-channel sEMG signals, Phinyomark et al. [Reference Phinyomark, Phukpattaranont and Limsakul13] proposed a detrended fluctuation analysis (DFA) method for pattern recognition of forearm pronation and supination and wrist radial ulnar deflection, with a recognition rate of 90%. DFA could effectively improve the recognition rate with fewer channels. Zhang et al. [Reference Zhang, Li, Bie and Han14] proposed a method combining a single-channel sEMG decomposition strategy with a long short-term memory cyclic neural network to solve the problem of the low recognition rate of a single-channel sEMG acquisition mode. The action potential sequence of the multichannel motor unit was decomposed through the collected sEMG signals of six subjects’ four movements of extending palm, clenching fist, pinching index finger, and pinching middle finger, and the characteristics of absolute value mean value and zero crossing number were extracted. The pattern recognition results showed that the recognition rate of the proposed method was 90%, 11.53% higher than that of SVM. Xiong et al. [Reference Xiong, Zhang, Zhao, Han and Liu15] decomposed the single-channel sEMG signals into six motor unit action potential sequences by using a second-order difference filtering, threshold calculation, spike detection, hierarchical clustering, and other processes. They fully excavated the relationship between data and extracted absolute value integral, maximum value, nonzero median, half window energy, and other features. SVM was used for pattern recognition of five hand movements, and the recognition rate was more than 80%.
(2) There are two main technical routes to build a motion pattern classifier. One is to build a universal motion pattern classifier offline to face many patients. The recognition accuracy of the classifier depends on the size and diversity of datasets of patients. The accuracy of pattern recognition can be greatly improved by obtaining the sEMG signals and selecting an appropriate classifier [Reference Shi16]. In the early stage of the offline mode, it is necessary to collect EMG signals from different patients and environment. When the constructed fixed classifier is used for a new patient, the recognition rate of movements will often decline, resulting in the inconsistency between the rehabilitation training movement and the patient’s intention and needs. To solve such problems, some researchers used the method of transfer learning [Reference Matsubara and Morimoto17, Reference Cote-Allard, Fall, Campeau-Lecours, Gosselin, Laviolette and Gosselin18], that is, to build a new classifier by adding a small amount of data from new patients to the original classifier. Because the acquisition of sEMG signals is largely affected by the individual’s own state and the surrounding environment, sEMG signals not only have individual differences but also have dynamic variability of time periods and batches even for the same individual. This is affected by the current patch position, movement state, and environmental characteristics. Further, these seriously affect the repeatability of the specific response of sEMG signals and reduce the generalization ability of the classification algorithm. Based on this, another technical route is to build a personalized motion pattern classifier online. By timely collecting the patient’s sEMG signals, the classifier is trained online to adapt to the patient’s action pattern classification in the current period and overcome the cross-individual and cross-time variability of the sEMG signals. Kalani et al. [Reference Kalani, Moghimi and Akbarzadeh19] and Zhang et al. [Reference Zhang, Xu and Luo20] optimized the SVM classifier through online real-time collection of the required sEMG signals and adaptive sample update strategy. It was used to actively control the rehabilitation robot, drive the patient’s limbs to carry out the corresponding rehabilitation training, and improve the flexibility of human–computer interaction.
(3) Most of the existing EMG acquisition methods are wired devices. For example, Sun et al. [Reference Sun, Cheng, Dai, Chen, Liu and Xu21] recognized human motion intention based on EMG signals and angle signals. When a set of wired equipment is used to collect signals, it is necessary to hold PC to perform movements. This method limits the subjects’ movements to a certain extent. The wired equipment has a great limit on the subject in space. At the same time, in order to eliminate the power frequency interference caused by the electrode wire in the wired equipment and meet the various movements and postures of the subject, the wireless EMG acquisition equipment is more popular with researchers [Reference Chen and Wang22]. Zhou et al. [Reference Zhou, Yang, Liao, Liang and Ye23] designed a wireless system for the movement recognition of the ankle foot joint based on sEMG signals and acceleration sensors. The wireless acquisition system has the characteristics of small structure and strong adhesion. Three channels were used to collect the sEMG signals of lower limbs in dorsiflexion, plantar flexion, valgus, and varus. In addition, data fusion based on the principal component analysis (PCA) was used to complete the pattern recognition of different actions. Compared with SVM, ANN, and other algorithms, the highest recognition rate could reach 99.8%, which verified the stability and effectiveness of the designed system. Li et al. [Reference Li, Zhou, Liu and Ji24] used the portable wireless acquisition module and the acquisition of sEMG signals of the subjects to complete the real-time control of the virtual reality scene. One channel of the sEMG system was a module, so the modular design improved the flexibility of the system. The sEMG signals were separated by mean square and sliding window, and the features were extracted. Using SVM and a probabilistic NN classification, offline and online recognition rates could reach 95% and 90.31%, respectively, improving the immersion of the rehabilitation training. Bai et al. [Reference Bai, Liu, Liu, Cui, Cao, Shan, Yu, Yuan and Wang25] developed a flexible dry electrode with a wireless acquisition function by using a micro-electromechanical system (MEMS). The system was light in weight, small in space, and strong in endurance, with a maximum transmission distance of 50 m and a good biological compatibility.
Based on the above analysis, this paper aims to propose an online pattern recognition method of lower limb movements based on the personalized sEMG signals and performs the corresponding experimental researches in the rehabilitation training. The main contributions can be summarized as follows: (1) wireless sEMG acquisition instruments are used to conduct experiments. On the basis of less collection channels, the dimension of input variables in the sample data is expanded by using a variety of feature indicators, so as to improve the classification and recognition rate. (2) A personalized classifier is built for patients online. PSO-KNN and PSO-ELM algorithms are used to optimize the performance of the classifier. (3) Based on the wireless sEMG acquisition instrument, a novel online training classification and real-time recognition system for individual sitting and lying movements is developed.
The rest of this paper is organized as follows: in Section 2, the basic materials and methods are introduced. In Section 3, results and discussion are presented. Conclusions of this study and future works are drawn in Section 4.
2. Materials and methods
To ensure the reality of the experimental situations, the data collection and test were carried out on the multi-posture lower limb rehabilitation robot [Reference Wang, Ou, Xie, Wang, Yu, Fan and Chu26] developed by our research group.
2.1. System composition and workflow
For the rehabilitation problems of ankle joint, knee joint, and hip joint of lower limbs, we design rehabilitation movements in two modes.
One is the sitting mode, as shown in Fig. 1, including four movements: single movement of ankle (SMA), single movement of knee (SMK), combined movements of knee and ankle (CMKA), and sitting motionless (SM). The subject sits on the lower limb rehabilitation robot. The sEMG signals of four movements are collected in the sitting mode.
Figure 1. Four movements in the sitting mode. (a) SMA, (b) SAK, (c) CMKA and (d) SM.
The other is the lying mode, as shown in Fig. 2, including four movements of lying: low amplitude (LA), medium amplitude (MA), high amplitude (HA), and lying motionless (LM). The subject lies flat on the rehabilitation robot, where one side of the affected limb is tied to the lower limb of the rehabilitation robot, and the other lower limb of the healthy side is freely placed on the rehabilitation robot. Then, sEMG signals of the healthy side lower limb of four movements are collected.
Figure 2. Four movements in the lying mode. (a) LA, (b) MA, (c) HA, and (d) LM.
2.2. Software and circuit design
2.2.1 Hardware and software design of wireless EMG equipment
The hardware experimental equipment of this system includes silver/silver chloride electrode, wireless muscle acquisition device, Bluetooth adapter, wireless muscle acquisition device charger, wireless muscle acquisition device charger cable, USB serial port expander, and bandage. The size of the wireless sEMG acquisition device is 51 mm long, 36 mm wide, 20 mm high, and 35 g weight. The one-time power supply duration of the rechargeable lithium battery is 10 h. These parameters can be used in most environments. It is easy to wear without any interference. The collector uses an electrode piece to stick it on the skin surface and selects the tendon of the anterior tibialis muscle and the rectus femoris muscle as the placement positions of the electrode pieces in Channel 1 and Channel 2. Examples of the two channels’ unfixed bandages and fixed bandages are shown in Fig. 3. And the flowcharts for the design of hardware and software are shown in Figs. 4 and 5, respectively.
Figure 3. Positions of sEMG acquisition instruments.
Figure 4. Flowchart for the design of hardware.
Figure 5. Flowchart for the design of software.
The host computer software of this system is divided into two categories: online construction of a personalized classifier and real-time pattern recognition. Major elements of this system are the optimization design of the human–computer interface, coding of serial communication, real-time waveform display and data storage, simple processing of data, and calling Matlab program to extract the characteristics of each EMG signal, calling algorithms to optimize the channels and carry out the pattern recognition.
2.2.2 Data acquisition of the lower limb sEMG signals
Eight healthy subjects of different genders, ages, and heights were recruited. The eight subjects included six men and two women, aged between 23 and 30 years, and their heights ranged from 160 cm to 180 cm.
The system is mainly divided into two modules: one is the online construction of a personalized classifier and the other is the real-time motion recognition and rehabilitation training module.
The online personalized classifier module is divided into two modes: sitting and lying. For each subject, 24 groups of data are collected in 4 movements under one mode. Each movement has 6 groups of data, each group has 10,240 samples, and the sampling rate is 1024 Hz. The duration of data acquisition for each group is 10 s. The first 2 s are used as the test period, and the last 8 s are used as the formal data acquisition time. In the order of movement extraction, random acquisition is conducted according to the subject’s own intention and actions. The sEMG collection time for each subject under one mode is 240 s, that is, 4 min. The sEMG signals of No. subject 1 are presented in Figs. 6 and 7.
Figure 6. Four movements of the sitting mode.
Figure 7. Four movements of the lying mode.
It can be observed from Figs. 6 and 7 that the waveforms generated by sEMG signals are different for different movements under the same mode. The amplitude and frequency of sEMG signals of different movements differ greatly, indicating that sEMG signals can be used as a basis for the motion recognition.
2.2.3 Real-time motion recognition and the rehabilitation training system
In this paper, the implementation process of the online real-time rehabilitation training system is presented in Fig. 8. For any patient, first, we build a personalized classifier module online. Then the real-time rehabilitation training module is performed. At this time, the system collects the patient’s sEMG signals every 10 s in real time, and then the effective data segment is selected, and the time-domain and nonlinear features are extracted. Further, the personalized classifier is called to identify the action mode, and the corresponding action label instructions are generated to drive the lower limb rehabilitation robot to do the corresponding movements, that is, to conduct the rehabilitation training for the patient. The specific control system is depicted in Fig. 9. In the online construction of building a personalized classifier module, a preprocessing program for sEMG signals is written based on LabVIEW2018, and a program for feature extraction, pattern recognition, and channel optimization of sEMG signals is written based on MatlabR2018. In the real-time rehabilitation training module, the real-time decoding of sEMG signals is first performed based on the preprocessing program of LabVIEW2018, followed by the feature extraction of sEMG signals based on MatlabR2018. Finally, the trained pattern recognition program obtained from the personalized classifier module is used for the pattern recognition, and the results of the pattern recognition are sent to the central control system. In the central control system, based on LabVIEW2018, both the result display module and related motion mode command sending module generated by the upper computer are written.
Figure 8. Online real-time rehabilitation training process.
Figure 9. Hardware and software control system of the lower limb rehabilitation robot.
2.3. Algorithm design
2.3.1 KNN classification algorithm
In this paper, a KNN classification algorithm [Reference Narayan27] is used for the supervised learning. The advantage of KNN is that it is insensitive to abnormal values and suitable for multi-classification problems. There are two important issues to be addressed in KNN. One is the similarity measurement of data points to adjacent data points, and the other is the selection of
$K$
value [Reference Zhang, Li, Zong, Zhu and Cheng28]; however,
$K$
generally does not exceed 20.
Suppose that the characteristic vector of EMG signals is
$W^m$
in the
$m$
-dimensional space.
$y_i=(x_i^{(1)},x_i^{(2)},\ldots,x_i^{(m)} )^T$
and
$y_j=(x_j^{(1)},x_j^{(2)},\ldots,x_j^{(m)} )^T$
. Euclidean distance is used for the similarity measurement between
$y_i$
and
$y_j$
, that is,
\begin{equation} dist\left (y_{i},y_{j}\right )=\sqrt{\sum _{i=1}^{n}\left (y_{i}-y_{j}\right )^{2}}, \end{equation}
2.3.2 Extreme Learning Machine (ELM)
Here, extreme learning machine (ELM) [Reference Huang, Zhu and Siew29] is selected as a different pattern recognition algorithm to construct the classification algorithm.
Figure 10 presents that ELM is divided into three layers, namely the input layer, the hidden layer, and the output layer. The sEMG data acquisition equipment has two channels, and each channel has five features, so the joint feature channel is ten dimensions. The number of the training samples for each subject is 24 groups. Since the input layer needs to match the number of data types, the input layer
$n$
is 10. The number of the hidden layer should be less than the number of samples. Thus, the number of the hidden layer
$m$
is 20. The output layer is the number of the classification labels with
$k$
= 4.
Figure 10. Structure diagram of ELM.
For
$N$
training samples (
$X_i, t_i$
),
$X_i$
=
$[x_{i,1},x_{i,2},\ldots,x_{i,n}]$
,
$t_i$
=
$[t_{i,1},t_{i,2},\ldots,t_{i,k}]$
,
$y_i$
=
$[y_{i,1},y_{i,2},\ldots,y_{i,nk}]$
, ELM with
$m$
hidden points is
\begin{equation} \sum _{j=1}^{m} \beta _{i} g\left (\sum _{i=1}^{n} W_{i} \cdot X_{j}+b_{i}\right )=y_{i}, \end{equation}
where
$g(x)$
is the activation function.
$W_i$
=
$[w_{i,1},w_{i,2},\ldots,w_{i,m} ]^T$
denotes the weights of the input,
$b_i$
is the threshold of the
$i\text{th}$
hidden neuron, and
$W_i\cdot X_j$
represents the product of
$W_i$
and
$X_j$
.
To make the classification correct and the error is minimum, it can be expressed as
$\sum _{j=1}^{N}\left \|y_{i}-t_{j}\right \|=0$
. Namely, if
$\sum _{j=1}^{m} \beta _{i} g\left (\sum _{i=1}^{n} W_{i} \cdot X_{j}+b_{i}\right )=t_{i}$
exists, it can be expressed as
$H\beta =T$
, where
$T$
represents the desired output. The input weight
$W$
and the node threshold b of the hidden layer are random numbers between 0 and 1. After being randomly selected, the output expected value
$T$
is uniquely determined.
2.3.3 PSO-KNN algorithm
In this paper, three time-domain features and two nonlinear features are selected. The matrix composed of features of two channels is a 10-dimensional matrix, and there are interference items or redundancy items in the matrix, so it is necessary to optimize the characteristic components. Particle swarm optimization (PSO) [Reference Wang, Tan and Liu30, Reference Pano and Ouyang31] is a parallel global search strategy based on the population. Its advantages are less adjustment parameters, fast convergence speed, wide application range, and optimization in a high-dimensional space. Therefore, a classical PSO optimization algorithm is selected for the feature component selection, and the processes are as below:
(1) Initialize different parameters. The parameter settings are as follows: the population size
$n$
is 20, the particle’s velocity range
$V$
$\in$
[−10,10], the particle’s position range
$X$
$\in$
[−10,10],
$D$
-dimensional search space is 10, the number of evolution is 1, and the maximum iteration number is 50.
(2) The fitness function is defined as the classification accuracy, and the corresponding fitness function value of the population is calculated.
1) Set the position vector of the
$i$
th particle to
$X_i$
=
$(X_{i1},X_{i2},\ldots,X_{iD})$
. We use the Sigmaid function
$S(x)=1/((1+e^x))$
to map the position vector linearly. The weight value of 10 characteristic elements in the feature matrix is obtained, and the range is [0,1]. Then we set the threshold value as the median 0.5. When the position weight of a feature element is greater than 0.5, it is determined that the feature element is selected and will be assigned a value of 1. When the position weight value is less than or equal to 0.5, the feature element is not selected and will be assigned a value of 0.
2) For the selected feature elements, the KNN pattern recognition algorithm in Section 2.3.1 is used to calculate the fitness (classification accuracy) corresponding to the
$i$
th particle.
3) Repeat the above 1) and 2) processes for all n particles to obtain the population fitness.
Definition:
$P_i$
=
$(P_{i1},P_{i2},\ldots,P_{iD})$
is the optimal fitness position vector of the
$i$
th particle in the iteration process.
$P_g$
=
$(P_{g1},P_{g2},\ldots,P_{gD})$
is the global optimal solution, that is, the position vector corresponding to the maximum population fitness. Then the speed and position are updated (let the learning factors
$c_1$
=
$c_2$
= 1.49, inertia weight
$\lambda$
= 1).
(3) The processes will be ended when the number of evolution time is reached; otherwise, the above steps are repeated.
Pattern recognition based on KNN shall be carried out according to the sitting mode and lying mode, respectively. There are four movements in each mode, and each movement has six groups of data. Sixteen groups of data (four groups of data for each movement) are randomly selected as the training set, and the remaining eight groups of data (two groups of data for each movement) are selected as the test set, and finally the personal optimal feature components (OFCs) are screened. Personal optimal characteristic components, 24 groups of sEMG signals, and known tag numbers (24 groups of sEMG data) are taken as the input samples. Using the KNN algorithm with the parameters determined in advance, a personalized classifier is constructed and saved in the designated folder. When there are new sEMG signals, the parameters in the personalized classifier are compared with the features of the sEMG signals of the unknown action tag based on the KNN algorithm, and then the corresponding action tag number is generated.
2.3.4 PSO-ELM algorithm
The PSO-ELM algorithm is basically the same as PSO-KNN in principle, but the difference is that this algorithm uses the accuracy of the ELM pattern recognition as the fitness. Although the two classification algorithms have many similarities in the construction principle and data distribution, the characteristics and advantages of each optimization algorithm combined with different pattern recognition methods are different. The classification algorithm based on PSO-ELM is that the advantages of PSO such as fast computing speed, strong search ability, and good generalization performance of ELM are combined. It can quickly find the best joint feature vector for individuals to adapt to ELM and build a personalized classifier for patients.
2.4. Rehabilitation training experiments
In this paper, the hardware environment is a host PC which can support the operation of the online pattern recognition system. The algorithm design was implemented using LabVIEW 2018 and Matlab R2018a. The experimental setup involves an Intel(R) Core(TM) i5-6500 CPU with 8 GB RAM with Windows 10 Operating System.
2.4.1 Rehabilitation training experiment in the sitting mode
In the sitting mode, when the subject needs to perform the rehabilitation training for a certain movement, the healthy side lower limb is used to continue to perform a certain movement (SMA, SMK, CMKA, and SM) for 10 s. Then, the personalized classifier is used to conduct the real-time motion recognition, the corresponding label number is generated, and the rehabilitation robot is driven to connect with the affected side lower limb to perform the corresponding lower limb action rehabilitation training (Fig. 11). When the healthy side of the lower limb of the subject remains in a SM state, the recognition will be conducted once every 10 s. When the recognition result is a SM label value, the lower limb rehabilitation robot will continue to perform the last rehabilitation movement. The transformation of joint angle of each movement driven the rehabilitation robot in the sitting mode and the corresponding label value of each movement are presented in Table I.
Table I. Corresponding relationship with the rehabilitation robot in the sitting mode.
Figure 11. Real-time rehabilitation training in the sitting mode (“A” is the upper computer interface, “B” is the healthy side limb of the subject, “C” is the affected limb of the subject (hypothesis), and “D” is the left mechanism of the lower limb rehabilitation robot in the sitting training state).
2.4.2 Rehabilitation training experiments in the lying mode
When the subject needs to perform a certain amplitude (LA, MA, and HA) rehabilitation training on the affected lower limb in the lying mode, first the subject uses the healthy lower limb to perform a 10-s movement of this amplitude. Then the determined personalized classifier is adopted to conduct the real-time motion recognition, and the corresponding label number is generated. Further, the rehabilitation robot connected to the affected lower limb is driven to perform the corresponding rehabilitation movement (Fig. 12). When the healthy side of the lower limb of the subject remains in a motionless state, the recognition will be performed once every 10 s. When the recognition result is a motionless tag number, the lower limb rehabilitation robot will continue to perform the last rehabilitation movement. The four movements in the lying mode are labeled as 1, 2, 3, and 4, respectively, and the corresponding relationship is depicted in Table II.
Table II. Corresponding relationship between the lying mode and rehabilitation robot.
Figure 12. Real-time rehabilitation training in the lying mode (“A” is the upper computer interface, “B” is the healthy side limb of the subject, “C” is the affected limb of the subject (hypothesis), and “D” is the left mechanism of the lower limb rehabilitation robot in the lying training state).
We collected six sets of data for the four types of movements both in the sitting and lying modes, with the collection time accounting for 2/3 of the total time (6 min) for the two modes. The time for data processing, pattern recognition, and driving the robot accounts for 1/3 of the total time, so the collection of a single movement and a single group of movements always takes 10 s + 5 s = 15 s to drive the robot’s movement. The total time from the sEMG acquisition to the robot actuation of each cycle is 15 s.
3. Results and discussion
3.1. Feature analysis of the lower limb sEMG signals
3.1.1 Methods of feature extraction
How to extract useful features from sEMG signals is the crucial process of pattern recognition. Feature extraction is a necessary step for information prediction of human bioelectrical signals. The performance of any learning system is widely dependent on the selection of input signals and the quality of feature extraction [Reference Pancholi and Joshi32]. Here, the similarity and difference of eight subjects’ features between the same or different movements are analyzed by using the distribution maps of the features.
Time-domain features
Three features are selected: integrated EMG (IEMG) value, RMS value, and first-order difference (FD) absolute value mean. The formulas are given from Eqs. (3) to (5):
\begin{equation} \text{IEMG}=\frac{1}{N} \sum _{i=1}^{N}\left |x_{i}\right |, \end{equation}
\begin{equation} \text{FMS}=\sqrt{\frac{\sum _{i=1}^{N} x_{i}^{2}}{N}}, \end{equation}
\begin{equation} \text{FD}=\frac{1}{N-1} \sum _{i=1}^{N-1}\left |x_{(i+1)}-x_{i}\right |, \end{equation}
where
$x_i$
is the voltage amplitude at the
$i$
th point of a movement, and
$i=1, 2\ldots, N$
is a time sample sequence of sEMG signals with a length of
$N$
.
Nonlinear features
(1) Approximate entropy
The value of approximate entropy (AE) indicates the complexity of a group of data. A more complex data yields a greater AE, whereas a more regular data yields a smaller AE. The AE [Reference Fan34] is calculated as follows:
1) Let the original signals be
$\{x(1),x(2),\ldots,x(N)\}$
, with
$N$
points in total. Then follow the steps below to reconstruct a set of
$m$
-dimensional vectors
$y_m(i)$
, where
$y_m (i)=x(i),x(i+1),\ldots,x(i+m-1),i=1\sim (N-m+1)$
. The maximum contribution distance between the components of
$y(i)$
and
$y(j)$
is recorded as
$D\{y(i),y(j)\}$
, where
$j=1\sim (N-m+1)$
, and
$j\neq i$
.
2) Let the similarity threshold be denoted as
$r$
. Caculate the ratio
$C_i^m (r)$
of the number of vectors that meet the conditions
$D\{y(i),y(j)\}\lt r$
to the total number
$N-m+1$
. Then calculate the logarithmic mean value of
$C_i^m (r)$
and record it as
$ \Phi ^m (r)$
, namely:
\begin{equation} \Phi ^{m}(r)=\frac{1}{n-m+1} \sum _{i=1}^{n-m+1} \ln C_{i}^{m}(r), \end{equation}
3) Let increase the window length
$m$
to
$m+1$
and repeat the above steps to get
$C_i^{m+1} (r)$
and
$\Phi ^{m+1} (r)$
. The theoretical AE is
$ApEn(r)=\lim \limits _{N\rightarrow \infty }[\Phi ^m (r)-\Phi ^{m+1} (r)]$
. However, as
$N$
is 10,240, the AE can be simplified as:
\begin{equation} ApEn(m,r,N)=\Phi ^m(r)-\Phi ^{m+1}(r), \end{equation}
(2) Singular entropy of the wavelet (SEW)
The signal with length
$N$
is decomposed by
$S$
-layer wavelet, and a number of
$m$
wavelet coefficients with length
$n$
are obtained to form the time-frequency matrix
$W_{m\times n}$
. According to the singular value decomposition,
$W_{m\times n}$
can be decomposed into an
$(m\times l)$
-dimensional matrix
$U$
, an
$(l\times l)$
-dimensional matrix
$\Lambda$
, and an
$(l\times n)$
-dimensional matrix
$V$
. Therefore, the matrix
$W_{m\times n}$
can be decomposed into Eq. (8):
\begin{equation} W_{m\times n}=U_{m\times l}\Lambda _{l\times l}V_{l\times n}, \end{equation}
where
$m=2^S, n=N/2^S$
and
$\Lambda$
= diag
$(\lambda _1,\lambda _2,\ldots,\lambda _l)$
are the diagonal matrices of the singular values. The singular values satisfy the inequality
$\lambda _1\gt \lambda _2\ldots \gt \lambda _n$
.
To describe the signal complexity, the wavelet singular entropy is
\begin{equation} W_{i}=\sum _{i=1}^{l}\Delta p_i \end{equation}
where
$\Delta p_i$
is the incremental wavelet singular entropy of the
$i$
th nonzero singular value, defined as:
\begin{equation} \Delta p_{i}=-\left (\lambda _{i}/ \sum _{j=1}^{l} \lambda _{j}\right ) \log \left (\lambda _{i}/ \sum _{j=1}^{l} \lambda _{j}\right ) \end{equation}
3.1.2 Feature analysis of the lower limb EMG signals
The collected sample data of the sEMG signals are preprocessed and substituted into Eqs. (3)–(10) to calculate the corresponding features.
Feature analysis of the sitting mode
Feature extraction is performed by the sEMG signals of four sitting movements under the online training module. Here, the sEMG signals of eight subjects through dual channels were collected, including four movements: SMA, SMK, CMKA, and SM (the sampling rate is set to 1024 Hz, and the total number of samples is set to 10,240). A total of 24 sets of data were collected for each subject’s four movements (each with six sets of data), and the collection of sEMG signals during the sitting training took a total of 240 s or 4 min. To test the quality of the feature extraction results, the collected sEMG signals were preprocessed and brought into the five time-domain and nonlinear feature values determined in the paper. The average and standard deviation of the four time-domain and nonlinear feature values of eight subjects in the sitting mode were calculated. As shown in Table III, the first channel for each participant was selected to display the differences in the same characteristic values under different movements, as well as the differences in different characteristic values for the same movements. Further, Fig. 13 and Fig. 14, respectively, present the time-domain and nonlinear feature space distributions of four movements in the sitting mode.
Table III. Features of the four movements in the sitting mode.
Figure 13. Spatial distribution of the time-domain features of the four movements under the sitting mode.
Figure 14. Spatial distribution of the nonlinear features of the four movements under the sitting mode.
It can be noticed from Table III that the features of different subjects under the same movement have similarities, while the results of different movements have differences. The absolute value of the FD in the time-domain features exhibits the most obvious difference, while the IEMG values are not too obvious. The discriminability of the two nonlinear features is good, where the wavelet singular entropy has a better discriminability. In addition, Figs. 13 and 14 present that the characteristic values of RMS and FD of the eight subjects are clearly distinguished, and there is a small amount of superposition of IEMG. However, the distribution of two nonlinear features of the eight subjects is quite distinct, and there is no superposition of features.
Feature analysis of the lying mode
For the four movements in the lying mode under the online training module, the sampling rate and sampling number of LA, MA, HA, and LM are consistent with the parameters under the sitting mode. The number of the training groups and acquisition time for each movement are consistent, that is, the training time in the lying mode is still 4 min. In order to demonstrate the advantages and disadvantages of extracting the five characteristic values of the four movements in the lying mode in the time and nonlinear domains, the features of the first channel sEMG signals of each subject were selected for analysis, and the average and standard deviation of the five characteristic values of different movements of eight subjects were calculated, as shown in Table IV. Further, Fig. 15 and Fig. 16, respectively, show the time-domain and nonlinear feature space distributions of four movements in the lying mode.
Table IV. Features of the four movements in the lying mode.
Figure 15. Spatial distribution of the time-domain features of the four movements under the lying mode.
Figure 16. Spatial distribution of the nonlinear features of the four movements under the lying mode.
It can be intuitively observed from Table IV that the most obvious difference of the time-domain features between different movements of the eight subjects is the FD. The AE and wavelet singular entropy (WSE) perform better in nonlinear features, and the contribution of the time and nonlinear domains to different movements is different. Moreover, Figs. 15 and 16 present that there is no overlap in the time-domain feature distribution between different movements of each subject, which shows a good discrimination effect. In the nonlinear feature distribution of Fig. 16, except for the overlap between the middle and high lying postures of the eighth subject, all the other subjects performed well.
3.2. Analysis of the construction results of the personalized classifier
To simplify the representation of the optimal characteristic components of eight subjects under two modes, the corresponding abbreviations of the 10 dimensional features of the 2 channels are adopted, as shown in Table V.
Table V. Abbreviations of 10 dimensional features for 2 channels (here, “
$\alpha$
-
$\beta$
” denotes the
$\beta _{th}$
(
$\beta$
=1,2,…,5) feature of the
$\alpha _{th}$
(
$\alpha$
=1,2) channel.
3.2.1 The results of the experiments under the sitting mode
Table VI presents the results of the OFCs based on PSO-KNN and PSO-ELM classification algorithms for eight subjects in the sitting mode, as well as the recognition accuracy of each subject. In the PSO-KNN algorithm, the individual OFCs of five subjects are five dimensions and of other three subjects are six dimensions. In the PSO-ELM algorithm, the individual OFCs of four subjects are four dimensions and of other four subjects are three dimensions. The results of the OFC of each subject are different, and it can be observed that the OFCs in the sitting mode are personalized.
Table VI. Selection of optimal feature components of eight subjects in the sitting mode.
As shown in Table VI, when the classifier is tamed based on the PSO-KNN classification algorithm, the value of the optimal fitness for each subject (the accuracy of pattern recognition of KNN) is obtained, respectively, in which the highest recognition accuracy was 100%, the lowest recognition accuracy was 86%, the average accuracy (AA) was 96.50%, the high-frequency channel (HFC) is 2-FD, and the average time for domesticating the classifier (ATDC) is 61.0042 s. When the classifier is tamed based on the PSO-ELM classification algorithm, the value of the optimal fitness for each subject (the correct rate of the ELM pattern recognition) is obtained, respectively, in which the highest recognition accuracy is 100%, the lowest recognition accuracy is 94%, the AA is 97.38%, the HFC is 2-WSE, and the ATDC is 4.3115 s. Combined the recognition accuracy and calculation efficiency, PSO-ELM is selected to conduct the online pattern recognition of the sitting mode and the corresponding rehabilitation training.
3.2.2. The results of the experiment under the lying mode
Similarly, the results of the OFCs of each subject are different in the lying mode as well. It can be noticed that the OFCs are also personalized.
As shown in Table VII, when the classifier is tamed based on the PSO-KNN classification algorithm, the value of the optimal fitness for each subject (the accuracy of the pattern recognition of KNN) is obtained, respectively, in which the highest recognition accuracy was 100%, the lowest recognition accuracy was 86%, the average recognition accuracy was 96.50%, and the ATDC is 61.0042 s. When the classifier is tamed based on the PSO-ELM classification algorithm, the value of the optimal fitness for each subject (the accuracy of the ELM pattern recognition) is obtained, respectively, in which the highest recognition accuracy is 100%, the lowest recognition accuracy is 94%, the average recognition accuracy is 97.38%, and the ATDC is 4.3115 s. Combined the recognition accuracy and calculation efficiency, PSO-ELM is selected to conduct the online pattern recognition of the sitting mode and the corresponding rehabilitation training.
Table VII. Selection of optimal feature components of eight subjects in the lying mode.
3.3. Analysis of the online rehabilitation training results
Figure 17 presents the real-time recognition results of random sequence training test of eight subjects’ four movements in the sitting mode, and the test time is 1000 s.
Figure 17. Real-time recognition results of eight subjects in the sitting mode (the green and red lines indicate the label numbers for the pattern recognition and the real movement, respectively).
It can be noticed from Fig. 17 that the No. subject 6 has the highest recognition accuracy, and the real action label numbers are highly consistent with the ones of the pattern recognition, with a recognition accuracy of 99%. No. subject 1 has the lowest recognition accuracy, the recognition accuracy is 94%, and the average recognition accuracy of eight subjects is 96.63%. The real-time pattern recognition based on the personalized classifier constructed for each subject can better achieve the timeliness and recognition accuracy between switches of different movements.
In the real-time experiment of the No. subject 6, the rehabilitation training process of driving the lower limb rehabilitation robot to perform the corresponding movements is shown in Fig. 18. The label number corresponding to the human limb movements, the label number corresponding to the real-time recognition results, and the angle changes of the ankle and knee joints of the rehabilitation robot can be well matched. The real-time response of the rehabilitation robot’s joint movement mechanism meets the actual rehabilitation training requirements.
Figure 18. Real-time rehabilitation training results of No. subject 6 in the sitting mode.
For the four movements of LA, MA, HA, and LM in the lying mode, we conducted a random sequence training test on eight subjects, with the test time of 1000 s. The real-time recognition results of the training test are presented in Fig. 19.
Figure 19. Real-time recognition results of eight subjects in the lying mode (the green and red lines indicate the label numbers for the pattern recognition and the real movement, respectively).
It can be observed from Fig. 19 that the real action label numbers of the No. subject 7 are completely consistent with the ones of the pattern recognition, and the recognition accuracy is 100%. There are many inconsistencies for the No. subject 2, and the recognition accuracy is 94%. The average recognition accuracy of eight subjects in a random sequence is 97.25%, which can better realize the real-time movement recognition.
In the real-time experiment of the No. Subject 7, the lower limb rehabilitation robot was driven to perform the rehabilitation training process of the corresponding movements in the lying mode, as shown in Fig. 20. The label number corresponding to the human limb actions, the label number corresponding to the real-time recognition results, and the hip joint angle change of the lower limb rehabilitation robot can be highly consistent, which have the actual rehabilitation effect.
Figure 20. Real-time rehabilitation training results of No. subject 7 in the lying mode.
3.4. Discussions
There have been significant developments in the collection, analysis, and application of sEMG signals; however, there are still many problems in human–machine interactions and deciphering human motion intentions. There are differences between individuals, especially differences in sEMG signals between healthy individuals and patients. When a constructed fixed classifier encounters new testers, the accuracy of human motion recognition will greatly decrease. However, when faced with such problems, researchers usually choose to increase the training sample data size. For example, Fan et al. [Reference Fan34] used traditional methods to collect a large amount of lower limb EMG signal data from subjects through wired collectors, but the recognition rate still cannot be stable at over 90%, and it increases the workload of data collection in the early stage and increases the data dimension. Faced with the problem of large data collection and long classifier construction time, some researchers have adopted the transfer learning method [Reference Matsubara and Morimoto17, Reference Cote-Allard, Fall, Campeau-Lecours, Gosselin, Laviolette and Gosselin18] to reduce the amount of data, which adds a small amount of data from new testers to the original classifier to construct a new classifier. Although this method reduces data dimensions and data collection workload, it still requires continuous classifier construction.
The results in Sections 3.1 and 3.2 reveal that based on the wireless sEMG acquisition instrument and two muscles of lower limbs as the acquisition channels, a personalized classifier and online rehabilitation training system for individual sitting and lying movements is well developed in this paper. On the basis of less collection channels and the joint feature vector, the dimension of input variables in the sample data is expanded by using a variety of feature indicators. As there are redundant and similar information in the joint feature vector, this information will affect the classification recognition accuracy and efficiency [Reference Liu and Yu35–Reference Xue, Zhang, Browne and Yao37]. Thus, we have extracted the OFCs from the joint feature vector based on the PSO-KNN algorithm and PSO-ELM algorithm, so as to maximize the performance of the classifier. We observe that the technical requirements for building a personalized classifier online are effective (high accuracy of motion recognition) and fast (less data collection of sEMG signals and short training time of the classifier).
4. Conclusions and future works
The lower limb rehabilitation training experiments were conducted for eight different subjects to test the recognition accuracy and efficiency of the online personalized classifier. The identified motion results were generated as the corresponding tag numbers, and the lower limb rehabilitation robot was actively controlled to make the corresponding movements to complete the rehabilitation training. The experimental results indicate that the online pattern recognition and real-time rehabilitation training system for the lower limb movements based on the wireless sEMG sensors has the following advantages and characteristics:
(1) The wireless sEMG signal acquisition instrument is selected to get rid of the power frequency interference, space limitation, and other defects caused by the wired acquisition. Based on the wireless sEMG acquisition instrument, a multifunction host computer is developed to achieve the functions of real-time data storage, real-time display, online personalized classifier, and the real-time rehabilitation training.
(2) In terms of both recognition accuracy and recognition efficiency, we first use a few (only two) channels of sEMG signal acquisition modes. Then, multiple characteristic indexes (IEMG value in time domain, RMS value, absolute value of FD, and wavelet singular entropy and AE in nonlinear characteristic value) are used to form a joint feature vector to enrich the information of input variables in the sample data. Finally, based on the PSO-KNN and PSO-ELM algorithms, feature components are optimized to reduce redundant information in joint feature vectors, and the classification accuracy and recognition efficiency are both improved.
(3) The traditional offline method to build a universal motion pattern classifier has a large amount of data. In this paper, it takes a short time (6 min) to build a personalized classifier online, without offline processing of the data. This method reduces the workload before the rehabilitation training and the data is real time.
(4) Using the personalized classifier constructed online and based on the multi-position lower limb rehabilitation robot, the real-time motion pattern recognition and the corresponding rehabilitation training are carried out. The experimental results show that the real-time recognition accuracy of subjects is high (the average recognition accuracy is more than 95%), and the joint motion mechanism of the rehabilitation robot responds in a timely manner. These features meet the requirements of the actual rehabilitation training and have a strong practicability.
In summary, we have achieved many improvements in the traditional sEMG signals control of the rehabilitation robot, enabling the sEMG signals to be quickly trained online, recognized online, and obtain better recognition results. However, the clinical rehabilitation based on the sEMG signals using the rehabilitation robot is still not widely carried out. How to use the sEMG signals to obtain the patient’s movement intention more quickly and identify more and more complex movements is still a problem worth studying in this field.
Author contributions
Conceptualization: Y.Y., N.G.X., and M.X.Z.; methodology: Y.Y., N.G.X., and M.X.Z.; validation: M.X.Z and C.W.O.; formal analysis: C.W.O. and B. Z.W.; resources: C.W.O. and L.W.; data curation: Y.Y., N.G.X., and B. Z.W.; writing – original draft preparation: Y.Y., N.G.X. and M.X.Z.; writing – review and editing: C.W.O. and L.W.; supervision: Y.Y. and N.G.X.; project administration: M.X.Z.; funding acquisition: Y.Y. All authors have read and agreed to the published version of the manuscript.
Financial support
This work was supported by the Natural Science Key Foundation of Anhui Provincial Education Department of China (No.KJ2021A0412), the University Synergy Innovation Program of Anhui Province (No.GXXT-2021-044); the Excellent Scientific Research and Innovation Team of Anhui Provincial Education Department, China (No. 2022AH010027), Science and Technology Plan Project in Ma’anshan City, Anhui Province, China (No.YL-2022-3).
Data availability statement
Data are not publicly available due to privacy considerations. Data are available per request from Ye Ye ([email protected]).
Ethical approval
The ethical committee of Anhui University of Technology of China (AHUT), Anhui, China, approved this study, and written consent was obtained for all participants. This study was conducted on the premises of the AHUT after getting permission.
Informed consent statement
Informed consent was obtained from all subjects involved in the study.
Competing interests
The authors declare no competing interests.
