Nomenclature
- EEG
-
Electroencephalogram
- SA
-
Situation Awareness
- MATB
-
Multi-Attribute Task Battery
- HSL
-
High-SA Level
- LSL
-
Low-SA Level
- SAGAT
-
Situation Awareness Global Assessment Technology
- ANOVA
-
Analysis Of VAriance
- DT
-
Decision Tree
- LR
-
Logistic Regression
- RF
-
Random Forest
- ANN
-
Artificial Neural Network
- HF
-
Human Factors
- ECG
-
Electrocardiographic
- HRV
-
Heart Rate Variability
- PSD
-
Power Spectral Density
- SVM
-
Support Vector Machine
- ANN
-
Artificial Neural Network
- BNN
-
Bayesian Neural Network
- LDA
-
Linear Discriminant Analysis
- GNB
-
Gaussian Naive Bayes
- KNN
-
K-Nearest Neighbors
- SD
-
Standard Deviation
- EOG
-
Electrooculogram
- ICA
-
Independent Component Analysis
- FFT
-
Fast Fourier Transform
-
$\delta$
-
Delta
-
$\theta$
-
Theta
-
$\alpha$
-
Alpha
-
$\sigma_1$
-
Sigma1
-
$\sigma_2$
-
Sigma2
-
$\beta_1$
-
Beta1
-
$\beta_2$
-
Beta2
-
$\beta_3$
-
Beta3
- PCA
-
Principal Component Analysis
- PCs
-
Principal Components
1.0 Introduction
Situation awareness (SA) is commonly defined as ‘the perception of elements in the environment within a volume of time and space, the comprehension of their meaning, and the projection of their status in the near future’ [Reference Endsley1]. Seventy to eighty percent of aviation accidents can be attributed to human factors (HF) in the chain of causation [Reference Sant’Anna and Hilal2, Reference Wiegmann and Shappell3], among which 88% was found to be due to problems with SA [Reference Endsley, Fuller, Johnston and McDonald4]. Real-time measurement and accurate classification of the operator’s SA are critical issues in SA research, and related research can be used to support the design of triggering mechanisms for dynamic human-machine function allocation in complex industrial systems [Reference Schwarz and Fuchs5, Reference Schwerd and Schulte6], which can serve to improve the effectiveness of human-system integration and ensure human factors safety. Considering that operators may exhibit undesirable SA levels during the execution of a task, and there is still a relative lack of research on the exploration of sensitive physiological features and state classification of SA levels, this direction has received extensive and continuous attention from researchers and practitioners in the field of HF [Reference Feng, Liu, Wanyan, Chen, Min and Ma7, Reference Kästle, Anvari, Krol and Wurdemann8].
The measurement of SA can be categorised into inquiry method, subjective scoring, modeling and physiological measures [Reference Gawron9]. Among them, the situation awareness global assessment technique (SAGAT) of the inquiry method is commonly regarded as the gold standard of SA measurement. However, SAGAT suffers from the drawback of requiring task interruption or freezing, which prevents its adoption in real scenarios and frontline operational environments [Reference Smith, Endsley and Clark10]. On the other hand, subjective scoring and modeling methods suffer from shortcomings such as high subjectivity and poor scalability. Accompanied by the recent development of machine learning techniques for classifying critical human states (e.g., SA) and considering that physiological measures have the advantages of real-time signals and high sampling rates, the method has received increasing attention [Reference Bracken, Tobyne, Winder, Shamsi, Endsley, Ayaz, Asgher and Paletta11].
Typical physiological measures include electrocardiographic (ECG), eye movement, electroencephalogram (EEG), etc [Reference Zhang, Yang, Liang, Pitts, Prakah-Asante, Curry, Duerstock, Wachs and Yu12]. Heart rate variability (HRV) of ECG has a strong relationship with executive functioning (especially working memory), which is often considered an essential component of SA [Reference Mehta, Peres, Shortz, Hoyle, Lee, Saini, Chan and Pryor13, Reference Saus, Johnsen, Eid, Riisem, Andersen and Thayer14]. Previous studies have found that HRV decreases during task execution but less in SA-training groups [Reference Saus, Johnsen, Eid, Riisem, Andersen and Thayer14]. In addition, individuals with higher resting HRV typically have higher SA [Reference Saus, Johnsen, Eid, Riisem, Andersen and Thayer14]. Thus, HRV has the potential to predict and assess SA from the perspective of cognitive workload [Reference Smith, Endsley and Clark10, Reference Sætrevik15]. Eye tracking is essential for the visual perception of information, and researchers have noted its potential to characterise vision-related SA [Reference de Winter, Eisma, Cabrall, Hancock and Stanton16, Reference van de Merwe, van Dijk and Zon17]. An air traffic control study suggested that the attention allocation pattern (i.e. widely or narrowly allocated) affects the operator’s SA and that more fixations on a target aircraft lead to higher SA for the aircraft [Reference Moore and Gugerty18]. Additionally, in SA-related studies, eye movement measures are often considered to be correlated with direct SA measures [Reference Zhang, Yang, Liang, Pitts, Prakah-Asante, Curry, Duerstock, Wachs and Yu12]. Compared with the aforementioned physiological measures, EEG is becoming one of the most popular methods for measuring SA because of its high temporal resolution, objective measurement of the operator’s neural activity, and strong connection with cognitive processes related to SA [Reference Feng, Liu, Wanyan, Chen, Min and Ma7, Reference Li, Wang and Sourina19].
The literature suggests that
$\beta$
absolute or relative power may be associated with or senstive to SA. Kästle et al. [Reference Kästle, Anvari, Krol and Wurdemann8] observed a correlation between
$\beta$
band (absolute power) and SA [Reference Kästle, Anvari, Krol and Wurdemann8]. Feng et al. [Reference Feng, Liu, Wanyan, Chen, Min and Ma7] highlighted that
$\beta$
relative power, rather than
$\beta$
absolute power is senstive to SA [Reference Feng, Liu, Wanyan, Chen, Min and Ma7]. Absolute power provides information about the overall strength of neural activity within a specific frequency range but does not account for changes in the overall EEG spectrum. In contrast, relative power focuses on the contribution of activity within a specific frequency range relative to the overall EEG activity, making it inherently relative. The utility of
$\beta$
absolute or relative power in assessing SA requires further investigation. Furthermore, the
$\beta$
band can be subdivided into more refined categories such as
$\beta$
1,
$\beta$
2 and
$\beta$
3 [Reference Corace, Baysarowich, Willows, Baddeley, Schubert and Knott20, Reference Shokouh Alaei, Ghoshuni and Vosough21], and research on their sensitivity to SA is still lacking. Although several EEG features (e.g.,
$\beta$
waves) in various brain regions have been noted to be associated with or sensitive to SA [Reference Feng, Liu, Wanyan, Chen, Min and Ma7, Reference Kästle, Anvari, Krol and Wurdemann8], there is still a relative lack of SA-related EEG studies [Reference Zhang, Yang, Liang, Pitts, Prakah-Asante, Curry, Duerstock, Wachs and Yu12], particularly in regard to the sensitivity of more refined
$\beta$
1,
$\beta$
2 and
$\beta$
3 bands.
After the sensitive EEG features for SA were explored, they were further combined with machine learning algorithms to classify SA levels. Li et al. [Reference Li, Lan, Cui, Sourina and Wang22] selected power spectral density (PSD) from 1,674 samples of 11 participants as inputs and used the response time of performance as the label to carry out fatigue-related classification of high and low SA levels [Reference Li, Lan, Cui, Sourina and Wang22]. Compared with the results of classification algorithms such as logistic regression (LR) and support vector machine (SVM), it was found that the subject-matching framework proposed in the study obtained the best accuracy (74.32%) [Reference Li, Wang and Sourina19]. In addition, Kästle et al. [Reference Kästle, Anvari, Krol and Wurdemann8] used PSD from 2,880 samples of 32 participants as inputs, high and low SA groups divided by SAGAT scores as outputs and adopted random forest (RF) and boosted trees for model construction and reached an accuracy 67% [Reference Kästle, Anvari, Krol and Wurdemann8]. Feng et al. [Reference Feng, Liu, Wanyan, Chen, Min and Ma7] used SA-sensitive EEG features based on
$\beta$
-wave from 24 sample points of 46 participants as inputs, high and low SA groups classified by SAGAT scores as outputs and the model of Bayesian classification algorithm achieved an accuracy of 70.8% [Reference Feng, Liu, Wanyan, Chen, Min and Ma7]. Li et al. [Reference Li, Ng, Yu, Yiu and Lyu23] used EEG and eye tracking data from 2,240 samples of 28 participants as inputs, and high and low SA were classified by positional deviation or angular error of performance as labels [Reference Li, Ng, Yu, Yiu and Lyu23]. SVM, LR, RF, artificial neural network (ANN), and Bayesian neural network (BNN) were further used for classification. The results showed that the best accuracy for high and low SA classification was 76.1%, and the best accuracy for recognising low SA associated with high workload reached 82.7% [Reference Li, Ng, Yu, Yiu and Lyu23].
Based on the background mentioned above, it can be found that the current studies are still deficient in the following aspects. First, on SA-sensitive features, although some studies have pointed out that there is a correlation between the
$\beta$
-wave features of EEG and SA or that they are sensitive to SA, there still exists a gap for exploring the more refined
$\beta$
1,
$\beta$
2 and
$\beta$
3 wave features. Second, current SA classification models still differ in selecting input features and labels, which may bring considerations about the models’ interpretability and rationality. For inputs of classification models, PSD features or SA-sensitive EEG features [Reference Feng, Liu, Wanyan, Chen, Min and Ma7, Reference Kästle, Anvari, Krol and Wurdemann8, Reference Li, Wang and Sourina19, Reference Li, Lan, Cui, Sourina and Wang22] were often adopted, which may lead to differentiated classification results. Among them, the selection of various EEG features for SA classification remains to be further explored.
In addition, researchers chose performance features or SAGAT scores in a psychological paradigm [Reference Kästle, Anvari, Krol and Wurdemann8], and performance metrics in a driving context [Reference Li, Wang and Sourina19, Reference Li, Lan, Cui, Sourina and Wang22, Reference Li, Ng, Yu, Yiu and Lyu23] as output labels to generate relatively large-sample data for SA classification. Considering the unclear relationship between SA and performance, the latter may raise considerations about the rationality of labeling. For small-sample data in an aviation task scenario, less research concentrates on selecting SA-sensitive EEG features as model inputs and adopting SAGAT scores as labels for classification modeling. Last, supervised machine learning classification algorithms have been employed individually or partially in constructing SA classification models, with corresponding classification accuracies ranging from 67% to 82.7%, regardless of their interpretability. There is a lack of comparisons of the classification accuracies of generalised algorithms with different interpretability, including algorithms with easy interpretability (LR, decision tree (DT), linear discriminant analysis (LDA), linear kernel function based SVM, gaussian naive Bayes (GNB)) and algorithms with hard interpretability (RF, ANN, k-nearest neighbors (KNN)) [Reference Osisanwo, Akinsola, Awodele, Hinmikaiye, Olakanmi and Akinjobi24].
The research content in this paper mainly includes (1) exploring the EEG features sensitive to SA; (2) applying typical supervised machine learning algorithms on classifying SA levels based on the above sensitive EEG features; and (3) comparing the classification accuracies of algorithms with different interpretability. Based on the above scenario, the following hypotheses are proposed:
Hypothesis 1 (H1): The absolute power of
$\beta$
1,
$\beta$
2 and
$\beta$
3 is sensitive to changes in SA levels.
Hypothesis 2 (H2): The relative power of
$\beta$
1,
$\beta$
2 and
$\beta$
3 and the corresponding ratio features are sensitive to changes in SA levels.
Hypothesis 3 (H3): SA-sensitive EEG features associated with
$\beta$
1,
$\beta$
2 and
$\beta$
3 were selected to carry out small-sample classification of SA levels, and all of the above generalised supervised machine learning algorithms can obtain good classification accuracies.
2.0 Methods
2.1 Participants
Forty-eight graduate students (23.15 ± 0.72 years, mean ± standard deviation (SD)) with aviation backgrounds were selected as participants (including nine females) from Beihang University. They all had right-hand dominance, good health, normal or corrected-to-normal vision and at least 8 hours of sleep the night before the test. A comprehensive evaluation of participants’ sleep status was conducted in the current study, including: (a) pre-experiment inquiries on time to fall asleep, sleep duration, emotion and arousal states, diet, medication, etc.; (b) naturalistic observations during the experiment of the current state, drowsiness, expressions, body behaviours (e.g., sitting posture, yawning), task performance, etc.; and (c) post-experiment documentation of subjective status reporting, eye movement and EEG sampling status, and the aforementioned details. The Biological and Medical Ethics Committee at Beihang University gave its approval to this study (approval number: BM20220176).
2.2 Experimental platform and EEG equipment
System monitoring, target tracking, scheduling display and resource management are four common subtasks in multi-attribute task battery (MATB) II [Reference Santiago-Espada, Myer, Latorella and Comstock25]. Keyboard, mouse and flying joystick controls are available for interacting with the platform, as shown in Fig. 1. EEG data from 32 channels, including F7, FT7, T3, TP7, T5, A1, FP1, F3, FC3, C3, CP3, P3, O1, FZ, FCZ, CZ, CPZ, PZ, OZ, FP2, F4, FC4, C4, CP4, P4, O2, F8, FT8, T4, TP8, T6 and A2, were recorded using the Neuroscan Neuamps amplifier (Compumedics limited, Victoria, Australia). Ag/AgCl was used for all electrodes. The recorded wave was between 0.1 and 200Hz, and 1,000Hz was used for sampling. It was kept under 5kΩ for the electrode impedance. The left mastoid A1 was used as the reference point, and ground (GND) served as the ground. The electrooculogram (EOG)’s vertical and horizontal channels were concurrently recorded. Typical EEG procedures include: (a) ensuring participants have washed and blow-dried their hair before the experiment or instructing them to do so promptly; (b) selecting reference electrodes; (c) locating eye movement recording positions by placing electrodes 1cm from the outer corners of both eyes for horizontal movement and above the eyebrows and below the lower edge of the eye sockets for vertical movement; (d) choosing an electrode cap based on the participant’s head circumference, worn from front to back with the Cz point at the centre; (e) applying approximately 0.5ml of conductive gel to the ground and reference electrodes, followed by other electrode points; (f) configuring data collection parameters and initiating EEG recording once satisfactory impedance is achieved.
Figure 1. Platform, measurement and analysis system displays and experimental scenarios.
2.3 Experimental design
On the above basis, 48 participants were recruited to complete an SA standard test in an aviation task scenario based on MATB II, and the corresponding EEG data and SAGAT scores were recorded. The population with the top 25% of SAGAT scores was selected as the high-SA level (HSL) group, and the bottom 25% was selected as the low-SA level (LSL) group. The HSL group included 12 males and no females, whereas the LSL group comprised 7 males and 5 females in terms of gender distribution. The SA level was the independent variable in our experimental design, whereas the EEG features were the dependent variable. High- and low-SA groups were categorised based on SAGAT scores on the MATB II tasks, thus attaining the SA levels.
The SAGAT freezing interface occupied the entire screen, and participants responded to questions using a mouse, as shown in Fig. 1. The MATB II, derived from the primary flight task [Reference Santiago-Espada, Myer, Latorella and Comstock25], underwent a redesign and simplification, as follows: (a) in the tracking monitoring task, a two-dimensional random-input compensatory task displayed the circular reticle’s movement, with manual programming for mode switching. Participants were required to press the left arrow on the keyboard when transitioning from ‘auto-pilot’ to ‘manual’. (b) For the system monitoring task, participants identified any scale displaying a steady ‘extreme’ position at the upper or lower end by clicking on the corresponding scale. (c) In the resource monitoring task, participants responded to pump failures by clicking on the corresponding pump using the left mouse button. (d) In the communication monitoring task, participants monitored communication task quantities on the schedule line and pressed the right arrow on the keyboard when the upper edge of the communication block reached the zero lines (located on the left of the scheduling display, marked in green).
2.4 Experimental procedure
The experiment consisted of two stages: training and the formal experiment. In the training stage, all participants received adequate training and took an assessment test, of which the answers’ accuracy was required to be above 90% before they were allowed to participate in the formal experiment. Each participant had to perform the resting test with EEG sensors first. Before the experiment, all participants completed an informed consent form after receiving information about the procedure and instructions. In the formal stage, the computerised freezing technique-based SAGAT measurements were used to assess participants’ SA levels during MATB II tasks [Reference Smith, Endsley and Clark10]. Specifically, at some random time during the experiment, the MATB II interface was frozen and covered by the given inquiry interface. Six questions were asked at each interval of presentation freezing, which lasted around five minutes. Questions were generally shown for less than a minute, and there were more than two minutes between each freezing [Reference Endsley26]. No questions would be asked in the first three minutes, which guaranteed the participant to gather information in a secure environment [Reference Endsley26]. The participant used mouse clicks to provide an answer to the pop-up query that had been chosen from the SAGAT database.
3.0 Results
3.1 Quantitative β-based EEG analysis of different SA groups
3.1.1 Data analysis methods
The average of the bilateral mastoid electrodes offline was used as the new reference for the EEG signal. The bandpass filtering of 1–30Hz was used, and the analysis was carried out using the toolbox MATLAB R2020a eeglab v2023.0 (accessible via https://sccn.ucsd.edu/eeglab/download/daily/). Independent component analysis (ICA) was used to evaluate the EEG signals. The Adjust 1.1.1 toolbox, proven effective in mitigating EEG artifacts as indicated by various studies [Reference Billeci, Callara, Guiducci, Prosperi, Morales, Calderoni, Muratori and Santocchi27, Reference Nakuci, Covey, Shucard, Shucard and Muldoon28], was employed to remove artifacts, including horizontal eye movement, vertical eye movement, eye blink and generic discontinuity, as defined in Mognon et al. [Reference Mognon, Jovicich, Bruzzone and Buiatti29]. Access the toolbox at https://www.nitrc.org/frs/?group_id=739&release_id=3054#. As shown in Fig. 2, Fast Fourier transform (FFT) was carried out after segmenting the EEG data. After that, eight waves were created by averaging the segmented data: 1–4Hz for the delta (
$\delta$
) wave, 4–8Hz for the theta (θ) wave, 8–12Hz for the alpha (
$\alpha$
) wave, 12–14Hz for the sigma1 (
$\sigma$
1), 14–16Hz for the sigma2 (
$\sigma$
2), 16–20Hz for the beta1 (
$\beta$
1), 20–24Hz for the beta2 (
$\beta$
2) and 24–30Hz for the beta3 (
$\beta$
3) wave [Reference Shokouh Alaei, Ghoshuni and Vosough21, Reference Nayak and Anilkumar30, Reference Park, Che, Im, Jung, Bae and Lee31]. The total band power was calculated from the integration of the power across the defined spectrum. Microvolts squared per octave (μV2) served as the unit of power analysis.
Figure 2. Steps for EEG feature processing and classification of SA levels.
The EEG acquisition device was used throughout the experiment to capture the EEG data in real-time. The EEG spectral features were the eight bands’ absolute power (μV2), relative power (%) and
$\beta$
1
∼
$\beta$
3
related ratios. The ratio of a wave’s absolute power to the total wave’s absolute power (1–30Hz) is known as the relative power of that wave. In addition,
$\beta$
1
∼
$\beta$
3
related ratio feature selection was conducted according to the slow and fast wave features. Since δ oscillations are closely associated with states of diminished consciousness [Reference Frohlich, Toker and Monti32], and no significant main effects were found in the subsequent absolute and relative power, they were not conceived in this study. Based on the above, 72
$\beta$
1
∼
$\beta$
3
related ratio features in three categories were selected, including single, double and triple denominator categories. After EEG processing, subsequent steps involving data preprocessing, classifier application and performance evaluation are further conducted. Detailed descriptions for these processes are provided in the subsequent sections.
Previous studies showed that the relative power of the
$\beta$
wave in C and P regions is sensitive to changes in SA. Therefore, in our investigation, electrode placements in three different brain regions – central: C3, Cz, C4; central-parietal: CP3, CPz, CP4; and parietal: P3, Pz, P4 – were taken into account for analysis [Reference Feng, Liu, Wanyan, Chen, Min and Ma7, Reference Kästle, Anvari, Krol and Wurdemann8]. Additionally, studies frequently fixed on the differences between central, right and left brain literalities [Reference Trapsilawati, Herliansyah, Nugraheni, Fatikasari and Tissamodie33, Reference Wanyan, Zhuang, Lin, Xiao and Song34]; so three lateralities (left: C3, CP3, P3; central: Cz, CPz, Pz; right: C4, CP4, P4) were also investigated. Statistical analysis was conducted using Statistical Package for the Social Sciences (SPSS) Statistics 25.0 (IBM, United States), and the confidence levels for statistical tests were set to
$\alpha$
= 0.05. Repeated-measures analysis of variance (ANOVA) was adopted in three brain regions (central, central-parietal and parietal) × three brain lateralities (left, middle and right) × two SA levels (LSL and HSL) to investigate the interaction and main effects of SA levels on the features. When interaction and main effects were significant, the Bonferroni method was used for simple effects analysis and post hoc analysis. Sphericity was tested using the Mauchly method. If the hypothesis was not accepted, it was adjusted using the Greenhouse-Geisser method [Reference Armstrong35, Reference Mishra, Singh, Pandey, Mishra and Pandey36].
3.1.2 Absolute power of different SA levels
The topographical maps in Fig. 3 have clarified the average values of absolute and relative power of EEG spectrum features for all participants at two SA levels. Brighter hues (in this case, red) signify electrode sites with higher activations, while blue denotes electrode sites with lower activations.
The absolute power of EEG features under the LSL and HSL were selected for ANOVA, and the results indicated no significant interaction effect (ps > 0.05). For the θ absolute power, the main effect of SA levels was significant (F(1, 22) = 4.351, p = 0.049, η 2 = 0.165). Post hoc comparisons showed that the θ absolute power for HSL was significantly lower than that for LSL. As indicated in Table 1, the main effects of SA levels for the absolute power in the other waves were also insignificant (ps > 0.05).
Table 1. ANOVA results of absolute power for the two SA levels
Figure 3. Topographical representation of average values of absolute and relative power of EEG spectrum features (scores on the color bar indicate the logarithm for the spectral power represented in each topography).
3.1.3 Relative power results of different SA levels
Figure 4 depicts descriptive results of the relative power in eight waves for various SA levels, in which * represents that p ≤ 0.001. As shown in Table 2, ANOVA results demonstrated a significant interaction effect of the
$\beta$
3 relative power for the laterality × SA level (F(1.951, 42.911) = 4.313, p = 0.020, η
2 = 0.164). As the result of simple effect analysis, for HSL,
$\beta$
3 relative power on the right side of the brain was significantly higher than that on the left side and the midline of the brain (p = 0.017, p < 0.001), with no significant difference between the left side and the midline of the brain (p > 0.05). For LSL,
$\beta$
2 relative power on the left side of the brain was significantly higher than that on the midline of the brain (p = 0.026), with no other significant difference (ps > 0.05). The interaction effects of the relative power in the other waves were not significant. In addition, for θ,
$\beta$
1,
$\beta$
2 and
$\beta$
3 relative power, the main effects of SA levels were significant (p = 0.027, p = 0.027, p = 0.007 and p = 0.003). Post hoc comparisons revealed that
$\beta$
1,
$\beta$
2 and
$\beta$
3 relative power for LSL were significantly lower than those for HSL; and θ relative power for LSL was significantly higher than that for HSL.
Figure 4. Relative power for the two SA levels. (*: p ≤ 0.001).
3.1.4 Ratios results of different SA levels
For the ratio features, ANOVA showed that the main effects of SA levels were all significant (ps < 0.05) except six features, including
$\alpha/\beta_1, \sigma_1/\beta_1$
,
$\sigma_2/\beta_1, (\alpha+\sigma_1)/\beta_1, (\sigma_1+\sigma_2)/\beta_1, (\alpha+\sigma_1 +\sigma_2)/\beta_1$
and
$\alpha/(\beta_1+\beta_2)$
. Detailed table format of ratios can be found in a Supplementary Materials.
3.2 Classification of SA levels
An independent sample t-test was conducted for the above typical EEG features. Considering the large amount of corresponding SA-sensitive features, this study set p ≤ 0.001 as the selection criteria and obtained 57 model input features, as shown in Table 3. Furthermore, supervised learning algorithms based on these sensitive features were applied to classify of SA levels, including two main steps: data preprocessing and data processing, as shown in Fig. 2.
Table 2. ANOVA results of relative power for the two SA levels
Table 3. Selection of model inputs
3.2.1 Data preprocessing
As shown in Fig. 2, Data preprocessing was first conducted in the following steps: (1) min-max normalisation, i.e.,
${\rm{x'}} = {({\rm{x}} - \min ({\rm{x}}))} / (\max ({\rm{x}}) - \min ({\rm{x}}))$
; (2) Z-score standardisation, i.e.,
${\rm{Z}} = {{({\rm{x'}} - \mu )} / \sigma }$
; (3) principal component analysis (PCA) is a method employed for reducing data dimensionality by representing correlated datasets as uncorrelated principal components (PCs) (Ayesha et al., 2020), only PCs with eigenvalues exceeding 1 were selected here.; (4) Z-score standardisation for the PCs.
3.2.2 Data processing
As shown in Fig. 2, classifiers, including LDA, LR, GNB, DT, KNN, RF and ANN, were adopted to classify SA levels through the sklearn package in Python. The model inputs were the PCs of the preprocessed SA-sensitive EEG features. As a comparison, the PCs for the other seven categories of EEG features were also computed, including (1) absolute power; (2) relative power; (3) ratios; (4) absolute power versus relative power; (5) relative power versus ratios; (6) absolute power versus ratios; and (7) absolute power, relative power and ratios. The main steps were as follows: (1) found the optimal parameter settings for different classification models based on the results of five-fold cross-validation; (2) adopted the optimal parameter settings to calculate the classification results for each fold of cross-validation, including accuracy, precision, recall, F1 score, etc; (3) averaged the classification results of five-fold cross-validation to obtain the mean values; and (4) repeated the process 10 times to get the final classification accuracies of the above classifiers, as shown in Table 4. The interpretability of a classification model is contingent upon its underlying algorithms and discriminatory methods. Comprehensive information on this aspect can be found in relevant literature [Reference Osisanwo, Akinsola, Awodele, Hinmikaiye, Olakanmi and Akinjobi24, Reference Uddin, Khan, Hossain and Moni37] as well as in the MATLAB documentation on machine learning, accessible at https://ww2.mathworks.cn/help/releases/R2020a/stats/choose-a-classifier.html#bu62uj8.
Table 4. Classification results of typical classifiers
4.0 Discussion
This study explored EEG features sensitive to SA and further classified different SA levels. The results found that the difference between LSL and HSL was insignificant for the absolute power of
$\beta$
1,
$\beta$
2 and
$\beta$
3. The relative power of
$\beta$
1,
$\beta$
2 and
$\beta$
3, as well as most ratios associated with
$\beta$
1,
$\beta$
2 and
$\beta$
3, was sensitive to changes in SA levels, which is evidenced by the fact that these metrics were higher in HSL than in LSL. Furthermore, SA-sensitive EEG features were screened according to the criterion of ps ≤ 0.001 in the independent samples T-test, and the screened EEG features were sent into the classifiers for small-sample classification of SA levels. The results showed that LR, DT, LDA, SVM, GNB, RF, ANN and KNN models achieve good accuracy (greater than 75%). Last but not least, for the classification of SA levels in this study, LR and DT are recommended by combining the interpretability and accuracy of the models.
In contrast to previous studies that indicated that
$\beta$
wave activation is associated with changes in SA levels, this study further explored refined
$\beta$
1,
$\beta$
2 and
$\beta$
3 wave activities. In the present study, the difference between SA levels for the absolute power features of
$\beta$
1,
$\beta$
2 and
$\beta$
3 was insignificant. Therefore, H1 was rejected. That is, the absolute power of
$\beta$
1,
$\beta$
2 and
$\beta$
3 were insensitive to changes in SA levels, which suggests that these EEG features are limited in revealing SA levels. This result enriches the findings of previous studies in
$\beta$
absolute power [Reference Feng, Liu, Wanyan, Chen, Min and Ma7]. In addition, a significant difference was found in the absolute power of the θ wave, which was manifested by the fact that the absolute power of the θ wave of LSL was higher than that of HSL. θ oscillations are strongly associated with increased working-memory load or attention demands [Reference Li, Lan, Cui, Sourina and Wang22, Reference Ren, Xu, Zhao, Zhang, Wang, Xu, Zhou, Chen, Chen, Li and Shan38]. This indicates that LSL requires more memory or attention resources when facing the same operational task.
For EEG feature extraction, it has been ubiquitous to carry out relative power and ratio based on absolute power [Reference Hou, Chen and Zhou39, Reference Yi Wen and Mohd Aris40]. The present study found that the relative power of
$\beta$
1,
$\beta$
2, and
$\beta$
3 were higher in HSL than in LSL. Also, it found that most of
$\beta$
1,
$\beta$
2 and
$\beta$
3 related ratio features were sensitive to SA levels. Therefore, H2 was accepted.
$\beta$
wave oscillations are generally associated with higher levels of arousal or vigilance [Reference Yi Wen and Mohd Aris40, Reference Zhao, Zhao, Liu and Zheng41], which suggests that LSL has lower levels of arousal or vigilance. This result can be considered as an extension of previous studies [Reference Feng, Liu, Wanyan, Chen, Min and Ma7, Reference Kästle, Anvari, Krol and Wurdemann8], suggesting that the relative power of
$\beta$
1,
$\beta$
2 and
$\beta$
3 and their associated ratio features are sensitive in revealing changes in SA levels. This finding provides an ideal theoretical basis for subsequent classification of SA levels using the aforementioned SA-sensitive EEG features, which is conducive to improving the interpretability of model inputs. Finally, a significant difference between different SA levels was also found in the θ wave relative power, which manifested as significantly higher θ relative power in LSL than in HSL, once more suggesting the lack of resources exhibited by LSL when oriented to the same task demands.
In addition, this study constructed a classification model for SA. As demonstrated in the above results, this study used SA-sensitive EEG features as inputs and selected SA levels classified by SAGAT scores as outputs for small-sample classification of SA levels in an aviation task scenario. The results of the five-fold cross-validation showed that the classification accuracies of LR, DT, LDA, SVM, GNB, RF, ANN and KNN models were greater than 75%. Therefore, H3 was accepted. From the results in Table 4, on one hand, compared with the accuracy of the previous work in selecting the sensitive EEG features related to
$\beta$
-wave, the above classifiers obtained higher accuracies after selecting the refined sensitive EEG features related to
$\beta$
1,
$\beta$
2 and
$\beta$
3; On the other hand, the optimal classification accuracy using SA-sensitive EEG features as inputs was found to be higher compared to using the following data as input: (a) absolute power; (b) relative power; (c) ratios; (d) absolute power versus relative power; (e) relative power versus ratios; (f) absolute power versus ratios; and (g) absolute power, relative power and ratios. The results can provide some ideas for subsequent SA classification studies.
Moreover, considering the urgent need and application potential of cockpit transparency and human-machine trust in aviation, interpretability becomes one of the crucial references for model selection. Therefore, combining the characteristics of small-sample data makes it practical to compare the modeling effects using generalised supervised machine learning classifiers with different interpretability. The studies above show that the best classification accuracies obtained by LR and DT with easy interpretability (92%) are superior to those obtained by RF and ANN with hard interpretability (88.8% and 84%, respectively). The best classification accuracies in both types of models are better than 67% [Reference Kästle, Anvari, Krol and Wurdemann8], 70.6%–74.32% [Reference Li, Wang and Sourina19, Reference Li, Lan, Cui, Sourina and Wang22], 76.1%–82.7% [Reference Li, Ng, Yu, Yiu and Lyu23] and 70.8% [Reference Feng, Liu, Wanyan, Chen, Min and Ma7]. Analysing the reasons, on the one hand, may lie in the selection of refined SA-sensitive EEG features and the attempt at parameter tuning of multiple supervised machine learning classifiers. On the other hand, small sample data may be more suitable for using easily interpretable machine learning classifiers, while more complex network structures and hierarchical parameter settings may not be suitable for this context. For the classification of SA levels in this study, LR and DT are recommended by combining the interpretability and accuracy of the models.
Compared to prior studies on SA discrimination models, the present study focused on analysing SA levels using SA-sensitive EEG features in small-sample dataset. This included performing model construction and comparison with accuracy and interpretability. The study addresses specific considerations in the following aspects. The first is the input and output of the classifier. Differences in inputs may lead to differences in discriminatory results and trigger considerations in the interpretability of model inputs. Unlike prior studies using PSD features [Reference Kästle, Anvari, Krol and Wurdemann8, Reference Li, Wang and Sourina19, Reference Li, Lan, Cui, Sourina and Wang22], SA-sensitive
$\beta$
-features [Reference Feng, Liu, Wanyan, Chen, Min and Ma7] or a combination of EEG and eye movement features [Reference Li, Ng, Yu, Yiu and Lyu23], this study selected SA-sensitive, refined
$\beta$
1,
$\beta$
2 and
$\beta$
3 wave features as inputs. Utilising these EEG features resulted in higher classification accuracy compared to alternative feature combinations, as shown in Table 4. For output labels, the SAGAT method is employed due to its proven sensitivity and reliability in assessing SA across various experimental settings and domains [Reference Endsley42]. In line with prior studies [Reference Feng, Liu, Wanyan, Chen, Min and Ma7, Reference Kästle, Anvari, Krol and Wurdemann8], SAGAT scores are used to categorise high and low SA levels, differing from studies that used performance metrics as labels for fatigue/load-related SA [Reference Li, Wang and Sourina19, Reference Li, Lan, Cui, Sourina and Wang22, Reference Li, Ng, Yu, Yiu and Lyu23]. Sleep-related fatigue, often associated with reduced SA [Reference Hopko, Khurana, Mehta and Pagilla43, Reference Sneddon, Mearns and Flin44], needs consideration in SA-related experiments. Based on the experiment recordings of participants’ statuses in our study, we indeed observed phenomena that might be related to fatigue in some participants, such as three times reporting falling asleep after midnight the day before the experiment and three times showing occasional adjustments in sitting posture or yawning. Considering the relatively short experiment duration, and limited number of fatigue related participant reactions, SA is considered to remain unaffected by sleep-related fatigue in this study. However, exploring the impact of sleep-related fatigue on SA will be quite meaningful and should be investigated in long-duration experiment tasks carefully.
What’s more, the accuracies of classifiers with different interpretability are compared. Machine learning models with good transparency or easy interpretability enhance human-machine mutual trust, thus improving situation awareness, task performance and human factors safety [Reference Schmidt, Biessmann and Teubner45, Reference Win, Jacobs, Muttram, Olszewska, Rajabiyazdi, Theodorou, Underwood, Wortham and Watson46]. Some studies chose only discriminative models with easy interpretability [Reference Feng, Liu, Wanyan, Chen, Min and Ma7], others chose models with hard interpretability [Reference Kästle, Anvari, Krol and Wurdemann8, Reference Li, Wang and Sourina19] and still others used both types of models [Reference Li, Lan, Cui, Sourina and Wang22, Reference Li, Ng, Yu, Yiu and Lyu23]. In this study, classification algorithms such as LR and DT with easy interpretability were selected and compared with models such as RF and ANN with hard interpretability. In terms of accuracy, LR and DT obtained superior accuracy, outperforming 67%–82.7% of previous results. In conclusion, this study conducted a classification study of SA levels divided by SAGAT scores based on SA-sensitive EEG features with a small-sample dataset in an aviation task scenario. The results show that the selected generic supervised learning classifiers all obtained good classification accuracies, and the LR and DT are recommended by combining the interpretability and accuracy of the models.
Despite the previous findings, this study has some shortcomings. First, the platform is MATB II based on aviation task settings, which have specific differences from the actual flight scenario. It is still necessary to build task scenarios closer to actual flights to further verify the reliability of the conclusions. Secondly, considering the difference between students and real frontline operators, an improvement in subject selection is needed in the subsequent study. The results of this study can provide valuable references for the exploration of SA-sensitive features and the selection of SA classification models. The practical application in real-world settings still requires attention to the following aspects: (a) validating and optimising SA-sensitive EEG features in specific aviation domain; (b) considering factors like time frame and computational power for meaningful assessment of SA levels; (c) exploring the feasibility of wearable, real-time EEG measurements in flight, taking into account aviation regulations, previous practices and potential interferences in physiological measurement.
5.0 Conclusions
Based on the typical human-computer interaction task setting in this study, the following conclusions can be obtained:
(1) The refined relative power of
$\beta$
1,
$\beta$
2 and
$\beta$
3 and their corresponding ratio features are sensitive to SA and can provide effective inputs for subsequent classification of SA level.
(2) Small-sample classification of SA levels was carried out by using SA-sensitive EEG features, and it was found that the general supervised learning classifier, including LR, DT, RF, ANN and other classification algorithms, can obtain good classification accuracies (greater than 75%).
(3) For the classification of SA levels in this study, LR and DT are recommended by combining the interpretability and accuracy of the models.
In this study, the exploration of SA-sensitive EEG features and the classification of SA levels were carried out under different SA levels, and the corresponding conclusions can provide some references for online monitoring and classification optimisation of SA.
Supplementary Material
To view supplementary material for this article, visit https://doi.org/10.1017/aer.2024.36
Data availability statement
The data presented in this study are available on request from the corresponding author. The data are not publicly available due to privacy.
Acknowledgments
The authors gratefully acknowledge the agencies NSFC, CCAC and ASFC for the financial support. In addition, the authors acknowledge the participants for their participation.
Author contributions
All authors have read and agreed to the published version of the manuscript. Conceptualisation, formal analysis, investigation, methodology, validation, visualisation and writing – original draft, C.F.; conceptualisation, data curation, investigation, methodology, resources and writing – review and editing, S.L.; conceptualisation, data curation, methodology, resources, supervision and writing – review and editing, project administration, funding acquisition, X.WY.; methodology, validation and writing – original draft, Y.D.; visualisation, investigation and writing – original draft Z.W.; data curation, validation and writing – original draft, C.Q.
Funding
This research was funded by the joint program of the National Natural Science Foundation of China and Civil Aviation Administration of China (No. U1733118), the National Natural Science Foundation of China (No. 71301005), and the Aeronautical Science Foundation of China (No. 201813300002).
Competing interests
The authors declare no competing interests.
