2.1. Novelty of work

The model used in this research provides both feature selection and classification in one step, as opposed to separate feature selection and classification models. The model also provides shrinkage of the existing features, which is utilized to improve the interpretability of the results. Unlike other feature selection/extraction techniques, this model uses the original features within the model, rather than transformed features – for example, the features generated in PCA or JDA. Therefore, the work provides a novel implementation of a one-step feature selection and classification model, with the benefit of interpretable results. Most interestingly, the resultant features can be used to generalize to new, previously unobserved experiments.

3.1. The LASSO loss function

For both the independent learner and MTL, logistic regression is used for classification. To mathematically model binary classifications of “True” or “False” a linear regression is used in the form of $ {\boldsymbol{W}}^T{\boldsymbol{x}}^{(i)} $ , followed by an activation function. The Sigmoid function is used in this case, to generate a predicted value between zero and unity:

(3.1) $$ {\hat{y}}^{(i)}=\frac{1}{1+{e}^{-{\boldsymbol{W}}^T{\boldsymbol{x}}^{(i)}}} $$

where $ {\boldsymbol{x}}^{(i)}\in {\mathrm{\mathbb{R}}}^M $ is an observed set of readings of $ M $ features, $ \boldsymbol{W}\in {\mathrm{\mathbb{R}}}^M $ is the weight vector with a corresponding weight for each of the $ M $ features, and the superscript $ i $ refers to one of the $ N $ observed sets of readings, $ i\in \left\{1,2,\dots, N\right\} $ .

When the Sigmoid function is used, the resultant value from equation (3.1) will be a value between 0 and 1. The value can be interpreted as a probability that $ {\hat{y}}^{(i)} $ is “True”. To set the classification output, a threshold value must be selected; in this work, the typical value of $ 0.5 $ is used. The cross-entropy loss function for learning is dependent on the predicted value $ \hat{y} $ and the observed value $ y $ :

(3.2) $$ J\left({y}^{(i)},{\hat{y}}^{(i)};\boldsymbol{W}\right)=-\frac{1}{N}\sum \limits_{i=1}^N\left({y}^{(i)}\mathit{\log}\left({\hat{y}}^{(i)}\right)+\left(1-{y}^{(i)}\right)\mathit{\log}\left(1-{\hat{y}}^{(i)}\right)\right) $$

Equation (3.2) represents the empirical loss function for a single task. The empirical loss function is a measurement of error per sample, which is averaged over all $ N $ measurement sets. Without regularization, the model is likely to overfit; that is, it will perform well for training data but will not perform well when new data are tested. To reduce the likelihood of overfitting, a further term is added to the loss function. The Least Absolute Shrinkage and Selection Operator (LASSO) algorithm (Tibshirani, Reference Tibshirani1996) adds a regularization term to the empirical loss (3.2) in the form of an $ {\mathrm{\ell}}^1 $ norm. To understand the impact of the $ {\mathrm{\ell}}^1 $ norm on the loss function, it is useful to understand the general form of $ {\mathrm{\ell}}^p $ norms. If $ \boldsymbol{W} $ is a vector, $ \left[{w}_1,{w}_2,\dots {w}_M\right] $ , then the $ {\mathrm{\ell}}^P $ norm is given by,

(3.3) $$ {\Vert \boldsymbol{W}\Vert}_P={\left(\sum \limits_{j=1}^M{w}_j^P\right)}^{\frac{1}{P}} $$

To visualize the effects of LASSO, consider a simple two-feature optimization; Figure 2 shows three examples of such an optimization. $ \hat{W} $ refers to the optimal solution of the weights without any penalty. If implemented, this solution would fit the training data well but is likely to have high variance. Hence, a penalty is included. The ‘constraint surface’ in the three examples is represented by the pink shape centered on the origin. The penalties, for some $ c>0 $ , are visualized here for comparison as follows:

• • $ {\mathrm{\ell}}^1 $ (or LASSO) penalty: $ {\sum}_{j=1}^M\mid {w}_j\mid <c $ ,

• • $ {\mathrm{\ell}}^2 $ penalty: $ {\left({\sum}_{j=1}^M{w}_j^2\right)}^{\frac{1}{2}}<c $ ,

• • $ {\mathrm{\ell}}^{\infty } $ penalty: $ {\left({\sum}_{j=1}^M{w}_j^{\infty}\right)}^{\frac{1}{\infty }}<c $ .

Figure 2. A visualization of the shrinkage gained from the LASSO compared to other regularization methods.

The solution to the optimization is the point where the contour of the optimal solution meets the edge of the constraint surface.

For the three examples in Figure 2, the optimal weight is in the same position; the only difference is the shape representing the penalty term. For LASSO, the intersection is on the axis, that is, $ {w}_1 $ is 0, whereas for the other two penalties, the solution is not. To summarize, the visualization demonstrates how LASSO can shrink some of the features to 0, whereas this is less likely with other $ {\mathrm{\ell}}^n $ penalty terms. By implementing $ {\mathrm{\ell}}^1 $ regularization the resulting model is more likely to be representative when given test datasets. However, it should be noted that if the regularization dominates, then under-fitting will occur, and the model will perform poorly for both the training and the test datasets. The total loss function given by the standard independent LASSO algorithm is,

(3.4) $$ \Gamma ({y}^{(i)},{\hat{y}}^{(i)};\boldsymbol{W})=J({y}^{(i)},{\hat{y}}^{(i)};\boldsymbol{W})+\lambda \parallel \boldsymbol{W}{\parallel}_1 $$

where $ \lambda $ is a scalar known as the regularization parameter, and $ \parallel \boldsymbol{W}{\parallel}_1={\sum}_{j=1}^M\mid {w}_j\mid $ is the $ {\mathrm{\ell}}^1 $ norm of the weight vector.

This total loss function can increase sparsity as it applies the $ {\mathrm{\ell}}^1 $ norm. A large feature set is often less interpretable; therefore, sparsity is a desirable characteristic as it not only offers a reduction in processing requirements but has the additional benefit of better interpretability. In addition, if fewer features are required in a model, this helps to combat the curse of dimensionality (Bellman and Kalaba, Reference Bellman and Kalaba1959), which is often an issue for vibration-based SHM.

3.2. Multitask LASSO

To use the loss function across multiple tasks within one model, the loss function must consider all related tasks. Joint regularization with LASSO was introduced in MTL by Obozinski et al. (Reference Obozinski, Taskar and Jordan2006) to encourage features to share the same sparsity pattern among similar tasks by adding an $ {\mathrm{\ell}}^{2,1} $ constraint. The constraint is $ {\mathrm{\ell}}^2 $ across the different tasks, which is then combined into an $ {\mathrm{\ell}}^1 $ -norm across the features,

(3.5) $$ {\displaystyle \begin{array}{ll}\parallel {\boldsymbol{W}}_L{\parallel}_2={\left(\sum \limits_{l=1}^L{w}_{j,l}^2\right)}^{1/2}& \left({\mathrm{\ell}}^2\mathrm{across}\ \mathrm{the}\ \mathrm{tasks}\right)\\ {}\parallel {\boldsymbol{W}}_{\boldsymbol{M}}{\parallel}_{2,1}=\sum \limits_{j=1}^M\mid {\left(\sum \limits_{l=1}^L{w}_{j,l}^2\right)}^{1/2}\mid & \left({\mathrm{\ell}}^1\mathrm{across}\ \mathrm{the}\ \mathrm{features}\right)\end{array}} $$

where $ L $ is the number of tasks, and $ {\boldsymbol{W}}_L\in {\mathrm{\mathbb{R}}}^M $ .

For each feature, there is an $ {\mathrm{\ell}}^2 $ norm constraint between the tasks. The next layer of constraint is the $ {\mathrm{\ell}}^1 $ norm. For this research, if a feature has a zero weight with this constraint, then all of the tasks will have a zero weight for the given feature; that is, the weight matrix is shared and all of the tasks will share the same sparsity pattern. For multiple tasks the empirical loss function (3.2), becomes,

(3.6) $$ J\left({y}_l^{(i)},{\hat{y}}_l^{(i)};{\boldsymbol{W}}_l\right)=-\frac{1}{L}\sum \limits_{l=1}^L\frac{1}{N_l}\sum \limits_{i=1}^{N_l}\left({y}_l^{(i)}\mathit{\log}\left({\hat{y}}_l^{(i)}\right)+\left(1-{y}_l^{(i)}\right)\mathit{\log}\left(1-{\hat{y}}_l^{(i)}\right)\right) $$

where $ {\boldsymbol{W}}_l=\left[{w}_{1,l},{w}_{2,l},\dots {w}_{M,l}\right] $ refers to the weight vector, and $ {N}_l $ refers to the number of samples for a given task $ l $ . Using the empirical loss function defined above (3.6), the total loss function (3.4), becomes,

(3.7) $$ \Gamma \left({y}_l^{(i)},{\hat{y}}_l^{(i)};{\boldsymbol{W}}_l\right)=J\left({y}^{(i)},{\hat{y}}^{(i)};{\boldsymbol{W}}_{\mathbf{l}}\right)+\lambda \parallel {\boldsymbol{W}}_M{\parallel}_{2,1} $$

where $ \parallel {\boldsymbol{W}}_M{\parallel}_{2,1}={\sum}_{j=1}^M\mid {\left({\sum}_{l=1}^L{w}_{j,l}^2\right)}^{1/2}\mid $ .

3.3. Solving with gradient boosting

Further sparsity can be achieved in both independent learning and multitask learning by using gradient boosting to solve the algorithm (Obozinski et al., Reference Obozinski, Taskar and Jordan2006). Gradient boosting uses coordinate descent as opposed to gradient descent. Rather than updating all weights simultaneously in one iteration, gradient boosting only updates the weight that provides the largest reduction in the loss. Algorithm 1 details the methodology of gradient boosting for the multitask learner algorithm. As all of the weights are initially zero and only one weight is adjusted per iteration, if a feature has a low influence on the loss function, then it will remain zero, hence encouraging sparse solutions.

Algorithm 1 shows the equations that would be used for the MTL approach; however, the methodology is similar in the case of single-task learning models too (utilizing equations (3.2) and (3.4)). To initialize the algorithm, a step size needs to be selected ( $ \varepsilon $ ); this is the increment that one of the weights will change by in each iteration, it is a measure of the sensitivity of the resulting weight matrix. In addition, a tolerance needs to be specified ( $ \xi $ ), used to determine whether a forward or backward step will be taken. The condition specified in Zhao and Yu (Reference Zhao and Yu2004), is that $ \varepsilon \ge \xi $ . Larger values of these parameters will result in sparser weight matrices. Only the features that have a significant impact on the loss result in updated weights. With reduced sensitivity, the algorithm becomes less accurate, with less iterations, therefore, an increased level of sparsity. With smaller values of the hyperparameters there will be increased sensitivity, more iterations of the algorithm and hence a more accurate (but potentially less sparse) solution.

Following initialization, iterations continue by identifying the weight, which results in the largest reduction in total loss (independent learner: (3.4) and MTL: (3.7), Algorithm 1 line 7). There now exists the original weight matrix and the new weight matrix (with the alteration to the weight, which results in the largest reduction in total loss). The difference between the total loss with the new weight matrix and the total loss with the old matrix represents the loss, $ L $ (Algorithm 1 line 8).

A forward step is taken if $ L<-\xi $ ; the weight matrix is updated and there are no changes to the regularization parameter (Algorithm 1, line 10–11). However, if $ L\ge -\xi $ , then a backward step is taken. For a backward step, the original weight matrix is used and the weight that results in the largest reduction in empirical loss is identified (Algorithm 1, line 13). The weight matrix is updated with the alteration to the weight, which results in the largest reduction in empirical loss (Algorithm 1 line 14). The regularization parameter is also updated (Algorithm 1, line 15), as the minimum value between the original regularization parameter and a new regularization parameter. The iterations of the algorithm continue until the regularization parameter is less than, or equal to, 0. The result of gradient boosting is a sparse solution with weights that do not impact the loss remaining at 0.

3.4. Algorithm performance

Two elements will determine the success of this algorithm: firstly, how close the algorithm is to finding the true labels, and secondly, how sparse the resulting weight matrix is. In this case, we assume sparsity is a good indication of the interpretability of features. Often in machine-learning problems, it is only the accuracy of the algorithm that is assessed; however, a reduction in features can result in increased simplicity and interpretability, as well as reduced processing times, which would be desirable outcomes for SHM applications. Both studies demonstrate how sparse solutions can lead to more meaningful features and better generalization.

While iterating the algorithm, the value of the total loss function is used to assess performance via (3.4) and (3.7); however, this loss is not intuitive when comparing different models against each other. Therefore, to compare independent learning models and MTL models against each other, the F1 score is used. The F1 score is suitable as it is bounded between 0–100% and applies if class sizes are uneven. The F1 score is the harmonic mean of precision and recall, where precision is the percentage of accurate positive classifications from all of the predicted positive classifications and recall is the percentage of accurate positive classifications from all of the actual positive classifications.

Hurley and Rickard (Reference Hurley and Rickard2009) provide a comparison of different measures of sparsity; among several methodologies for measuring sparsity, the Gini Index was highlighted as the most robust measure. Originally used to measure wealth distribution by Farris (Reference Farris2010), it has since been applied to various disciplines including ecology (Cordonnier and Kunstler, Reference Cordonnier and Kunstler2015) and medicine (Bandara et al., Reference Bandara, Gurunayaka, Lakraj, Pallewatte, Siribaddana and Wansapura2022). In 2017, the Gini Index was introduced in encoder-based applications to assess the health condition of rotating machinery (Zhao and Lin, Reference Zhao and Lin2018). Hurley and Rickard (Reference Hurley and Rickard2009) defines the Gini Index as,

(3.8) $$ G\left(\boldsymbol{W}\right)=1-2\sum \limits_{j=1}^M\frac{w_j}{{\Vert \boldsymbol{W}\Vert}_1}\left(\frac{M-j+\frac{1}{2}}{M}\right) $$

where the $ \boldsymbol{W} $ vector used in equation (3.8) has been ordered by magnitude such that $ \mid {w}_x\mid <\mid {w}_{x+1}\mid $ for $ x\in \left\{1,\dots, M\right\} $ . The Gini Index will not only consider if a weight has been activated or not, but the magnitude of the weight too; this measure gives a result $ G\left(\boldsymbol{W}\right)\in \left[0,1\right] $ .

3.5. Benchmark tests

To provide a benchmark that relates closely to MTL LASSO, the performance will be analyzed against JDA, which is supervised in both the source and target domains (using the F1 score). JDA aims to find a space by using the kernel-trick and then projecting the data into a space where they become aligned across domains. To provide classification, a two-step process is required: feature extraction, via JDA, followed by a classifier (in this case, k-nearest neighbor (kNN)). JDA does not encourage sparsity, since the linear projection on the kernel is not regularized in the same way. Hence, sparsity will not be assessed for the benchmark.

4.1. Dataset generation

Two tailplanes labeled A and B from a PA-28 ‘Arrow’ aircraft were used to create the dataset. The tailplane had elevators and wing tips removed and each was cut in half to create a port and a starboard side. Tailplanes A and B have more or less the same geometry (although B was cut asymmetrically). Had the tailplanes been cut symmetrically, then they would be classed as a homogeneous population. A population is considered as homogeneous if the structures are nominally identical (Gosliga et al., Reference Gosliga, Hester, Worden and Bunce2022) and there is structural equivalence (Gosliga et al., Reference Gosliga, Gardner, Bull, Dervilis and Worden2021). Thus, tailplanes from A and B are not homogeneous as they were cut asymmetrically and are considered here as weakly homogeneous.

The task analyzed as part of this case study is to determine which data are from which tailplane (A or B) using the frequency response function (FRF). This classification is viewed as similar to reviewing a structure in normal condition (A tailplane) and some other condition such as operating, environmental or damage conditions (B tailplane), or vice versa. As there are port and starboard parts of the tailplane, there are two classification tasks.

During the experiments, Gaussian white noise excitation was applied to each of the tailplanes over a frequency bandwidth of 1 kHz with a resolution of 0.3125 Hz; this resulted in 3200 points in the frequency spectrum. The useful range of the FRF was deemed to be between 33.75 Hz and 217.1875 Hz (points 107 to 695). Frequencies lower than this were considered to be influenced too strongly by rigid body movements and higher frequencies were highly influenced by noise. The port and starboard sections each had 180 measurement response points, which were averaged and then normalized across each tailplane, to generate the normalized FRF. The resulting FRF can be seen in the top row of Figure 3.

Figure 3. Port tailplane measurements (Task 1, left, blue for A and green for B) and starboard measurements (Task 2, right, sky blue for A and purple for B), based on the average response of the 180 measurement points. Upper: Mean values only. Lower: Mean values and all 750 sample points per structure shown as small translucent points.

In the frequency range, there are 588 measured frequencies, each corresponding to one feature. Only one data point exists for each frequency (feature) corresponding to the summed average across all of the 180 response locations. Jain and Waller (Reference Jain and Waller1978) proposed that for uncorrelated features, the optimal sample size is $ N=M+1 $ (where $ N $ is the total number of samples and $ M $ is the total number of features), whereas for highly correlated features the optimal sample size is $ N={M}^2 $ . This relationship is further backed up by research of Hua et al. (Reference Hua, Xiong, Lowey, Suh and Dougherty2005). To increase the dataset from 1 to $ N $ , a demo dataset was generated using Monte-Carlo sampling, with the mean from the experiments and the variance estimated using the coherence function as in Worden (Reference Worden1998),

(4.1) $$ \sigma \left({H}_p\left(\omega \right)\right)=\frac{\sqrt{1-{\gamma}_p^2\left(\omega \right)}}{\mid {\gamma}_p\left(\omega \right)\mid \sqrt{2n}}{H}_p\left(\omega \right) $$

where $ {H}_p\left(\omega \right) $ is the measured frequency response at a given frequency, $ {\gamma}_p\left(\omega \right) $ is the standard coherence function and $ n=6 $ is the number of values used to compute the FRF (that is, the number of average values used to generate the resulting measured value).

Random sampling (10,000 samples used here) can be used to generate a dataset if the results are assumed to have a Gaussian distribution. As sparsity is encouraged in the algorithm, there is the implicit assumption that correlation between features exists, for the task of distinguishing A from B. The number of meaningful features is unknown at this stage in the statistical model development.

It is hoped that there will be high correlation between the features, so that a sparse solution can be found. A high correlation between tasks would suggest that the number of samples required would be 354,744 ( $ {588}^2 $ , or 172,872 per class). However, such a high number of samples drastically increases the processing time required by the algorithm. Instead, a sample size of 1500 per task (that is 750 in each class) was generated for training and validation.

To increase the training and validation datasets, fivefold cross-validation was implemented. Fivefold cross-validation is generally accepted as the lower end of the permissible values for k-fold cross-validation (Priddy and Keller, Reference Priddy and Keller2005; Mirkin, Reference Mirkin2011). As opposed to separate training and validation sets, fivefold cross-validation reduces bias within the model, hence enabling optimal values for the hyperparameters to be selected. A further 500 samples per task (250 per class) were generated for testing purposes. The hyperparameters that need to be selected are the step size, $ \varepsilon $ , and tolerance, $ \xi $ . All three models were trained with the same hyperparameters, $ \varepsilon \in \left\{\mathrm{1,0.3,0.1,0.03}\right\} $ , and $ \xi \in \left\{\mathrm{0.1,0.01,0.001}\right\} $ , and all combinations of $ \varepsilon $ and $ \xi $ were tried subject to $ \varepsilon >\xi $ .

Three models were produced: an independent learner for Task 1, an independent learner for Task 2 and a multitask learner for Tasks 1 and 2. To aid the review of the relative performance of the three models, the FRF is split into 2 windows. The first window contains frequencies less than 125 Hz, it does not contain a lot of information about the structures; however, the second window contains a lot more information about the structures.

4.2. Trained model comparison

Following cross-validation, the resulting values are: $ \varepsilon =0.3 $ and $ \xi =0.01 $ for both the independent and the multitask models. Figure 4 shows the FRF overlaid onto vertical lines, which represent the activated nonzero weights, and therefore the activated frequencies of the solution.

Figure 4. FRF and activated weights for test data for both Task 1 (upper, Class 1 blue and Class 2 green) and Task 2 (lower, Class 1 sky blue and Class 2 purple) for (a) LASSO and (b) Joint Feature Selection with LASSO, $ \varepsilon =0.3 $ and $ \xi =0.01 $ . The activated weights are shown as vertical lines and the color of the vertical line represents the value of the weight. The dashed black vertical line represents the boundary for the two windows.

The independent learners (Figure 4(a)) both have activated frequencies at the lower end of the frequencies analyzed. The large number of activated weights at this lower end is not informative of the differences in the structure, but rather of the differences in the rigid body movements created because of the experimental setup. For Window 2 (frequencies $ > $ 125 Hz), there appear to be similarities in the activated weights between Task 1 and Task 2. This is to be anticipated, as the samples are from the same aircraft and the motivation for using a multitask learner on this dataset! The benefit of the MTL will become evident in the next section when the model is applied to an unseen task.

Figure 4(a) shows that near-perfect F1 scores are obtained for both tasks and both windows. However, the Gini Index for Window 1 across both tasks is 10% points lower than the Gini Index for Window 2. For Task 1, it could be argued that the feature set from either of the windows could be used; Window 1 has a perfect F1 score; however, there is a marked improvement in Gini Index in Window 2, which is arguably worth the small reduction in F1 score. The selection of the window for Task 2 would be Window 2 as it has a perfect F1 score and a high Gini Index.

For Figure 4(b), the F1 scores are lower than the independent learner equivalent. For Window 1, the frequencies that have been activated are at the higher end of the frequency range, and there is only one frequency activated less than 80 Hz. The absence of low frequencies being activated indicates that the experimental setup (which would be different across all four samples), is no longer useful to differentiate between the two classes. Window 2 outperforms Window 1 for both tasks, with higher F1 scores but a lower Gini Index.

4.3. Transfer of results

To determine the success of the models in finding general features, rather than experiment-specific ones, it is useful to implement the weight matrix (features selected) from the two independent learners and the multitask learner on the two existing tasks and on a third, unseen task. This can be viewed as a form of transfer learning.

In the tailplane dataset, there is a third tailplane, C, which is taken from a PA-28 ‘Cherokee’ aircraft. The structure set is no longer weakly homogeneous, rather it is heterogeneous, as C is a different variant of aircraft. The starboard side of the tailplane (C2) was damaged and the port side of the tailplane (C1) was not damaged. To formulate a task that has similarities with the original task set, a useful classification is to differentiate from the port side of tailplane C and tailplane B. Figure 5 shows the FRF for the two original tasks (upper and middle) and the third task (lower), which has the FRF data from C1.

Figure 5. FRF for the three different tasks showing plus and minus one standard deviation (shaded band). Upper: A1 vs B1, Middle: A2 vs B2, and Lower: C1 vs B1.

There are three tasks and there are also three potential weight matrices plus the benchmark JDA; this results in 12 different models in which the F1 score can be analyzed, Figure 6 shows the results for Window 2. Figure 6 shows that the multitask learner is successful in the classification of the third task, to the same accuracy as the two tasks that it was trained on. The independent learner weight matrix for ‘A1 vs B1’ (Task 1) scores perfectly on the task it was trained for, however, there is a drop in the F1 score (to below the performance of the multitask learner) when applied to the unseen third task of ‘C1 vs B1’. This is because the independent model learned experiment-specific features, as opposed to features relating to the physics of the structure. As the features were less structurally meaningful, the model fails to generalize to the third unseen task.

Figure 6. F1 results for three different weight matrices and JDA (y-axis) applied to three different tasks (x-axis) for Window 2.

For Task 1 and Task 2, there appears to be an issue with labeling, such that the classification of Task 1 is perfect with the weight matrix trained on Task 1; however, when the weight matrix for Task 1 is applied to Task 2 there is almost a perfect misclassification. When looking at the FRF and weight matrices for comparison (Figure 4), it becomes apparent that, although the same data are used in all models, the features selected for Task 2 are at frequencies that provide the opposite classification for Task 1. This is particularly apparent at around 150 Hz in Figure 4. While JDA outperforms MTL for the tasks that it was trained on (that is, Task 1 and Task 2) for the third unseen task, the JDA features result in a decline in kNN performance, demonstrating that the transformation learned by JDA is less general.

In applying the matrices from the three models to a third, unseen task, it is shown that good generalization has occurred in the case of the multitask learner. The multitask learner did not see a reduction in the F1 score when applied to the third task, which demonstrates that the selected features are representative of the physics of the structure as opposed to differences in experimental setup. Positive transfer, to a previously unobserved experimental setup, demonstrates the advantage of more general feature selection by using MTL in SHM.

5.1. The dataset

To further demonstrate how joint feature selection can aid interpretability and general features, a subset of the GNAT dataset used by Manson et al. (Reference Manson, Worden and Allman2003b) will be tested. Ground-vibration tests were conducted on the wing by applying white Gaussian excitation via an electrodynamic shaker. Figure 7 shows a schematic of the wing; it contained nine panels, which were removed and replaced in a series of experiments to mimic damage followed by maintenance, as described in Manson et al. (Reference Manson, Worden and Allman2003b).

Figure 7. GNAT plane schematic recreated from Gardner et al. (Reference Gardner, Bull, Dervilis and Worden2021).

Each panel has an associated transmissibility, that is, the ratio of the response transducer spectrum with the reference transducer spectrum. The transmissibilities were measured and converted into magnitudes (Manson et al., Reference Manson, Worden and Allman2003b). The lower frequencies were deemed to be insensitive to damage (Manson et al., Reference Manson, Worden and Allman2003a) and hence the frequency range of the spectrum is 1024 Hz to 2048 Hz, with 1024 spectral lines.

There were 25 iterations of removing and replacing panels, each iteration containing 100 samples. For this work, rather than readings from all 9 transmissibilities, only transmissibilities and damage associated with panel 1 (A1) will be reviewed. There will be 1024 features corresponding to the frequency transmissibility measurements for A1. A lot of samples relate to damage occurring at different panels, and hence the dataset will be reduced to the following classes:

• • Normal 1: Prerepair normal condition – The initial normal condition with all plates in place ( $ \boldsymbol{X}\in {\mathrm{\mathbb{R}}}^{100\times 1024} $ )

• • Damage 1: Prerepair Panel 1 removed – Plate P1 removed ( $ \boldsymbol{X}\in {\mathrm{\mathbb{R}}}^{100\times 1024} $ )

• • Normal 2: Postrepair normal condition – The normal condition after plate P1, plate P2 and plate P3 have been sequentially removed and replaced. ( $ \boldsymbol{X}\in {\mathrm{\mathbb{R}}}^{100\times 1024} $ )

• • Damage 2: Postrepair Panel 1 removed – plate P1 removed again after plate P1, plate P2 and plate P3 have been sequentially removed and replaced. ( $ \boldsymbol{X}\in {\mathrm{\mathbb{R}}}^{100\times 1024} $ )

• • Normal 3: Post-2 ^nd-repair normal condition – The normal condition after plate P1, plate P2 and plate P3 have been sequentially removed and replaced for a second time. ( $ \boldsymbol{X}\in {\mathrm{\mathbb{R}}}^{100\times 1024} $ )

Initially, the data will be split into two tasks: Prerepair and Postrepair (omitting the 2 ^nd postrepair class). Each task will have a normal condition class and a damage class. Figure 8 shows the two tasks (the data undergo a log base-10 transformation). This transformation is useful for FRF analysis, with different magnitudes of responses. As before, for MTL, two hyperparameters need to be selected; the step size $ \varepsilon $ , and tolerance $ \xi $ .

Figure 8. The transmissibility prerepair (Task 1, upper) and postrepair (Task 2, lower) of reference transducer AR to response transducer A1 for normal condition and panel 1 removed. Prerepair, Task 1, has Class 1 as normal condition (purple) and Class 2 as Panel 1 removed (sky blue) and postrepair, Task 2 has Class 1 as normal condition (green) and Class 2 as Panel 1 removed (orange). One standard deviation of banding is shown for each class.

Three models were produced; an independent learner for prerepair, an independent learner for postrepair and a multitask learner for task pre- and postrepair. All three models were trained with the same hyperparameters, the values of the step size, $ \varepsilon $ , were selected to be approximately three times smaller than the previous value, $ \varepsilon \in \left\{\mathrm{10,3,1,0.3,0.1}\right\} $ , and the tolerance was trialed for two different orders of magnitude, $ \xi \in \left\{\mathrm{1,0.1}\right\} $ . The value of $ \xi $ has less of an impact on the final results than $ \varepsilon $ , hence less sensitivity analysis for $ \xi $ . All combinations of $ \varepsilon $ and $ \xi $ were tried subject to $ \varepsilon >\xi $ .

5.2. Trained model comparison

Following tuning of the hyperparameters, the results are: $ \varepsilon =0.3 $ and $ \xi =0.1 $ for both the independent and the multitask models. Figure 9 shows the transmissibilities overlaid onto vertical lines, which represent the activated weights and therefore the activated frequencies of the solution. The prerepair independent learner (Figure 9(a), top), has the lowest F1 score of the models and also the highest Gini Index. The majority of the weights activated are within a band at around spectral line 1500 and there is another band of activated frequencies at around spectral line 1280. For the independent learner for postrepair (Figure 9(a), bottom) there are more spectral lines that have been selected and there appears to be less grouping of the activated spectral lines.

Figure 9. Transmissibilities and activated weights for test data for both Task 1 (upper, normal condition purple and panel 1 removed blue) and Task 2 (lower, normal condition green and panel 1 removed orange) for (a) LASSO and (b) Joint Feature Selection with LASSO, $ \varepsilon =0.1 $ and $ \xi =0.1 $ . The activated weights are shown as vertical lines and the color of the vertical line represents the value of the weight.

Figure 9(b) shows that the MTL has improved F1 scores for the prerepair task over the independent equivalent. However, as with the previous example dataset, the Gini Index is lower for the MTL than either of the two independent learners.

5.3. Domain adaptation as a result of the models

To understand how the multitask model aids the discovery of a general feature space, it is useful to visualize the PCA subspaces of the full features. In this paper domain adaptation refers to multiplying the original data by the weights that have been calculated in the corresponding models (either weights from the single-task learning models or weights from MTL). Figure 10 shows the four classes, following PCA transformation for three different scenarios.

Figure 10. PCA of the four classes: prerepair normal condition (purple), prerepair Panel 1 removed (blue), postrepair normal condition (green) and postrepair Panel 1 removed (red). (a) PCA of normalized data. (b) PCA after data is multiplied by the weights and bias for the two independent learners (on the corresponding task), concatenated, and then normalized. (c) PCA after data has been multiplied by the weights and bias of the MTL, for both tasks, and then normalized. (d) Shared domain created with JDA.

Figure 10(a) shows the normalized data without any domain adaptation. There are four distinct clusters that are present. The clusters that are closest to each other are the Postrepair Normal Condition and the Postrepair Panel 1 Removed. The classification boundary between normal condition and panel removal is not linear.

The impact of the independent learning models on the clusters is shown in Figure 10(b). Alike clusters are closer together; note that the scale on (b) is different to the scale on (a). In addition, the clusters appear evenly spaced from one another, with each class having its own distinct cluster, as in (a).

Figure 10(c) shows how MTL has adapted the domains such that the two classes classified as Normal Condition overlap and the two classes classified as Panel 1 Removed overlap as well. This implies that the similarities between the two Normal Condition classes have been identified by the MTL, as have the similarities between the two Panel 1 Removed classes. In effect, the datasets and tasks have been harmonized. A linear classification boundary can be readily applied to differentiate between the normal condition and panel removal, unlike the original data and single-task learning. In addition, the F1 score of the MTL shows that the model can accurately classify between the two different classes; this is a promising result, as MTL intends to learn similarities between the tasks for each class and then uses these similarities to select features that can accurately perform the binary classification. Figure 10(d) is included to show how MTL achieves similar domain adaptation to JDA, while also learning the classification task at the same time, and selecting from the original feature set, rather than transforming them.