Applications of deep learning to physical simulations such as Computational Fluid Dynamics have recently experienced a surge in interest, and their viability has been demonstrated in different domains. However, due to the highly complex, turbulent, and three-dimensional flows, they have not yet been proven usable for turbomachinery applications. Multistage axial compressors for gas turbine applications represent a remarkably challenging case, due to the high-dimensionality of the regression of the flow field from geometrical and operational variables. This paper demonstrates the development and application of a deep learning framework for predictions of the flow field and aerodynamic performance of multistage axial compressors. A physics-based dimensionality reduction approach unlocks the potential for flow-field predictions, as it re-formulates the regression problem from an unstructured to a structured one, as well as reducing the number of degrees of freedom. Compared to traditional “black-box” surrogate models, it provides explainability to the predictions of the overall performance by identifying the corresponding aerodynamic drivers. The model is applied to manufacturing and build variations, as the associated performance scatter is known to have a significant impact on $ \mathrm{C}{\mathrm{O}}_2 $ emissions, which poses a challenge of great industrial and environmental relevance. The proposed architecture is proven to achieve an accuracy comparable to that of the CFD benchmark, in real-time, for an industrially relevant application. The deployed model is readily integrated within the manufacturing and build process of gas turbines, thus providing the opportunity to analytically assess the impact on performance with actionable and explainable data.

Recent efforts in interpreting convolutional neural networks (CNNs) focus on translating the activation of CNN filters into a stratified Answer Set Program (ASP) rule-sets. The CNN filters are known to capture high-level image concepts, thus the predicates in the rule-set are mapped to the concept that their corresponding filter represents. Hence, the rule-set exemplifies the decision-making process of the CNN w.r.t the concepts that it learns for any image classification task. These rule-sets help understand the biases in CNNs, although correcting the biases remains a challenge. We introduce a neurosymbolic framework called NeSyBiCor for bias correction in a trained CNN. Given symbolic concepts, as ASP constraints, that the CNN is biased toward, we convert the concepts to their corresponding vector representations. Then, the CNN is retrained using our novel semantic similarity loss that pushes the filters away from (or toward) learning the desired/undesired concepts. The final ASP rule-set obtained after retraining, satisfies the constraints to a high degree, thus showing the revision in the knowledge of the CNN. We demonstrate that our NeSyBiCor framework successfully corrects the biases of CNNs trained with subsets of classes from the Places dataset while sacrificing minimal accuracy and improving interpretability.

The identification of fishing vessel operations holds significant importance in addressing fishing industry issues, such as overfishing and illegal, unreported and unregulated fishing (IUUF). Many countries utilise data from vessel monitoring systems (VMSs) or automatic identification systems (AISs) to monitor fishing activities. These data include vessel trajectories, headings and speeds, among others. We aimed to analyse the fishing behaviours of three types of fishing gear used by vessels (trawl, purse seine and gill net) and identify the types of gear employed by the vessels. Therefore, a 1D CNN-LSTM fishing vessel operational behaviour prediction model was proposed by combining a one-dimensional convolutional (1D CNN) neural network and a long short-term memory (LSTM) neural network. The model utilises 1D CNN to extract local features from fishing vessel trajectories and employs LSTM to capture the time series information in the data, eventually classifying fishing gears. The results show that the proposed model achieves a classification accuracy of 92% in categorising fishing vessel operational trajectories. This study significantly contributes to preventing IUUF, curtailing overfishing, and enhancing fisheries management strategies.

Language identification (LID) is a crucial preliminary process in the field of Automatic Speech Recognition (ASR) that involves the identification of a spoken language from audio samples. Contemporary systems that can process speech in multiple languages require users to expressly designate one or more languages prior to utilisation. The LID task assumes a significant role in scenarios where ASR systems are unable to comprehend the spoken language in multilingual settings, leading to unsuccessful speech recognition outcomes. The present study introduces convolutional recurrent neural network (CRNN)-based LID, designed to operate on the mel-frequency cepstral coefficient (MFCC) characteristics of audio samples. Furthermore, we replicate certain state-of-the-art methodologies, specifically the convolutional neural network (CNN) and Attention-based convolutional recurrent neural network (CRNN with Attention), and conduct a comparative analysis with our CRNN-based approach. We conducted comprehensive evaluations on thirteen distinct Indian languages, and our model resulted in over 98 per cent classification accuracy. The LID model exhibits high-performance levels ranging from 97 per cent to 100 per cent for languages that are linguistically similar. The proposed LID model exhibits a high degree of extensibility to additional languages and demonstrates a strong resistance to noise, achieving 91.2 per cent accuracy in a noisy setting when applied to a European Language (EU) dataset.

Deep learning (DL) has been widely used in bearing fault diagnosis. In particular, convolutional neural networks (CNNs) improve diagnosis accuracy by extracting excellent fault features. However, CNN lacks an explicit learning mechanism to distinguish between different fault characteristics in the input signal to the diagnosis results. This article presents a new end-to-end depth framework called multi-head self-attention convolution neural network (MSA-CNN) for bearing fault diagnosis. Firstly, we adopt a data pre-processing method that directly converts one-dimensional (1D) original signals into two-dimensional (2D) grayscale images, which is simple to implement and preserves the complete information of the original signal. Secondly, multi-head self-attention (MSA) is first constructed to aggregate the global information and adaptively assign weights to the input signal's features. Thirdly, the CNN with small-scale kernels extracted detailed local features. Finally, the learned high-level representations are fed into the full connect (FC) layer for fault diagnosis. The performance of the MSA-CNN is validated on different datasets. The results show that the proposed MSA-CNN can significantly improve fault diagnosis accuracy compared with the other state-of-the-art methods and has excellent noise immunity performance.

We present a data-driven emulator, a stochastic weather generator (SWG), suitable for estimating probabilities of prolonged heat waves in France and Scandinavia. This emulator is based on the method of analogs of circulation to which we add temperature and soil moisture as predictor fields. We train the emulator on an intermediate complexity climate model run and show that it is capable of predicting conditional probabilities (forecasting) of heat waves out of sample. Special attention is payed that this prediction is evaluated using a proper score appropriate for rare events. To accelerate the computation of analogs, dimensionality reduction techniques are applied and the performance is evaluated. The probabilistic prediction achieved with SWG is compared with the one achieved with a convolutional neural network (CNN). With the availability of hundreds of years of training data, CNNs perform better at the task of probabilistic prediction. In addition, we show that the SWG emulator trained on 80 years of data is capable of estimating extreme return times of order of thousands of years for heat waves longer than several days more precisely than the fit based on generalized extreme value distribution. Finally, the quality of its synthetic extreme teleconnection patterns obtained with SWG is studied. We showcase two examples of such synthetic teleconnection patterns for heat waves in France and Scandinavia that compare favorably to the very long climate model control run.

The seismic response of a wide variety of structures, from small but irreplaceable museum exhibits to large bridge systems, is characterized by rocking. In addition, rocking motion is increasingly being used as a seismic protective strategy to limit the amount of seismic actions (moments) developed at the base of structures. However, rocking is a highly nonlinear phenomenon governed by non-smooth dynamic phases that make its prediction difficult. This study presents an alternative approach to rocking estimation based on a physics-informed convolutional neural network (PICNN). By training a group of PICNNs using limited datasets obtained from numerical simulations and encoding the known physics into the PICNNs, important predictive benefits are obtained relieving difficulties associated with over-fitting and minimizing the requirement for a large training database. Two models are created depending on the validation of the deep PICNN: the first model assumes that state variables including rotations and angular velocities are available, while the second model is useful when only acceleration measurements are known. The analysis is initiated by implementing K-means clustering. This is followed by a detailed statistical assessment and a comparative analysis of the response-histories of a rocking block. It is observed that the deep PICNN is capable of effectively estimating the seismic rocking response history when the rigid block does not overturn.

Very metal-poor (VMP, [Fe/H]<-2.0) stars serve as invaluable repositories of insights into the nature and evolution of the first-generation stars formed in the early galaxy. The upcoming China Space Station Telescope (CSST) will provide us with a large amount of spectral data that may contain plenty of VMP stars, and thus it is crucial to determine the stellar atmospheric parameters ($T_{\textrm{eff}}$, $\log$ g, and [Fe/H]) for low-resolution spectra similar to the CSST spectra ($R\sim 200$). This study introduces a novel two-dimensional Convolutional Neural Network (CNN) model, comprised of three convolutional layers and two fully connected layers. The model’s proficiency is assessed in estimating stellar parameters, particularly metallicity, from low-resolution spectra ($R \sim 200$), with a specific focus on enhancing the search for VMP stars within the CSST spectral data. We mainly use 10 008 spectra of VMP stars from LAMOST DR3, and 16 638 spectra of non-VMP stars ([Fe/H]>-2.0) from LAMOST DR8 for the experiments and apply random forest and support vector machine methods to make comparisons. The resolution of all spectra is reduced to $R\sim200$ to match the resolution of the CSST, followed by pre-processing and transformation into two-dimensional spectra for input into the CNN model. The validation and practicality of this model are also tested on the MARCS synthetic spectra. The results show that using the CNN model constructed in this paper, we obtain Mean Absolute Error (MAE) values of 99.40 K for $T_{\textrm{eff}}$, 0.22 dex for $\log$ g, 0.14 dex for [Fe/H], and 0.26 dex for [C/Fe] on the test set. Besides, the CNN model can efficiently identify VMP stars with a precision rate of 94.77%, a recall rate of 93.73%, and an accuracy of 95.70%. This paper powerfully demonstrates the effectiveness of the proposed CNN model in estimating stellar parameters for low-resolution spectra ($R\sim200$) and recognizing VMP stars that are of interest for stellar population and galactic evolution work.

Human activity recognition (HAR) is an emerging challenge among researchers. HAR has many possible uses in various fields, including healthcare, sports, and security. Furthermore, there are only a few publicly accessible datasets for classifying and recognizing physical activity in the literature, and these datasets comprise fewer activities. We created and compared our dataset with available datasets, that is, NTU-RGBD, UP-FALL, UR-Fall, WISDM, and UCI HAR. The proposed dataset consists of seven activities: eating, exercise, handshake, situps, vomiting, headache, and walking. The activities were collected from 20 people between the ages of 25 and 40 years using Kinect V2 sensor at 30 FPS. For classification, we use deep learning architectures based on convolutional neural network (CNN) and long short-term memory (LSTM). Additionally, we developed a novel hybrid deep learning model by combining a CNN, a bidirectional LSTM unit, and a fully connected layer for activity identification. The suggested model builds unique guided features using the preprocessed skeleton coordinates and their distinctive geometrical and kinematic aspects. Results from the experiment are contrasted with the performance of stand-alone CNNs, LSTMs, and ConvLSTM. The proposed model’s accuracy of 99.5% surpasses that of CNN, LSTM, and ConvLSTM, which have accuracy rates of 95.76%, 97%, and 98.89%, respectively. Our proposed technique is invariant of stance, speed, individual, clothes, etc. The proposed dataset sample is accessible to the general public.

Radar-based hand gesture recognition is a potential noncontact human–machine interaction technique. To enhance the recognition performance of hand gesture, a multidomain fusion-based recognition method using frequency-modulated continuous wave radar is proposed in this article. The received raw echo data of gestures is preprocessed to obtain the range–time matrix, Doppler–time matrix, and range–Doppler–frame tensor. The obtained three-domain radar data corresponding to each gesture are input into the three-channel convolutional neural network for feature extraction. In particular, the extracted features from three-domain data are fused with learnable weight matrices to obtain the final gesture classification results. The experimental results have shown that the classification accuracy of the proposed multidomain fusion network based on learning weight matrix-based fusion is 98.45%, which improves the classification performance compared with the classic average-based fusion and concatenation fusion.

Weeds contribute to biodiversity and a wide range of ecosystem functions. It is crucial to map different weed species and analyze their physiological activities. Remote sensing techniques for plant identification, especially hyperspectral imaging, are being developed using spectral response patterns to vegetation for detection and species identification. A library of hyperspectral images of 40 urban weed species in northeast China was established in this study. A terrestrial hyperspectral camera was used to acquire 435 hyperspectral images. The hyperspectral information for each weed species was extracted and analyzed. The spectral characteristics and vegetation indices of different weeds revealed the differences between weed species in the cities of northeast China and indirectly characterized the growth and physiological activity levels of different species, but could not effectively distinguish different species. Five methods—first derivative spectrum (FDS), second derivative spectrum (SDS), standard normal variate (SNV), moving averages (MA), and Savitzky-Golay (SG) smoothing—were used to pretreat the spectral curves to maximize the retention of spectral characteristics while removing the influence of noise. We investigated the application of a convolutional neural network (CNN) with terrestrial hyperspectral remote sensing to identify urban weeds in northeast China. A CNN classification model was established to distinguish weeds from the hyperspectral images and demonstrated a test accuracy of 95.32% to 98.15%. The accuracy of the original spectrum was 97.45%; SNV had the best accuracy (98.15%) and SG was the least accurate (95.32%). This provides a baseline for understanding the hyperspectral characteristics of urban weed species and monitoring their growth. It also contributes to the development of a hyperspectral imaging database with global applicability.

The goal of few-shot semantic segmentation is to learn a segmentation model that can segment novel classes in queries when only a few annotated support examples are available. Due to large intra-class variations, the building of accurate semantic correlation remains a challenging job. Current methods typically use 4D kernels to learn the semantic correlation of feature maps. However, they still face the challenge of reducing the consumption of computation and memory while keeping the availability of correlations mined by their methods. In this paper, we propose the adaptively mining correlation network (AMCNet) to alleviate the aforementioned issues. The key points of AMCNet are the proposed adaptive separable 4D kernel and the learnable pyramid correlation module, which form the basic block for correlation encoder and provide a learnable concatenation operation over pyramid correlation tensors, respectively. Experiments on the PASCAL VOC 2012 dataset show that our AMCNet surpasses the state-of-the-art method by $0.7\%$ and $2.2\%$ on 1-shot and 5-shot segmentation scenarios, respectively.

In laparoscopic surgery, surgeons should develop several manual laparoscopic skills before carrying out real operative procedures using a low-cost box trainer. The Fundamentals of Laparoscopic Surgery (FLS) program was developed as a program to assess fundamental knowledge and surgical skills, required for basic laparoscopic surgery. The peg transfer task is a hands-on exam in the FLS program that assists a trainee to understand the relative minimum amount of grasping force necessary to move the pegs from one place to another place without dropping them. In this paper, an autonomous, sequential assessment algorithm based on deep learning, a multi-object detection method, and, several sequential If-Then conditional statements have been developed to monitor each step of a surgeon’s performance. Images from three different cameras are used to assess whether the surgeon executes the peg transfer task correctly and to display a notification on any errors on the monitor immediately. This algorithm improves the performance of a laparoscopic box-trainer system using top, side, and front cameras and removes the need for any human monitoring during a peg transfer task. The developed algorithm can detect each object and its status during a peg transfer task and notifies the resident about the correct or failed outcome. In addition, this system can correctly determine the peg transfer execution time, and the move, carry, and dropped states for each object by the top, side, and front-mounted cameras. Based on the experimental results, the proposed surgical skill assessment system can identify each object at a high score of fidelity, and the train-validation total loss for the single-shot detector (SSD) ResNet50 v1 was about 0.05. Also, the mean average precision (mAP) and Intersection over Union (IoU) of this detection system were 0.741, and 0.75, respectively. This project is a collaborative research effort between the Department of Electrical and Computer Engineering and the Department of Surgery, at Western Michigan University.

Twelve climate models and observations are used to attribute the global mean surface temperature (GMST) changes from 1900 to 2014 to external climate forcings. The external forcings are decomposed into the effects of the well-mixed greenhouse gas concentration variation, the effects of anthropogenic aerosol concentration changes, and the effects of natural forcings. First, a convolutional neural network (CNN) is trained to estimate the simulated historical GMST from single-forcing experiments using outputs from the multi-model ensemble. We then use this CNN to solve the attribution problem using an original variational inversion approach. The variational inversion is first validated using historical climate simulations as pseudo-observations. Then we perform an inversion from observations. This provides a distribution of the GMST resulting from the three forcings. For 2014, inversions estimate that the greenhouse gases changes are responsible for a GMST anomaly within [0.8 $ {}^{\circ } $ C,1.9 $ {}^{\circ } $ C], while anthropogenic aerosols and natural forcings anomalies are within [−0.7 $ {}^{\circ } $ C,−0.1 $ {}^{\circ } $ C] and [−0.1 $ {}^{\circ } $ C,0.3 $ {}^{\circ } $ C], respectively. The method designed here can be adapted and extended to attribute the changes of other variables or to focus on the regional scale.

Robotic assistive or rehabilitative devices are promising aids for people with neurological disorders as they help regain normative functions for both upper and lower limbs. However, it remains challenging to accurately estimate human intent or residual efforts non-invasively when using these robotic devices. In this article, we propose a deep learning approach that uses a brightness mode, that is, B-mode, of ultrasound (US) imaging from skeletal muscles to predict the ankle joint net plantarflexion moment while walking. The designed structure of customized deep convolutional neural networks (CNNs) guarantees the convergence and robustness of the deep learning approach. We investigated the influence of the US imaging’s region of interest (ROI) on the net plantarflexion moment prediction performance. We also compared the CNN-based moment prediction performance utilizing B-mode US and sEMG spectrum imaging with the same ROI size. Experimental results from eight young participants walking on a treadmill at multiple speeds verified an improved accuracy by using the proposed US imaging + deep learning approach for net joint moment prediction. With the same CNN structure, compared to the prediction performance by using sEMG spectrum imaging, US imaging significantly reduced the normalized prediction root mean square error by 37.55% ($ p $ < .001) and increased the prediction coefficient of determination by 20.13% ($ p $ < .001). The findings show that the US imaging + deep learning approach personalizes the assessment of human joint voluntary effort, which can be incorporated with assistive or rehabilitative devices to improve clinical performance based on the assist-as-needed control strategy.

Reproducibility of a deep-learning fully convolutional neural network is evaluated by training several times the same network on identical conditions (database, hyperparameters, and hardware) with nondeterministic graphics processing unit operations. The network is trained to model three typical time–space-evolving physical systems in two dimensions: heat, Burgers’, and wave equations. The behavior of the networks is evaluated on both recursive and nonrecursive tasks. Significant changes in models’ properties (weights and feature fields) are observed. When tested on various benchmarks, these models systematically return estimations with a high level of deviation, especially for the recurrent analysis which strongly amplifies variability due to the nondeterminism. Trainings performed with double floating-point precision provide slightly better estimations and a significant reduction of the variability of both the network parameters and its testing error range.

The capabilities of teleoperated robots can be enhanced with the ability to recognise and reproduce human-like behaviour. The proposed framework presents motion recognition for a Kinect-based NAO teleoperation. It allows the NAO robot to recognise the human motions and act as a human motion imitator. A stable whole-body imitation is still a challenging issue because of the difficulty in dynamic balancing of centre of mass (CoM). In this paper, a novel adaptive balancing technique for NAO (ABTN) is proposed to control the whole body in single as well as double supporting phases. It targets dynamic balancing of the humanoid robot by solving forward kinematics and applying a weighted average of mass with the CoMs of individual links with respect to the previous joint frames, which provides us with the dynamic CoM of the whole body. Our novel approach uses this dynamic CoM and calculates joint angles using proposed pitch and roll control algorithm to keep the dynamic CoM inside the stable region. Additionally, the NAO robot is capable of recognising human motions using the proposed 7-layer one-dimensional convolutional neural network (1D-CNN). To solve the problem of variable length of time sequences, Zero padding is introduced with 1D-CNN. It attains a recognition accuracy of 95% as compared to the hidden Markov model and neural network. The experimental results demonstrate that the developed teleoperation framework is robust and serves as potential support for the development and application of teleoperated robots.