Climate models are primary tools for investigating processes in the climate system, projecting future changes, and informing decision makers. The latest generation of models provides increasingly complex and realistic representations of the real climate system, while there is also growing awareness that not all models produce equally plausible or independent simulations. Therefore, many recent studies have investigated how models differ from observed climate and how model dependence affects model output similarity, typically drawing on climatological averages over several decades. Here, we show that temperature maps of individual days drawn from datasets never used in training can be robustly identified as “model” or “observation” using the CMIP6 model archive and four observational products. An important exception is a prototype storm-resolving simulation from ICON-Sapphire which cannot be unambiguously assigned to either category. These results highlight that persistent differences between simulated and observed climate emerge at short timescales already, but very high-resolution modeling efforts may be able to overcome some of these shortcomings. Moreover, temporally out-of-sample test days can be assigned their dataset name with up to 83% accuracy. Misclassifications occur mostly between models developed at the same institution, suggesting that effects of shared code, previously documented only for climatological timescales, already emerge at the level of individual days. Our results thus demonstrate that the use of machine learning classifiers, once trained, can overcome the need for several decades of data to evaluate a given model. This opens up new avenues to test model performance and independence on much shorter timescales.

Studies agree on a significant global mean sea level rise in the 20th century and its recent 21st century acceleration in the satellite record. At regional scale, the evolution of sea level probability distributions is often assumed to be dominated by changes in the mean. However, a quantification of changes in distributional shapes in a changing climate is currently missing. To this end, we propose a novel framework quantifying significant changes in probability distributions from time series data. The framework first quantifies linear trends in quantiles through quantile regression. Quantile slopes are then projected onto a set of four orthogonal polynomials quantifying how such changes can be explained by independent shifts in the first four statistical moments. The framework proposed is theoretically founded, general, and can be applied to any climate observable with close-to-linear changes in distributions. We focus on observations and a coupled climate model (GFDL-CM4). In the historical period, trends in coastal daily sea level have been driven mainly by changes in the mean and can therefore be explained by a shift of the distribution with no change in shape. In the modeled world, robust changes in higher order moments emerge with increasing $ {\mathrm{CO}}_2 $ concentration. Such changes are driven in part by ocean circulation alone and get amplified by sea level pressure fluctuations, with possible consequences for sea level extremes attribution studies.

Monitoring river water levels is essential for the study of floods and mitigating their risks. River gauges are a well-established method for river water-level monitoring but many flood-prone areas are ungauged and must be studied through gauges located several kilometers away. Taking advantage of river cameras to observe river water levels is an accessible and flexible solution but it requires automation. However, current automated methods are only able to extract uncalibrated river water-level indexes from the images, meaning that these indexes are relative to the field of view of the camera, which limits their application. With this work, we propose a new approach to automatically estimate calibrated river water-level indexes from images of rivers. This approach is based on the creation of a new dataset of 32,715 images coming from 95 river cameras in the UK and Ireland, cross-referenced with gauge data (river water-level information), which allowed us to train convolutional neural networks. These networks are able to accurately produce two types of calibrated river water-level indexes from images: one for continuous river water-level monitoring, and the other for flood event detection. This work is an important step toward the automated use of cameras for flood monitoring.

The growing need for agricultural products and the challenges posed by environmental and economic factors have created a demand for enhanced agricultural systems management. Machine learning has increasingly been leveraged to tackle agricultural optimization problems, and in particular, reinforcement learning (RL), a subfield of machine learning, seems a promising tool for data-driven discovery of future farm management policies. In this work, we present the development of CropGym, a Gymnasium environment, where a reinforcement learning agent can learn crop management policies using a variety of process-based crop growth models. As a use case, we report on the discovery of strategies for nitrogen application in winter wheat. An RL agent is trained to decide weekly on applying a discrete amount of nitrogen fertilizer, with the aim of achieving a balance between maximizing yield and minimizing environmental impact. Results show that close to optimal strategies are learned, competitive with standard practices set by domain experts. In addition, we evaluate, as an out-of-distribution test, whether the obtained policies are resilient against a change in climate conditions. We find that, when rainfall is sufficient, the RL agent remains close to the optimal policy. With CropGym, we aim to facilitate collaboration between the RL and agronomy communities to address the challenges of future agricultural decision-making.

Oceans will play a crucial role in our efforts to combat the growing climate emergency. Researchers have proposed several strategies to harness greener energy through oceans and use oceans as carbon sinks. However, the risks these strategies might pose to the ocean and marine ecosystem are not well understood. It is imperative that we quickly develop a range of tools to monitor ocean processes and marine ecosystems alongside the technology to deploy these solutions on a large scale into the oceans. Large arrays of inexpensive cameras placed deep underwater coupled with machine learning pipelines to automatically detect, classify, count, and estimate fish populations have the potential to continuously monitor marine ecosystems and help study the impacts of these solutions on the ocean. In this paper, we successfully demonstrate the application of YOLOv4 and YOLOv7 deep learning models to classify and detect six species of fish in a dark artificially lit underwater video dataset captured 500 m below the surface, with a mAP of 76.01% and 85.0%, respectively. We show that 2,000 images per species, for each of the six species of fish is sufficient to train a machine-learning species classification model for this low-light environment. This research is a first step toward systems to autonomously monitor fish deep underwater while causing as little disruption as possible. As such, we discuss the advances that will be needed to apply such systems on a large scale and propose several avenues of research toward this goal.

Cyclones are a severe storm system with a defined center, occurring in the tropical regions. Upon landfall, it causes massive damage to both lives and the economy. With the increase in frequency and intensity of tropical cyclones occurring over the years and growing coastal settlements, the study of cyclone landfall remains of paramount importance for disaster control and mitigation. Cyclones experience rapid changes, with various environmental factors modulating the trajectory and intensity. Thus predicting cyclone landfall demands a highly precise technique coupled with knowledge of environmental parameters. With the complexity and nonlinearity of the cyclone track data, determining parameters conducive for the landfall prediction of a cyclone remains crucial for precision and knowledge of the storm system. While numerous methods have been employed for detecting causal interactions among weather systems like Granger Causality and Transfer Entropy, each comes with its limitation and computational overhead. In this work, we investigate the where and when of a cyclone landfall by studying the influencing factors regulating the location and time of a cyclone landfall over the North Indian Ocean with mutual information (MI). We utilize dilated recurrent neural network with gated recurrent unit cells coupled with feature selection via MI criterion for predicting the cyclone landfall location and intensity between 12 and 36 training hours. The model efficacy is validated further on the landfall data of a recent devastating storm—Fani.

Complex machine learning architectures and high-dimensional gridded input data are increasingly used to develop high-performance geoscience models, but model complexity obfuscates their decision-making strategies. Understanding the learned patterns is useful for model improvement or scientific investigation, motivating research in eXplainable artificial intelligence (XAI) methods. XAI methods often struggle to produce meaningful explanations of correlated features. Gridded geospatial data tends to have extensive autocorrelation so it is difficult to obtain meaningful explanations of geoscience models. A recommendation is to group correlated features and explain those groups. This is becoming common when using XAI to explain tabular data. Here, we demonstrate that XAI algorithms are highly sensitive to the choice of how we group raster elements. We demonstrate that reliance on a single partition scheme yields misleading explanations. We propose comparing explanations from multiple grouping schemes to extract more accurate insights from XAI. We argue that each grouping scheme probes the model in a different way so that each asks a different question of the model. By analyzing where the explanations agree and disagree, we can learn information about the scale of the learned features. FogNet, a complex three-dimensional convolutional neural network for coastal fog prediction, is used as a case study for investigating the influence of feature grouping schemes on XAI. Our results demonstrate that careful consideration of how each grouping scheme probes the model is key to extracting insights and avoiding misleading interpretations.

This paper derives statistical models for predicting wintertime subseasonal temperature over the western US. The statistical models are trained on two separate datasets, namely observations and dynamical model simulations, and are based on least absolute shrinkage and selection operator (lasso). Surprisingly, statistical models trained on dynamical model simulations can predict observations better than observation-trained models. One reason for this is that simulations involve orders of magnitude more data than observational datasets.

The winter stratospheric polar vortex (SPV) exhibits considerable variability in magnitude and structure, which can result in extreme SPV events. These extremes can subsequently influence weather in the troposphere from weeks to months and thus are important sources of surface predictability. However, the predictability of the SPV extreme events is limited to 1–2 weeks in state-of-the-art prediction systems. Longer predictability timescales of SPV would strongly benefit long-range surface prediction. One potential option for extending predictability timescales is the use of machine learning (ML). However, it is often unclear which predictors and patterns are important for ML models to make a successful prediction. Here we use explainable multiple linear regressions (MLRs) and an explainable artificial neural network (ANN) framework to model SPV variations and identify one type of extreme SPV events called sudden stratospheric warmings. We employ a NN attribution method to propagate the ANN’s decision-making process backward and uncover feature importance in the predictors. The feature importance of the input is consistent with the known precursors for extreme SPV events. This consistency provides confidence that ANNs can extract reliable and physically meaningful indicators for the prediction of the SPV. In addition, our study shows a simple MLR model can predict the SPV daily variations using sequential feature selection, which provides hints for the connections between the input features and the SPV variations. Our results indicate the potential of explainable ML techniques in predicting stratospheric variability and extreme events, and in searching for potential precursors for these events on extended-range timescales.

Artificial intelligence has provided many breakthroughs in the field of computer vision. The fully convolutional networks U-Net in particular have provided very promising results in the problem of retrieving rain rates from space-borne observations, a challenge that has persisted over the past few decades. The rain intensity is estimated from the measurement of the brightness temperatures on different microwave channels. However, these channels are slightly different depending on the satellite. In the case where a retrieval model has been developed from a single satellite, it may be advantageous to use domain adaptation methods in order to make this model compatible with all the satellites of the constellation. In this proposed feasibility study, a Cycle Generative Adversarial Nets model is used for adapting one set of brightness temperature channels to another set. Results of a toy experiment show that this method is able to provide qualitatively good precipitation structure but still could be improved in terms of precision.

Representation learning has proven to be a powerful methodology in a wide variety of machine-learning applications. For atmospheric dynamics, however, it has so far not been considered, arguably due to the lack of large-scale, labeled datasets that could be used for training. In this work, we show how to sidestep the difficulty and introduce a self-supervised learning task that is applicable to a wide variety of unlabeled atmospheric datasets. Specifically, we train a neural network on the simple yet intricate task of predicting the temporal distance between atmospheric fields from distinct but nearby times. We demonstrate that training with this task on the ERA5 reanalysis dataset leads to internal representations that capture intrinsic aspects of atmospheric dynamics. For example, when employed as a loss function in other machine-learning applications, the derived AtmoDist distance leads to improved results compared to the $ {\mathrm{\ell}}_2 $-loss. For downscaling one obtains higher resolution fields that match the true statistics more closely than previous approaches and for the interpolation of missing or occluded data the AtmoDist distance leads to results that contain more realistic fine-scale features. Since it is obtained from observational data, AtmoDist also provides a novel perspective on atmospheric predictability.

Streamflow predictions are vital for detecting flood and drought events. Such predictions are even more critical to Sub-Saharan African regions that are vulnerable to the increasing frequency and intensity of such events. These regions are sparsely gaged, with few available gaging stations that are often plagued with missing data due to various causes, such as harsh environmental conditions and constrained operational resources. This work presents a novel workflow for predicting streamflow in the presence of missing gage observations. We leverage bias correction of the Group on Earth Observations Global Water and Sustainability Initiative ECMWF streamflow service (GESS) forecasts for missing data imputation and predict future streamflow using the state-of-the-art temporal fusion transformers (TFTs) at 10 river gaging stations in the Benin Republic. We show by simulating missingness in a testing period that GESS forecasts have a significant bias that results in poor imputation performance over the 10 Beninese stations. Our findings suggest that overall bias correction by Elastic Net and Gaussian Process regression achieves superior performance relative to traditional imputation by established methods. We also show that the TFT yields high predictive skill and further provides explanations for predictions through the weights of its attention mechanism. The findings of this work provide a basis for integrating Global streamflow prediction model data and the state-of-the-art machine learning models into operational early-warning decision-making systems in resource-constrained countries vulnerable to drought and flooding due to extreme weather events.

Robust feature selection is vital for creating reliable and interpretable machine-learning (ML) models. When designing statistical prediction models in cases where domain knowledge is limited and underlying interactions are unknown, choosing the optimal set of features is often difficult. To mitigate this issue, we introduce a multidata (M) causal feature selection approach that simultaneously processes an ensemble of time series datasets and produces a single set of causal drivers. This approach uses the causal discovery algorithms PC$ {}_1 $ or PCMCI that are implemented in the Tigramite Python package. These algorithms utilize conditional independence tests to infer parts of the causal graph. Our causal feature selection approach filters out causally spurious links before passing the remaining causal features as inputs to ML models (multiple linear regression and random forest) that predict the targets. We apply our framework to the statistical intensity prediction of Western Pacific tropical cyclones (TCs), for which it is often difficult to accurately choose drivers and their dimensionality reduction (time lags, vertical levels, and area-averaging). Using more stringent significance thresholds in the conditional independence tests helps eliminate spurious causal relationships, thus helping the ML model generalize better to unseen TC cases. M-PC$ {}_1 $ with a reduced number of features outperforms M-PCMCI, noncausal ML, and other feature selection methods (lagged correlation and random), even slightly outperforming feature selection based on explainable artificial intelligence. The optimal causal drivers obtained from our causal feature selection help improve our understanding of underlying relationships and suggest new potential drivers of TC intensification.

Uncertainty quantification (UQ) plays a crucial role in data assimilation (DA) since it impacts both the quality of the reconstruction and near-future forecast. However, traditional UQ approaches are often limited in their ability to handle complex datasets and may have a large computational cost. In this paper, we present a new ensemble-based approach to extend the 4DVarNet framework, an end-to-end deep learning scheme backboned on variational DA used to estimate the mean of the state along a given DA window. We use conditional 4DVarNet simulations compliant with the available observations to estimate the 4DVarNet probability density function. Our approach enables to combine both the efficiency of 4DVarNet in terms of computational cost and validation performance with a fast and memory-saving Monte-Carlo based post-processing of the reconstruction, leading to the so-called En4DVarNet estimation of the state pdf. We demonstrate our approach in a case study involving the sea surface height: 4DVarNet is pretrained on an idealized Observation System Simulation Experiment (OSSE), then used on real-world dataset (OSE). The sampling of independent realizations of the state is made among the catalogue of model-based data used during training. To illustrate our approach, we use a nadir altimeter constellation in January 2017 and show how the uncertainties retrieved by combining 4DVarNet with the statistical properties of the training dataset lead to a relevant information providing in most cases a confidence interval compliant with the Cryosat-2 nadir alongtrack dataset kept for validation.

Cirrus clouds are key modulators of Earth’s climate. Their dependencies on meteorological and aerosol conditions are among the largest uncertainties in global climate models. This work uses 3 years of satellite and reanalysis data to study the link between cirrus drivers and cloud properties. We use a gradient-boosted machine learning model and a long short-term memory network with an attention layer to predict the ice water content and ice crystal number concentration. The models show that meteorological and aerosol conditions can predict cirrus properties with R2 = 0.49. Feature attributions are calculated with SHapley Additive exPlanations to quantify the link between meteorological and aerosol conditions and cirrus properties. For instance, the minimum concentration of supermicron-sized dust particles required to cause a decrease in ice crystal number concentration predictions is 2 × 10−4 mg/m3. The last 15 hr before the observation predict all cirrus properties.

Transparent, understandable, and persuasive recommendations support the electricity consumers’ behavioral change to tackle the energy efficiency problem. This paper proposes an explainable multi-agent recommendation system for load shifting for household appliances. First, we extend a novel multi-agent approach by designing and implementing an Explainability Agent that provides explainable recommendations for optimal appliance scheduling in a textual and visual manner. Second, we enhance the predictive capacity of other agents by including weather data and applying state-of-the-art models (i.e., k-nearest-neighbors, extreme gradient boosting, adaptive boosting, Random Forest, logistic regression, and explainable boosting machines). Since we want to help the user understand a single recommendation, we focus on local explainability approaches. In particular, we apply post-model approaches local, interpretable, model-agnostic explanation and SHapley Additive exPlanations as model-agnostic tools that can explain the predictions of the chosen classifiers. We further provide an overview of the predictive and explainability performance. Our results show a substantial improvement in the performance of the multi-agent system while at the same time opening up the “black box” of recommendations.

Canopy habitats challenge researchers with their intrinsically difficult access. The current scarcity of climatic data from forest canopies limits our understanding of the conditions and environmental variability of these diverse and dynamic habitats. We present 307 days of climate records collected between 2019 and 2020 in the tropical rainforest canopy of the Yasuní National Park, Ecuador. We monitored climate with a 10-min temporal resolution in the middle crowns of eight canopy trees. The distance between canopy climate stations ranged from 700 m to 10 km. Apart from air temperature, relative humidity, leaf wetness, and photosynthetically active radiation (PAR), measured in each canopy climate station, global radiation, rainfall, and wind speed were measured in different subsets of them. We processed the eight data series to omit erroneous records resulting from sensor failures or lack of the solar-based power supply. In addition to the eight original data series, we present three derived data series, two aggregating canopy climate for valleys or for ridges (from four stations each), and one overall average (from the eight stations). This last derived data series contains 306 days, while the shortest of the original data series covers 22 days and the longest 296 days. In addition to the data, two open-source tools, developed in RStudio, are presented that facilitate data visualization (a dashboard) and data exploration (a filtering app) of the original and aggregated records.

Coarse spatial resolution in gridded precipitation datasets, reanalysis, and climate model outputs restricts their ability to characterize the localized extreme rain events and limits the use of the coarse resolution information for local to regional scale climate management strategies. Deep learning models have recently been developed to rapidly downscale the coarse resolution precipitation to the high local scale resolution at a much lower cost than dynamic downscaling. However, these existing super-resolution deep learning modeling studies have not rigorously evaluated the model’s skill in producing fine-scale spatial variability, particularly over topographic features. These current deep-learning models also have difficulty predicting the complex spatial structure of extreme events. Here, we develop a model based on super-resolution deconvolution neural network (SRDN) to downscale the hourly precipitation and evaluate the predictions. We apply three versions of the SRDN model: (a) SRDN (no orography), (b) SRDN-O (orography only at final resolution enhancement), and (c) SRDN-SO (orography at each step of resolution enhancement). We assess the ability of SRDN-based models to reproduce the fine-scale spatial variability and compare it with the previously used deep learning model (DeepSD). All the models are trained and tested using the Conformal Cubic Atmospheric Model (CCAM) data to perform a 100 to 12.5 km of hourly precipitation downscaling over the Australian region. We found that SRDN-based models, including orography, deliver better fine-scale spatial structures of both climatology and extremes, and significantly improved the deep-learning downscaling. The SRDN-SO model performs well both qualitatively and quantitatively in reconstructing the fine-scale spatial variability of climatology and rainfall extremes over complex orographic regions.

This paper proposes an approach to diagnosing the skill of a machine-learning prediction model based on finding combinations of variables that minimize the normalized mean square error of the predictions. This technique is attractive because it compresses the positive skill of a forecast model into the smallest number of components. The resulting components can then be analyzed much like principal components, including the construction of regression maps for investigating sources of skill. The technique is illustrated with a machine-learning model of week 3–4 predictions of western US wintertime surface temperatures. The technique reveals at least two patterns of large-scale temperature variations that are skillfully predicted. The predictability of these patterns is generally consistent between climate model simulations and observations. The predictability is determined largely by sea surface temperature variations in the Pacific, particularly the region associated with the El Nino-Southern Oscillation. This result is not surprising, but the fact that it emerges naturally from the technique demonstrates that the technique can be helpful in “explaining” the source of predictability in machine-learning models.

Estimates of global and national emissions of carbon dioxide (CO2) are important for scientific understanding and public policy on global climate change. Estimates published annually often see revisions of estimates from previous years. Revisions of data on CO2 emissions reflect revisions of the energy data from which CO2 emissions are estimated. Learning is taking place as missing values are compiled, estimated values are revised, and data management systems are updated. Revisions are a frequent feature of the database. Revisions are widespread among countries, commodities, and transactions. We have examined 11 annual reports of the United Nations Energy Statistics Database (those published from 2010 to 2020) to see in the detailed statistics what values are being changed and what are the magnitudes and patterns of change. They are most common in recent years, among developed countries, and among data on liquid fuels. Revisions are generally small and there are no indications of systematic manipulation or bias. Revisions of specific numbers are believed to represent improvements in accuracy but lack of revisions does not point toward accuracy. This examination of revisions does not permit by itself a quantitative estimate of the data uncertainty but it does suggest that the estimates of global and national totals of CO2 emissions are generally consistent and that both absolute values and trends are reliable over time and sufficiently accurate for scientific understanding and public policy.