The way networks grow and change over time is called network evolution. Numerous off-the-shelf algorithms have been developed to study network evolution. These can give us insight into the way systems grow and change over time. However, what off-the-shelf algorithms often lack are knowledge of the behavioral details surrounding a specific problem. Here we will develop a simple case that we will revisit over the next few chapters: How do children learn words from exposure to a sea of language? One possibility is that the words children learn first influence the words they learn next. Another possibility is that the structure of language itself facilitates the learning of some words over others. Indeed, we know that adults speak differently to children in ways that facilitate language learning, with semantically informative words tending to appear more often around words that children learn earliest. This invites the question: To what extent does the semantic structure of language predict word learning? This chapter will provide a general framework for building and competing models against one another with a specific application to the network evolution of child vocabularies.

Network science is a broadly interdisciplinary field, pulling from computer science, mathematics, statistics, and more. The data scientist working with networks thus needs a broad base of knowledge, as network data calls for—and is analyzed with—many computational and mathematical tools. One needs good working knowledge in programming, including data structures and algorithms to effectively analyze networks. In addition to graph theory, probability theory is the foundation for any statistical modeling and data analysis. Linear algebra provides another foundation for network analysis and modeling because matrices are often the most natural way to represent graphs. Although this book assumes that readers are familiar with the basics of these topics, here we review the computational and mathematical concepts and notation that will be used throughout the book. You can use this chapter as a starting point for catching up on the basics, or as reference while delving into the book.

There is a daunting array of statistical “methods” out there – regression, ANOVA, loglinear models, GLMMS, ANCOVA, etc. They often are treated as different data analysis approaches. We take a more holistic view. Most methods biologists use are variations on a central theme of generalized linear models – relating a biological response to a linear combination of predictor variables. We show how several common “named” methods are related, based on classifying biological response and predictor variables as continuous or categorical. We use simple regression, single-factor ANOVA, logistic regression, and two-dimensional contingency tables to show how these methods all represent generalized linear models with a single predictor. We describe how we fit these models and outline their assumptions.

In this chapter, we introduce the design of statistical anomaly detectors. We discuss types of data – continuous, discrete categorical, and discrete ordinal features – encountered in practice. We then discuss how to model such data, in particular to form a null model for statistical anomaly detection, with emphasis on mixture densities. The EM algorithm is developed for estimating the parameters of a mixture density. K-means is a specialization of EM for Gaussian mixtures. The Bayesian information criterion (BIC) is discussed and developed – widely used for estimating the number of components in a mixture density. We also discuss parsimonious mixtures, which economize on the number of model parameters in a mixture density (by sharing parameters across components). These models allow BIC to obtain accurate model order estimates even when the feature dimensionality is huge and the number of data samples is small (a case where BIC applied to traditional mixtures grossly underestimates the model order). Key performance measures are discussed, including true positive rate, false positive rate, and receiver operating characteristic (ROC) and associated area-under-the-curve (ROC AUC). The density models are used in attack detection defenses in Chapters 4 and 13. The detection performance measures are used throughout the book.

Approximate Bayesian analysis is presented as the solution for complex computational models where no explicit maximum likelihood estimation is possible. The activation-suppression racemodel (ASR), which does have a likelihood amenable to Markov chain Monte Carlo methods, is used to demonstrate the accuracy with which parameters can be estimated with the approximate Bayesian methods.

A good model aims to learn the underlying signal without overfitting (i.e. fitting to the noise in the data). This chapter has four main parts: The first part covers objective functions and errors. The second part covers various regularization techniques (weight penalty/decay, early stopping, ensemble, dropout, etc.) to prevent overfitting. The third part covers the Bayesian approach to model selection and model averaging. The fourth part covers the recent development of interpretable machine learning.

As probability distributions form the cornerstone of statistics, a survey is made of the common families of distributions, including the binomial distribution, Poisson distribution, multinomial distribution, Gaussian distribution, gamma distribution, beta distribution, von Mises distribution, extreme value distributions, t-distribution and chi-squared distribution. Other topics include maximum likelihood estimation, Gaussian mixtures and kernel density estimation.

The Halphen type B (Hal-B) frequency distribution has been employed for frequency analyses of hydrometeorological and hydrological extremes. This chapter derives this distribution using entropy theory and discusses the estimation of its parameters with the use of the constraints used for their derivation. The distribution i+L13s tested using entropy and the methods of moments and maximum likelihood estimation.

Several generalized frequency distributions have been employed in environmental and water engineering over the years. These distributions are quite versatile and can apply to frequency analysis of a wide variety of random variables, such as flood peaks, volume, duration, and inter-arrival time; extreme rainfall amount, duration, spatial coverage, and inter-arrival time; drought duration, severity, spatial extent, and inter-arrival time; wind speed, duration, direction, and spatial coverage; water quality parameters; and sediment concentration, discharge, and yield. However, because of their relatively complex form, these distributions have not become as popular as the simpler distributions. These distributions have at least three but usually more parameters, which have been estimated using the methods of moments, maximum likelihood, probability weighted moments, and L-moments. In some cases, entropy theory has been used to estimate parameters. This chapter provides a snapshot of the generalized distributions that will be discussed in this book. Moreover, a short discussion of the methods of parameter estimation, goodness-of-fit statistics, and confidence intervals is provided.

The problem of tracking the system frequency is ubiquitous in power systems. However, despite numerous empirical comparative studies of various algorithms, the underlying links and commonalities between frequency tracking methods are often overlooked. To this end, we show that the treatment of the two best known frequency estimation methodologies: (i) tracking the rate of change of the voltage phasor angles, and (ii) fixed frequency demodulation, can be unified, whereby the former can be interpreted as a special case of the latter. Furthermore, we show that the frequency estimator derived from the difference in the phase angle is the maximum likelihood frequency estimator of a nonstationary sinusoid. Drawing upon the data analytics interpretation of the Clarke and related transforms in power system analysis as practical Principal Component Analyzers (PCA), we then set out to explore commonalities between classic frequency estimation techniques and widely linear modeling. The so-obtained additional degrees of freedom allow us to arrive at the adaptive Smart Clarke and Smart Park transforms (SCT and SPT), which are shown to operate in an unbiased and statistically consistent way for both standard and dynamically unbalanced smart grids. Overall, this work suggest avenues for next generation solutions for the analysis of modern grids that are not accessible from the Circuit Theory perspective.

This chapter deals with

• Spatial econometrics notions

• Handling spatial dependence and spatial heterogeneity through spatial econometric models

• Diagnostics for spatial dependence (Lagrange multiplier tests)

• Spatial autoregressive model (SAR)

• Spatial error model

• Spatial filtering

• Spatial regression with regimes

• Spatial two-stage least squares (S2SLS)

• Maximum likelihood

• Generalized method of moments (GMM)

After a thorough study of the theory and lab sections, you will be able to

• Distinguish which spatial regression method to use in order to account for spatial dependence and spatial heterogeneity

• Interpret spatial diagnostics used in OLS regression that can guide you to the appropriate spatial regression model

• Build spatial regression models and interpret results

• Identify if spatial dependence or spatial heterogeneity is accounted for by the adopted method

• Conduct spatial econometric analysis through GeoDa space