A standard probability formalism is introduced, including a definition of the probability density function (PDF) and its first four moments. Most basic PDFs such as the uniform and Gaussian PDFs, are defined. The fundamentals of the Bayes’ formula derivation and its formulation in terms of PDFs are also presented. More importantly, data assimilation is described as a recursive Bayes’ formula, which connects the standard Bayes’ formula from different analysis times by using transition PDFs. A basic introduction to Shannon information theory is presented, followed by a definition of uncertainty in terms of entropy, and therefore establishing a mathematical basis for interpreting data assimilation in terms of information processing that is used throughout this book. The multivariate Gaussian data assimilation framework, most often used in practice, is described. Common analysis solutions that include maximum a posteriori and minimum variance methods are derived, which include a formulation of the cost function and posterior probability.