In this chapter we will discuss some important inequalities used in probability and statistics and their applications. They include the Cauchy–Schwarz inequality, Jensen's inequality, Markov and Chebyshev inequalities. We then discuss Chernoff's bounds, followed by an introduction to large deviation theory.

Inequalities frequently used in probability theory

Cauchy–Schwarz inequality

The Cauchy–Schwarz inequality is perhaps the most frequently used inequality in many branches of mathematics, including linear algebra, analysis, and probability theory. In engineering applications, a matched filter and correlation receiver are derived from this inequality. Since the Cauchy–Schwarz inequality holds for a general inner product space, we briefly review its properties and in particular the notion of orthogonality. We assume that the reader is familiar with the notion of field and vector space (e.g., see Birkhoff and MacLane [28] and Hoffman and Kunze [153]). Briefly stated, a field is an algebraic structure with notions of addition, subtraction, multiplication, and division, satisfying certain axioms. The most commonly used fields are the field of real numbers, the field of complex numbers, and the field of rational numbers, but there is also a finite field, known as a Galois field. Any field may be used as the scalars for a vector space.