Chapter 3 discusses the field of machine learning from a theoretical perspective. The review will advance the discussion of advanced metrics in Chapter 5 and error estimation methods in Chapter 6. The specific concepts surveyed in this chapter include loss functions, empirical risk, generalization error, empirical and structural risk minimization, regularization, and learning bias. The unsupervised learning paradigm is also reviewed and the chapter concludes with a discussion of the bias/variance tradeoff.