Introduction

This chapter describes three common methods of statistical estimation: least squares (LS), maximum likelihood (ML), and generalized estimating equations (GEE). In-depth understanding of these methods inherently requires detailed statistical knowledge. However, key concepts can be illustrated using a less technical approach.

The intent is to focus on those key concepts in a nonmathematical manner. Least squares is explained first because it is perhaps the most intuitive method, and then the other methods are explained in relation to least squares. However, prior to discussing these methods, it is useful to consider the frameworks used to draw inference from results.

The three general frameworks for inference are frequentist, Bayesian, and likelihood-based. With frequentist inference, true or false conclusions are drawn from significance tests, or results are expressed in terms of a sample-derived confidence interval. With Bayesian inference, results are expressed in terms of a probability distribution for the parameter being estimated. Likelihood-based inference arises from the assertion that all the information in a sample is contained in probability distributions, called likelihood functions. The extent to which the evidence supports one parameter value or hypothesis against another is therefore equal to the ratio of their likelihoods. From these likelihoods (probabilities) confidence intervals and hypothesis tests can be constructed.