This chapter begins to develop the framework for analyzing the success of particular natural experiments, focusing on the key definitional feature: the random or as-if random assignment of units to treatment, that is, to different categories of the independent variable. Chapters 9 and 10 will then discuss two other dimensions on which different natural experiments may vary: the credibility of statistical and causal models and the substantive or theoretical relevance of the intervention.

In an important sense, the plausibility of as-if random assignment stands logically prior to the analysis of data from a natural experiment. After all, if the claim of random or as-if random assignment is not compelling, then the simple and transparent techniques discussed in previous chapters may substantially lose their credibility. For example, if confounding is an issue, then simple differences of means may not suffice as compelling evidence of causal effects. The material in this chapter could thus be placed before the discussion in Chapters 5–7. This is also one reason why, in published research reports, quantitative and qualitative diagnostics for assessing as-if random should usually precede the reporting of results. However, because one goal is to place the discussion of as-if random specifically in relation to the two desiderata discussed in the next chapter—the credibility of statistical models and the substantive relevance of intervention—it is useful to discuss this issue in the final part of the book.