Researchers who generate data often optimize efficiency and robustness by choosing stratified over simple random sampling designs. Yet, all theories of inference proposed to justify matching methods are based on simple random sampling. This is all the more troubling because, although these theories require exact matching, most matching applications resort to some form of ex post stratification (on a propensity score, distance metric, or the covariates) to find approximate matches, thus nullifying the statistical properties these theories are designed to ensure. Fortunately, the type of sampling used in a theory of inference is an axiom, rather than an assumption vulnerable to being proven wrong, and so we can replace simple with stratified sampling, so long as we can show, as we do here, that the implications of the theory are coherent and remain true. Properties of estimators based on this theory are much easier to understand and can be satisfied without the unattractive properties of existing theories, such as assumptions hidden in data analyses rather than stated up front, asymptotics, unfamiliar estimators, and complex variance calculations. Our theory of inference makes it possible for researchers to treat matching as a simple form of preprocessing to reduce model dependence, after which all the familiar inferential techniques and uncertainty calculations can be applied. This theory also allows binary, multicategory, and continuous treatment variables from the outset and straightforward extensions for imperfect treatment assignment and different versions of treatments.

Our research investigates the effects of residential energy efficiency audit programs on subsequent household electricity consumption. Here there is a one-time interaction between households, which participate voluntarily, and the surveyors. Our research objective is to determine whether and to what extent the surveys lead to behavioral changes. We then examine how persistent the intervention is over time and whether the effects decay or intensify. The main evaluation problem here is survey participants’ self-selection, which we address econometrically via several non-parametric estimators involving kernel-based propensity-score matching. In the first method we use difference-in-differences (DID) estimation. Our second estimator is quantile DID, which produces estimates on distributions. The comparison group consists of households who were not yet participating in the survey but participated later. Our evidence is that the customers who participated in the survey reduced their electricity consumption by about 7%, on average compared to customers who had not yet participated in the survey. Considering the total number of high-usage households participating in the survey in 2009, we estimate that electricity consumption was reduced by an aggregate of 2 million kWh per year, which is approximately equal to the monthly consumption of 3500 typical households in California with an estimated 1527 metric tons less of carbon dioxide emissions. Because the energy audit program is inexpensive ($10–$20 per household) a key issue is that while the program is cost-effective, is it regressive? We find that as the quantiles of the outcome distribution increase, high-use households save proportionally less electricity than do low-use customers. Overall, our results imply that program designers can better target low-use and low-income households, because they are more likely to benefit from the programs through energy savings.