Appendix 2.1

The Rubin/Holland analysis, which is also widely adopted by economists discussing RCTs, begins with a singular counterfactual difference: that between the value that the outcome (say x_k(i)) would have in the individual case i were i subject to the treatment (x_k^T(i)) and the value it would have in i were i subject to the control (x_k^C(i)). It is assumed that the possible values x_k can take for i are determinedFootnote ¹⁸ by a complete set of possible causes of x_k that might act on i during the relevant time period given the actual situation of i, including possibly the treatment T (which in this simple case gets value 1 in the treatment group and 0 in the control). This gets represented in the potential outcomes equation:

In this equation the variables on the left represent the targeted outcome; c= signifies that the two sides of the equation are equal and that the factors on the right are causes of those on the left. T(i) may or may not genuinely appear there (i.e., α_T(i) may be zero). The equation represents the possible value the outcome can take given various combinations of values a complete set of causes for it take. Besides the treatment there are J possible additive causes as well as those that make up the interactive factor α_T(i) (which may turn out to be 0), most unknown or unobservable.

Now consider treatment (T = 1) and control (T = 0) groups and calculate averages. Imagine that random assignment and blinding have succeeded as hoped in ensuring that T is orthogonal in the mean to the net effect of other causal factors (α_T(i) and ∑α_j(i)x_j(i)), in which case, using Exp for ‘expectation’

$Exp\left(x_k^T\right(i)-x_k^C(i\left)\right)=Exp{x}_k^T\left(i\right)-Exp{x}_k^C\left(i\right)=Exp{\alpha }_T\left(i\right)$(1)

So the middle term (the difference in means between the treatment and control groups, which is observable) is an unbiased estimator of Exp α_T(i). Given the causal interpretation proposed of the potential outcomes equation, a genuinely positive effect size shows that α_T(i) ≠ 0 for some i: that is, that (in the long run of experiment repetitions) the treatment will have caused the outcome in at least some individuals in the treatment group. Note that the observed effect size is the estimated mean of the individual treatment effects, which in turn is the estimated mean of α_T(i). By inspection α_T(i) represents the net effect of the interactive/support factors that fix whether, and to what degree, T can contribute to x_k in individual case i.