Effect sizes, like relative risks and Cohen’s d, are shown to be important, rather than mere statistical significance.

All null hypothesis statistical significance test (NHST) procedures follow eight steps: (1) forming groups in the data, (2) define the null hypothesis (H0), (3) set alpha (α), (4) choose a one-tailed or a two-tailed test, (5), calculate the observed value, (6) find the critical value, (7), compare the observed value and the critical value, and (8) calculate an effect size. For a z-test the effect size is Cohen’s d. This can be interpreted as the number of standard deviations between the two means.

A z-test is the simplest NHST and tests the H0 that a sample’s dependent variable mean and a population’s dependent variable mean are equal. If H0 is retained, then the difference between means is no greater than what would be expected from sampling error. If the H0 is rejected, it is not a good statistical model for the data.

When conducting NHSTs, it is possible to make the wrong decision about the H0. A Type I error occurs when a person rejects a H0 that is actually true. A Type II error occurs when a person retains a H0 that is actually false. It is impossible to know whether a correct decision has been made or not.

One of the most frequent research designs in the social sciences is to compare two groups’ scores. When there is no pairing between sets of scores, it is necessary to conduct an unpaired two-sample t-test. The steps of this NHST are the same eight steps of the previous statistical tests because all of these are members of the GLM. There are some modifications that make the unpaired two-sample t-test unique:

• The null hypothesis for an unpaired two-sample t-test is always.

• The equation for the number of degrees of freedom is different: df = (n1 – 1) + (n2 – 1).

• The formula to calculate the observed value is more complex:.

• The effect size for an unpaired two-sample t-test is still Cohen’s d, but the formula has been modified:.

The unpaired two-sample t-test shares many characteristics with other NHSTs: the sensitivity to sample size, the necessity of calculating an effect size, the arbitrary nature of selecting an α value, and the need to worry about Type I and Type II errors.

Paired scores occur when each score in one sample corresponds to a single score in another sample. Whenever paired scores occur, researchers can measure whether the difference between them is statistically significant through a paired-samples t-test. A paired-samples t-test is similar to a one-sample t-test. The major difference is that the sampling distribution consists of differences between group means. This necessitates a few minor changes to the t-test procedure. The major differences are:

• The null hypothesis for a paired-samples t-test is now, whereis the average difference between paired scores.

• n now refers to the number of score pairs in the data.

• The Cohen’s d formula is now, where the numerator is the difference between the two sample means and is the pooled standard deviation.

All of the other aspects of a paired-samples t-test are identical to a one-sample t-test, including the default α value, the rationale for selecting a one- or a two-tailed test, how to compare the observed and critical values in order to determine whether to reject the null hypothesis, and the interpretation of the effect size. The caveats of all NHSTs apply.