Main findings

This study found very small between-group differences in both primary and secondary outcomes favouring both assessment and feedback (group 1) and assessment only (group 2) in comparison with no contact (group 3). These differences were consistently in the anticipated direction, and some attained statistical significance. They provide evidence of a population-level effect obtained through very brief and simple individual-level interventions. However, this statement should not be interpreted to suggest that there is strong evidence of intervention benefit and any effects beyond those obtained after 3 months should be expected to deteriorate in the longer term. ^{Reference Moyer, Finney, Swearingen and Vergun24} That many small differences were not statistically significant, even with such a large sample size, demonstrates just how small any such effects actually are. This study was highly naturalistic, comprising an unobtrusive evaluation of a national system in which student health services send emails to university students. We were aware at the outset that our evaluation strategy was conservative with populations randomised and compared regardless of their need for intervention, interest in participation and motivation to change, and also that the identification of even small effects is not without public health significance. ^{Reference McCambridge, Bendtsen, Bendtsen and Nilsen11,Reference Bendtsen, McCambridge, Bendtsen, Karlsson and Nilsen18} This study design and the findings are novel not only for alcohol, but also for other health behaviours such as tobacco smoking for which it is desirable to intervene with individuals for population-level prevention purposes, now facilitated by the internet. ^{Reference Borland, Balmford and Benda25}

Strengths and limitations

A highly pragmatic evaluation was designed to minimise interference by research artefacts stemming from intervention study participation. This naturalistic study context, although having certain methodological advantages, also imposed limitations, most notably in participation rates and consequent potential for selection biases. Offering feedback in the initial email led to greater uptake in group 1 compared with group 2. Differential participation rates at baseline could bias outcomes away from the null in comparisons of groups 1 and 2, increasing the risk of selection bias in the per-protocol analysis in particular, whereas differential participation rates at follow-up are more difficult to interpret but may bias comparisons of all three groups. The overall follow-up rate of 52% is fairly unremarkable in the context of e-health trials, being higher than many and not as high as some. ^{Reference Eysenbach26} Although other analyses suggested missing data were missing at random, the higher reported mean heavy episodic drinking among responders via email, compared with the large majority who responded via the online survey, provides further grounds for caution in interpreting these results. This suggests either a missing not at random mechanism, with heavier drinkers being disproportionately missing, or a mode effect.

Table 3 Main outcomes (intention-to-treat analyses of groups 1-3)Footnote ^a

				Group 1 v. 2			Group 1 v. 3			Group 2 v. 3
					Model, P			Model, P			Model, P
	Group 1 (n = 2546)	Group 2 (n = 2594)	Group 3 (n = 2669)	Effect size or odds ratioFootnote b (95% CI)	IFootnote c	IIFootnote d	Effect size or odds ratioFootnote b (95% CI)	IFootnote c	IIFootnote d	Effect size or odds ratioFootnote b (95% CI)	IFootnote c	IIFootnote d
AUDIT score, geometric mean (s.d.)Footnote e
AUDIT-C	3.46 (3.09)	3.44 (3.17)	3.60 (3.14)	0.005 (–0.050 to 0.060)	0.854	0.773	–0.046 (–0.101 to 0.008)	0.094	0.073	–0.051 (–0.105 to 0.003)	0.065	0.039
AUDIT 1	1.50 (1.00)	1.50 (1.02)	1.55 (1.01)	0.003 (–0.052 to 0.057)	0.927	0.891	–0.048 (–0.102 to 0.007)	0.086	0.088	–0.050 (–0.104 to 0.005)	0.072	0.067
AUDIT 2	1.07 (1.15)	1.08 (1.18)	1.10 (1.16)	–0.009 (–0.063 to 0.045)	0.749	0.885	–0.029 (–0.083 to 0.026)	0.299	0.205	–0.020 (–0.074 to 0.035)	0.480	0.256
AUDIT 3	1.06 (1.03)	1.06 (1.04)	1.12 (1.04)	0.001 (–0.053 to 0.056)	0.960	0.906	–0.052 (–0.107 to 0.002)	0.059	0.044	–0.054 (–0.108 to 0.001)	0.052	0.036
Risky drinking,Footnote f n (%)	1136 (44.6)	1194 (46.0)	1288 (48.3)	0.95 (0.85 to 1.05)	0.310	0.334	0.86 (0.78 to 0.96)	0.008	0.006	0.91 (0.82 to 1.02)	0.105	0.079

AUDIT, Alcohol Use Disorders Identification Test; AUDIT-C, three consumption questions from the AUDIT.

a. Group 1: assessment and feedback group; group 2: assessment-only group; group 3: no contact group (i.e. neither assessment nor feedback).

b. Cohen's d for AUDIT-C scores and odds ratio for risky drinking, unadjusted for baseline covariates.

c. Univariate analysis (t-test for AUDIT scores and χ² test for risky drinking).

d. Multivariate analysis adjusted for age, gender, university and term.

e. Approximate standard deviation back-calculated from the log-scale.

f. Risky drinking: heavy episodic drinking ⩾1 a month or weekly consumption >14 drinks for men and >9 for women (i.e. according to Swedish national guidelines).

The AUDIT-C provides limited capacity to identify intervention effects, although such a short instrument served masking well. Outcomes were self-reported and although computerised data collection may minimise social desirability bias, ^{Reference Turner, Ku, Rogers, Lindberg, Pleck and Sonenstein27} there is a need to study the validity of self-reported data in brief alcohol intervention trials as validity established in treatment contexts provides only limited reassurance. ^{Reference Noknoy, Rangsin, Saengcharnchai, Tantibhaedhyangkul and McCambridge28,Reference Del Boca and Darkes29} The multiplicity of analyses should be borne in mind when interpreting study findings. Heterogeneity of findings from previous studies partly results from the detailed content of interventions evaluated, for example in the study by Moreira and colleagues ^{Reference Moreira, Oskrochi and Foxcroft30} normative feedback arrived some weeks after the assessment. There are also differences among trial design and methods used, meaning that even the most rigorous estimates of intervention effects are contingent on the detailed characteristics of the evaluation studies. ^{Reference Lipsey and Wilson31}

This study has randomised and retained many more participants than any other alcohol study of which we are aware. Other study strengths include complete automation and minimisation of the potential for subversion of randomisation and observer bias in ascertainment of study outcomes. No previous study has been undertaken to evaluate a possible population-level impact of brief alcohol interventions. ^{Reference Heather32} Indeed, there are no randomised studies and meagre time series data on alcohol programmes of any duration for any target group having an impact at a population level. ^{Reference Heather32} Any such impact is thus noteworthy, especially when the costs of obtaining it are so low (annual cost per university is 6000 SEK, approximately £600 or US$920). Even if one assumes that the intervention was only delivered to those in groups 1 and 2, who participated at both baseline and follow-up (and thus ignoring subsequent delivery to all groups), the costs per intervention delivered are lower than in all previous brief intervention studies. ^{Reference Bray, Zarkin, Hinde and Mills33} This calculation does not include the initial developmental costs, ^{Reference McCambridge, O'Donnell, Godfrey, Khadjesari, Linke and Murray34} and although we did not include an economic evaluation (another study limitation to be borne in mind) it is reasonable to assume cost-effectiveness would be demonstrated for these effects given that the costs per brief intervention delivered are extremely low. Reductions in risky drinking and AUDIT scores can also be translated into possible effects on the prevalence of clinical diagnoses. ^{Reference Foxcroft, Kypri and Simonite35}

Implications

The effects of assessment alone observed here are striking. These findings extend previous randomised evaluations of assessment reactivity, almost entirely obtained in studies of hazardous and harmful drinkers. ^{Reference McCambridge and Kypri14} It appears that students who are drinking at levels that are not hazardous may change their behaviour after thinking about it when prompted by answering questions. It also appears that feedback may be additionally useful to such thinking among those who are drinking at hazardous or harmful levels. These findings are compatible with self-regulation theory, ^{Reference Clifford and Maisto36} and there is a need to develop conceptual frameworks to guide further studies of these effects. ^{Reference McCambridge and Kypri14,Reference McCambridge37}

This study was also designed to address key methodological issues. The differences between groups 2 and 3 are important because it has been shown that effect estimation depends on whether comparison is made with an assessed or an unassessed control group. Previous brief alcohol intervention studies have all used control conditions similar to group 2 rather than group 3. Group 2 constitutes a control condition that suffers fundamentally from contamination, as the data gathered for group 2 are entirely necessary to the delivery of the intervention received by group 1. ^{Reference McCambridge and Kypri14,Reference McCambridge, Butor-Bhavsar, Witton and Elbourne38}

Table 4 Per-protocol analyses of groups 1 and 2Footnote ^a among risky drinkers at study entry

	Group 1 (n = 805)	Group 2 (n = 728)	Effect sizeFootnote b (95% CI)	Model, P
				IFootnote c	IIFootnote d
Primary outcomes
AUDIT-C, geometric mean (s.d.)Footnote e	5.44 (2.44)	5.67 (2.42)	–0.095 (–0.195 to 0.006)	0.066	0.069
Risky drinking at follow-up, n (%)Footnote f	552 (68.6)	526 (72.3)	0.84 (0.67 to 1.04)	0.115	0.161
Secondary outcomes
AUDIT 1, geometric mean (s.d.)Footnote e	1.96 (0.69)	2.02 (0.68)	–0.088 (–0.188 to 0.013)	0.076	0.079
AUDIT 2, geometric mean (s.d.)Footnote e	1.70 (1.18)	1.79 (1.24)	–0.074 (–0.175 to 0.026)	0.155	0.201
AUDIT 3, geometric mean (s.d.)Footnote e	1.73 (0.94)	1.79 (0.96)	–0.063 (–0.164 to 0.037)	0.193	0.212
Additional outcomeFootnote g
Weekly consumption g/week	65.90 (68.27)	73.40 (74.92)	–0.105 (–0.205 to −0.005)	0.041	0.040

AUDIT, Alcohol Use Disorders Identification Test; AUDIT-C, three consumption questions from the AUDIT.

a. Group 1: assessment and feedback group; group 2: assessment-only group; group 3: no contact group (i.e. neither assessment nor feedback).

b. Cohen's d for AUDIT-C scores and odds ratio for risky drinking, unadjusted for baseline covariates.

c. Univariate (t-test for AUDIT-C and weekly consumption and ω² test for risky drinking).

d. Multivariate adjusted for gender, age, university, term and log (weekly consumption at baseline +1).

e. Approximate standard deviation back-calculated from the log-scale

f. Risky drinking: heavy episodic drinking ⩾1 a month or weekly consumption >14 drinks for men and >9 for women (i.e. according to Swedish national guidelines).

g. The weekly consumption analysis was not pre-specified.

This study has attempted both to evaluate a part of a national programme and to address important methodological problems in behavioural intervention trials. It makes clear the need to overcome certain methodological challenges associated with rigorous evaluation in behavioural intervention trials where the sought effects are subtle and thus vulnerable to interference by the research process. From an intervention perspective, study findings point towards the need to further develop content, delivery and tailoring methods, applying insights into behaviour change gained elsewhere. Experimentation with intervention timing as well as media such as smartphone applications may also be useful. This study provides useful evidence of both the need for, and potential benefit of, so doing. It indicates also the likely need to integrate these individual-level e-health interventions with other means to address high levels of alcohol consumption and problems associated with student drinking at a population level. ^{Reference Room, Babor and Rehm6}

To address problems within the general population, increasing the price of alcohol, better controlling its availability and restricting marketing to change the cultural acceptability of heavy drinking are most likely to be effective. ^{Reference Babor, Caetano, Casswell, Edwards, Giesbrecht and Graham5} Student drinking may even be more price sensitive than that in adult populations, and university campuses provide environments that lend themselves to outlet and marketing controls. It is a key weakness of the existing literature that we do not possess multilevel studies that explore the synergy or otherwise of individual-level interventions with those implemented at the community or population level in any group. Although this particular intervention targeted university students, evaluation of universal applications for adult general populations may be warranted, for example in primary care. The global burden of alcohol-related harm is likely to grow in the coming years. Very modest interventions similar to those evaluated here appear capable of making a small but important contribution to moving the entire distribution of alcohol consumption and related problems to the left, and thus to the strategy of preventive medicine. ^{Reference Rose39}