Participants

Participants were 525 adults with elevated scores on a depressive symptom assessment scale. They were recruited by means of a postal screening questionnaire sent to 27000 individuals aged 18–52 years who had been randomly selected from the Canberra electoral roll using the Statistical Package for the Social Sciences (SPSS) SELECT CASES random selection option. A total of 6130 people (22.7% response rate; 24.6% excluding those whose questionnaire was returned to sender) completed the screening questionnaire; their mean age was 37.3 years (s.d.=9.9) and 35.8% were male. Respondents were not eligible for inclusion in the trial if they

1. (a) scored below 12 on the 10-item Kessler Psychological Distress Scale (K10; Reference Andrews and SladeAndrews & Slade, 2001) (74.1% of the screening questionnaire respondents);

2. (b) indicated that they did not have internet access at home or at work (11.4%);

3. (c) indicated that they did not wish to participate in an intervention (52.1%).

Of the 822 respondents who were potentially eligible based on the screening survey, 24 (2.9%) were found not to have suitable internet access during the recruitment call. In addition, 41 (5%) reported that they were receiving psychiatric or psychological care and were excluded from the trial to ensure that it did not interfere with their treatment. Of the remaining respondents, 657 indicated a willingness to participate in the study and were sent consent and pre-intervention forms. Of these, 525 participants returned completed forms and were randomised to the interventions. The sample comprised 150 men and 375 women. The mean respondent age was 36.4 years (s.d.=9.4) and the mean K10 score was 17.8 (s.d.=5.2).

Design

Eligible individuals who completed consent forms were randomly assigned to access a depression literacy website (BluePages, n=165), a cognitive–behavioural therapy skills training site (MoodGYM, n=182) or an attention control condition (control, n=178). Participants were randomised to interventions by a statistician using the SPSS SELECT CASES random selection option and enrolled in the trial by the project manager. Those enrolled completed pre-intervention surveys by post and began the intervention an average of 1.7 weeks (s.d.=1) later. Screening and interventions were staggered between August 2002 and May 2003. The project protocol was approved by the human ethics committee of the Australian National University.

Interventions

The BluePages website (http://www.bluepages.anu.edu.au) provided information about depression, including symptoms, general and specific sources of help, evidence-based information about medical, psychological and alternative/lifestyle treatments for depression, and information about depression prevention. The site indicated that depression is an illness, provided information that moderate depression is as disabling as multiple sclerosis, emphasised that depression can and should be treated and stressed the importance and wisdom of seeking help. It also included short biographies of famous people who have experienced depression. Descriptions of the rationale for many of the treatments incorporated biomedical explanations of depression (e.g. neurotransmitter depletion), but some emphasised non-biomedical factors or approaches to depression (e.g. conflict reduction in interpersonal therapy, changing negative thoughts and improving social skills in cognitive–behavioural therapy). The intervention consisted of directing participants to read a different specified section of the BluePages site each week for 5 weeks.

The MoodGYM intervention (http://www.moodgym.anu.edu.au) comprised five interactive modules that covered cognitive therapy, behavioural methods for over-coming dysfunctional thinking, relaxation, problem solving, assertiveness and self-esteem training, and strategies for coping with relationship break-up. The intervention did not emphasise biomedical factors but rather focused on training the participant to develop strategies for coping with events and circumstances that might activate depression. As with the BluePages site, participants were directed to read a different specified module of the MoodGYM site each week for 5 weeks.

The control intervention involved weekly contact with an interviewer, who asked open-ended questions about factors that might influence depression, including physical and artistic activities, hobbies and educational activities, relationships, social support and financial management, work stress and relationships, health and nutrition and alcohol intake. The contacts were made at weekly intervals over 5 weeks and were designed to last for approximately 10 min.

Procedures

Individuals who indicated in their screening questionnaire that they might be willing to participate in an intervention trial were contacted by telephone by a trained interviewer, who described each of the three interventions and asked if the respondent would be willing to be randomised to one of them. Participants who agreed were advised that they would soon receive a consent form and pre-intervention questionnaire through the post.

Participants randomised to the two website conditions were provided with a log-in identification number and a manual containing information about the relevant website. This booklet also outlined the sections or modules of the websites that were to be completed for each of the 5 weeks. At the end of the intervention period, participants were sent a post-intervention questionnaire. Lay interviewers were not given access to questionnaire results.

Interviewers maintained weekly telephone contact with participants over the period of the intervention (a total of six contacts of approximately 10 min each: one at the outset of the intervention and one at the end of each of the 5 weeks of the intervention). During these contacts, participants using the websites were asked standard questions about their use of the website over the past week and reminded of their task for the following week. Interviewers were provided with separate instruction booklets for each participant, containing verbatim instructions for each of the weekly contacts. These booklets also included a form for recording telephone calls to participants.

Prior to the study, all interviewers attended a 1-day training session on the trial. Of the six interviewers, none had formal qualifications in mental health or a related field, although four had many years of experience in conducting survey interviews for the Centre for Mental Health Research; the other two were undergraduates without prior interviewing experience.

Measures

Demographic and clinical history, psychological distress, stigma, level of depressive symptoms and mental health knowledge were assessed by postal questionnaire.

Depression stigma scale

Changes in stigma were assessed using an 18-item test that we constructed for the purpose (Table 1; further details available from the authors upon request). Items were derived from major recurring broad themes extracted from websites on depression and stigma. In particular, themes were based on the first 100 results returned using the Google search engine and the keywords STIGMA DEPRESSION and on the Google depression directory using the keyword STIGMA. (Google was selected for the search because of its extensive coverage of the web and its demonstrated superiority in returning relevant content: Reference Hawking, Craswell and BaileyHawking et al, 2001). Themes reflected in the items included status of depression as an illness (‘depression is not a real medical illness’), extent to which depression is under personal control/fault (‘people could snap out of depression if they wanted’), character flaw (‘sign of weakness’), dangerousness, unpredictability, shame/concealment (‘would not tell anyone’), avoidance (‘avoid people with depression’) and discrimination (‘not vote for politician with depression’, ‘not employ someone with depression’). Half of the items required the participant to rate how strongly they personally agreed with a statement about depression (e.g. ‘people with depression are unpredictable’). The other half of the items asked the participant to indicate what they thought most other people believed about the same issue (e.g. ‘most people believe that people with depression are unpredictable’). Ratings were made on a five-point Likert scale. A principal components analysis on the data of the trial participants showed that the scale had two factors:

Table 1 Item-total correlations for the personal and perceived depression stigma scales for the trial group prior to randomisation (n=525)

Item	Personal stigma	Correlation1	Item	Perceived stigma	Correlation1
1	People with depression could snap out of it if they wanted	0.40	10	Most people believe that people with depression could snap out of it if they wanted	0.58
2	Depression is a sign of personal weakness	0.47	11	Most people believe that depression is a sign of personal weakness	0.62
3	Depression is not a real medical illness	0.50	12	Most people believe that depression is not a real medical illness	0.63
4	People with depression are dangerous	0.44	13	Most people believe that people with depression are dangerous	0.52
5	It is best to avoid people with depression so you don't become depressed yourself	0.48	14	Most people believe that it is best to avoid people with depression so you don't become depressed yourself	0.49
6	People with depression are unpredictable	0.29	15	Most people believe that people with depression are unpredictable	0.45
7	If I had depression I would not tell anyone	0.29	16	If they had depression, most people would not tell anyone	0.38
8	I would not employ someone if I knew they had been depressed	0.59	17	Most people would not employ someone they knew had been depressed	0.55
9	I would not vote for a politician if I knew they had been depressed	0.56	18	Most people would not vote for a politician they knew had been depressed	0.51

1. (a) personal stigma (reflecting the participants’ personal attitudes, and comprising nine items);

2. (b) perceived stigma (reflecting the participants’ beliefs about the attitudes of others, and also comprising nine items).

Scores on the total scale can range from 0 to 36 for the full scale and 0 to 18 for each of the two nine-item sub-scales, with higher scores indicating greater stigma. Cronbach's α values for the total, personal and perceived depression stigma scales were 0.78, 0.76 and 0.82 respectively and the correlation between the scores on the personal and perceived stigma scales was 0.10 (n=525; P=0.02). Individual item total correlations are shown in Table 1.

Analysis

Baseline characteristics of the three groups were compared using one-way analyses of variance (ANOVAs) for continuous measures and chi-squared analyses for categorical variables. The effects of the interventions were assessed using both intent-to-treat and completer analyses. In the intent-to-treat analyses, participants who did not respond to the post-intervention questionnaire were allocated their baseline score for the post-test score. Personal and perceived stigma were analysed separately using repeated-measures ANOVAs. Where a significant intervention × time interaction effect was obtained, the effect of the intervention on stigma was compared between the conditions using one-way ANOVAs of the change in stigma score (post-intervention score minus pre-intervention score) followed by Bonferroni-adjusted multiple comparisons. All effects were tested at the P < 0.05 level. Effect sizes were calculated using Cohen's d (standardised mean difference; Reference CohenCohen, 1988).

Potential mediators of the intervention programmes were tested using techniques and criteria described by Baron & Kenny (Reference Baron and Kenny1986) and MacKinnon (Reference Mackinnon1994). This approach involves the following steps:

1. (a) testing whether the programme affects the outcome of interest (i.e. whether the website effects the change in stigma) (Conclusion 1);

2. (b) testing whether the programme affects the potential mediator or mediators (i.e. whether the website effects change in depression, change in depression literacy, change in cognitive–behavioural therapy literacy, change in dysfunctional thoughts) (Conclusion 2);

3. (c) testing whether the potential mediators affect the outcome controlling for exposure to the intervention (i.e. whether change in depression literacy, change in cognitive–behavioural therapy literacy or change in dysfunctional thoughts is associated with a change in stigma controlling for exposure to the website) (Conclusion 3);

4. (d) if these conclusions are satisfied, a test is conducted to determine whether the mediated effect is statistically significant, using the Sobel test (Reference Sobel and LeinhartSobel, 1982) (Conclusion 4).

Analyses were conducted separately for the BluePages and MoodGYM interventions for those who completed the intervention. In addition, analyses were made using both single mediator models (in which the effect of a potential mediator might have been confounded by the contribution of other potential mediators) and multiple mediator models (where the effect of each potential mediator was computed independently of the effect of other potential mediators).

Participation

Of the 525 participants, 435 (83%) completed the post-intervention survey and 414 (79%) completed both the allocated intervention and the post-intervention survey (Table 2). Participants who returned their questionnaires did not differ in age (F _(1,522)=2.68, P=0.10), gender (χ²(1)=1.84, P=0.18) or years of education (F _(1,508)=1.81, P=0.18) from those who failed to return their questionnaire. Nor did they differ in baseline personal or perceived stigma scores (personal, F _(1,523)= 0.59, P=0.44; perceived, F _(1,523)=3.35, P=0.07), although there was a trend towards higher perceived stigma among those who did not complete the questionnaire. The latter group also had higher psychological distress and depression scores at baseline (K10, F _(1,523)=7.64, P=0.006; CES–D, F _(1,523)=5.15, P=0.024). Questionnaire return rates differed across the intervention groups (χ²(2)=14.18, P < 0.001); in particular, a significantly greater proportion of participants enrolled in the MoodGYM intervention failed to complete the post-intervention survey compared with the BluePages (χ²=5.5, P=0.02) or control (χ²=12.97, P < 0.001) conditions.

Test-retest reliability of depression stigma sub-scales

The test–retest reliabilities based on pre- and post-test data for the combined, control, BluePages and MoodGYM conditions were 0.71 (n=435), 0.66 (n=159), 0.79 (n=136) and 0.70 (n=140) respectively for the personal stigma scale, and 0.67 (n=0.434), 0.67 (n=159), 0.63 (n=135) and 0.73 (n=140) respectively for the perceived stigma scale (P < 0.001 in each case).

Effects for personal stigma

Table 2 shows the mean personal stigma scores for participants who were randomised to an intervention condition (intent-to-treat). The main effect for time was not significant (F _(1,522)=2.95, P > 0.05), but there was a significant interaction between time and intervention group (F _(2,522)=4.36, P=0.016), indicating that the interventions had different effects on stigma. An analysis of the change in stigma over time demonstrated a significant main effect for intervention (F _(2,522)=4.36, P=0.013). In particular, stigma reduction was significantly greater in both the BluePages and MoodGYM groups than in the control condition after Bonferroni correction: BluePages v. control groups, mean difference 0.94, 95% CI 0.07–1.82, P=0.031; MoodGYM v. control groups, mean difference 0.90, 95% CI 0.043–1.75, P=0.036. There was no significant difference between the two websites in stigma reduction (mean difference 0.04, 95% CI -0.83 to 0.92, P=1.0). Re-analysis of the results using education as a covariate yielded the same pattern of findings as above, except the greater stigma reduction in the MoodGYM condition relative to the control only approached significance (P=0.054) after Bonferroni correction. The pattern of results for those who completed both the intervention and the post-intervention questionnaire was similar to that for the intent-to-treat groups, but the effect for MoodGYM (which was significant using an unadjusted t-test) was not significant following Bonferroni adjustment (mean difference 0.96, 95% CI -0.12 to 2.04, P=0.10).

Effects for perceived stigma

The mean perceived stigma scores for each intervention before and after the intervention are shown in Table 2. The main effect for time was not significant (F _(1,522)=0.3, P > 0.05), but there was a significant interaction between time and intervention group (F _(2,522)=4.16, P=0.016), indicating that the interventions had different effects on perceived stigma. There was a significant effect for intervention on change in stigma (F _(2,522)=4.16, P=0.016). In particular, stigma increased in the MoodGYM group relative to the control group (MoodGYM v. control, mean difference -1.07, 95% CI -1.96 to -0.177, P=0.012), but there was no other significant difference in stigma reduction for the interventions (BluePages v. control, mean difference 0.60, 95% CI -1.5 to 0.32, P=0.35; BluePages v. MoodGYM, mean difference -0.47, 95% CI -1.38 to 0.44, P=0.64). Re-analysis of the results using education as a covariate yielded the same pattern of results as above, notably a significant interaction between intervention and time and a significantly greater increase in stigma in the MoodGYM group.

Size of the effects

The pre- minus post-effect sizes for personal stigma were 0.12 (BluePages), 0.11 (MoodGYM) and -0.07 (control) for the intent-to-treat group, and 0.13, 0.10 and -0.09 respectively for those who completed the trial. The corresponding pre–post effect sizes for perceived stigma were 0.01, -0.09 and 0.14 for the intent-to-treat group and 0.02, -0.14 and 0.15 for those who completed the trial.

Analyses of mediation effects

The following analyses included change in depressive symptoms, and change in depression and cognitive–behavioural therapy literacy as potential mediators in the model. However, dysfunctional thoughts were excluded from the model since there was a high correlation between the ATQ and CES–D change scores (BluePages r=0.70, MoodGYM r=0.59, control r=0.71).

Perceived stigma

Since, as noted above, the BluePages intervention did not lead to a change in perceived stigma (Conclusion 1), no mediation analysis was conducted for this intervention. The MoodGYM intervention, however, was associated with an increase in perceived stigma relative to the control group: β=-1.41 (s.e.=0.49), t=2.87, P=0.004 (Conclusion 1 met). In addition, the relationship between the mediators and change in stigma was significant, as documented above (Conclusion 2 met). Controlling for intervention, greater depression literacy improvement was associated with less perceived stigma reduction for both the single and multiple mediator models: for MoodGYM, multiple model, ΔCES–D, β=0.04 (s.e.=0.03), P=0.14; ΔD–Lit=-0.21 (s.e.=0.10), P=0.04; Δ CBT–Lit=0.04 (s.e.=0.12), P=0.77; single model, Δ CES–D, β=0.04 (s.e.=0.03), P=0.10; Δ D–Lit=-0.22 (s.e.=0.10), P=0.03; Δ CBT–lIT=-0.02 (s.e.=0.12), P=0.89 (Conclusion 3 met for depression literacy change). However, the mediated effect was not significant (ratio of the indirect to the direct effect -0.10, Sobel test 1.65, P=0.10). The other mediators (change in symptoms, depression literacy) did not lead to perceived stigma change (Conclusion 3 not met). In conclusion, there was no evidence that the perceived stigma increase for MoodGYM relative to the control group was associated with change in depression or change in depression or cognitive–behavioural therapy literacy.

Limitations of the study

The major limitation of this study is that the effect sizes were small. There are a number of possible reasons for this. First, pre-intervention personal stigma was not marked, the approximate mean score being 10 (out of a maximum score of 36). This might have placed a floor on the potential for participant improvement. Second, the intervention might have been suboptimal for the current purposes. In particular, neither the exercises nor the other content of MoodGYM specifically addressed issues of self-stigma. Similarly, BluePages was not specifically designed as an anti-stigma package but rather as a site about depression in general, with an emphasis on evidence-based treatments. BluePages did incorporate an illness/biomedical model of depression and the strong message that depression is treatable, with the aim (in part) of reducing controllability and stability attributions, since social attribution theory predicts that reducing these attributions should reduce stigma (Reference Corrigan, River and LundinCorrigan et al, 2000). However, it has been argued that the biomedical model of mental illness might increase stigma (e.g. Reference Walker and ReadWalker & Read, 2002; Reference Dietrich, Beck and BujantugsDietrich et al, 2004). Inclusion of the model in BluePages might therefore have reduced the effect size of the intervention. However, any effects on stigma of a biomedical explanation could operate differently for people with and without depression, and more targeted research on the role of the biomedical model is needed (Reference Griffiths and ChristensenGriffiths & Christensen, 2004).

Another limitation of our study is that stigma measured by means of self-completed questionnaires may lack ecological validity, possibly measuring participant cognitions or hypotheses rather than their attitudes in a real-life context (Reference HaghighatHaghighat, 2001). In addition, apparent changes in self-reported stigma may reflect social desirability bias (Reference HaghighatHaghighat, 2001).

Because our study was conducted in the broader context of a depression intervention trial (Reference Christensen, Griffiths and JormChristensen et al, 2004), the control condition was designed to be credible as an intervention for depression. It did not therefore incorporate a neutral website, a limitation for the purposes of this study because the nature of the interviewer contacts for the control group differed from those for the website groups. Another potential weakness of our findings is that the response rate for the screening survey was low. However, as we have noted in a previous paper, the more relevant factor in a design such as this is the response rate among people with a high level of depressive symptoms (Reference Jorm, Griffiths and ChristensenJorm et al, 2003). Using data from the Australian National Survey of Mental Health and Well-Being (Reference Andrews and SladeAndrews & Slade, 2001), it is estimated that 1831 people in the original sample of 27 000 would have obtained a K10 score in the required range. In fact, 1586 (86.6%) people in this range returned completed surveys. Thus, the response rate to the screening survey was high among the target group of interest. Only 33.1% (525) of respondents with a high level of depressive symptoms met other necessary criteria for inclusion in the trial, including having internet access, not currently receiving treatment from a psychologist or psychiatrist and being willing to participate. It seems likely, however, that the attributes of the intervention sample reflect the subset of the community who would be willing to undertake an internet intervention to decrease psychological distress.

Another limitation of our study is that testing occurred soon after completion of the interventions. Clearly, a meaningful intervention must have longer-term sustainable effects. In addition, since the current personal stigma scale clustered into one factor, it is not possible to determine if the intervention differentially affected different aspects of personal stigma. Moreover, it is not possible to judge from the study which aspects of the websites’ content were effective in reducing personal stigma. Finally, it remains to be seen if BluePages can reduce personal stigma among people without depressive symptoms.

Future research

The internet is used by a range of organisations and individuals to deliver public health messages designed to combat stigmatising attitudes to mental health disorders. However, the effect of these sites on stigma has not previously been evaluated. The results of this study suggest that the internet warrants further investigation as a potentially effective means of delivering stigma reduction programmes for depression. The challenge is to develop programmes that exert larger effects on stigma. This would involve systematically testing and identifying the content and contexts which are most effective in reducing stigma. There would also be value in comparing the efficacy of different delivery methods, including face-to-face interviews, internet websites and printed material. Finally, there is a need to explore the effectiveness of educational programmes in reducing stigma among family members of people with depression, among people without depressive symptoms and among people with high levels of stigma about depression.

CLINICAL IMPLICATIONS

1. ▪ Depression literacy and cognitive–behavioural therapy programmes may be helpful in decreasing personal stigma in people with depression.

2. ▪ The internet offers an opportunity to disseminate such programmes widely and at low cost.

3. ▪ Care should be taken to ensure that cognitive–behavioural programmes do ot increase perceived stigma among clients.

LIMITATIONS

1. ▪ The demonstrated effect sizes were small.

2. ▪ Further research is needed to identify which messages are most effective.

3. ▪ It is not known if the positive effects of these psychoeducational interventions are sustainable over time.