Over the past two decades, the internet has increasingly been used to access and exchange mental-health-related information. Reference Mager1–Reference Wong, Fu, Yau, Ma, Law and Chang3 Among the variety of suicide-related resources that may be found on the internet, Reference Wong, Fu, Yau, Ma, Law and Chang3–Reference Till and Niederkrotenthaler5 there are many educative websites hosted by professional organisations. An important, but still under-researched question is whether these websites have the potential to have an impact on the risk factors for suicide among online users. Case studies have indicated that online media may be used as a tool for learning about suicide methods. Reference Becker and Schmidt6–Reference Baume, Cantor and Rolfe8 Additionally, some websites may trigger additional suicides (the so-called Werther effects). Reference Phillips9 These findings have fuelled debates regarding whether educative websites have any beneficial impact. Reference Beautrais, Fergusson and Coggan10 Recent research findings related to traditional media, however, have indicated that suicide-related resources may have a positive impact if their focus is on constructive ways of coping with adverse circumstances (known as the Papageno effect). Reference Niederkrotenthaler, Voracek, Herberth, Till, Strauss and Etzersdorfer11 Accordingly, a recent laboratory experiment demonstrated a decrease in the risk of suicide after exposure to a fictional film featuring a suicidal protagonist overcoming his crisis. This effect was particularly pronounced among viewers with some degree of vulnerability to suicide. Reference Till, Strauss, Sonneck and Niederkrotenthaler12 In the present study, a randomised controlled trial (RCT) was conducted to examine the effects of the non-interactive components of three German-language educative suicide prevention websites targeting adolescents and young adults. We evaluated the impact of these websites on suicidal ideation (the primary outcome) and the following secondary outcomes: mood, suicide-prevention-related knowledge and attitudes towards suicide/seeking professional help. In addition to these quantitative analyses, qualitative interviews with 12 participants were conducted to gain deeper insight into how participants experienced the websites.
For the primary outcome, the following three a priori hypotheses were tested; (a) educative websites will reduce suicidal ideation; (b) these effects will vary for different educative websites; and (c) the effects on the primary outcome will vary depending on an individual's vulnerability to suicide. The third hypothesis was generated after the completion of a topically related investigation conducted by this research group, in which individual vulnerability was found to mediate the effects of suicide-related movies. Reference Till, Strauss, Sonneck and Niederkrotenthaler12 This hypothesis was generated after trial commencement but before data analysis and is disclosed here in accordance with the CONSORT 2010 statement.
Method
Participants
A single-blinded RCT was conducted between March 2014 and January 2015. Participants included were 161 individuals currently residing in Austria. Of these individuals, 108 were women (67.1%) and 53 men (32.9%). We recruited medical, psychology and communication studies undergraduates from the University of Vienna and the Medical University of Vienna. Participants were allowed to invite family and friends to join them in participating in their respective session. The mean age of participants was 24.5 years (s.d. = 5.8) and, therefore, in accordance with the target population of the selected mental health organisations.
Power analysis
Based on the results of an a priori power analysis conducted using G*Power 3.1.2, Reference Faul, Erdfelder, Lang and Buchner13 an ANOVA model with three repeated measures, four groups and an assumed correlation of 0.80 among repeated measures would require a sample-size of 156 participants to detect a medium-sized effect of 0.25. This sample-size estimate should be viewed as conservative. More specifically, an analysis based on data from an earlier study Reference Till, Strauss, Sonneck and Niederkrotenthaler12 was performed using GLIMMPSE, Reference Kreidler, Barón and Glueck14 which indicated that a linear mixed model with three repeated measures, four study groups and a dichotomous variable reflecting low v. high baseline vulnerability would require 136 participants (P < 0.05, 80% power).
Materials
We searched the internet using the Google search engine to identify German-language websites on suicide-related education and prevention among adolescents and young adults. Reference Till and Niederkrotenthaler5 The search terms used were Selbstmord, Hilfe and Krise (suicide, help, crisis). The following three mental health service providers, all based in Germany, were identified within the first five pages of the search results: Reference Jansen and Spink15 Youth-Life-Line (www.youth-life-line.de), U25-Freiburg (www.u25-freiburg.de), and Freunde furs Leben (‘Friends for Life’, www.frnd.de). The Youth-Life-Line website provides email counselling by peers for young people in crisis for adolescents and young adults and is based in Tubingen. U25-Freiburg is based in Freiburg im Breisgau and offers email counselling by peers for young individuals with psychological problems. Friends for Life is an organisation based in Berlin that provides information on depression and suicide. All websites provide information on suicide prevention.
To identify the perceived differences across these websites, we conducted a pre-test with 15 raters. Participants were instructed to search for information related to suicide and suicide prevention on each website; subsequently, they were requested to rate the structural and stylistic aspects of and how strongly specific aspects of suicide were emphasised on each website using a survey prepared for this pre-test (see online Table DS1). In addition, the raters were requested to write a short summary of the primary message of each website. The summaries varied only slightly between websites. No significant differences were found among website characteristics, as assessed using the survey.
Procedure
We conducted an RCT with three intervention groups and a control group exposed to a website unrelated to suicide or mental health. Group 1 viewed the Youth-Life-Line website; group 2 was exposed to the U25-Freiburg website, and group 3 participants were presented with the Friends for Life website. The control group viewed the website of the Austrian boy scouts (www.ppoe.at).
Participants were informed that they would participate in a study on website effects, without further specification. In total, 165 individuals volunteered to participate anonymously and were recruited in blocks of four or eight participants per session. All sessions occurred at the Center for Public Health, Medical University of Vienna, Austria. Based on ethical considerations, participants who were suicidal (defined as individuals scoring >33 on the German short version of the Beck Hopelessness Scale, Reference Krampen16,Reference Beck and Steer17 as assessed at T 1) were offered counselling by a psychologist (B.T.) and excluded from the study. Reference McMillan, Gilbody, Beresford and Neilly18 A total of four individuals scored above this cut-off. Participants (n = 161) were randomly assigned in blocks of four individuals to one of the four groups. The participants were free to select any of the available computer workstations in the lab, which were pre-programmed with one of the four websites, using an allocation ratio of 1:1:1:1 (three intervention groups and one control group). For the final session, the allocation ratio was adapted to achieve a balanced distribution of participants across groups. Figure 1 shows the study flow chart.
Fig. 1 Study flow chart.
All participants were instructed that they had 10 min to browse the website, to search for information on suicide and suicide prevention and to learn about all aspects of the topic. Questionnaires on suicidal ideation, mood, suicide-related knowledge, and attitudes towards suicide and seeking professional help were completed before (T 1), immediately after exposure (T 2), and approximately 1 week after exposure to the website (T 3). Participants were instructed not to search for information on suicide or suicide prevention between T 2 and T 3 or discuss the study with other participants.
After the experiment, we offered psychological counselling to all participants to facilitate coping with potential trial-related distress. Participants received a gift voucher of €5 for participation. The study was approved by the Research Ethics Board at the Medical University of Vienna (study protocol 2063/2013, date: 2014-01-27). The trial was registered with the American Economic Association's registry for randomised controlled trials (www.socialscienceregistry.org), as RCT-ID:0000924. The trial data for this study are available at figshare.com (https://figshare.com/articles/Dataset_sav/4898147).
Primary outcome measure
Reasons for Living Inventory
Suicidal ideation was assessed with the 48-item Reasons for Living Inventory Reference Linehan, Goodstein, Nielsen and Chiles19 (for example ‘I am afraid of the actual “act” of killing myself’). Respondents rated their beliefs and expectations for not dying by suicide on a scale ranging from 1 (not at all important) to 6 (extremely important). Scale scores were reverse-coded, resulting in higher scores indicating higher suicidal ideation.
Secondary outcome measures
Mood subscale
Mood was assessed with the mood subscale of the Affective State Scale, Reference Becker20 which uses responses to eight adjectives such as merry or sad, scored on a scale from 1 (not at all) to 4 (highly). Higher scores indicate better mood.
Revised Facts on Suicide Quiz
Suicide-prevention-related knowledge was assessed with 5 items of the 34-item Revised Facts on Suicide Quiz. Reference Voracek, Tran and Sonneck21 Twenty-nine items were excluded, based on information not appearing on all three suicide-prevention websites. The remaining five items were Items 1, 3, 12, 27 and 32 (for example ‘People who talk about suicide rarely commit suicide’), and were answered with true, false or don't know, or with content-wise appropriate multiple-choice formats of three response alternatives each. Respondents received one point for each correct answer. Higher scores indicate more knowledge.
Cognitions Concerning Suicide Scale
Attitudes towards suicide were measured with the Cognitions Concerning Suicide Scale by Biblarz and colleagues, Reference Biblarz, Brown, Biblarz, Pilgrim and Baldree22 consisting of 20 items (for example ‘Everyone has the right to commit suicide’) rated on 6-point scales ranging from 0 (disagree) to 5 (agree). Higher scores indicate higher approval of suicide.
Short Attitudes toward Seeking Professional Help Scale
This is a 10-item self-report measure Reference Fischer and Farina23 to assess attitudes towards seeking professional psychological help (for example ‘I would want to get psychological help if I were worried or upset for a long period of time’). Items are rated on a scale from 0 (disagree) to 3 (agree). Higher scores represent higher approval of help-seeking.
Data analysis
For each participant, the mean scores across all items of the respective questionnaire were calculated. The scores for each dependent measure were subjected to a study group (intervention group, control group) × test condition (pre-exposure, post-exposure, 1 week later) × suicidal ideation (baseline suicidal ideation below v. above the median) analysis using linear mixed models. Group differences were tested using contrast tests. We report Bonferroni-Holm corrected P-values for contrast tests to correct for multiple comparisons. Linear mixed models are particularly well-suited for repeated-measure analysis, providing greater flexibility than traditional analysis of variance (ANOVA). Reference West, Welch and Gatecki24 For instance, linear mixed models can properly account for the correlation between repeated measures of the same participant and handle missing data. Reference Gueorguieva and Krystal25 Studies have suggested that the application of mixed models has greatly increased and that these models have increasingly replaced traditional ANOVA models in psychiatric research. Reference Gueorguieva and Krystal25
To differentiate the effects of exposure in participants with lower and participants with higher baseline suicidal ideation (our third hypothesis), we applied a median split to the sample, using the suicidal ideation scores (median 3) observed prior to website exposure. Accordingly, we stratified the participants into two groups, with one group including those with lower suicidal ideation (i.e. suicidal ideation scores <3) (n = 82, median 2.79, interquartile range (IQR) = 0.41, minimum 2.13, maximum 3.00) and the other group including those with higher suicidal ideation (i.e. suicidal ideation scores >3) (n = 79, median 3.48, IQR = 0.46, minimum 3.02, maximum 4.50). Reference Till, Strauss, Sonneck and Niederkrotenthaler12
Interviews
To get deeper insight into how participants perceived website contents, we conducted a qualitative study with 12 participants (6 women and men each; median age 24 years). Reference Hjelmeland and Knizek26 Two women and men each were allocated to each of the three prevention websites. Interviews lasted 30–60 mins and were mp3-recorded.
Interview schedule
The interview schedule was designed to capture information on participants' perception of the emotional impact and usefulness of the websites. Participants were asked to talk freely about any thoughts related to the website. A set of predetermined questions (for example ‘Please describe what you have seen here’, ‘To what extent were you emotionally involved in the text?’) were used to start and continue the conversation.
Qualitative analysis
Audio recordings were transcribed and analysed following the documentary method. Reference Bohnsack27 This method considers both conversation content and the form in which it is presented. Reference Przyborski and Slunecko28 The documentary method is appropriate to collect and analyse data on topics not easily accessible for self-disclosure (such as media effects), and/or subject to stigmatisation. Reference Till29
Results
There were no significant differences in terms of the participants' gender, age, highest completed education or baseline suicidal ideation across the four groups, as indicated by χ2 and ANOVA tests. See Table 1 for an overview of the demographic characteristics of participants in each group.
Table 1 Descriptive demographic statistics across all four groups (n = 161)
| Demographic variable | Youth-Life-Line | U25 | Friends for Life | Control group | χ2 (d.f. = 3) | ANOVA, F
(d.f. = 3,156) |
|---|---|---|---|---|---|---|
| Women, n (%) | 31 (75.6) | 26 (65.0) | 23 (57.5) | 28 (70.0) | 3.25 | |
| Age, years: mean (s.d.) | 24.5 (6.7) | 23.9 (3.0) | 24.5 (6.0) | 25.0 (6.8) | 0.23 | |
| Highest completed education, n (%) | ||||||
| College | 9 (22.0) | 10 (25.0) | 5 (12.5) | 10 (25.0) | 2.52 | |
| High school | 30 (73.2) | 29 (72.5) | 32 (80.0) | 29 (72.5) | 0.84 | |
| Below high school | 2 (4.9) | 1 (2.5) | 3 (7.5) | 1 (2.5) | 1.64 | |
RCT
An analysis of outcomes in the three intervention groups demonstrated no significant differences between groups (online Table DS2), resulting in the rejection of our second hypothesis regarding the presence of a difference in the impact of the different prevention websites. Based on this finding and the results of the pre-test, which demonstrated that users did not identify website differences in terms of structure, style or content, these three groups were collapsed into a single intervention group for further analysis and the overall presentation of results. Additional information regarding the results obtained for the individual prevention websites is presented in online Tables DS2, DS3 and DS4.
Table 2 displays the means values and corresponding 95% confidence intervals for all study variables before (T 1), immediately after (T 2), and 1 week after website exposure (T 3). Table DS3 presents these data for the three initial intervention groups.
Table 2 Risk factors for suicide and suicide-prevention-related knowledge in the audience before (T 1) and after (T 2) website exposure as well as 1 week later (T 3) (n = 161)
| Mean (95% CI) | |||
|---|---|---|---|
| T 1 | T 2 | T 3 | |
| Primary outcome | |||
| Suicidal ideation (α = 0.85, score range: 1–6) | |||
| Intervention group | 3.11 (3.02–3.20) | 3.03 (2.94–3.11) | 3.07 (2.98–3.15) |
| Control group | 3.04 (2.89–3.19) | 3.01 (2.85–3.17) | 2.97 (2.81–3.13) |
| Secondary outcomes | |||
| Mood (α = 0.82, score range: 1–4) | |||
| Intervention group | 3.28 (3.19–3.37) | 2.98 (2.88–3.08) | 3.29 (3.21–3.38) |
| Control group | 3.21 (3.10–3.31) | 3.12 (2.98–3.26) | 3.32 (3.18–3.45) |
| Attitudes towards suicide (α = 0.78, score range: 0–5) | |||
| Intervention group | 1.83 (1.72–1.94) | 1.80 (1.69–1.90) | 1.80 (1.69–1.91) |
| Control group | 1.73 (1.56–1.90) | 1.80 (1.63–1.97) | 1.80 (1.62–1.97) |
| Attitudes towards seeking professional help (α = 0.76, score range: 0–3) | |||
| Intervention group | 2.10 (2.01–2.19) | 2.14 (2.05–2.23) | 2.14 (2.05–2.24) |
| Control group | 2.06 (1.88–2.23) | 2.09 (1.93–2.26) | 2.12 (1.95–2.29) |
| Suicide-prevention-related knowledge (α = 0.39, score range: 0–1) | |||
| Intervention group | 0.47 (0.42–0.52) | 0.76 (0.72–0.81) | 0.75 (0.71–0.79) |
| Control group | 0.46 (0.38–0.54) | 0.47 (0.38–0.56) | 0.48 (0.40–0.56) |
The mean values and their corresponding 95% confidence intervals stratified by baseline suicidal ideation below and above the sample median (the third hypothesis) are provided in Table 3.
Table 3 Risk factors for suicide and suicide-prevention-related knowledge in the audience before (T 1) and after (T 2) website exposure as well as 1 week later (T 3) stratified for baseline suicidal ideation below and above sample median (n = 161)
| Mean (95% CI) | ||||||
|---|---|---|---|---|---|---|
| Baseline suicidal ideation below the median | Baseline suicidal ideation above the median | |||||
| T 1 | T 2 | T 3 | T 1 | T 2 | T 3 | |
| Primary outcome | ||||||
| Suicidal ideation (score range: 1–6) | ||||||
| Intervention group | 2.69 (2.63–2.76) | 2.70 (2.63–2.78) | 2.77 (2.68–2.86) | 3.51 (3.42–3.60) | 3.33 (3.24–3.42) | 3.34 (3.23–3.45) |
| Control group | 2.70 (2.58–2.82) | 2.67 (2.53–2.79) | 2.66 (2.52–2.80) | 3.50 (3.37–3.63) | 3.47 (3.32–3.62) | 3.40 (3.22–3.58) |
| Secondary outcomes | ||||||
| Mood (score range: 1–4) | ||||||
| Intervention group | 3.35 (3.26–3.45) | 3.04 (2.90–3.18) | 3.33 (3.20–3.45) | 3.21 (3.07–3.35) | 2.92 (2.77–3.07) | 3.26 (3.14–3.39) |
| Control group | 3.29 (3.16–3.43) | 3.25 (3.09–3.41) | 3.36 (3.20–3.52) | 3.09 (2.92–3.26) | 2.95 (2.71–3.19) | 3.26 (3.01–3.50) |
| Attitudes towards suicide (score range: 0–5) |
||||||
| Intervention group | 1.67 (1.52–1.82) | 1.64 (1.50–1.78) | 1.64 (1.48–1.80) | 1.98 (1.84–2.13) | 1.95 (1.80–2.10) | 1.94 (1.79–2.09) |
| Control group | 1.62 (1.34–1.89) | 1.66 (1.39–1.93) | 1.58 (1.31–1.85) | 1.89 (1.72–2.06) | 2.00 (1.85–2.15) | 2.09 (1.95–2.22) |
| Attitudes towards seeking professional help (score range: 0–3) |
||||||
| Intervention group | 2.10 (1.98–2.22) | 2.10 (1.98–2.22) | 2.11 (1.97–2.24) | 2.10 (1.97–2.24) | 2.18 (2.05–2.31) | 2.18 (2.05–2.31) |
| Control group | 2.15 (1.98–2.32) | 2.18 (2.01–2.35) | 2.26 (2.08–2.43) | 1.93 (1.57–2.29) | 1.97 (1.64–2.30) | 1.94 (1.61–2.27) |
| Suicide-prevention-related knowledge (score range: 0–1) |
||||||
| Intervention group | 0.41 (0.34–0.48) | 0.74 (0.67–0.80) | 0.72 (0.66–0.79) | 0.52 (0.46–0.58) | 0.79 (0.73–0.84) | 0.77 (0.72–0.83) |
| Control group | 0.44 (0.33–0.56) | 0.46 (0.33–0.59) | 0.46 (0.34–0.58) | 0.48 (0.35–0.61) | 0.48 (0.35–0.61) | 0.51 (0.38–0.63) |
Table 4 gives an overview of the results of the linear mixed models for all outcome variables. Website exposure had an effect on the primary outcome, i.e. suicidal ideation (significant main effect of time; Table 4). However, this effect was qualified by interactions with baseline suicidal ideation and study group. The interaction between study group with time was not significant. Taken together, these findings led to the rejection of the first hypothesis regarding the primary outcome, as website exposure appeared to have no significant effect on the entire intervention group.
Table 4 Findings from linear mixed models for all outcome variables (n = 161) a
| F (d.f.1, d.f.2) p | |||||||
|---|---|---|---|---|---|---|---|
| Group | Baseline suicidal ideation |
Time | Group × baseline suicidal ideation |
Group × time | Baseline suicidal ideation × time |
Group × baseline suicidal ideation × time |
|
| Primary outcome | |||||||
| Suicidal ideation | 0.02 (1,157.04) 0.887 | 173.35 (1,157.04) < 0.001 | 4.40 (2,156.69) 0.014 | 0.98 (1,157.04) 0.323 | 2.37 (2,156.69) 0.097 | 4.09 (2,156.69) 0.019 | 3.53 (2,156.69) 0.032 |
| Secondary outcomes | |||||||
| Mood | 0.04 (1,156.94) 0.847 | 4.62 (1,156.94) 0.033 | 16.41 (2,156.69) < 0.001 | 0.39 (1,156.94) 0.533 | 3.30 (2,156.69) 0.040 | 0.91 (2,156.69) 0.405 | 0.35 (2,156.69) 0.706 |
| Attitudes towards suicide | 0.00 (1,156.95) 0.993 | 11.63 (1,156.95) 0.001 | 0.47 (2,156.56) 0.626 | 0.13 (1,156.95) 0.724 | 2.70 (2,156.56) 0.070 | 1.56 (2,156.56) 0.213 | 2.55 (2,155.48) 0.081 |
| Attitudes towards seeking professional help | 0.37 (1,156.92) 0.542 | 1.17 (1,156.92) 0.282 | 2.41 (2,156.48) 0.093 | 2.96 (1,156.92) 0.087 | 0.12 (2,156.48) 0.886 | 1.06 (2,156.48) 0.348 | 1.81 (2,156.48) 0.166 |
| Suicide-prevention-related knowledge | 21.71 (1,157.01) < 0.001 | 1.70 (1,157.01) 0.194 | 29.30 (2,156.80) < 0.001 | 0.19 (1,157.01) 0.668 | 24.30 (2,156.80) < 0.001 | 0.46 (2,156.80) 0.632 | 0.31 (2,156.80) 0.738 |
a. Values are F- and P-values with degrees of freedom (d.f.1, d.f.2) given in parentheses from linear mixed models representing the change of the respective outcome variable with regard to group (intervention v. control group), baseline suicidal ideation (low v. high suicidal ideation/vulnerability), time (T 1, T 2, T 3), and interactions between these factors.
With regard to the secondary outcomes, a short-term worsening of mood was observed immediately after exposure to the suicide prevention websites (contrast test T 2 v. T 1: Bonferroni-corrected P < 0.001, d = −0.59, 95% CI −0.75 to −0.43) that was not sustained at 1 week after website exposure (contrast test T 3 v. T 1. Bonferroni-corrected P = 1.00, d = 0.03, 95% CI −0.13 to 0.19). Among control group participants, no worsening of mood was identified (contrast test T 2 v. T 1: Bonferroni-corrected P = 0.59, d = −0.10, 95% CI −0.26 to 0.05); contrast test T 3 v. T 1. Bonferroni-corrected P = 0.33, d = 0.13, 95% CI −0.03 to 0.29). The analysis of the impact of website exposure also revealed a significant group × test condition interaction for suicide-prevention-related knowledge. In the intervention group, an increase in knowledge was observed after exposure to the website (contrast test T 2 v. T 1: Bonferroni-corrected P < 0.001, d = 1.12, 95% CI 0.97–1.28). This effect was preserved 1 week after website exposure (contrast test T 3 v. T 1: Bonferroni-corrected P < 0.001, d = 1.12, 95% CI 0.96–1.28). In the control group, knowledge related to suicide prevention did not change (contrast test T 2 v. T 1. Bonferroni-corrected P = 1.00, d = 0.02, 95% CI −0.14 to 0.18); contrast test T 3 v. T 1: Bonferroni-corrected P = 1.00, d = 0.04, 95% CI−0.11 to 0.20).
In the intervention group, there were no significant short-term or sustained effects of website exposure on attitudes towards suicide (contrast test T 2 v. T 1: Bonferroni-corrected P = 0.56, d = −0.11, 95% CI −0.26 to 0.05); contrast test T 3 v. T 1: Bonferroni-corrected P = 0.87, d = −0.08, 95% CI −0.24 to 0.07) or on attitudes towards seeking professional help (contrast test T 2 v. T 1. Bonferroni-corrected P = 0.16, d = 0.16, 95% CI −0.002 to 0.31); contrast test T 3 v. T 1: Bonferroni-corrected P = 0.44, d = 0.12, 95% CI −0.04 to 0.28).
With regard to the primary outcome, a significant three-way interaction between group, time and baseline suicidal ideation was identified. The contrast tests indicated an immediate reduction in suicidal ideation after exposure to the prevention websites among participants with baseline suicidal ideation scores above the median (contrast test T 2 v. T 1: Bonferroni-corrected P < 0.001, d = −0.55, 95% CI −0.71 to −0.39), whereas no impact on suicidal ideation was observed among individuals with baseline scores below the median (contrast test T 2 v. T 1: Bonferroni-corrected P = 1.00, d = 0.04, 95% CI −0.12 to 0.20) or among control group participants (with baseline suicidal ideation above the median: contrast test T 2 v. T 1: Bonferroni-corrected P = 1.00, d = −0.04, 95% CI −0.20 to 0.12); with baseline suicidal ideation below the median: contrast test T 2 v. T 1: Bonferroni-corrected P = 1.00, d = −0.06, 95% CI −0.22 to 0.09). The reduction in suicidal ideation among intervention group participants with comparatively high baseline suicidal ideation scores remained significant 1 week after website exposure (contrast test T 3 v. T 1: Bonferroni-corrected P < 0.001, d = −0.34, 95% −0.50 to −0.19). Thus, our third hypothesis was supported. Of note, the changes observed in suicidal ideation between T 2 and T 3 in this subgroup were negligible (contrast test T 3 v. T 2: Bonferroni-corrected P = 1.00, d = 0.03, 95% CI −0.13 to 0.18).
We performed an additional analysis to determine whether the decrease identified in suicidal ideation in the subgroup of more vulnerable individuals would also be present when the three groups exposed to suicide prevention websites were not collapsed into one intervention group. The three-way interaction between group, time and baseline suicidal ideation was also significant in this additional analysis (Table DS4).
Interview findings
When directly asked about the impact of websites, nearly all interviewees highlighted that website exposure did not have a significant impact on their mood. However, despite this generally reported absence of emotional impact, some participants in the course of the interviews noted that several website aspects (for example reports on the prevalence of suicide, stories of the bereaved) resulted in feelings of discomfort and negative emotions:
‘I am still surprised or shocked how frequent this [suicide] is as I read those numbers …’ (Male 1, U25)
There was broad agreement among interviewees that the websites were helpful for individuals with suicidal ideation and for family and friends seeking information on suicide:
‘I think ifs certainly relieving to see something like this [the website] and to see that there is help and there are people who feel the same way.’ (Female 1, Youth-Life-Line)
Most participants mentioned that they had learned information that was new to them. See online Table DS5 for additional examples of the narratives produced by the interviewees.
Discussion
This study indicates that the educative components of suicide prevention websites supported by professional organisations and designed to target adolescents and young adults had a beneficial impact on users. In particular, individuals with suicidal ideation scores above the sample median demonstrated a reduction in suicidal ideation that partially persisted 1 week after exposure. A sustained increase in suicide-prevention-related knowledge was also observed.
Interpretation of our findings
There are several potential explanations for the observed reduction in suicidal ideation among study participants with baseline suicidal ideation scores above the median. In particular, the educative websites may have empowered participants (particularly those with increased vulnerability) to modify their perceptions regarding their ability to cope constructively with suicidal ideation. This effect may have been derived from the participants learning about what can be done should they ever be unable to cope with a difficult life situation. This explanation is supported by the fact that the reductions observed in suicidal ideation were particularly prominent on the survival and coping beliefs subscale of the Reasons for Living Scale, Reference Linehan, Goodstein, Nielsen and Chiles19 which was used to assess suicidal ideation. This subscale comprises items that primarily gauged the perceptions of hope and empowerment among participants (such as ‘I believe I can learn to adjust or cope with my problems’). Reference Linehan, Goodstein, Nielsen and Chiles19 The notion that the website materials could reinforce the perception of ‘not being alone’ when involved in a crisis situation was also brought up during the qualitative interviews. Previous research has demonstrated that users of suicide-related websites may feel less alienated and suicidal because of their online activities. Reference Harris, McLea and Sheffield30
In a similar vein, the present results lend further support to the presence of the Papageno effect, Reference Niederkrotenthaler, Voracek, Herberth, Till, Strauss and Etzersdorfer11 which (as opposed to the Werther effect) describes media as having a suicide-protective impact. Potentially suicide-protective effects were first described in association with newspaper stories of individuals coping with adverse circumstances. Reference Niederkrotenthaler, Voracek, Herberth, Till, Strauss and Etzersdorfer11 The websites evaluated in the present study clearly highlighted the feet that individuals with suicidal ideation or experiencing adverse circumstances were not alone and further provided examples of individuals with lived experience of suicidality. These characteristics may have contributed to an increase in survival beliefs and corresponding reduction in suicidal ideation.
The sustainability of the beneficial effects observed regarding suicidal ideation remains to be fully elucidated. Adverse rebound effects on the discontinuation of interventions designed to prevent suicide have been described elsewhere and need to be considered. Reference Smith, Austin, Kim, Miller, Eisen and Christiansen31,Reference Tondo and Baldessarini32 In the present study, the increase observed in suicidal ideation 1 week after exposure to the prevention website was marginal, non-significant and disproportionate to the initially observed decrease. A decrease in initial effects may be expected after a one-time, media-based intervention. Future research should, therefore, be conducted to investigate the longer-term effects of repeated exposure to educative materials.
Our finding indicating that the more vulnerable individuals in the sample benefitted most from the intervention suggests that further studies are warranted to test the effects in individuals who currently experience psychosocial crises or mental illness. These individuals may experience the greatest reduction in suicidal ideation when exposed to a prevention website, and the present findings strengthen the case for trials focusing on at-risk populations.
The websites did not only have an impact on suicidal ideation but also on mood. Participants appeared to be sadder after exposure to the prevention website than they were prior to intervention exposure. This finding is in line with evidence demonstrating a worsening of mood after exposure to films that conclude with the protagonist's suicide. Reference Till, Strauss, Sonneck and Niederkrotenthaler12,Reference Biblarz, Brown, Biblarz, Pilgrim and Baldree22,Reference Till29,Reference Till, Niederkrotenthaler, Herberth, Vitouch and Sonneck33,Reference Till, Niederkrotenthaler, Herberth, Voracek, Sonneck and Vitouch34 However, in contrast with the findings related to suicidal ideation in the present study, the worsening of mood was short-lived, and mood scores had rebounded back to baseline levels when assessed 1 week later. This observation is consistent with research demonstrating that the effects of media exposure on mood are temporary. Reference Biblarz, Brown, Biblarz, Pilgrim and Baldree22,Reference Tan35 In the qualitative interviews, participants generally tended to trivialise the impact of the websites on their mood when directly asked about this intervention effect, a finding which may be the result of psychological defence mechanisms and stem from the third-person effect, which suggests that individuals exposed to mass media tend to underestimate the media's impact on their own emotions, attitudes and behaviours. Reference Davison36 However, negative impacts on mood were acknowledged during the interviews when specific aspects presented on the websites were discussed.
The increase observed in suicide-prevention-related knowledge indicates that the information provided on the websites was learned and remembered by participants, showing that websites can effectively be used to educate the public. Educative websites, however, did not influence the users' attitudes towards suicide and towards seeking professional psychological help, which was similar to findings reported in previous research. Reference Biblarz, Brown, Biblarz, Pilgrim and Baldree22,Reference Till29,Reference Till, Niederkrotenthaler, Herberth, Vitouch and Sonneck33,Reference Till, Niederkrotenthaler, Herberth, Voracek, Sonneck and Vitouch34 Attitudes and beliefs may be too robust to change after brief exposure to prevention material. Reference Erber, Hodges, Wilson, Petty and Krosnick37
Limitations
The study has some limitations. First, individuals at high risk for suicide as indicated by a hopelessness score >33 were excluded from participation. Reference Krampen16–Reference McMillan, Gilbody, Beresford and Neilly18 It remains unknown whether and how our findings would generalise to suicidal individuals. A further limitation was the fact that our internet search for suicide-prevention websites was not exhaustive. We were able to test the effects of three such websites, which were rated as relatively similar in terms of structure and content. However, there may be other professional educative websites that approach suicide prevention in a different way that, in turn, may lead to different effects. Further, the characteristics of study participants, although being in accordance with the age of the target population of the organisations hosting the included websites, were not representative of the general population, as young, female and highly educated individuals were overrepresented. The reliability (in terms of Cronbach α, i.e. lower-bound reliability) of the items assessing suicide-prevention-related knowledge was low (α = 0.39). This is a consequence of content heterogeneity across this subset of items and, more broadly, a known psychometric limitation of the entire scale. Reference Voracek, Tran and Sonneck21
Another study limitation was the relatively short follow-up interval of 1 week. Further studies are warranted to evaluate the longer-term effects of the intervention when exposure is repeated over time. Further, trial registration did not occur prior to the recruitment of participants. The present study employed a single-blinded design, but masking was not assessed through a guess test or any other means. It is also possible that participants may have been prone to bias their responses related to suicide-related outcomes, given that this may have been the noticeable focus of the study. To reduce this bias, participants were not informed about the study aim until the end of their participation. Finally, with 40 participants in the control group, the sample size available for analysis was limited although not underpowered. Hence, larger replication and extension studies are warranted.
Implications
Mental health organisations are employing online resources to educate the public about suicide and offer help to individuals in need. The present study highlights the fact that the educative components of websites targeting young individuals in crisis may demonstrate suicide-preventive effects among individuals with some degree of vulnerability. Further, these websites seem effective in yielding a sustainable increase in suicide-related knowledge. The educative components of the tested websites appear to be relevant additions to the clinical and other interactive services offered for suicide prevention.
Funding
This work was funded by the Austrian Science Fund (FWF), grant number , to T.N. The funder had no influence on study design, the collection, analysis and interpretation of data, the writing of the report; or the decision to submit the article for publication.
eLetters
No eLetters have been published for this article.