Hostname: page-component-586b7cd67f-rcrh6 Total loading time: 0 Render date: 2024-11-29T05:33:33.719Z Has data issue: false hasContentIssue false

Strategies for preventing occupational stress in healthcare workers: past evidence, current problems

Published online by Cambridge University Press:  23 April 2021

Riccardo De Giorgi*
Affiliation:
Wellcome Trust Doctoral Training Fellow in the Department of Psychiatry, University of Oxford, and an Honorary Clinical Fellow with Oxford Health NHS Foundation Trust, UK. He works on experimental medicine trials in mood disorders. He is interested in the evidence-based treatment of mental illness and the neuroscientific underpinnings of psychopharmacology.
Bianca M. Dinkelaar
Affiliation:
holds an MPhil from the Faculty of Classics, University of Oxford, UK, and works on psychological and sociological approaches to ancient philosophy and religion. Her research interests and publication topics include medical ethics and mental health.
*
Correspondence Riccardo De Giorgi. Email: [email protected]
Rights & Permissions [Opens in a new window]

Summary

The problem of occupational stress in healthcare workers is hardly new, but effective interventions in this area are lacking despite being sorely needed – especially in the ongoing COVID-19 pandemic. The results of a Cochrane review suggest that cognitive–behavioural therapy and mental and physical relaxation reduce stress more than no intervention but not more than alternative interventions, and that changing work schedules may lead to a reduction of stress. Other organisational interventions showed no effect on stress levels. However, the evidence is of low quality owing to risk of bias and lack of precision. This commentary critically appraises the review and attempts to put its findings into the current real-world context.

Type
Round the corner
Copyright
Copyright © The Authors 2021

Stress is a state of mental or emotional strain or tension the paraphysiological function of which is to trigger a fight-or-flight reaction from the body in the face of a threat (National Institute of Mental Health 2020). However, long-term stress prevents the body from returning to its normal functioning and is typically characterised by exhaustion, a sense of reduced effectiveness and decreased motivation, leading to the development of dysfunctional attitudes and behaviours in a range of functional domains (Ruotsalainen Reference Ruotsalainen, Verbeek and Mariné2015). When stress is either (or both) caused by or expressed in someone's work environment, we consider this ‘occupational stress’. Since occupational stress has both a health and an economic impact, it is in the interest of governments and healthcare institutions to find the most effective ways of avoiding it.

Occupational stress in healthcare workers: an underresearched problem showing a compelling new interest

Studies have shown that levels of occupational stress in healthcare workers are high, with the healthcare sector having one of the highest estimated prevalence rates of work-related stress of all occupations in the UK (Health and Safety Executive 2020). Occupational stress may lead to burnout and psychosomatic illness, and therefore to reduced quality of life in healthcare workers and worse healthcare service provision (Weinberg Reference Weinberg and Creed2000). This could have a devastating ‘snowball effect’ on the overall health and quality of life of the general public.

Research in this area has hitherto been little and insufficient; however, this issue has recently come to the foreground owing to an increased interest in the well-being of healthcare workers during the COVID-19 pandemic (Preti Reference Preti, Di Mattei and Perego2020). Nevertheless, among the plethora of papers advocating the need for immediate action, the number of concrete intervention strategies proposed remains disappointingly low.

Preventing versus treating stress (and the importance of its causes)

The development of preventive approaches (Box 1) for mental illness is considered the ‘holy grail’ of psychiatric care (Rice-Oxley Reference Rice-Oxley2019): an idyllic goal that is more desirable than treatment itself, yet hard to achieve, especially since we have not been able to determine specific causative factors for most mental disorders. However, stress (and stress-related disorders) is unique in the sense that its source, no matter how complex, can usually be identified: for example, a significant traumatic event in post-traumatic stress disorder. Therefore, stress-related disorders may be avoided through targeted preventive interventions. Indeed, the adoption of preventive strategies in the workplace is advocated as crucial for the effective management of occupational stress (Jordan Reference Jordan, Gurr and Tinline2003).

BOX 1 Prevention

Prevention is often described in lay terms as a strategy applied to a non-ill population to avoid the onset of that illness in that population; for example, smoking-cessation aids are a preventive measure against lung cancer in smokers who have not developed lung cancer yet. In epidemiology, however, the concept of prevention is more complex and is subdivided in primary, secondary and tertiary prevention.

Primary prevention includes any approach that protects the health of people who have not yet become sick – in practice, what we described just above.

Secondary prevention refers to the early recognition and management of a paraphysiological or pathological abnormality that would otherwise almost inevitably lead to a full-blown disorder; for example, reversing high blood pressure to avoid a later myocardial infarction. Arguably, secondary prevention could already qualify as a form of ‘treatment’.

Tertiary prevention involves measures that curb or sometimes remove the complications of an illness only after that illness has already occurred; for example, physical rehabilitation following a stroke. Again, tertiary prevention can be seen as another form of ‘treatment’.

The causes of stress in healthcare workers may differ from the causes of stress in other occupations, for example the risk presented by close contact with diseases, the emotional response to contact with death and suffering, and specific organisational problems and conflicts (Ruotsalainen Reference Ruotsalainen, Verbeek and Mariné2015). We may question from the start whether interventions directed at stress prevention in general can be at all effective when they are not tailored to the specific underlying causes of stress. The authors of this month's Cochrane Corner review (Ruotsalainen Reference Ruotsalainen, Verbeek and Mariné2015) seem to acknowledge the importance of this problem and therefore aimed to include only studies on healthcare workers; however, it is unclear whether the interventions had been devised with the specific needs of this population in mind.

Moreover, the COVID-19 pandemic has brought with it additional causes of stress for healthcare workers. Not only have workload and personal health risk increased, but healthcare workers are also confronted with societal stigmas: a fear of healthcare workers spreading the virus has led to social ostracism and even harassment (Bouchard Reference Bouchard2020); conversely, the promotion of ‘NHS heroes’ has built unrealistic expectations of the duty of healthcare workers to the public (Cox Reference Cox2020). Thankfully, the increased fame of healthcare professionals has also been accompanied by a renewed interest in their mental health: over recent months new research has been published and novel prevention strategies proposed (Blake Reference Blake, Bermingham and Johnson2020).

The ‘PICO’ of the Cochrane review

The objective of the review (Ruotsalainen Reference Ruotsalainen, Verbeek and Mariné2015) was to assess the effectiveness of preventing occupational stress (outcome) in healthcare workers (population) comparing work- and person-directed interventions (intervention) with no intervention or alternative intervention (comparison). Below, we go into the details of this ‘PICO’ (Box 2).

BOX 2 ‘PICO’

PICO stands for population, intervention(s), comparison(s) and outcome(s). It is the best way to formulate a clear clinical query to which a research study would like to find a meaningful answer. Today, best research practice recommends that both the study title and aims should follow the PICO structure, to ensure that they are most informative.

Participants were further defined as healthcare workers who had not received treatment for mental illness (e.g. burnout, depression, anxiety disorder), which is appropriate for a study that focuses on prevention.

The review authors included the following interventions: cognitive–behavioural techniques (person-directed, providing better ways to think, behave and feel in stressful situations), relaxation techniques (person-directed, averting attention from stress and building resilience) and organisational interventions (work-directed, reducing the occurrence or impact of stressful events by amending work practices). It is unclear why other potentially useful interventions, such as counselling or psychodynamic therapy (Reynolds Reference Reynolds2000), were not considered. Mixed interventions, for example cognitive–behavioural plus relaxation techniques, were accounted for. A further distinction between physical and mental relaxation techniques was described (and analysed accordingly) only in the Results section, thus appearing as a post hoc assessment; however, findings were also presented for relaxation techniques as a whole. The review authors themselves recognised that, for some interventions, such as organising peer-support groups or mentoring schemes, categorisation as either person- or work-directed was hard. We would also argue that pooling or comparing these different interventions can be problematic: physical relaxation techniques such as massage are more easily seen as a treatment rather than a preventive strategy, whereas mental relaxation (e.g. mindfulness), cognitive–behavioural techniques and organisational interventions are more likely to have both preventive and treatment facets.

The outcomes were assessed at <1 month, 1–6 months and >6 months – a sensible choice since the interventions involved would likely show quite different effects at short versus longer follow-ups; yet, the review authors later acknowledged that measuring all outcomes at either less or more than 1 month would be reasonable too.

For the primary outcome, any previously validated self-reported questionnaire evaluating occupational stress or burnout (most commonly, the Maslach Burnout Inventory (Maslach Reference Maslach, Jackson and Leiter1996)) was used. Most of the secondary outcomes, whose discussion would go beyond the space of this commentary, similarly involved subjective measures of anxiety and depressive symptoms. The review authors deliberately excluded outcomes that, in their opinion, did not directly assess stress or its consequences for individuals; however, we argue that some of these measures, such as absenteeism, could have gauged the interventions’ effectiveness very objectively, hence reducing bias issues – particularly with masking (‘blinding’), a very important concern with this kind of research. None of the included studies reported figures on cost-effectiveness, which is unfortunate since these data could more immediately support the implementation of beneficial interventions, especially at an organisational level.

The ‘nuts and bolts’ of the review

This review (Ruotsalainen Reference Ruotsalainen, Verbeek and Mariné2015) was an update of a previous one (Marine Reference Marine, Ruotsalainen Jani and Serra2006) by the same research group. Their revised search strategy appears very thorough: more electronic databases were explored and the additional hand-search comprised reference lists as well as all issues of the specialist journal Work & Stress.

Publication bias was assessed in intervention comparisons containing five or more studies, but the assessment may not have been sufficiently powered, as the Cochrane Handbook recommends a minimum of ten studies (Higgins Reference Higgins and Thomas2020).

The study types included only randomised controlled trials (RCTs) for cognitive–behavioural and relaxation techniques, whereas for organisational interventions cluster RCTs and prospective cohort studies were considered too. Cross-over trials were not excluded, provided that they had a ‘sufficient wash-out period’ according to the review authors; however, the very notion of ‘wash-out’ seems questionable for interventions that aim to teach, possibly with lasting efficacy, strategies for preventing stress.

Data were collected, extracted and analysed by calculating effect sizes (generally, standardised mean difference or s.m.d.) with 95% confidence intervals (CI), in agreement with best practice. The review authors made a commendable effort to obtain missing data from numerous studies.

Heterogeneity was calculated and considered ‘substantial’ when >50%. The review authors perplexingly asserted that, in the latter case, the ‘most likely explanation is that there are data input errors’.

The assessment of the risk of bias was performed according to the standard at the time (Higgins Reference Higgins and Green2011). Items for ‘blinding’ were not used because none of the included studies did mask participants or intervention providers and most outcomes were from self-reported questionnaires. The review authors agreed with the studies’ researchers in maintaining that ‘blinding is impossible’ in this type of study; however, we argue that masking would be ‘difficult’ rather than impossible, particularly with the small sample sizes of the included trials, as demonstrated by other studies employing comparable psychosocial interventions.

Sensitivity analyses (Box 3) were conducted with the exclusion of studies deemed at high risk of bias. Subgroup analysis (Box 3) was performed to differentiate between nurses, physicians and other healthcare professionals; we believe that this subgrouping was scarcely informative, whereas distinguishing between ‘clinical settings’ would lead to more valuable findings, as it is plausible that occupational stress is very dissimilar in, for example, emergency services compared with paediatrics or general medicine.

BOX 3 Sensitivity analysis versus subgroup analysis

A sensitivity analysis is a statistical tool that allows the interrogation of data with ‘What if?’ questions. The resulting findings will not reflect the entirety of the collected data, but may strengthen the validity of the study's results or shed some light on additional research questions. For example, in this Cochrane review the authors asked themselves ‘What if we only had data from higher-quality studies?’. They therefore eliminated highly biased studies from their analysis; their sensitivity analysis results remained in line with the findings from their primary analysis.

A subgroup analysis instead asks the same research question of the primary analysis by probing data for specific subsets of the whole study population. For example, in this Cochrane review the authors performed several additional subgroup analyses to see whether the included interventions showed any different efficacy for the subgroups of nurses, physicians and other healthcare professionals.

The results of the search

The literature search yielded 58 records. The studies’ flowchart as well as the main text of the review detail the screening and selection process in a somewhat confusing fashion (an example of a well-presented flowchart is shown in Fig. 1). Some of the studies had been published before the previous Cochrane review (Marine Reference Marine, Ruotsalainen Jani and Serra2006) and yet had not been captured, thus confirming the higher sensitivity of this updated review (Ruotsalainen Reference Ruotsalainen, Verbeek and Mariné2015). Most of the articles were from after 2010, possibly showing a growing interest in this topic – a trend easily confirmed by performing a similar search today. Three trials (Spoor Reference Spoor, De Jonge and Hamers2010; Gómez-Gascón Reference Gómez-Gascón, Martín-Fernández and Gálvez-Herrer2013; Niks Reference Niks, De Jonge and Gevers2013) were ongoing at the time; of these, only Niks et al's results have been made available (Niks Reference Niks, de Jonge and Gevers2018), reporting the benefit of an organisational intervention on healthcare staff (refer to the paper for further details).

FIG 1 A flowchart or flow diagram represents the best practice, according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, for describing the process that led from the initial literature search to the choice of studies included in the final analysis. Today, all well-performed systematic reviews and meta-analyses would report a PRISMA flowchart (usually as ‘Fig. 1’) in the body of the article. A detailed flowchart should allow any researcher to replicate the findings, in terms of the included articles, of the systematic review at a specified time. A PRISMA flowchart usually comprises at least four rows (as shown in this example). The first row (‘Identification’) often depicts two boxes: one containing the number of records retrieved via electronic databases and the other those added through a manual search. These are often merged in a single box, with duplicate records from different databases removed. The second row (‘Screening’) should have a box with the number of records whose title and abstract have been screened, according to pre-specified inclusion/exclusion criteria, and a box with those records excluded on this basis, with reason. The third row (‘Eligibility’) replicates what was done in the second row, but this time the full text of the articles is assessed. Finally, the fourth row (‘Included’) reports the number of studies that will eventually be analysed.

An adequate number of studies for each outcome's timescale was retrieved (i.e. <1 month: 24 studies; 1–6 months: 22 studies; >6 months: 12 studies). The review authors may be praised for their industry in distinguishing the (sometimes) ambiguous intervention and comparison arms so that more data could be used for their meta-analyses.

The review's population consisted of 7188 participants, of whom 3592 were in the various intervention groups and 3596 in the control groups. Sample sizes were very diverse across studies, ranging from <20 to >300. It is worth noting that, and unclear why, most interventions were directed at nurses, with very few studies involving physicians.

The evidence: do cognitive–behavioural techniques, relaxation techniques and organisational interventions prevent occupational stress in healthcare workers?

For a detailed report of all pooled results, we refer to the review's full text (Ruotsalainen Reference Ruotsalainen, Verbeek and Mariné2015). It should be noted that, even for the primary outcome, many different scales were used.

Cognitive–behavioural techniques were more effective in preventing occupational stress than no intervention, but not than any alternative intervention, only at follow-ups longer than 1 month, namely at 1–6 months (8 studies, 549 participants, s.m.d. = −0.38, 95% CI −0.59 to −0.16) and even more so at >6 months (2 studies, 157 participants, s.m.d. = −1.04, 95% CI −1.37 to −0.70).

Relaxation techniques showed similar effects already at <1 month (4 studies, 97 participants, s.m.d. = −0.48, 95% CI −0.89 to 0.08), as well as at 1–6 months (12 studies, 521 participants, s.m.d. = −0.49, 95% CI −0.78 to −0.21) and >6 months (1 study, 40 participants, s.m.d. = −1.89, 95% CI −2.65 to −1.13). Physical relaxation techniques showed a slightly more positive, yet non-significant better trend than their mental relaxation counterparts.

Organisational interventions differed very much from each other and therefore the large majority of data could not be pooled for meta-analysis. In their results summary, the review authors mentioned two studies showing that a change in work schedules reduced occupational stress; however, we could only retrieve three separate records that reported a significant effect on stress levels for three different interventions: respectively, Ewers et al (Reference Ewers, Bradshaw and McGovern2002) (training programme for handling behaviourally problematic patients, 20 participants, s.m.d. = −1.23, 95% CI −2.21 to −0.26;); Bourbonnais et al (Reference Bourbonnais, Brisson and Vézina2011) (changing working conditions, 488 participants, s.m.d. = −0.38, 95% CI −0.56 to −0.20); and Peterson et al (Reference Peterson, Bergstrom and Samuelsson2008) (peer-support groups, 131 participants, s.m.d. = −0.38; 95% CI −0.73 to −0.03).

Comparisons of cognitive–behavioural techniques, relaxation techniques and organisational interventions with any other intervention were mostly hard to calculate, inconsistent or non-significantly different – the last tentatively suggesting that ‘doing something/anything’ is better than ‘doing nothing’ for preventing occupational stress.

Sensitivity and subgroup analyses, where performed, did not change the outcomes as described.

The quality of the evidence was mainly in the low range, especially due to the elevated risk of bias. Most of the included studies were methodologically poor. However, one study (Günüsen Reference Günüsen and Ustun2010) was of high methodological quality thanks to a low risk of bias in randomisation, allocation concealment, incomplete outcome data and selective reporting – in our view, proving that high-quality studies may be difficult, but still feasible, in this research area.

Conclusions

In summary, the interventions included in this Cochrane review (Ruotsalainen Reference Ruotsalainen, Verbeek and Mariné2015), namely cognitive–behavioural techniques, relaxation techniques and organisational changes, could be useful for preventing stress in healthcare workers – although from a statistical perspective and in the presence of high risk of bias.

The interpretation of the clinical significance of these findings, however, is more challenging. Even the most commonly used and widely validated scale for measuring occupational stress, namely the Maslach Burnout Inventory (Maslach Reference Maslach, Jackson and Leiter1996), has no generally accepted change regarded as clinically relevant. The Cochrane review authors went to the effort of contacting the inventory's authors, but could not clarify this matter – hence, we would reiterate our suggestion for future studies to consider adding proxy outcome measures, such as days of sick leave due to stress. However, the review authors correctly used standardised mean differences to gauge the significance of the effect sizes.

The question of adherence to these interventions, especially if not targeted at the healthcare workers’ specific needs and circumstances, is in our view of crucial importance.

Overall, the prevention of occupational stress in healthcare workers remains a problem of great relevance, especially in the context of the COVID-19 pandemic; therefore, further research is certainly warranted.

Author contributions

R.D.G. is responsible for the ideation, design and write up of the manuscript. B.M.D. is responsible for the literature review and for supporting with the design of the manuscript and its revision.

Declaration of interest

None.

ICMJE forms are in the supplementary material, available online at https://doi.org/10.1192/bja.2020.90.

Footnotes

Commentary on… Preventing occupational stress in healthcare workers (Cochrane Corner). See this issue.

References

Blake, H, Bermingham, F, Johnson, G, et al. (2020) Mitigating the psychological impact of COVID-19 on healthcare workers: a digital learning package. International Journal of Environmental Research and Public Health, 17(9): 2997.CrossRefGoogle ScholarPubMed
Bouchard, JP (2020) Covid-19: les soignants entre héroïsation et ostracisation [Covid-19: health care workers between heroism and ostracism]. Revue de L'infirmière, 69: 31–2.CrossRefGoogle Scholar
Bourbonnais, R, Brisson, C, Vézina, M (2011) Long-term effects of an intervention on psychosocial work factors among healthcare professionals in a hospital setting. Occupational and Environmental Medicine, 68(7): 479–86.CrossRefGoogle ScholarPubMed
Cox, CL (2020) ‘Healthcare Heroes’: problems with media focus on heroism from healthcare workers during the COVID-19 pandemic. Journal of Medical Ethics, 46: 510–3.CrossRefGoogle ScholarPubMed
Ewers, P, Bradshaw, T, McGovern, J, et al. (2002) Does training in psychosocial interventions reduce burnout rates in forensic nurses? Journal of Advanced Nursing, 37: 470–6.CrossRefGoogle ScholarPubMed
Gómez-Gascón, T, Martín-Fernández, J, Gálvez-Herrer, M, et al. (2013) Effectiveness of an intervention for prevention and treatment of burnout in primary health care professionals. BMC Family Practice, 14(1): 173.CrossRefGoogle ScholarPubMed
Günüsen, N, Ustun, B (2010) An RCT of coping and support groups to reduce burnout among nurses. International Nursing Review, 57: 485–92.CrossRefGoogle ScholarPubMed
Health and Safety Executive (2020) Work-Related Stress, Anxiety or Depression Statistics in Great Britain, 2020. Health and Safety Executive (https://www.hse.gov.uk/statistics/causdis/stress.pdf).Google Scholar
Higgins, JPT, Green, S (eds) (2011) Cochrane Handbook for Systematic Reviews of Interventions, Version 5.1.0. Cochrane Collaboration.Google Scholar
Higgins, JPT, Thomas, J (eds) (2020) Cochrane Handbook for Systematic Reviews of Interventions, Version 6.1. Cochrane Collaboration.Google Scholar
Jordan, J, Gurr, E, Tinline, G, et al. (2003) Beacons of Excellence in Stress Prevention (Research Report 133). Health and Safety Executive.Google Scholar
Marine, A, Ruotsalainen Jani, H, Serra, C, et al. (2006) Preventing occupational stress in healthcare workers. Cochrane Database of Systematic Reviews, 11: CD002892. Available from: https://doi.org/10.1002/14651858.CD002892.pub2.CrossRefGoogle Scholar
Maslach, C, Jackson, SE, Leiter, SP (1996) Maslach Burnout Inventory Manual (3rd Edn). Consulting Psychologists Press.Google Scholar
National Institute of Mental Health (2020) 5 Things You Should Know about Stress. NIMH (https://www.nimh.nih.gov/health/publications/stress/index.shtml). Accessed 5 October 2020.Google Scholar
Niks, IMW, De Jonge, J, Gevers, JMP, et al. (2013) Design of the DISCovery project: tailored work-oriented interventions to improve employee health, well-being, and performance-related outcomes in hospital care. BMC Health Services Research, 13: 66.CrossRefGoogle ScholarPubMed
Niks, IMW, de Jonge, J, Gevers, JMP, et al. (2018) Work stress interventions in hospital care: effectiveness of the DISCovery method. International Journal of Environmental Research and Public Health, 15(2): 332.CrossRefGoogle ScholarPubMed
Peterson, U, Bergstrom, G, Samuelsson, M, et al. (2008) Reflecting peer-support groups in the prevention of stress and burnout: randomized controlled trial. Journal of Advanced Nursing, 63: 506–16.CrossRefGoogle ScholarPubMed
Preti, E, Di Mattei, V, Perego, G, et al. (2020) The psychological impact of epidemic and pandemic outbreaks on healthcare workers: rapid review of the evidence. Current Psychiatry Reports, 22(8): 43.CrossRefGoogle Scholar
Reynolds, S (2000) Interventions: what works, what doesn't? Occupational Medicine, 50: 315–9.CrossRefGoogle ScholarPubMed
Rice-Oxley, M (2019) Prevention: the new holy grail of treating mental illness. The Guardian, 5 Oct (https://www.theguardian.com/society/2019/jun/08/prevention-the-new-holy-grail-of-treating-mental-illness).Google Scholar
Ruotsalainen, JH, Verbeek, JH, Mariné, A, et al. (2015) Preventing occupational stress in healthcare workers. Cochrane Database of Systematic Reviews, 4: CD002892. Available from: https://doi.org/10.1002/14651858.CD002892.pub5.Google Scholar
Spoor, E, De Jonge, J, Hamers, JP (2010) Design of the DIRECT-project: interventions to increase job resources and recovery opportunities to improve job-related health, well-being, and performance outcomes in nursing homes. BMC Public Health, 10: 293.CrossRefGoogle ScholarPubMed
Weinberg, A, Creed, F (2000) Stress and psychiatric disorder in healthcare professionals and hospital staff. Lancet, 355: 533–7.CrossRefGoogle ScholarPubMed
Figure 0

FIG 1 A flowchart or flow diagram represents the best practice, according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, for describing the process that led from the initial literature search to the choice of studies included in the final analysis. Today, all well-performed systematic reviews and meta-analyses would report a PRISMA flowchart (usually as ‘Fig. 1’) in the body of the article. A detailed flowchart should allow any researcher to replicate the findings, in terms of the included articles, of the systematic review at a specified time. A PRISMA flowchart usually comprises at least four rows (as shown in this example). The first row (‘Identification’) often depicts two boxes: one containing the number of records retrieved via electronic databases and the other those added through a manual search. These are often merged in a single box, with duplicate records from different databases removed. The second row (‘Screening’) should have a box with the number of records whose title and abstract have been screened, according to pre-specified inclusion/exclusion criteria, and a box with those records excluded on this basis, with reason. The third row (‘Eligibility’) replicates what was done in the second row, but this time the full text of the articles is assessed. Finally, the fourth row (‘Included’) reports the number of studies that will eventually be analysed.

Supplementary material: PDF

De Giorgi and Dinkelaar supplementary material

De Giorgi and Dinkelaar supplementary material

Download De Giorgi and Dinkelaar supplementary material(PDF)
PDF 1.3 MB
Submit a response

eLetters

No eLetters have been published for this article.