Hostname: page-component-586b7cd67f-dlnhk Total loading time: 0 Render date: 2024-11-23T02:17:24.041Z Has data issue: false hasContentIssue false
  • English
  • Français

Digital mental health apps and the therapeutic alliance: initial review

Published online by Cambridge University Press:  29 January 2019

Philip Henson ,
Hannah Wisniewski ,
Chris Hollis ,
Matcheri Keshavan  and
John Torous
Philip Henson
Affiliation:
Research Assistant, Division of Digital Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, USA
Hannah Wisniewski
Affiliation:
Research Assistant, Division of Digital Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, USA
Chris Hollis
Affiliation:
Chair of Child and Adolescent Psychiatry and Director, NIHR MindTech MedTech Co-operative, NIHR Nottingham BRC Mental Health; and Technology Theme Lead, University of Nottingham and Institute of Mental Health, UK
Matcheri Keshavan
Affiliation:
Stanley Cobb Professor and Vice-Chair for Public Psychiatry, Department of Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, USA
John Torous*
Affiliation:
Director of Digital Psychiatry, Departments of Psychiatry and Clinical Informatics, Beth Israel Deaconess Medical Center, Harvard Medical School, USA
*
Correspondence: John Torous, MD, Departments of Psychiatry and Clinical Informatics,, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston 02446, MA, USA. Email: [email protected]
Rights & Permissions [Opens in a new window]

Abstract

Background

As mental healthcare expands to smartphone apps and other technologies that may offer therapeutic interventions without a therapist involved, it is important to assess the impact of non-traditional therapeutic relationships.

Aims

To determine if there were any meaningful data regarding the digital therapeutic alliance in smartphone interventions for serious mental illnesses.

Method

A literature search was conducted in four databases (PubMed, PsycINFO, Embase and Web of Science).

Results

There were five studies that discuss the therapeutic alliance when a mobile application intervention is involved in therapy. However, in none of the studies was the digital therapeutic alliance the primary outcome. The studies looked at different mental health conditions, had different duration of technology use and used different methods for assessing the therapeutic alliance.

Conclusions

Assessing and optimising the digital therapeutic alliance holds the potential to make tools such as smartphone apps more effective and improve adherence to their use. However, the heterogeneous nature of the five studies we identified make it challenging to draw conclusions at this time. A measure is required to evaluate the digital therapeutic alliance.

Keywords

SmartphoneallianceIndividual Psychotherapy
Type
Review
Information
BJPsych Open , Volume 5 , Issue 1 , January 2019 , e15
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.
Copyright
Copyright © The Royal College of Psychiatrists 2019

Among the 44.7 million adults living with mental illness in the USA, only 43.1% received mental health treatment in the prior year.1 As technology attempts to bridge the healthcare access gap, it is increasingly important that quality of treatment remains both high and consistent. One element of such high-quality care is a strong therapeutic relationship, defined as the working alliance between the patient and therapist that is composed of shared goals, agreement with tasks and development of a bond.Reference Bordin2 Generalisable to all psychotherapies, the therapeutic alliance that is established during face-to-face therapy is considered predictive of positive outcomesReference Ardito and Rabellino3 and essential to medical care.Reference Suchman and Matthews4

Changes in the delivery of therapy and the impact on the therapeutic alliance

The introduction of computers into therapy and clinical visits has brought new challenges for providers including balancing the management of the electronic health record with establishing a meaningful relationship with the patient.Reference Rosen, Nakash and Alegria5 Although the introduction of technology may in some cases lead to overall improvements in quality and efficiency of care,Reference Chaudhry, Wang, Wu, Maglione, Mojica and Roth6 an inverse relationship exists between clinician computer use and the quality of the therapeutic alliance.Reference Ardito and Rabellino3

More recently, researchers have begun to investigate the usability and efficacy of smartphone interventions in mental health as the popularity of mobile applications (apps) continues to expand. Although smartphone apps have the potential to aid people with depression,Reference Firth, Torous, Nicholas, Carney, Pratap and Rosenbaum7 bipolar disorderReference Nicholas, Larsen, Proudfoot and Christensen8 and suicide ideation,Reference Larsen, Nicholas and Christensen9 identifying quality, evidence-based apps still remains a challenge. People who seek support for their well-being not only use health apps on a daily basis,Reference Rubanovich, Mohr and Schueller10 but also have over 10 000 mental health apps to choose from that are available from commercial marketplaces.Reference Torous and Roberts11 As more patients turn to,Reference Aschbrenner, Naslund, Grinley, Bienvenida, Bartels and Brunette12 or are referred to mental health apps, it is important to understand the efficacy of these appsReference Nicholas, Fogarty, Boydell and Christensen13 and the impact of the frequent lack of human support in their use.Reference Bennion, Hardy, Moore and Millings14 Therapists recognise that the unsupervised use of apps is concerning,Reference Stawarz, Preist, Tallon, Wiles and Coyle15 and that there is a need to understand the nature of the therapeutic relationship that patients have with these apps, which we will refer to in this article as the digital therapeutic alliance.

Initial attempts to address this question have focused on the impact of computer and internet interventions on a patient–online therapist alliance, suggesting online therapy may generate a similar therapeutic relationship to that in face-to-face therapies,Reference Berger16 but there is little consensus as to how to measure the digital therapeutic alliance for smartphone apps and mobile devices.Reference Sucala, Schnur, Constantino, Miller, Brackman and Montgomery17 One study modified a validated scale, the Working Alliance Inventory (WAI), by changing some of the language, for example replacing ‘therapist’ with ‘online therapist,’ but the authors noted the limitation of using a scale originally designed for face-to-face therapy.Reference Andersson, Paxling, Wiwe, Vernmark, Felix and Lundborg18 Both patients and clinicians seek clarity on this topic with the digital therapeutic alliance being voted as a top ten research priority in a recent national study involving 600 mental health stakeholders in the UK.Reference Hollis, Sampson, Simons, Davies, Churchill and Betton19

Aims of our literature review

Given the importance of the therapeutic alliance in therapy, the scalability of mobile phone interventions and the popularity of mental health apps, we conducted a literature search to determine if there were any meaningful data regarding the digital therapeutic alliance in smartphone interventions for serious mental illness. We realise that our results may be limited because we have only included studies that involve smartphones, but we determined that it was critical to establish a baseline in order to know where the field currently stands regarding the digital therapeutic alliance in mental health apps.

Method

Search strategy

A literature search was conducted in four databases (PubMed, PsycINFO, Embase and Web of Science) on 19 July 2018. The specific search terms (see Appendix 1) were generated with the help of a librarian and included the mention of alliance (or support, bond, relationship) in the title or abstract, a serious mental illness and a mobile app or smartphone. The search resulted in 2501 abstracts. An additional study was identified through other sources and added to the search results.

Selection strategy

Duplicates were removed to yield 1507 abstracts, which were subsequently screened by two reviewers (P.H. and H.W.) using the software RayyanReference Ouzzani, Hammady, Fedorowicz and Elmagarmid20 with the following inclusion criteria: (a) individuals are at risk for or are diagnosed with a serious mental illness (such as depression, bipolar disorder, schizophrenia or schizoaffective disorder); (b) individuals are interacting with the software outside of a laboratory environment; and (c) therapeutic alliance (or support, bond, relationship) is mentioned in the title or abstract.

The exclusion criteria were: (a) in a foreign language; (b) reviews or study protocols; and (c) software was not smartphone-focused. One study was replaced by another from the same authors that more properly fitted our criteria but was not returned in the search results. All conflicts were discussed with a third reviewer (J.T.) until consensus was reached.

This strategy resulted in five studies that discuss the therapeutic alliance when a mobile application intervention is involved in therapy (see Fig. 1 for the PRISMA 2009 flow diagramReference Moher, Liberati, Tetzlaff, Altman and Group21 for our study).

Fig. 1 PRISMA 2009 flow diagram.

Data extraction

Study characteristics were obtained from the five studies and included: (a) authors name(s); (b) diagnosis of patients enrolled in study; (c) sample size and mean age of patients; (d) intervention used and length of treatment; (d) outcome(s) measured; (e) therapeutic alliance scale used; and (f) study quality, which was evaluated using a 27-item checklist designed for assessing both randomised and non-randomised studies.Reference Downs and Black22

Results

Basic study characteristics and quality scores are outlined in Table 1.

Table 1 Study characteristics

a. Addiction Comprehensive Health Enhancement Support System.

b. Mobile Tracking Young People's Experiences.

Forchuck et al Reference Forchuk, Reiss, Eichstedt, Singh, Collins and Rudnick23 investigated usability and acceptability of a smartphone accessible electronic personal health record (ePHR) system called Lawson SMART record (LSR) among young people with depressive symptoms and found via thematic analysis the potential for the LSR to enhance the patient–therapist alliance.

Richard & SimpsonReference Richards and Simpson24 investigated the effects of a web-based technology (goACT) as an adjunct to psychotherapy. They used a short form of the Agnew Relationship Measure (ARM-5) and did not find significant changes in the therapeutic alliance between therapist and patient before and after using goACT.

Mackie et al Reference Mackie, Dunn, MacLean, Testa, Heisel and Hatcher25 studied the effectiveness of a smartphone application (ACHESS) originally designed to treat harmful substance use in men presenting with self-harm who were veterans. They found via thematic analysis that the therapeutic alliance with technology is related to app engagement and a function of trust and communication.

Reid et al Reference Reid, Kauer, Hearps, Crooke, Khor and Sanci26 monitored young people with emotional mental health issues using a mobile phone application. The control group used a similar version of the mobile phone application that did not contain mental health questions. The therapeutic alliance between the patient and physician was measured via the General Practice Assessment Questionnaire and the Trust in Physician Scale and the authors found no major differences between the experimental and control groups.

Bauer et al Reference Bauer, Iles-Shih, Ghomi, Rue, Grover and Kincler27 monitored depression and anxiety symptoms using the Ginger.io smartphone-based platform. They found via thematic analysis that the care manager's relationship with the patient affected the feasibility and acceptability of a mobile health platform adjunct.

Discussion

Main findings

The lack of studies exploring the impact of mobile technology on the therapeutic relationship is surprising given the potential impact the therapeutic relationship may have on the efficacy of smartphone and digital-based interventions. The heterogeneous nature of the five studies we identified, including differences in individuals' mental health condition, duration of technology use and method of therapeutic alliance assessment, make it challenging to draw conclusions at this time.

Interpretation of our findings

Of the five studies, two used scales to measure the therapeutic alliance,Reference Richards and Simpson24, Reference Reid, Kauer, Hearps, Crooke, Khor and Sanci26 and only one alludes to a digital therapeutic alliance.Reference Mackie, Dunn, MacLean, Testa, Heisel and Hatcher25 None of the studies attempted to quantify the digital therapeutic alliance. Although all papers sought to evaluate the feasibility and acceptability of a new mobile tool for mental health intervention, the therapeutic alliance was never the primary outcome or central concern. In Mackie et al,Reference Mackie, Dunn, MacLean, Testa, Heisel and Hatcher25 the interviewer asks one participant if he felt supported by the app ‘during times that you were away, not directly in the face-to-face therapy’. The participant replies, ‘It was a non-issue. It [the app] didn't give me any security because it didn't work’. This case of faulty technology leading to a lack of security suggests that the digital therapeutic alliance in part many depend on good app design and functionality. Yet the digital therapeutic alliance is not purely technical and understanding the interplay between smartphone app features like privacy and safety, efficacy, engagement and data sharingReference Torous, Chan, Gipson, Kim, Nguyen and Luo29 along with patient's personal and clinical goals remains the challenge.

In general, the studies seemed to agree that smartphones as an adjunct to therapy can lead to increased engagement and adherence,Reference Forchuk, Reiss, Eichstedt, Singh, Collins and Rudnick23, Reference Mackie, Dunn, MacLean, Testa, Heisel and Hatcher25, Reference Bauer, Iles-Shih, Ghomi, Rue, Grover and Kincler27 and that key factors in the therapeutic alliance involve being able to communicate and share information with a clinician outside the normal therapy window.Reference Forchuk, Reiss, Eichstedt, Singh, Collins and Rudnick23–Reference Mackie, Dunn, MacLean, Testa, Heisel and Hatcher25, Reference Bauer, Iles-Shih, Ghomi, Rue, Grover and Kincler27 The question still remains how the relationship may change with varying degrees of clinician interaction, and what constitutes the key factors of the therapeutic alliance when interventions are delivered solely by smartphone. The lack of any literature on this latter point is notable given the rapid expansion of apps directed towards self-guided treatments.Reference Parker, Bero, Gillies, Raven, Mintzes and Jureidini30

Today's smartphone apps that often attempt to translate cognitive–behavioural therapy manuals into smartphone formats have yielded less impressive results than expectedReference Firth, Torous, Nicholas, Carney, Rosenbaum and Sarris31 and a better understanding of the digital therapeutic alliance may offer a solution to increase efficacy and better realise the true potential of digital mental health.

Scales for measuring the digital therapeutic alliance

The lack of studies reporting on the digital therapeutic alliance may also be in part a result of the lack of any practical scale to assess a digital therapeutic alliance. Simply replacing ‘therapist’ with ‘online therapist’ on alliance scales such as the WAI is limited in that important factors in face-to-face therapies may not be equally important in internet-based interventions.Reference Andersson, Paxling, Wiwe, Vernmark, Felix and Lundborg18 Berry et al has started to bridge that gap by introducing a new version of the ARM for digital health interventions,Reference Berry, Salter, Morris, James and Bucci32 but more research and consensus must be developed to test the implementation of these new measurements. At this point it remains unclear if measurements of the digital therapeutic alliance will be universal or whether it will be necessary to customise them by region, culture, age and technology literacy.

A research agenda exploring patients' relationships to their own smartphones, the use of smartphones in clinical care and the use of smartphone apps for self-care is necessary and urgent given the rapid expansion of mobile technology for care. We propose the use of a simple, easy to implement scale for measuring the therapeutic alliance based on the successful validation of a short form of the WAI (WAI-SR).Reference Hatcher and Gillaspy33 The Digital Working Alliance Inventory (D-WAI, see Appendix 2) would concisely assess the same core factors as WAI-SR – goals, tasks and bond – to quickly measure the therapeutic relationship between patient and app. Future research is required to test the D-WAI in a real-world setting.

Previous systematic reviews of the therapeutic alliance in face-to-face therapy have found positive relationships between alliance and outcome but in general studies focused on the patient–therapist alliance rather than the patient–software alliance, and included a broader search of internet-delivered interventions.Reference Sucala, Schnur, Constantino, Miller, Brackman and Montgomery17, Reference Pihlaja, Stenberg, Joutsenniemi, Mehik, Ritola and Joffe34

Limitations

It should be mentioned that our study has several weaknesses. The nature of our search terms may have limited our results to studies that discuss the therapeutic alliance but do not highlight it in their paper. In addition, our search terms could have been modified to include words such as ‘psychotherapy’, ‘psychology’ or ‘counseling’ which are relevant components to the therapeutic alliance and likely would have aided the screening process.

We focused on smartphones interventions because of the scalability of distribution for mobile apps, but in doing so may have missed insightful analyses on the digital therapeutic alliance with other internet-delivered interventions. This research also raises a fundamental question about the nature of the therapeutic relationship as an agent of change in the next generation of digital interventions using artificial intelligence, ‘chatbots,’ and ‘virtual human’ therapists.Reference Hollis, Sampson, Simons, Davies, Churchill and Betton19, Reference Cavanagh and Millings35 These genuinely interactive interventions attempt to humanise the therapy experience and may present challenges in evaluating the digital therapeutic alliance.

In conclusion, as innovative technology is adapted to fit healthcare needs it is important that the focus expands from product development to care delivery.Reference Torous and Hsin36 Among the many mental health apps available for download, many of them place responsibility on mental well-being in the hands of the patient.Reference Parker, Bero, Gillies, Raven, Mintzes and Jureidini30 Thus, assessing and optimising the digital therapeutic relationship holds the potential to make tools like smartphone apps more effective and improve adherence to their use.

Appendix 1

Search terms for literature review

PubMed: (alliance[TIAB] OR relationship[TIAB] OR bond[TIAB] OR support[TIAB]) AND (mental health OR psychiatry OR bipolar disorder OR schizophrenia OR schizoaffective OR depression) AND (‘Mobile Applications’[Mesh] OR ‘Cell Phone’[mesh] OR ‘Computers, Handheld’[mesh] OR ((app OR apps OR application* OR technology OR software) AND (phone[tiab] OR phones OR handheld* OR mobile)) OR smartphone* OR mobile device* OR handheld device* OR iphone* OR ipad* OR android*).

PsycInfo: (mental health OR psychiatry OR bipolar disorder OR schizophrenia OR schizoaffective OR depression) AND (‘Mobile Applications’ OR ‘Cell Phone’ OR ‘Handheld Computers’ OR ((app OR apps OR application* OR technology OR software) AND (TI phone OR AB phone OR phones OR handheld* OR mobile)) OR smartphone* OR mobile device* OR handheld device* OR iphone* OR ipad* OR android*)) AND ((TI alliance OR AB alliance OR TI relationship OR AB relationship OR TI bond OR AB bond OR TI support OR AB support).

Embase: (‘mental health’/exp OR ‘mental health’ OR (mental AND (‘health’/exp OR health)) OR ‘psychiatry’/exp OR psychiatry OR ‘schizophrenia’/exp OR schizophrenia OR ‘schizoaffective’/exp OR schizoaffective OR ‘depression’/exp OR depression) AND ((‘mobile applications’/exp OR ‘mobile applications’ OR ‘cell phone’/exp OR ‘cell phone’ OR ‘handheld computers’ OR ((appOR apps OR application* OR ‘technology’/exp OR technology OR ‘software’/exp OR software) AND (‘phone’:ab,ti OR phones OR handheld* OR mobile)) OR ‘smartphone* OR mobile’ OR (smartphone*or AND mobile AND device*) OR handheld) AND device* OR iphone* OR ipad* OR android*) AND (‘alliance’:ab,ti OR ‘relationship’:ab,ti OR ‘bond’:ab,ti OR ‘support’:ab,ti).

Web of Science: (TS = ((alliance OR relationship OR bond OR support) AND (mental health OR psychiatry OR bipolar disorder OR schizophrenia OR schizoaffective OR depression) AND (‘Mobile Applications’ OR ‘Cell Phone’ OR ‘Handheld Computers’ OR ((app OR apps OR application* OR technology OR software) AND (phone OR phones OR handheld* OR mobile)) OR smartphone* OR mobile device* OR handheld device* OR iphone* OR ipad* OR android*))).

Appendix 2

Digital Working Alliance Inventory (D-WAI)

  1. 1. I trust the app to guide me towards my personal goals [Goals]

  2. 2. I believe the app tasks will help me to address my problem [Tasks]

  3. 3. The app encourages me to accomplish tasks and make progress [Bond]

  4. 4. I agree that the tasks within the app are important for my goals [Goals]

  5. 5. The app is easy to use and operate [Tasks]

  6. 6. The app supports me to overcome challenges [Bond]

Goals: 1 and 4

Tasks: 2 and 5

Bond: 3 and 6

References

1National Institute of Mental Health. Mental Illness. National Institute of Mental Health, 2017 (https://www.nimh.nih.gov/health/statistics/mental-illness.shtml#part_154910).Google Scholar
2Bordin, ES. The generalizability of the psychoanalytic concept of the working alliance. Psychother Theory Res Pract 1979; 16: 252–60.Google Scholar
3Ardito, RB, Rabellino, D. Therapeutic alliance and outcome of psychotherapy: historical excursus, measurements, and prospects for research. Front Psychol 2011; 2: 270.Google Scholar
4Suchman, AL, Matthews, DA. What makes the patient-doctor relationship therapeutic? Exploring the connexional dimension of medical care. Ann Intern Med 1988; 108: 125–30.Google Scholar
5Rosen, DC, Nakash, O, Alegria, M. The impact of computer use on therapeutic alliance and continuance in care during the mental health intake. Psychotherapy (Chic) 2016; 53: 117–23.Google Scholar
6Chaudhry, B, Wang, J, Wu, S, Maglione, M, Mojica, W, Roth, E, et al. Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med 2006; 144: 742–52.Google Scholar
7Firth, J, Torous, J, Nicholas, J, Carney, R, Pratap, A, Rosenbaum, S, et al. The efficacy of smartphone-based mental health interventions for depressive symptoms: a meta-analysis of randomized controlled trials. World Psychiatry 2017; 16: 287–98.Google Scholar
8Nicholas, J, Larsen, ME, Proudfoot, J, Christensen, H. Mobile apps for bipolar disorder: a systematic review of features and content quality. J Med Internet Res 2015; 17: e198.Google Scholar
9Larsen, ME, Nicholas, J, Christensen, H. A systematic assessment of smartphone tools for suicide prevention. PLoS One 2016; 11: e0152285.Google Scholar
10Rubanovich, CK, Mohr, DC, Schueller, SM. Health app use among individuals with symptoms of depression and anxiety: a survey study with thematic coding. JMIR Ment Heal 2017; 4: e22.Google Scholar
11Torous, J, Roberts, L. Needed innovation in digital health and smartphone applications for mental health: transparency and trust. JAMA Psychiatry 2017; 74: 437–8.Google Scholar
12Aschbrenner, KA, Naslund, JA, Grinley, T, Bienvenida, JCM, Bartels, SJ, Brunette, M. A survey of online and mobile technology use at peer support agencies. Psychiatr Q 2018; 89:539–48.Google Scholar
13Nicholas, J, Fogarty, AS, Boydell, K, Christensen, H. The reviews are in: a qualitative content analysis of consumer perspectives on apps for bipolar disorder. J Med Internet Res 2017; 19: e105.Google Scholar
14Bennion, MR, Hardy, G, Moore, RK, Millings, A. E-therapies in England for stress, anxiety or depression: what is being used in the NHS? A survey of mental health services. BMJ Open 2017; 7: e014844.Google Scholar
15Stawarz, K, Preist, C, Tallon, D, Wiles, N, Coyle, D. User experience of cognitive behavioral therapy apps for depression: an analysis of app functionality and user reviews. J Med Internet Res 2018; 20: e10120.Google Scholar
16Berger, T. The therapeutic alliance in internet interventions: a narrative review and suggestions for future research. Psychother Res 2017; 27: 511–24.Google Scholar
17Sucala, M, Schnur, JB, Constantino, MJ, Miller, SJ, Brackman, EH, Montgomery, GH. The therapeutic relationship in e-therapy for mental health: a systematic review. J Med Internet Res 2012; 14: e110.Google Scholar
18Andersson, G, Paxling, B, Wiwe, M, Vernmark, K, Felix, CB, Lundborg, L, et al. Therapeutic alliance in guided internet-delivered cognitive behavioural treatment of depression, generalized anxiety disorder and social anxiety disorder. Behav Res Ther 2012; 50: 544–50.Google Scholar
19Hollis, C, Sampson, S, Simons, L, Davies, EB, Churchill, R, Betton, V, et al. Identifying research priorities for digital technology in mental health care: results of the James Lind Alliance Priority Setting Partnership. Lancet Psychiatry 2018; 5: 845–54.Google Scholar
20Ouzzani, M, Hammady, H, Fedorowicz, Z, Elmagarmid, A. Rayyan – a web and mobile app for systematic reviews. Syst Rev 2016; 5: 210.Google Scholar
21Moher, D, Liberati, A, Tetzlaff, J, Altman, DG, Group, TP. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLOS Med 2009; 6: 1–6.Google Scholar
22Downs, SH, Black, N. The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. J Epidemiol Community Health 1998; 52: 377–84.Google Scholar
23Forchuk, C, Reiss, J, Eichstedt, J, Singh, D, Collins, K, Rudnick, A, et al. The youth-mental health engagement network: an exploratory pilot study of a smartphone and computer-based personal health record for youth experiencing depressive symptoms. Int J Ment Health 2016; 45: 205–22.Google Scholar
24Richards, P, Simpson, S. Beyond the therapeutic hour: an exploratory pilot study of using technology to enhance alliance and engagement within face-to-face psychotherapy. Br J Guid Counc 2015; 43: 57–93.Google Scholar
25Mackie, C, Dunn, N, MacLean, S, Testa, V, Heisel, M, Hatcher, S. A qualitative study of a blended therapy using problem solving therapy with a customised smartphone app in men who present to hospital with intentional self-harm. Evid Based Ment Heal 2017; 20: 118–22.Google Scholar
26Reid, SC, Kauer, SD, Hearps, SJC, Crooke, AHD, Khor, AS, Sanci, LA, et al. A mobile phone application for the assessment and management of youth mental health problems in primary care: health service outcomes from a randomised controlled trial of mobiletype. BMC Fam Pract 2013; 14: 84.Google Scholar
27Bauer, AM, Iles-Shih, M, Ghomi, RH, Rue, T, Grover, T, Kincler, N, et al. Acceptability of mHealth augmentation of collaborative care: a mixed methods pilot study. Gen Hosp Psychiatry 2018; 51: 22–9.Google Scholar
28Kauer, SD, Reid, SC, Crooke, AHD, Khor, A, Hearps, SJC, Jorm, AF, et al. Self-monitoring using mobile phones in the early stages of adolescent depression: randomized controlled trial. J Med Internet Res 2012; 14: e67.Google Scholar
29Torous, JB, Chan, SR, Gipson, SY-MT, Kim, JW, Nguyen, T-Q, Luo, J, et al. A hierarchical framework for evaluation and informed decision making regarding smartphone apps for clinical care. Psychiatr Serv 2018; 69: 498–500.Google Scholar
30Parker, L, Bero, L, Gillies, D, Raven, M, Mintzes, B, Jureidini, J, et al. Mental health messages in prominent mental health apps. Ann Fam Med 2018; 16: 338–42.Google Scholar
31Firth, J, Torous, J, Nicholas, J, Carney, R, Rosenbaum, S, Sarris, J. Can smartphone mental health interventions reduce symptoms of anxiety? A meta-analysis of randomized controlled trials. J Affect Disord 2017; 218: 15–22.Google Scholar
32Berry, K, Salter, A, Morris, R, James, S, Bucci, S. Assessing therapeutic alliance in the context of mHealth interventions for mental health problems: development of the mobile Agnew relationship measure (mARM) questionnaire. J Med Internet Res 2018; 20: e90.Google Scholar
33Hatcher, RL, Gillaspy, JA. Development and validation of a revised short version of the Working Alliance Inventory. Psychother Res 2006; 16: 12–25.Google Scholar
34Pihlaja, S, Stenberg, J-H, Joutsenniemi, K, Mehik, H, Ritola, V, Joffe, G. Therapeutic alliance in guided internet therapy programs for depression and anxiety disorders – a systematic review. Internet Interv 2018; 11: 1–10.Google Scholar
35Cavanagh, K, Millings, A. (Inter)personal computing: the role of the therapeutic relationship in E-mental health. J Contemp Psychother 2013; 43: 197–206.Google Scholar
36Torous, J, Hsin, H. Empowering the digital therapeutic relationship: virtual clinics for digital health interventions. NPJ Digit Med 2018; 1: 16.Google Scholar
Figure 0

Fig. 1 PRISMA 2009 flow diagram.

Figure 1

Table 1 Study characteristics

Submit a response

eLetters

No eLetters have been published for this article.