Study characteristics

Table 1 and Table 2 show the characteristics of the studies included in the review.

Table 1 Study characteristics

ST, schema therapy; COP, clarification-oriented psychotherapy; MBT, mentalisation-based therapy; JCP, joint crisis plans; CBT, cognitive–behavioural therapy; SC, stepped care; PEPS, psychoeducation with problem-solving; DBT, dialectical behaviour therapy; NID, nidotherapy; DBT-I, in-patient DBT; DBT-OP, out-patient DBT; DH, day hospital; LOP, long-duration OP; SDH, short-duration DH; LDH, long-duration DH; SIP, short-duration IP; LIP, long-duration IP; MACT, manual-assisted cognitive therapy; TFP, transference-focused psychotherapy; TAU, treatment as usual; AO, assertive outreach; AOR, assertive outreach and rehabilitation; CEA, cost-effectiveness analysis; CUA, cost-utility analysis.

Table 2 Further study characteristics

DRG, diagnosis-related group; QALY, quality adjusted life year; EQ-5D, EuroQol's EQ-5D; n.a., not applicable; NSSI, non-suicidal self-injury; ED, emergency department; BDI-II, Beck Depression Inventory II; BSS, Beck Scale for Suicidal Ideation; STAI-Y, Spielberger State–Trait Anxiety Inventory Form Y; BSI, Brief Symptom Inventory; CSRI, Client Service Receipt Inventory; TiC-P, Treatment Inventory Cost in Psychiatric Patients; GAF, Global Assessment of Functioning; BPRS, Brief Psychiatric Rating Scale; SFSUS, Secure Facilities Use Schedule; SCID-II, Structured Clinical Interview for DSM-IV for Axis II Disorders; AD-SUS, Adult Service Use Schedule; BPDSI, Borderline Personality Disorder Severity Index.

Studies were conducted in the UK, Europe and Australia, with sample sizes ranging from 34 to 642 (mean 195.2, s.d. = 167.7). The youngest sample had a mean age of 28.4 years and the oldest sample's mean was 39.2 years. The percentage of women in the samples ranged from 25 to 95%. The lowest rate of employment in a sample was 9.5% and the highest rate of employment was 68.5%, though seven studies did not report this.

Five of the studies included patients with any personality disorder diagnosis.^{Reference Priebe, Bhatti, Barnicot, Bremner, Gaglia and Katsakou17,Reference Grenyer, Lewis, Fanaian and Kotze21,Reference Kvarstein, Arnevik, Halsteinli, Rø, Karterud and Wilberg22,Reference Tyrer, Milošeska, Whittington, Ranger, Khaleel and Crawford25,Reference McMurran, Crawford, Reilly, Delport, McCrone and Whitham30} Three of the evaluations looked at cluster C personality disorders, described as avoidant, dependent or obsessive–compulsive,^{Reference Bamelis, Arntz, Wetzelaer, Verdoorn and Evers14,Reference Soeteman, Verheul, Meerman, Ziegler, Rossum and Delimon28} and one study looked at cluster B personality disorders, described as antisocial, borderline, histrionic or narcissistic.^{Reference Soeteman, Verheul, Delimon, Meerman, van den Eijnden and Rossum27} Seven studies only recruited participants with a diagnosis of borderline personality disorder.^{Reference Pasieczny and Connor16, Reference Davidson, Tyrer, Norrie, Palmer and Tyrer18,Reference Palmer, Davidson, Tyrer, Gumley, Tata and Norrie19,Reference Sinnaeve, van den Bosch, Hakkaart-van Roijen and Vansteelandt23,Reference van Asselt, Dirksen, Arntz, Giesen-Bloo, van Dyck and Spinhoven26,Reference Borschmann, Barrett, Hellier, Byford, Henderson and Rose29,Reference Blankers, Koppers, Laurenssen, Peen, Smits and Luyten31} Nine of the studies required a recent episode of self-harm, attendance at an emergency department or in-patient stay.^{Reference Murphy, Bourke, Flynn, Kells and Joyce15,Reference Priebe, Bhatti, Barnicot, Bremner, Gaglia and Katsakou17–Reference Grenyer, Lewis, Fanaian and Kotze21,Reference Sinnaeve, van den Bosch, Hakkaart-van Roijen and Vansteelandt23,Reference Borschmann, Barrett, Hellier, Byford, Henderson and Rose29} Two studies only included patients with other comorbid severe mental illness^{Reference Ranger, Tyrer, Miloseska, Fourie, Khaleel and North24} and one of the two also required substance dependence.^{Reference Tyrer, Milošeska, Whittington, Ranger, Khaleel and Crawford25} One study included a general sample of which less than half had complex emotional needs that met personality disorder diagnostic criteria.^{Reference Tyrer, Tom, Byford, Schmidt, Jones and Davidson20}

The interventions evaluated in the 18 studies are described in the supplementary Table 1. Psychotherapeutic interventions included dialectical behaviour therapy (n = 3), types of cognitive therapy (n = 3), nidotherapy (n = 2), schema-focused therapy (n = 2), psychoeducation with problem-solving (n = 1) and mentalisation-based therapy delivered in a day-hospital setting (MBT) (n = 1). Other interventions involved altering the setting in which care was delivered (n = 2), adopting a stepped-care approach (n = 3) and developing joint crisis plans (n = 1). Treatment as usual was the comparative intervention in eight of the included papers. Other comparators included psychotherapy provided by out-patient services (n = 4), psychotherapy provided by day-hospital services (n = 2), care provided by assertive outreach services (n = 2), transference-focused psychotherapy (n = 1), and baseline data (n = 1). More than three-quarters of the included papers employed a randomised controlled trial (RCT) design (n = 14), two used Markov models, one a wait-list controlled trial and one a quasi-experimental design.

In total, 33 different outcome measures were used across the 18 studies (Table 3), with inconsistent approaches to discounting outcomes (5 studies applied a discount rate of between 3 and 5% and 13 studies applied no discounting). Over half of the papers employed a CUA (n = 10). Other approaches used were CCAs (n = 5) and CEAs (n = 5).

Table 3 Outcomes used in studies

a. If data for the primary outcome were missing, back-up data were used instead.

SCID-II, Structured Clinical Interview for DSM-IV Axis II Disorders; ADP-IV, Assessment of DSM-IV Personality Disorder Questionnaire; BPDSI, Borderline Personality Disorder Severity Index; EQ-5D-3L, EuroQuol EQ-5D-3L; EQ-5D-5L, EuroQuol EQ-5D-5L; ADSHI, Acts of Deliberate Self Harm; LPC, Life-time Parasuicide Count; LPHI, Lineham's Parasuicide History Interview; HADS, Hospital Anxiety and Depression Scale; WAI, Working Alliance Inventory; CSQ, Client Satisfaction Questionnaire; SES, Service Engagement Scale; EAS, Engagement and Assessment Scale; WEMWS, Warwick–Edinburgh Mental Well-being Scale; WSAS, Work and Social Adjustment Scale; SFQ, Social Functioning Questionnaire; IIP-32, Inventory of Interpersonal Problems - Short Form 32; TES, Treatment Experience Scale; GAF, Global Assessment of Functioning; BDI, Beck Depression Inventory; STAI, Spielberger State–Trait Anxiety Inventory; YSQ, Young Schema Questionnaire; BSSI, Beck Scale for Suicidal Ideation; BSI, Brief Symptom Inventory; BPRS, Brief Psychiatric Rating Scale; ZRS-BPD, Zanari Rating Scale for Borderline Personality Disorder; MSA-QoL, Manchester Short Assessment of Quality of Life; PAS-Q, Quick Personality Assessment Schedule.

Outcomes and costs reported by all studies are reported in Table 4.

Table 4 Study results

QALY, quality-adjusted life year; DBT, dialectical behaviour therapy; TAU, treatment as usual; CBT-PD, cognitive–behavioural therapy for personality disorders; CBT, cognitive–behavioural therapy; MACT, manual-assisted cognitive therapy; GAF, Global Assessment of Functioning; BPRS, Brief Psychiatric Rating Scale; ST, schema therapy; WTP, willingness to pay; MBT, mentalisation-based treatment.

Dialectical behaviour therapy

Each of the three studies of DBT used a different method of economic evaluation. Murphy et al conducted a quasi-experimental study using a CUA,^{Reference Murphy, Bourke, Flynn, Kells and Joyce15} Pasieczny & Connor adopted a CCA for their waiting-list controlled trial^{Reference Pasieczny and Connor16} and Priebe et al performed a CEA based on an RCT.^{Reference Priebe, Bhatti, Barnicot, Bremner, Gaglia and Katsakou17} The mean follow-up period for these studies was 12 months (s.d. = 6 months). Murphy et al and Pasieczny & Connor adopted a narrow healthcare perspective, whereas Priebe et al took a broader view including lost employment costs.

Two of the studies^{Reference Pasieczny and Connor16,Reference Priebe, Bhatti, Barnicot, Bremner, Gaglia and Katsakou17} reported significant improvements in clinical outcomes. Pasieczny & Connor reported fewer suicide attempts, emergency department visits, admissions and in-patient days.^{Reference Pasieczny and Connor16} Priebe et al reported a reduction in self-harm events (rate ratio RR = 0.78; 95% CI 0.76–0.80).^{Reference Priebe, Bhatti, Barnicot, Bremner, Gaglia and Katsakou17} Pasieczny & Connor and Murphy et al reported DBT to be less costly than the control condition, although neither reported statistical significance.^{Reference Murphy, Bourke, Flynn, Kells and Joyce15} Murphy et al reported an incremental cost-effectiveness ratio (ICER) for DBT of €1965 per QALY, with the sensitivity analysis suggesting a 62% probability that DBT was cost-effective at a threshold (i.e. how much society is considered to value a QALY) of €45 000 EUR per QALY. Priebe et al reported the ICER for DBT to be £36 per percentage point decrease in the incidence of self-harm.^{Reference Priebe, Bhatti, Barnicot, Bremner, Gaglia and Katsakou17}

Cognitive therapies

All three studies evaluating cognitive therapies used data from RCTs. Palmer et al carried out a CUA, whereas Davidson et al and Tyrer et al employed a CCA.^{Reference Davidson, Tyrer, Norrie, Palmer and Tyrer18–Reference Tyrer, Tom, Byford, Schmidt, Jones and Davidson20} Tyrer et al evaluated manual-assisted cognitive therapy (MACT), with the other two evaluating cognitive–behavioural therapy (CBT). The follow-up periods for these studies were the most varied for a specific intervention category (mean 36 months, s.d. = 31.75). Tyrer et al and Palmer et al took a broad perspective and Davidson et al employed a health, social care and criminal justice perspective. All three used a version of the Client Service Receipt Inventory to record resource use, although Davidson et al and Palmer et al also used hospital records.

Only Tyrer et al reported a change in outcome that favoured cognitive therapies: the incidence rate of a first episode of self-harm per person year was 0.584 in the MACT group and 0.71 in the treatment as usual (TAU) group.^{Reference Tyrer, Tom, Byford, Schmidt, Jones and Davidson20} The frequency of self-harm events remained relatively unchanged (2.84 per year for MACT versus 2.54 for TAU, after outliers had been excluded). Davidson et al and Palmer et al reported the difference in costs between CBT and TAU, although they were not found to be significant. Palmer et al found CBT to be less costly but also less effective and reported an ICER of £6376 per QALY. When the WTP is below the ICER, CBT has a higher probability of being cost-effective, but when the WTP is above the ICER, TAU has a higher probability of being cost-effective. At the UK threshold of £30 000 per QALY, the probability of CBT being cost-effective was only ~25%, whereas the probability of TAU being cost-effective was ~75%, making TAU the more cost-effective choice. Neither Davidson et al nor Tyrer et al calculated an ICER.

Stepped care

All three studies on stepped-care interventions used data from RCTs. Sinnaeve et al carried out a CUA, Kvarstein et al employed a CEA and Grenyer et al conducted a CCA.^{Reference Grenyer, Lewis, Fanaian and Kotze21–Reference Sinnaeve, van den Bosch, Hakkaart-van Roijen and Vansteelandt23} The mean follow-up period was 32 months (s.d. = 18.33 months). The perspective of Kvarstein et al and Sinnaeve et al included healthcare and one other sector's costs (social care and employment respectively), whereas Grenyer et al took the narrowest perspective of all the studies included in the review, only including in-patient stays.

The model of stepped care varied between studies. Grenyer et al considered a brief intervention clinic delivered within 36 h of a crisis presentation, followed by longer-term community-based psychological therapy. A significant reduction in in-patient days was reported. Sinnaeve et al evaluated 3 months of DBT delivered as a residential intervention, followed by 6 months of out-patient DBT, and compared this with 12 months of standard out-patient DBT. Sinnaeve et al reported QALYs with mean (s.d.) utility scores; these were higher for the step-down DBT care group than the comparator group, but this difference was not statistically significant (mean 0.65 (s.d. = 0.33) for stepped-care DBT versus mean 0.62 (s.d. = 0.28) for out-patient DBT). Kvarstein et al evaluated intensive psychotherapy in day hospitals followed by out-patient individual and group therapy. Lower improvements in functioning were reported (measured on the Global Assessment of Functioning scale, GAF) compared with standard out-patient care.

None of the studies reported significant differences in costs. Sinnaeve et al reported an ICER of €278 067 per QALY; at a threshold of €80 000 per QALY there was a 21% likelihood that stepped care was the most cost-effective option. Kvarstein et al reported an ICER for out-patient care compared with stepped care for the subgroup with avoidant personality disorder (but not the subgroup with borderline personality disorder) which was €1092 per additional point gained on the GAF (favouring out-patient care). No uncertainty analysis was reported.

Nidotherapy

Two studies reported economic evaluations of nidotherapy using data from a single pilot RCT in a population with difficult-to-manage needs, personality disorder diagnoses and other comorbid severe mental illness. Ranger et al evaluated the pilot using a CEA.^{Reference Ranger, Tyrer, Miloseska, Fourie, Khaleel and North24} Tyrer et al later evaluated nidotherapy in a subgroup of patients with substance misuse problems, employing a CCA using data from the same trial.^{Reference Tyrer, Milošeska, Whittington, Ranger, Khaleel and Crawford25} The studies used a broad perspective and a follow-up period of 12 months.

Ranger et al^{Reference Ranger, Tyrer, Miloseska, Fourie, Khaleel and North24} did not observe any significant effects on outcomes, with a trend towards reduced symptoms in the intervention group. Tyrer et al^{Reference Tyrer, Milošeska, Whittington, Ranger, Khaleel and Crawford25} reported a significant reduction in bed-days in secondary subgroup analyses (54 days compared with 139 for the comparator) but found costs not to be significantly different. Ranger et al still found nidotherapy to be dominant (i.e. it resulted in lower costs and better secondary outcomes than the comparator, although these results were not significant) and even at a threshold of £0 per point of improvement on the Brief Psychiatric Rating Scale it had a 60% likelihood of being cost-effective.

Schema therapy

Two studies evaluated schema therapy using RCTs, and both conducted a CUA and CEA from a broad perspective.^{Reference Bamelis, Arntz, Wetzelaer, Verdoorn and Evers14,Reference van Asselt, Dirksen, Arntz, Giesen-Bloo, van Dyck and Spinhoven26} Bamelis et al^{Reference Bamelis, Arntz, Wetzelaer, Verdoorn and Evers14} compared schema therapy with TAU and van Asselt et al^{Reference van Asselt, Dirksen, Arntz, Giesen-Bloo, van Dyck and Spinhoven26} compared schema therapy with transference-focused psychotherapy. Both studies used the same measure of effectiveness, the proportion of recovered patients, and report this to be higher in the schema therapy group than in the comparator group. For Bamelis et al the proportions were 81.4 v. 51.2% respectively (‘recovered’ defined as a score ≤15 on the Structured Clinical Interview for DSM-IV Axis II Disorders version II) and van Asselt et al reported this to be 52% v. 29%, respectively (‘recovered’ defined as a score ≤15 on the Borderline Personality Disorder Severity Index). Bamelis et al found that schema therapy produced a greater median gain in number of QALYs than TAU, although this was not significant (2.34 v. 2.23; P = 0.51). van Asselt et al reported that total QALYs for the schema therapy group was lower than for the transference-focused psychotherapy group, this difference was also not significant (2.15 v. 2.27 respectively; 95% CI −0.51 to 0.28). van Asselt et al reported an incremental difference in mean costs: schema therapy was €8969 less costly than transference-focused psychotherapy, although this difference was not significant (95% CI −21 775 to 3546). Bamelis et al also reported a lower mean cost for schema therapy compared with TAU: €23 805 (95% CI 21 014–26 791) v. €26 333 (95% CI 22 384–30 605).

Both studies found that schema therapy was dominant with regard to cost per recovered patient and cost per QALY. Bamelis et al showed schema therapy to have an 80% probability of being cost-effective at a WTP threshold of 0, for both cost per recovered patient and cost per QALY. van Asselt et al reported that the likelihood of schema therapy being cost-effective, in terms of recovered patients and QALYs, was over 90% when WTP = 0. Both Bamelis et al and van Asselt et al observed that, as the threshold for recovered patients increased, the probability of schema therapy being cost-effective also increased; however, as the threshold for QALYs increased, the likelihood of schema therapy being cost-effective decreased.

Interventions defined by setting

Two evaluations conducted by Soeteman et al employed Markov models to evaluate the effect of care being provided in different settings for samples with either cluster B or cluster C ‘personality disorder’ diagnoses,^{Reference Soeteman, Verheul, Delimon, Meerman, van den Eijnden and Rossum27} using data from the SCEPTRE trial.^{Reference Bartak, Spreeuwenberg, Andrea, Holleman, Rijnierse and Rossum32,Reference Bartak, Andrea, Spreeuwenberg, Ziegler, Dekker and Rossum33} Markov models were used in these CUAs comparing out-patient, day-hospital and in-patient care. One of the studies varied the duration of day-hospital and in-patient care between long and short term.^{Reference Soeteman, Verheul, Meerman, Ziegler, Rossum and Delimon28} Both studies adopted a broad perspective, as well as repeating the models taking a narrower, health service provider perspective.

Soeteman et al reported that for a cluster B personality disorder group, care at a day hospital was associated with the greatest QALY gains, and for cluster C personality disorder group, short-term in-patient care produced the greatest estimated number of QALYs. For cluster B patients, out-patient care was the least costly and also dominated other settings (being less costly and more effective). For cluster C patients, short-term day-hospital treatment was the least costly option and dominated all long-term options. Estimated cost-effectiveness acceptability curves (CEAC) for cluster B showed out-patient care to be 84% likely to be cost-effective when the threshold was €0 per QALY. For cluster C patients the estimated CEAC showed short-term in-patient care to be 60% likely to be cost-effective at a high WTP threshold of €80 000 per QALY compared with short-term day-hospital care.

Other interventions

Four studies were identified which looked at other interventions: joint crisis plans,^{Reference Borschmann, Barrett, Hellier, Byford, Henderson and Rose29} psychoeducation with problem-solving,^{Reference McMurran, Crawford, Reilly, Delport, McCrone and Whitham30} clarification-oriented psychotherapy (COP)^{Reference Bamelis, Arntz, Wetzelaer, Verdoorn and Evers14} and mentalisation-based treatment delivered in a day hospital (MBT).^{Reference Blankers, Koppers, Laurenssen, Peen, Smits and Luyten31} All of them used RCT data and employed a CUA, and two of these studies (Blankers et al^{Reference Blankers, Koppers, Laurenssen, Peen, Smits and Luyten31} and Bamelis et al^{Reference Bamelis, Arntz, Wetzelaer, Verdoorn and Evers14}) also conducted a CEA. All of these studies took a broad perspective.

Borschmann et al^{Reference Borschmann, Barrett, Hellier, Byford, Henderson and Rose29} did not report a significant difference in costs or outcomes. However, their economic analysis found joint crisis plans to be dominant over usual care, with over 80% probability of being cost-effective when the threshold was £0 per QALY. This is not unusual as even in the absence of a significant clinical effect, there can still be a finding of cost-effectiveness. Cost-effectiveness combines point estimates of cost and outcome differences and in some cases, as here, can show high probabilities of interventions being cost-effective. McMurran et al^{Reference McMurran, Crawford, Reilly, Delport, McCrone and Whitham30} found that psychoeducation with problem-solving was also dominant, having a 64% chance of being more cost-effective than usual care with a threshold of £20 000–£30 000 per QALY.

Bamelis et al^{Reference Bamelis, Arntz, Wetzelaer, Verdoorn and Evers14} found COP to be inferior to TAU and schema-focused therapy. Blankers et al^{Reference Blankers, Koppers, Laurenssen, Peen, Smits and Luyten31} found that MBT was dominated by TAU in the CUA but also that in the CEA the ICER per patient in remission was €29 314. At a threshold of €45 000 the chance of MBT being cost-effective (when considering ‘remission’ as an outcome) was only slight (55%).

Quality appraisal of studies

Study quality, based on the CHEERS checklist, is detailed in Table 5. There was substantial variation between studies. Of the 18 studies, 10 met over 80% of checklist items. On the basis of this metric, the highest-quality studies were the two that evaluated schema therapy, with both having 95% of items met. The three that evaluated DBT were more varied, with the Priebe et al study^{Reference Priebe, Bhatti, Barnicot, Bremner, Gaglia and Katsakou17} having the highest rating. This was also the case for the three that evaluated CBT, with Palmer et al^{Reference Palmer, Davidson, Tyrer, Gumley, Tata and Norrie19} scoring highest. The stepped-care evaluations scored the lowest in terms of items met. Overall, studies that scored low tended to do so particularly for reporting of methods.

Table 5 Quality appraisal of papers