Hostname: page-component-78c5997874-m6dg7 Total loading time: 0 Render date: 2024-11-06T04:49:46.324Z Has data issue: false hasContentIssue false

Concerns regarding an evaluation of MTFC-A for adolescents in English care

Published online by Cambridge University Press:  02 January 2018

Gordon T. Harold
Affiliation:
School of Psychology, University of Sussex. Email: [email protected]
David S. DeGarmo
Affiliation:
Prevention Science Institute, University of Oregon
Rights & Permissions [Opens in a new window]

Abstract

Type
Columns
Copyright
Copyright © Royal College of Psychiatrists, 2014 

We are writing to highlight concerns regarding conclusions offered by Green et al in their evaluation of Multidimensional Treatment Foster Care for Adolescents (MTFC-A) relative to usual care for at-risk youth in English foster care. Reference Green, Biehal, Roberts, Dixon, Kay and Parry1 We commend the authors for undertaking an independent review of MTFC-A. However, we offer some observations to help contextualise the efficacy of the evaluation with respect to the primary conclusion that MTFC-A did not result in better outcomes than usual care.

Green et al’s evaluation employed a two-arm, single-blinded (assessor) randomised controlled trial embedded within an observational quasi-experimental case-control study. An intent-to-treat (ITT) analysis was employed specific to the MTFC-A versus usual care comparison. The authors state that the study was intended to be powered at β = 0.80 to detect half a standard deviation difference between ITT and usual care (with a target n of 130), and was powered β = 0.95 to detect the same effect between ITT and usual care in the quasi-experimental study (with a target n of 90). However, the target allocation for the trial was not met. The trial randomly allocated only 34 participants (n = 20 MTFC-A and n = 14 usual care). Based on these numbers, we estimate the study was actually powered at β = 0.29 in the ITT analysis to detect half a standard deviation difference between conditions assuming equal variances, and at β = 0.28 assuming unequal variances.

Substantive conclusions therefore seem to be based on a substantially underpowered design (as far as we can tell from the detail presented in the original manuscript). Further, the quasi-experimental arm was described as a case-control design. However, it was not a matched case-control design. This is evident from multiple baseline differences between groups, some of which remained after an intensive set of propensity-score weights was applied and after elimination of cases with probability of assignment to MTFC-A above 0.95 and below 0.05. Depending on the distribution of assignment probabilities, this may have resulted in relatively limited ‘data trimming’ in order to attain desired allocation probabilities near 0.50. The observed differences included not only age but also the primary outcome scores.

Notwithstanding concerns regarding statistical power for the trial, the authors reported intervention by baseline risk interactions in the only adequately powered arm of the study (see Table 5). Given prior demonstration of MTFC-A intervention by baseline risk interactions, Reference Leve, Chamberlain, Smith, Harold, Miller, Leve and Kerig2 these results may have been more appropriately presented as a hypothesised replication. Statistical power is also a concern for the reported analyses of offending; β = 0.034 to detect the observed ITT odds ratio of 1.24 using an allocation of 20 and 14 cases, and β = 0.031 in the quasi-experimental arm to detect the observed ITT odds ratio of 1.07 with 93 and 92 cases. Interpretation of effects should therefore be treated with caution.

We raise one additional point of clarification regarding prior MTFC-A implementations. The authors state that the context of intervention in the UK differs significantly from that in the originating US studies, since ‘these were focused on convicted delinquent youth where the alternative [to MTFC-A] was incarceration’, thereby concluding that the ‘control condition in the US studies approximated […] to juvenile custody’. Actually, similar to the usual care condition in the Green et al study, the standard control condition in US MTFC-A studies is group care, Reference Chamberlain3 not incarceration.

We offer these points by way of lending interpretation to the efficacy of Green et al’s results and to suggest caution in accepting the conclusion that MTFC-A may not result in better outcomes than usual care among at-risk adolescents in English care.

Footnotes

Conflict of interest: The authors have collaborated with US colleagues on projects using the MTFC-A programme.

References

1 Green, JM, Biehal, N, Roberts, C, Dixon, J, Kay, C, Parry, E, et al. Multidimensional Treatment Foster Care for Adolescents in English care: randomised trial and observational cohort evaluation. Br J Psychiatry 2014; 204: 214–21.CrossRefGoogle ScholarPubMed
2 Leve, LD, Chamberlain, P, Smith, DK, Harold, GT. Multidimensional treatment foster care as an intervention for juvenile justice girls in out-of-home care. In Delinquent Girls: Contexts, Relationships, and Adaptation (eds Miller, S, Leve, L, Kerig, P). Springer Press, 2011.Google Scholar
3 Chamberlain, P. Treating Chronic Juvenile Offenders: Advances Made through the Oregon Multidimensional Treatment Foster Care Model. American Psychological Association, 2003.CrossRefGoogle Scholar
Submit a response

eLetters

No eLetters have been published for this article.