Facts are stubborn things, but statistics are pliable

This special themed edition of the BJPsych takes on precision medicine and personalised healthcare in psychiatry. A bit like Sue Gray's parliamentary report and Wordle, some scepticism has arisen about whether this might be all hype, a waste of time that will not produce meaningful change in our lives. Practical gains seem forever just over the next hill. Another criticism is that it is the domain of academic converts lost in the weeds, and that the data are inexplicable to the average-intelligence reader. Fortunately, at the BJPsych, for these Highlights we've been able to team up both an expert who really understands the issues and that modal-psychiatrist to try to unpick the issues and gains. To save his blushes, we won't say which of them has served as that somewhat head-scratching average individual, except to note that he has been overheard asking whether deep neural networks (DNNs) were ‘the thing that went wrong with the robots in Blade Runner’.

In this issue, Cearns et al (pp. 219–228) draw attention to the clinical application of pharmacogenomic testing, while Kambeitz-Illankovic et al (pp. 175–178) ask more broadly ‘what is in reach?’ Risk assessment, diagnostics and targeted prescribing are offered as natural topics for machine learning, but it is noted that more multimodal approaches are necessary, better combining neuroimaging, genetic and blood markers. As well as the need for larger, longitudinal, prospective data, the call that ‘engaging clinicians’ also means showing that models can exceed their clinical heuristics felt apposite. On the second, Dwyer and Krishnadas (pp. 169–171) helpfully offer five points to consider when reading a translational machine learning paper: acknowledge our interest, appreciate the limitations, understand ‘overfitting’ (see below), consider sample representativeness and consider real-world utility.

Google Docs

Self-harm is an obvious target for prediction in mental health, hitherto plagued by the facts that such actions are regrettably common and the associated variables numerous. Van Velzen et al (pp. 210–218) tackle this in children, utilising a large cohort of over 5000 that were categorised as having suicidal thinking and behaviour, or mental illness but no suicidal thinking, or as healthy controls. These were further stratified using sociodemographic, clinical, cognitive, brain imaging and genomic data. Binomial penalised logistic regression minimised ‘overfitting’ – where a statistical model learns a classification ‘too well’ in one set of data and so would not be able to work in another – and was able to distinguish the suicidal thinking and behaviour group from clinical and healthy controls, something that was replicated in an initially withheld sample. The area under the receiver operator curve performance was between 0.70 and 0.80: findings of 0.5 suggests performance no better than tossing a coin, and figures over 0.7 are considered to indicate acceptable discrimination.

Response to medication is another potential opportunity, and Cearns et al (pp. 219–228) used genetic and clinical data to establish whether this could be done for response to lithium in patients with bipolar disorder. In unimodal models, clinical data outperformed genomic data; however, when patients were first stratified by polygenic risk score, genomic data improved prediction of response to treatment. In such a genomics-first approach, parsing heterogeneity may increase the utility of biological data.

Deep learning – effectively, teaching computers classifications that come naturally to us – is another area of contemporary interest, and Supekar and colleagues are at the forefront of this in psychiatry (pp. 202–209). They take a biomedical-first approach further, to explore neuroimaging data in individuals with an autism spectrum disorder (ASD). Unsupervised classification – for example, by DNNs – aims to find patterns in data without reference to predetermined labelling, but this can be challenging owing to the high dimensionality of functional neuroimaging data. Here, a novel DNN was developed with multisite imaging data, that were able to distinguish sex-specific differences in ASD compared with healthy controls. This is important, as such differences would not otherwise be detectable by conventional univariate analysis. This has real relevance, highlighting organisational brain differences in males and females that are related to clinical presentations.

Psychological therapies might initially seem a counter-intuitive area for precision medicine, but this is a growing field, and Bennemann and colleagues (pp. 192–201) demonstrate the breadth of the potential of machine learning, identifying predictive models of drop-out from therapy. Specifically, they tested over 20 models and ensembles in 2500 out-patients treated with cognitive–behavioural therapy. Decision tree and boosted algorithms – where key steps in prediction break down data into smaller and smaller subsets – appeared most accurate. Interestingly, key variables were education status and personality characteristics. If robust, the applications are clear: offering therapy to patients most likely to stay the course might seem the most obvious, but we could also target enhanced engagement strategies for those most at risk of drop-out.

First-episode psychosis represents a critical time for optimising care, and it is thus not surprising that this is also one of the most utilised areas of machine learning and predictive algorithms. In this issue, Lee et al (pp. 179–191) completed a systematic review of prediction models for outcome, reporting 13 studies and 31 models. Lack of independent or external validation and poor reporting of calibration and discrimination measures limit the burgeoning activity in this field, although the paper highlights some notable exceptions: this is also seen in Antonucci et al (pp. 229–245), where the potential limitations are addressed, including testing of models in independent cohort validation. With an ability to predict functional outcome in patients with clinical high risk with reasonable accuracy over 12 months, the authors again demonstrate that combining multimodal clinical, environmental and biomedical data may lead to more accurate predictions.

Brave new world?

The field of precision psychiatry is advancing, and gains are beginning to approach the realm of clinical utility. They might not yet be part of your out-patient clinic, but to deny that they ever will be is to deny the science. Beware of proponents (or commercial enterprises!) selling clinical-applicable prediction tools that are not able to show the rigour, validation and model testing outlined here. The current challenge is also around what predictions patients, clinicians and providers want to make, what we might be able to address and the degree of precision needed. As with all investigations and tests, machine learning models should be there to help inform, not determine, clinical decision-making. We argue that it is incumbent upon all psychiatrists to remain updated in this field. It feels appropriate to end with something we suspect might have been a lingering concern for many: what are the ethical implications? Lane and Broome (pp. 172–174) provide a very thoughtful account of this, not least in how personalised accuracy tends to be overestimated, and that machine learning processing needs high standards of reporting to maintain transparency.