Introduction
Patients with body dysmorphic disorder (BDD) experience distressing or impairing preoccupations with non-existent or slight defects in their appearance, which is accompanied by repetitive behaviors (i.e. rituals, compulsions), such as mirror checking, excessive grooming, reassurance-seeking, and skin picking; these behaviors are intended to reduce distress but may increase it (Phillips et al., Reference Phillips, Coles, Menard, Yen, Fay and Weisberg2005a, Reference Phillips, Menard, Fay and Weisberg2005b). BDD has a current prevalence of 1.7–2.9% in the general population (Brohede, Wingren, Wijma, & Wijma, Reference Brohede, Wingren, Wijma and Wijma2015; Buhlmann et al., Reference Buhlmann, Glaesmer, Mewes, Fama, Wilhelm, Brähler and Rief2010; Koran, Abujaoude, Large, & Serpe, Reference Koran, Abujaoude, Large and Serpe2008; Rief, Buhlmann, Wilhelm, Borkenhagen, & Brähler, Reference Rief, Buhlmann, Wilhelm, Borkenhagen and Brähler2006; Schieber, Kollei, de Zwaan, & Martin, Reference Schieber, Kollei, de Zwaan and Martin2015). BDD is associated with substantial impairment in psychosocial functioning (Phillips, Quinn, & Stout, Reference Phillips, Quinn and Stout2008) and high rates of suicidality (Angelakis, Gooding, & Panagioti, Reference Angelakis, Gooding and Panagioti2016; Snorrason, Beard, Christensen, Bjornsson, & Björgvinsson, Reference Snorrason, Beard, Christensen, Bjornsson and Björgvinsson2019).
Serotonin-reuptake inhibitors (SRIs) are the first-line pharmacologic treatment for BDD and are often efficacious (Phillips, Reference Phillips and Phillips2017). However, not all patients achieve response or remission, and thus it is important to identify predictors of treatment outcomes. For SRI treatment, intent-to-treat nonresponse rates range from 27% to 47%, and a completer analysis of the current study (the only medication study to report completer analyses) yielded a non-response rate of 19% (Phillips, Reference Phillips and Phillips2017). Likewise, for cognitive behavioral therapy (CBT) for BDD, intent-to-treat non-response rates range from 46% to 60% (Harrison, de la Cruz, Enander, Radua, & Mataix-Cols, Reference Harrison, de la Cruz, Enander, Radua and Mataix-Cols2016), and a completer analysis yielded non-response rates of 15% to 17% across two different sites in a recent trial (Wilhelm et al., Reference Wilhelm, Phillips, Greenberg, O'Keefe, Hoeppner, Keshaviah and Schoenfeld2019). Only a few SRI studies have examined predictor variables. Moreover, this research has focused principally on co-morbidity as the primary predictor rather than a more comprehensive set of constructs and variables that have relevance to BDD. Phillips, Dwight, and McElroy (Reference Phillips, Dwight and McElroy1998) found that comorbid major depressive disorder (MDD) and obsessive–compulsive disorder (OCD) did not predict the treatment response of BDD to fluvoxamine (n = 30). A randomized placebo-controlled trial of fluoxetine in 67 patients with BDD similarly found that treatment response was independent of the presence of comorbid MDD and OCD as well as severity and duration of BDD (Phillips, Albertini, & Rasmussen, Reference Phillips, Albertini and Rasmussen2002). Likewise, a small open-label trial of citalopram for BDD (n = 15) found that treatment response was as likely for those with and without MDD (Phillips & Najjar, Reference Phillips and Najjar2003). And in a double-blind cross-over trial of the SRI clomipramine v. the non-SRI antidepressant desipramine (n = 29), treatment efficacy was not moderated by the presence of comorbid MDD, OCD, or social anxiety disorder (SAD) (Hollander et al., Reference Hollander, Allen, Kwon, Aronowitz, Schmeidler, Wong and Simeon1999). In addition, in all of these studies, delusionality/insight of BDD beliefs did not predict treatment response.
There is, however, some limited evidence that personality disorder (PD) pathology might predict poorer outcomes of SRI treatment response in BDD, although findings are mixed. In two studies, comorbid PD did not predict response to the SRIs fluoxetine (n = 67) or fluvoxamine (n = 30), although this might be attributable to type II error (Phillips & McElroy, Reference Phillips and McElroy2000; Phillips et al., Reference Phillips, Albertini and Rasmussen2002). However, fluvoxamine responders had significantly fewer PDs than non-responders at the study baseline. And although the latter study did not find that neuroticism predicted SRI response, another, the larger study did (Fang, Porth, Phillips, & Wilhelm, Reference Fang, Porth, Phillips and Wilhelm2019). The primary report from the current study briefly noted that the only baseline variable that predicted BDD response was the presence of a PD (Phillips et al., Reference Phillips, Keshaviah, Dougherty, Stout, Menard and Wilhelm2016).
To our knowledge, these are the only studies that have examined predictors of SRI outcomes in BDD. None of these reports used supervised machine learning approaches, which have multiple advantages when examining predictors of the treatment outcome (see below). In addition, all prior reports examined treatment response but not remission, most were limited by small sample sizes, and most examined just a few potential predictors. Furthermore, given recent initiatives in promoting precision medicine frameworks in psychiatry and clinical psychology (Bernardini et al., Reference Bernardini, Attademo, Cleary, Luther, Shim, Quartesan and Compton2017; Hayes et al., Reference Hayes, Hofmann, Stanton, Carpenter, Sanford, Curtiss and Ciarrochi2019; Hofmann, Curtiss, & Hayes, Reference Hofmann, Curtiss and Hayes2020), it would be profitable to determine whether machine learning approaches would enhance our ability to predict who would be most likely to benefit from pharmacotherapy for BDD. Traditionally, researchers have used familiar statistical procedures such as ordinary least squares methods to test whether a small number of hypothesized psychological or demographic variables moderate treatment outcomes. Instead of examining variables individually as moderators of treatment outcome, an alternative approach is to leverage supervised machine learning to simultaneously investigate all potential predictors of interest. Machine learning can uncover patterns within a densely multivariate dataset to bolster a model's prediction accuracy in an independent dataset (Kuhn & Johnson, Reference Kuhn and Johnson2013). Such data analytic strategies are better poised to optimize the prediction of treatment outcomes at the individual level. More specifically, machine learning models can facilitate the development of ‘prediction calculators’, whereby a clinician may input certain demographic information and scores from clinical assessments into the calculator to determine the probability of a treatment being successful for an individual patient.
Although several prior studies have examined a limited number of predictor variables, as mentioned above, there has been no research employing state-of-the-art predictive modeling approaches such as machine learning to develop more refined predictive clinical tools for BDD treatment. Machine learning can assist in determining: (a) whether meaningful tools can be developed to predict individual treatment outcomes, and (b) what individual predictors contribute most to the accurate classification of treatment outcomes.
The current report is the first to focus on predictors of medication treatment response in BDD and, more specifically, to utilize machine learning to determine whether baseline clinical and demographic characteristics can predict treatment outcome with an SRI for BDD. This report leverages data from the largest study of SRI treatment for BDD (Phillips et al., Reference Phillips, Keshaviah, Dougherty, Stout, Menard and Wilhelm2016). Consistent with the primary goals of a machine learning approach, the current study adopted a wide array of predictors that have generally been studied in prior BDD studies and were available in the current dataset (Phillips, Reference Phillips and Phillips2017). Specifically, support vector machines (SVM) were used to predict three outcomes of interest: responder status, partial remission status, and full remission status. Being able to differentially predict whether any given patient with BDD will achieve one of these three outcomes can facilitate decision-making processes about what course of treatment is most advisable. To optimize our ability to predict treatment outcomes, recursive feature elimination (RFE) procedures were implemented to identify the best performing model using the most successful predictors. Furthermore, feature importance analyses were conducted to complement the primary SVM analyses to determine which baseline variables contributed most to prediction performance. The predictive modeling approach adopted in the current study may be able to promote better precision medicine tools for BDD treatment. Specifically, machine learning procedures may be able to inform an applied framework such as an online prediction calculator, whereby patient scores can be inputted to determine the likelihood of achieving a certain treatment outcome for SRI treatment. By identifying whether a patient will likely achieve remission, partial remission, or no response at all from SRI treatment, these tools can inform clinical decisions about whether SRI treatment is likely to be sufficient or whether additional interventions might be indicated (e.g. CBT). However, translating machine learning algorithms for use in clinical practice necessitates circumspection insofar as these prediction tools may be undermined by poor model performance and by being validated on non-representative samples (Senior, Fanshawe, Fazel, & Fazel, Reference Senior, Fanshawe, Fazel and Fazel2021). The current study provides the initial steps in leveraging machine learning in a precision medicine context for BDD treatment.
Methods
Participants
Participants in the study were 100 adults with a diagnosis of DSM-IV BDD. Diagnoses were obtained using the Structured Clinical Interview for DSM-IV Axis I PDs (SCID-I; First, Spitzer, Gibbon, & Williams, Reference First, Spitzer, Gibbon and Williams1997) and the Structured Clinical Interview for DSM-IV Axis II PDs (SCID-II; First, Gibbon, Spitzer, Williams, & Benjamin, Reference First, Gibbon, Spitzer, Williams and Benjamin1997). Further inclusion criteria included a score of at least ⩾24 on the Yale-Brown Obsessive–Compulsive Scale Modified for BDD (BDD-YBOCS, Phillips et al., Reference Phillips, Hollander, Rasmussen, Aronowitz, DeCaria and Goodman1997; Phillips, Hart, & Menard, Reference Phillips, Hart and Menard2014), reflecting BDD of at least moderate severity, and a score of at least moderate on the CGI Severity Scale (Guy, Reference Guy1976). Exclusion criteria included current or past bipolar disorder or a psychotic disorder, current clinically significant suicidality or a suicide attempt within the past year, substance abuse or dependence within the past 3 months, concurrent CBT, and use of psychotropic medication during the study or for 2 weeks before baseline assessment (6 weeks for fluoxetine). The current research is a secondary data analysis of the original trial (Phillips et al., Reference Phillips, Keshaviah, Dougherty, Stout, Menard and Wilhelm2016), and full inclusion and exclusion criteria are more fully described in the original publication.
Procedure
Full details about the experimental procedures for this clinical trial are reported in the original study (Phillips et al., Reference Phillips, Keshaviah, Dougherty, Stout, Menard and Wilhelm2016). As previously reported, participants were recruited from one of two sites (i.e. Massachusetts General Hospital or Butler Hospital and then Rhode Island Hospital, both affiliated with Brown University). The clinical trial consisted of two phases. In phase I, all participants received open-label escitalopram treatment (up to 30 mg/day) for 14 weeks. In phase II, responders to the initial treatment were randomly assigned to either 6 months of continuation of escitalopram treatment or to discontinuation of escitalopram and substitution with pill placebo. The current report uses data from phase I of the study, specifically pre-treatment baseline data and response and remission status after 14 weeks of escitalopram treatment.
Measures
Predictor variables
Several demographic variables were considered for the analysis, but the final models included gender and race because they were the two variables most likely to be associated with the outcome variables after a pre-screening process (see online Appendix I of Supplementary Methods). With respect to gender, all the participants identified as either male or female. Because there was a small proportion of participants who did not identify as white (16%), the race variable was binary coded to represent white and non-white, as further divisions would lead to categories with very low percentages that could not be analyzed.
Clinical variables included the following. The BDD-YBOCS (Phillips et al., Reference Phillips, Hollander, Rasmussen, Aronowitz, DeCaria and Goodman1997, Reference Phillips, Hart and Menard2014) is a 12-item semi-structured rater-administered scale adapted from the Yale-Brown Obsessive–Compulsive Scale, rating past-week BDD severity. Interrater reliability (intraclass correlations) on the BDD-YBOCS for all scale items and the total score was greater than 0.9 (Phillips et al., Reference Phillips, Keshaviah, Dougherty, Stout, Menard and Wilhelm2016). Reliability (Cronbach's alpha) of the scale in the current study was α = 0.78. The Clinical Global Impression Scale (CGI; Guy, Reference Guy1976) is a global rating scale that ranges from 1 (normal, not ill at all) to 7 (among the most extremely ill patients), which was used to assess BDD severity. The Brown Assessment of Beliefs Scale (BABS; Eisen et al., Reference Eisen, Phillips, Baer, Beer, Atala and Rasmussen1998) is a 7-item semi-structured rater-administered scale, assessing past-week BDD-related insight/delusional beliefs (e.g. ‘I am ugly’). Interrater reliability (intraclass correlations) on the BABS for all scale items and the total score was >0.9 (Phillips et al., Reference Phillips, Keshaviah, Dougherty, Stout, Menard and Wilhelm2016). Reliability (Cronbach's alpha) in the current dataset was α = 0.76. The Hamilton Depression Rating Scale (HAM-D; Miller, Bishop, Norman, & Maddever, Reference Miller, Bishop, Norman and Maddever1985) is a 17-item semi-structured instrument assessing the current severity of depressive symptoms (α = 0.78). The Q-LES-Q Short Form (Endicott, Nee, Harrison, & Blumenthal, Reference Endicott, Nee, Harrison and Blumenthal1993) assesses the quality of life in social, leisure, household, work, emotional well-being, physical, and school domains (α = 0.89). The Beck Depression Inventory II (BDI-II; Beck, Steer, & Brown, Reference Beck, Steer and Brown1996) is a 21-item, self-report questionnaire assessing depression (α = 0.92). The Beck Hopelessness Scale (BHS; Beck & Steer, Reference Beck and Steer1988) is a 20-item, self-report questionnaire assessing major aspects of hopelessness including feelings about the future, loss of motivation, and expectations (α = 0.92). The Brief Symptom Inventory (BSI; Derogatis & Melisaratos, Reference Derogatis and Melisaratos1983) is a 53-item, self-report instrument assessing a broad range of general psychopathology and distress (α = 0.97). Finally, the presence of certain comorbid Axis I diagnoses [i.e. SAD, MDD, and OCD] and the presence of any Axis-II PD were assessed with the SCID-I and SCID-II. Thus, a total of 14 demographic and clinical predictors were used.
Outcome variables
The primary outcomes of interest in the current machine learning study were response status (i.e. a clinically meaningful improvement of symptoms relative to the beginning of treatment), partial remission status (i.e. a meaningful improvement in symptoms and no longer meeting full diagnostic criteria for BDD, yet residual symptoms remain), and full remission status (i.e. being BDD-free). We examined treatment response because this is the standard outcome in pharmacotherapy trials. However, response in BDD is defined as only 30% or greater improvement in BDD symptoms (Fernández de la Cruz et al., Reference Fernández de la Cruz, Enander, Rück, Wilhelm, Phillips, Steketee and Veale2021; Phillips et al., Reference Phillips, Hollander, Rasmussen, Aronowitz, DeCaria and Goodman1997, Reference Phillips, Hart and Menard2014); while this degree of improvement is clinically meaningful, patients may have substantial remaining BDD symptoms. Thus, it is important to also examine more substantial improvement, such as partial remission and full remission (i.e. symptom-free). We defined these treatment outcomes in a manner consistent with previous research that has empirically characterized these states in BDD (Fernández de la Cruz et al., Reference Fernández de la Cruz, Enander, Rück, Wilhelm, Phillips, Steketee and Veale2021; Phillips et al., Reference Phillips, Hollander, Rasmussen, Aronowitz, DeCaria and Goodman1997, Reference Phillips, Hart and Menard2014). Specifically, treatment response is characterized by a 30% or greater reduction in BDD-YBOCS scores, whereas achieving at least partial remission is defined as a BDD-YBOCS score of less than or equal to 16. Consistent with Fernández de la Cruz et al. (Reference Fernández de la Cruz, Enander, Rück, Wilhelm, Phillips, Steketee and Veale2021), response and partial remission were stipulated as lasting for at least one week, which differs from the operationalization by Phillips et al. (Reference Phillips, Keshaviah, Dougherty, Stout, Menard and Wilhelm2016) requiring a reduction in symptoms being preserved for at least two consecutive assessment points. Because a cutpoint for full remission has not been established for the BDD-YBOCS, the field's primary treatment outcome measure, we characterized full remission by a score of less than or equal to 2 on the Psychiatric Status Rating Scale for Body Dysmorphic Disorder (BDD-PSR), which is a reliable global 7-point scale measuring BDD severity and diagnostic status. This cutpoint on the BDD-PSR was used in the primary paper from this study to determine full remission (Phillips, Pagano, Menard, & Stout, Reference Phillips, Pagano, Menard and Stout2006).
Data analysis
To predict treatment response, partial remission, and full remission, machine learning algorithms were evaluated using all the aforementioned predictor variables. Initially, several model algorithms were considered and compared, as indicated in the online Supplementary Methods section. Overall, a radial kernel SVM algorithm was evaluated to the outcomes of interest as it exhibited the best performance compared to other algorithms (see online Supplementary Tables S1–S3). SVM features a number of advantages that make it particularly suited to the current project. Specifically, SVM procedures attempt to maximize generalizability (i.e. how accurately the model performs), are suitable for situations in which there are relatively smaller sample sizes and a larger number of predictor variables, and are robust to outliers (Boehmke & Greenwell, Reference Boehmke and Greenwell2019). Machine learning models were examined using 10-fold cross-validation, which partitions the sample into 10 subsets, of which nine are used in the training process and predictions are made in the remaining subset. This process is repeated for each of the remaining 10 subsets, with each of the 10 subsets being used exactly once as the testing data. Results of the 10 folds are averaged to produce a single estimate.
To appraise classification performance, receiver operator characteristics (ROC) and area under the curve (AUC) metrics were calculated. An AUC value of 0.5 denotes discrimination between classes at the chance level, and values greater than 0.5 denote successful classification (i.e. maximize the true positive rate and minimize the false positive rate). Although the exact meaning of AUC values must be interpreted in relation to the particular classification problem of interest, we will adopt the following generally accepted AUC framework as an interpretive guideline: AUC = 0.50 reflects no discrimination, 0.70 ⩽ AUC ⩽ 0.80 reflects acceptable discrimination, and AUC ⩾ 0.80 reflects excellent discrimination (Hosmer & Lemeshow, Reference Hosmer and Lemeshow1999). Also, standard ROC metrics were evaluated such as sensitivity (i.e. the proportion of positives correctly identified) and specificity (i.e. the proportion of negatives that are correctly identified). These metrics range between 0 and 1 such that larger values indicate better performance. Accuracy, which is the percentage of total items classified correctly, was estimated as well.
When predicting categorical classification outcomes using machine learning, it is important to consider whether there are class imbalances in the outcome variable. Large class imbalances can bias machine learning results toward favoring prediction of the more frequent outcome. Because there appeared to be class imbalances in two of the three outcomes of interest (i.e. response status and full remission status), a procedure was adopted to improve machine learning performance for these two unbalanced outcomes. Specifically, a procedure referred to as the Synthetic Minority Over-sampling Technique (SMOTE) was implemented to improve class balance (Chawla, Bowyer, Hall, & Kegelmeyer, Reference Chawla, Bowyer, Hall and Kegelmeyer2002). In brief, this technique combines over-sampling of the minority class and under-sampling of the majority class to create a more balanced class structure for the outcome variable.
Because the primary objective of machine learning is to optimize predictive performance, it can be helpful to determine which subset of predictors results in the best classification performance. Thus, RFE procedures were implemented to identify the best performing model using the most successful predictors. In brief, RFE is an iterative process by which a machine learning model is trained and tested with varying subsets of the predictor variables, and a final model is fitted with the optimal subset of predictors (Kuhn & Johnson, Reference Kuhn and Johnson2013). In the current study, the RFE algorithm was estimated for subset sizes ranging from 1 to 14 (i.e. up to the total number of predictors available). Results are presented for the best-performing, final model containing the most optimal subset of predictors.
Furthermore, feature importance analyses were conducted to determine the ranked order of predictors in terms of their predictive power. For each predictor, an individual AUC value was estimated to indicate individual predictive performance. Feature importance values are presented for both the predictors in the final best performing model, as well as all other predictors used to build the initial SVM model prior to RFE procedures being implemented.
Because SVM algorithms are often considered ‘black-box’ models, for which there is no readily meaningful interpretation of the predictor weights, partial dependence plots (PDP) were estimated to visualize the relationship between a given predictor and the outcome (Boehmke & Greenwell, Reference Boehmke and Greenwell2019). PDPs display the probability of the outcome variable being a certain value (e.g. responder status) for each value of a predictor variable. PDPs were estimated for the top three predictors in each of the final models for each of the three outcomes. Analyses were performed in the R Caret package (Kuhn, Reference Kuhn2008).
Results
Participant characteristics
A total of 100 participants with BDD were included in the intent-to-treat population in Phase I, and the 97 subjects with a postbaseline assessment (i.e. were in the study long enough to have an assessment after the initial intake assessment) were included in the current study. Demographic characteristics are presented in Table 1. The mean age was 33.5 (s.d. = 12.4) with a range between 18 and 68, and 64% of the participants identified as female. With respect to race, 84% of participants identified as White, 7% as Black, 2% as Asian, 1% as American Indian, 1% as Alaskan Native, and 5% as multi-racial. Regarding ethnicity, 12% identified as Latinix. The mean baseline BDD-YBOCS score was 32.72 (s.d. = 5.43) with a range between 24 and 46, which reflects moderate to severe symptoms. Overall, in the intent-to-treat sample, 72% of participants achieved a response, 51% achieved partial remission, and 20% achieved full remission [these percentages differ slightly from those in this study's primary report (Phillips et al., Reference Phillips, Keshaviah, Dougherty, Stout, Menard and Wilhelm2016), reflecting slight differences in definitions used for treatment response]. Full demographic and clinical characteristics are also provided in the primary report from this study (Phillips et al., Reference Phillips, Keshaviah, Dougherty, Stout, Menard and Wilhelm2016).
Table 1. Demographic characteristics
N, number; s.d., standard deviation.
Note: Values are based on the original full sample of 100 patients.
Machine-learning prediction of response status
Results of the 10-fold cross-validation SVM analysis using RFE procedures revealed that the best performing model for treatment response contained five predictors (i.e. BHS, PD, CGI-BDD, BDI-II, and BSI). This final model exhibited acceptable classification performance with an AUC of 0.77. Sensitivity was 0.77, and specificity was 0.63. Accuracy was 0.73 (95% CI 0.63–0.82). The best model performance was associated with a cost parameter of 0.5 and a sigma tuning parameter of 0.2732069. Results of the feature importance analysis revealed that hopelessness (BHS), having a PD, and BDD severity (CGI) were the most important three predictors. Feature importance results for all predictors, as well as those emphasized in the final model, are presented in Table 2. PDPs for the top three predictors are displayed in Fig. 1. In general, higher scores of hopelessness and having a PD were associated with lower probabilities of achieving treatment response, whereas higher scores of BDD symptom severity on the CGI were associated with a greater probability of achieving treatment response.
Fig. 1. PDP of top features for a response.
Note: In the partial dependence plots, the y axis (i.e. yhat) denotes the probability of predicting response status given a particular value of the predictor. In general, higher scores of hopelessness and having a personality disorder were associated with lower probabilities of achieving treatment response, whereas higher scores of BDD symptom severity on the CGI were associated with a greater probability of achieving treatment response. PDP, partial dependence plot; BHS, Beck Hopelessness Scale; PD, personality disorder diagnosis; CGI-BDD, clinical global impression of BDD severity.
Table 2. Feature Importance for response, partial remission, and full remission
FI, feature importance; PD, having any personality disorder; SAD, social anxiety disorder; MDD, major depressive disorder; OCD, obsessive–compulsive disorder.
Note: The final models identified by RFE are highlighted in grey. For response status, RFE included the top 5 predictors. For partial remission status, RFE included the top 11 predictors. For full remission status, RFE included the top 2 predictors. Values denote area under the ROC curve for each predictor, indicating relative predictive power. Higher values indicate a better classification of responder status.
Machine-learning prediction of partial remission status
RFE procedures revealed that the best performing model contained eleven predictors (i.e. BSI, BDI-II, QLESQ, CGI-BDD, BDD-YBOCS, PD, BHS, BABS, HAM-D, OCD, and MDD). The final SVM model exhibited acceptable classification performance with an AUC of 0.75. Sensitivity was 0.67, and specificity was 0.73. Accuracy was 0.70 (95% CI 0.60–0.79). The best model performance was associated with a cost parameter of 1 and a sigma tuning parameter of 0.07505707. Results of the feature importance analysis of the final model revealed that general psychopathology (BSI), depression (BDI-II), and quality of life (QLESQ) were the most important predictors, respectively. Feature importance results for all predictors, as well as those emphasized in the final model, are presented in Table 2. PDPs for the top three predictors are displayed in Fig. 2. In general, higher scores of overall psychopathology and of depression were associated with lower probabilities of achieving partial remission, whereas higher scores of quality of life were associated with a greater probability of achieving partial remission.
Fig. 2. PDP of top features for partial remission.
Note: In the partial dependence plots, the y axis (i.e. yhat) denotes the probability of predicting partial remission status given a particular value of the predictor. In general, higher scores of overall psychopathology and of depression were associated with lower probabilities of achieving partial remission, whereas higher (better) scores of quality of life were associated with a greater probability of achieving partial remission. PDP, partial dependence plot; BSI, Brief Symptom Inventory; BDI, Beck Depression Inventory; QOL, quality of life.
Machine-learning prediction of full remission status
Predicting only those patients who achieved full remission with RFE procedures resulted in the best performing model with two predictors (i.e. QLESQ and BSI). The final SVM model yielded acceptable classification performance with an AUC of 0.79. Sensitivity was 0.70, and specificity was 0.79. Accuracy was 0.76 (95% CI 0.64–0.85). The best model performance was associated with a cost parameter of 0.25 and a sigma tuning parameter of 1.574244. Results of the feature importance analysis revealed that quality of life (QLESQ) and general psychopathology (BSI) were the only important predictors in the final model. Feature importance results for all predictors, as well as those emphasized in the final model, are presented in Table 2. PDPs for the only two predictors in the final model are displayed in Fig. 3. In general, higher scores of overall psychopathology were associated with lower probabilities of achieving full remission, whereas higher scores of quality of life were associated with a greater probability of achieving full remission.
Fig. 3. PDP of top features for full remission.
Note: In the partial dependence plots, the y axis (i.e. yhat) denotes the probability of predicting full remission status given a particular value of the predictor. In general, higher scores of overall psychopathology were associated with lower probabilities of achieving full remission, whereas higher (better) scores of quality of life were associated with a greater probability of achieving full remission. PDP, partial dependence plot; BSI, Brief Symptom Inventory; QOL, quality of life.
Discussion
The current study provides the first evidence that machine learning algorithms can successfully predict treatment outcomes for pharmacotherapy for BDD. In the BDD literature, very little prognostic information exists about what factors are predictive of successful pharmacotherapy treatment. Results of the final SVM models identified using RFE procedures indicated acceptable prediction of each of the three primary outcomes (i.e. response, partial remission, and full remission), as AUC values were all above 0.70. Furthermore, feature importance analyses supported constructs such as quality of life, depression symptoms, general psychopathology symptoms, and hopelessness as most predictive of treatment outcomes. The presence of a PD was more strongly predictive of poorer treatment response and partial remission rather than full remission. Demographic variables such as gender and race were the least predictive of treatment outcome. By probing the PDP plots, it appears that higher levels of some psychopathology measures (e.g. depression, general psychopathology, hopelessness, PD) were associated with less favorable treatment outcomes, whereas the better quality of life and more severe BDD as assessed by the CGI were predictive of better outcomes.
By embracing precision medicine methodologies such as machine learning, much more rigorous clinical tools can be developed to predict whether a given patient will attain a successful treatment outcome. Within the broader context of machine learning in psychiatry, model performance in the current study is similar to or better than that of other machine learning studies predicting outcomes for other disorders such as depression (Chekroud et al., Reference Chekroud, Zotti, Shehzad, Gueorguieva, Johnson, Trivedi and Corlett2016; Nie, Vairavan, Narayan, Ye, & Li, Reference Nie, Vairavan, Narayan, Ye and Li2018) and social anxiety (Hoogendoorn, Berger, Schulz, Stolz, & Szolovits, Reference Hoogendoorn, Berger, Schulz, Stolz and Szolovits2016). Furthermore, the successful performance of the machine learning models in the present study is noteworthy given that it relied only on baseline self-report and clinical interview data, which is more feasible than requiring costly neuroimaging or genetic data as has been emphasized in other machine learning studies in psychiatry (Lee et al., Reference Lee, Ragguett, Mansur, Boutilier, Rosenblat, Trevizol and McIntyre2018).
AUC values for the overall models were in the acceptable range for all three treatment outcomes, although relatively few individual predictor variables were in the acceptable range (above 0.70). This is consistent with prior studies, which did not consistently identify any predictors of BDD response to an SRI, although most prior studies had limited statistical power for this purpose. This attests to the value of leveraging machine learning models which can uncover patterns across individual predictors in the dataset to bolster overall prediction accuracy.
The current machine learning models may provide a platform for facilitating a more informed decision-making process about the potential utility of escitalopram for BDD for individual patients. However, using such models also incur potential risks for misclassification. For example, patients might decide to forgo the first-line medication treatment (an SRI) for this severe disorder when it in fact might actually prove efficacious for them. Because none of the models exhibits perfect classification performance and might differ in different patient populations or with different medications, clinicians and patients can collectively deliberate on the role of uncertainty in these predictions and collaborate on the extent to which it is advisable to follow a machine learning model's recommendation.
Greater general psychopathology on the BSI did predict poorer outcomes for partial and full remission (but not treatment response), with AUCs in the acceptable range. A possible explanation for this finding is that the BSI contains some items that might interfere with treatment response. For instance, feeling others are to blame for most of your troubles, or having ideas that someone else can control your thoughts, might inhibit social interaction and functioning, which is assessed by the BDD-YBOCS. It is unclear why more severe depression on the BDI-II reached the acceptable range for prediction of partial and full remission (but not for response), as our clinical impression is that depression severity does not affect improvement with SRI treatment. Moreover, comorbid MDD was one of the weaker predictors of treatment outcome. Perhaps, the BDI-II had more power to predict the outcomes because it is a continuous variable, whereas the dichotomous nature of the MDD variable conveys less information, which may undermine its predictive power.
Regarding predictors of treatment response, it is unclear why hopelessness was the most salient predictor. However, hopelessness has also been shown to predict poorer response to SRI treatment in MDD (e.g. Papakostas et al., Reference Papakostas, Petersen, Homberger, Green, Smith, Alpert and Fava2007). It is possible that greater hopelessness is associated with less expectation of improvement, which in turn might be associated with poorer outcomes (Papakostas et al., Reference Papakostas, Petersen, Homberger, Green, Smith, Alpert and Fava2007). Indeed, some studies of CBT for BDD have found that a lower expectancy of improvement predicts poorer treatment outcomes (Greenberg, Phillips, Steketee, Hoeppner, & Wilhelm, Reference Greenberg, Phillips, Steketee, Hoeppner and Wilhelm2019). It is worth noting that greater BDD severity was the third strongest predictor of response to treatment, although BDD severity did not predict SRI response in the only studies that have examined this variable (Phillips et al., Reference Phillips, Albertini and Rasmussen2002, Reference Phillips, Keshaviah, Dougherty, Stout, Menard and Wilhelm2016).
Our finding that comorbid MDD, OCD, and SAD did not predict treatment response much better than chance is consistent with clinical experience and with results from prior studies. The presence of comorbid PD was overall a stronger predictor of treatment response and partial remission than Axis I comorbidity. Studies of a broad range of Axis I disorders indicate that PDs tend to have an adverse effect on treatment response (Reich & Vasile, Reference Reich and Vasile1993), although BDD studies that have examined this issue have had mixed findings (Phillips et al., Reference Phillips, Albertini and Rasmussen2002, Reference Phillips, Keshaviah, Dougherty, Stout, Menard and Wilhelm2016).
Our finding that BDD-related delusionality/insight, as assessed by the BABS, did not predict response and full remission much better than chance is consistent with prior studies indicating that SRI monotherapy is equally efficacious for both delusional and nondelusional BDD (Phillips, Reference Phillips and Phillips2017). This has been a consistent and interesting finding in all prior BDD pharmacotherapy studies and is worth highlighting because SRI monotherapy is not considered efficacious for other disorders that are often characterized by delusional beliefs.
A principal strength of the current study is that machine learning models were validated across three different levels of the treatment outcome (i.e. response, partial remission, and full remission). SVM algorithms accomplished acceptable classification discrimination in predicting all outcomes. Furthermore, both sensitivity and specificity across all three models were largely balanced, indicating that the models exhibited acceptable ability in both ruling-in and ruling-out each of the three outcomes. The model predicting response status has slightly higher sensitivity than specificity, indicating it might be slightly more well-suited in ruling out someone being a ‘responder’. Moreover, the algorithm predicting full remission possessed slightly better specificity than sensitivity, perhaps indicating this model is somewhat better at ruling in an outcome (i.e. being a ‘full-remitter’). Another important aspect of the current study was the feature importance analyses, which determine which predictors are the most important in contributing to classification performance.
Notwithstanding the strengths of the current study, certain limitations warrant mention. First, from a clinical perspective, it is not known whether similar findings would pertain to the higher doses of escitalopram that can be used to treat BDD in clinical practice (Phillips, Reference Phillips and Phillips2017). Second, the sample size is relatively modest in the context of traditional machine learning studies. That notwithstanding, successful machine learning studies have been accomplished using smaller sample sizes (e.g. Flygare et al., Reference Flygare, Enander, Andersson, Ljótsson, Ivanov, Mataix-Cols and Rück2020), and approaches were adopted in the current study to optimize performance, given the sample size. Specifically, SVM algorithms have utility in smaller sample sizes (Boehmke & Greenwell, Reference Boehmke and Greenwell2019), and 10-fold cross-validation was employed to prevent biasing the model training on any single subsample of the dataset. Future research would benefit from replicating these results in more diversified samples in terms of race and ethnicity to enhance generalizability. Third, for two of the outcomes (i.e. response and full remission status), the outcomes were somewhat imbalanced. To mitigate model bias, the SMOTE procedure was utilized to create a more balanced class profile on which to train the models. Nonetheless, appropriate caution should be undertaken when evaluating model results for the response and full remission outcomes, and future studies with larger samples sizes are needed to determine whether the current results are replicated. Fourth, SVM is often regarded as a ‘black-box’ modeling technique insofar as the predictor weights do not have an inherent interpretive meaning. In an effort to better explicate the directionality of the relationship between the top predictors and treatment outcomes in each model, PDPs were produced to visualize relationships between predictors and outcomes. Fifth, the number of predictors in the current study was relatively modest compared to what might be typical for machine learning studies leveraging big data. It would be profitable for future studies to model much larger numbers of predictors to better identify which features are most predictive of treatment success for BDD patients. Furthermore, it may be profitable to consider individual item level data as predictors in addition to aggregate sum scores. Sixth, a traditional 10-fold cross-validation procedure was used such that each of the 10 subsets of the data was used exactly once as the testing data. This is a limitation, as performance estimates for training cross-validation tend to be optimistic relative to validation set performance. In future machine learning research, it would be especially beneficial to consider strategies to leverage multiple trial datasets to permit model testing in whole, novel datasets that were not used for training purposes. Finally, it is rare that any machine learning model yields perfectly accurate predictions, and this is true of the models validated in the current study. There is always a risk of misclassification, and, therefore, it would be necessary for clinicians to discuss the potential consequences of misclassification (e.g. waste of time in expecting treatment response, potential medication side-effects without intended therapeutic benefit, forgoing a medication that would actually be beneficial, etc.) with the patient and prescriber collaboratively making an informed decision. Importantly, a broader range of clinical factors than were examined in this report must be considered when deciding whether to treat with an SRI. For example, patients with a high degree of suicidality, which is common in BDD (Snorrason et al., Reference Snorrason, Beard, Christensen, Bjornsson and Björgvinsson2019) but who were not included in this study, should always receive an SRI (Phillips, Reference Phillips and Phillips2017). Thus, machine learning models are best viewed as tools that provide useful information for treatment planning rather than rigid prescriptions about treatment courses.
Overall, these results support the utility of machine learning in predicting three important treatment outcomes for pharmacotherapy for BDD. Consistent with precision medicine initiatives in psychiatry (Bernardini et al., Reference Bernardini, Attademo, Cleary, Luther, Shim, Quartesan and Compton2017), the current study provides the foundation for personalized pharmacotherapy strategies for patients with BDD, although further studies are needed. Future directions include replicating the machine learning framework of the current study for SRI treatment and other types of treatment for BDD (e.g. other medications, CBT, combined pharmacotherapy and CBT). By doing so, it may be possible to develop online prediction calculators to determine the likelihood of response and remission for any given treatment, which can facilitate shared decision making with a clinician. Differential merits about individual interventions can be evaluated to determine the most appropriate course of treatment for a single patient with BDD. A noteworthy benefit of the current study is that successful predictive modeling for BDD treatment response can be accomplished using accessible and cost-effective data from self-report and clinical interview assessments. By facilitating the dual goals of leveraging precision medicine and data feasibility, machine learning approaches to BDD treatment are poised to advance the current standard of patient care and improve outcomes at the individual level.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S0033291721005390
Financial support
This original trial presented in this paper was funded by a Collaborative R01 grant from the National Institute of Mental Health to Dr Phillips (R01 MH072917) and Dr Wilhelm (R01 MH072854).
Conflict of interest
K.A.P. has received royalties from the following publishers: Oxford University Press, International Creative Management, Inc., UpToDate/Wolter's Kluwer, Guilford Publications, American Psychiatric Association Publishing. Furthermore, she has received speaking honoraria from academic institutions and professional societies. S.W. is a presenter for the Massachusetts General Hospital Psychiatry Academy in educational programs supported through independent medical education grants from pharmaceutical companies; she has received royalties from Elsevier Publications, Guilford Publications, New Harbinger Publications, Springer, and Oxford University Press. S.W. has also received speaking honoraria from various academic institutions and foundations, including the International Obsessive–Compulsive Disorder Foundation, Tourette Association of America, and Brattleboro Retreat. In addition, she received payment from the Association for Behavioral and Cognitive Therapies for her role as Associate Editor for the Behavior Therapy journal, as well as from John Wiley & Sons, Inc. for her role as Associate Editor on the journal Depression & Anxiety. Dr Wilhelm has also received honoraria from One-Mind for her role on the PsyberGuide Scientific Advisory Board. S.W is also on the Scientific Advisory Board for Koa Health and the Scientific Advisory Board for Noom. J.E.C. has received book royalties from New Harbinger Press. E.E.B. has no conflicts of interest to declare.
