Hostname: page-component-cd9895bd7-dzt6s Total loading time: 0 Render date: 2025-01-03T16:07:20.619Z Has data issue: false hasContentIssue false
  • English
  • Français

A practical risk calculator for suicidal behavior among transitioning U.S. Army soldiers: results from the Study to Assess Risk and Resilience in Servicemembers-Longitudinal Study (STARRS-LS)

Published online by Cambridge University Press:  09 March 2023

Jaclyn C. Kearns ,
Emily R. Edwards ,
Erin P. Finley ,
Joseph C. Geraci ,
Sarah M. Gildea ,
Marianne Goodman ,
Irving Hwang ,
Chris J. Kennedy ,
Andrew J. King  and
Alex Luedtke
...Show all authors
Jaclyn C. Kearns
Affiliation:
National Center for PTSD, VA Boston Healthcare System, Boston, MA, USA Department of Psychiatry, Boston University School of Medicine, Boston, MA, USA
Emily R. Edwards
Affiliation:
Transitioning Servicemember/Veteran And Suicide Prevention Center (TASC), VISN 2 Mental Illness Research, Education and Clinical Center, James J. Peters VA Medical Center, Bronx, New York, NY, USA Department of Psychiatry, Icahn School of Medicine at Mount Sinai, New York, NY, USA
Erin P. Finley
Affiliation:
Center of Excellence for Research on Returning War Veterans, VISN 17, Doris Miller VA Medical Center, Waco, TX, USA Center for the Study of Healthcare Innovation, Implementation, and Policy (CSHIIP), VA Greater Los Angeles Healthcare System, Los Angeles, CA, USA
Joseph C. Geraci
Affiliation:
Transitioning Servicemember/Veteran And Suicide Prevention Center (TASC), VISN 2 Mental Illness Research, Education and Clinical Center, James J. Peters VA Medical Center, Bronx, New York, NY, USA Department of Psychiatry, Icahn School of Medicine at Mount Sinai, New York, NY, USA Center of Excellence for Research on Returning War Veterans, VISN 17, Doris Miller VA Medical Center, Waco, TX, USA Resilience Center for Veterans & Families, Teachers College, Columbia University, New York, NY, USA
Sarah M. Gildea
Affiliation:
Department of Health Care Policy, Harvard Medical School, Boston, MA, USA
Marianne Goodman
Affiliation:
Transitioning Servicemember/Veteran And Suicide Prevention Center (TASC), VISN 2 Mental Illness Research, Education and Clinical Center, James J. Peters VA Medical Center, Bronx, New York, NY, USA Center of Excellence for Research on Returning War Veterans, VISN 17, Doris Miller VA Medical Center, Waco, TX, USA
Irving Hwang
Affiliation:
Department of Health Care Policy, Harvard Medical School, Boston, MA, USA
Chris J. Kennedy
Affiliation:
Department of Psychiatry, Massachusetts General Hospital, Boston, MA, USA
Andrew J. King
Affiliation:
Department of Health Care Policy, Harvard Medical School, Boston, MA, USA
Alex Luedtke
Affiliation:
Department of Statistics, University of Washington, Seattle, WA, USA Vaccine and Infectious Disease Division, Fred Hutchinson Cancer Research Center, Seattle, WA, USA
Brian P. Marx
Affiliation:
National Center for PTSD, VA Boston Healthcare System, Boston, MA, USA Department of Psychiatry, Boston University School of Medicine, Boston, MA, USA
Maria V. Petukhova
Affiliation:
Department of Health Care Policy, Harvard Medical School, Boston, MA, USA
Nancy A. Sampson
Affiliation:
Department of Health Care Policy, Harvard Medical School, Boston, MA, USA
Richard W. Seim
Affiliation:
Center of Excellence for Research on Returning War Veterans, VISN 17, Doris Miller VA Medical Center, Waco, TX, USA
Ian H. Stanley
Affiliation:
Department of Emergency Medicine, University of Colorado School of Medicine, Aurora, CO, USA Department of Emergency Medicine, Center for COMBAT Research, University of Colorado School of Medicine, Aurora, CO, USA
Murray B. Stein
Affiliation:
Department of Psychiatry, University of California San Diego, La Jolla, CA, USA School of Public Health, University of California San Diego, La Jolla, CA, USA VA San Diego Healthcare System, La Jolla, CA, USA
Robert J. Ursano
Affiliation:
Department of Psychiatry, Center for the Study of Traumatic Stress, Uniformed Services University of the Health Sciences, Bethesda, MD, USA
Ronald C. Kessler*
Affiliation:
Department of Health Care Policy, Harvard Medical School, Boston, MA, USA
*
Author for correspondence: Ronald C. Kessler, E-mail: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

Background

Risk of suicide-related behaviors is elevated among military personnel transitioning to civilian life. An earlier report showed that high-risk U.S. Army soldiers could be identified shortly before this transition with a machine learning model that included predictors from administrative systems, self-report surveys, and geospatial data. Based on this result, a Veterans Affairs and Army initiative was launched to evaluate a suicide-prevention intervention for high-risk transitioning soldiers. To make targeting practical, though, a streamlined model and risk calculator were needed that used only a short series of self-report survey questions.

Methods

We revised the original model in a sample of n = 8335 observations from the Study to Assess Risk and Resilience in Servicemembers-Longitudinal Study (STARRS-LS) who participated in one of three Army STARRS 2011–2014 baseline surveys while in service and in one or more subsequent panel surveys (LS1: 2016–2018, LS2: 2018–2019) after leaving service. We trained ensemble machine learning models with constrained numbers of item-level survey predictors in a 70% training sample. The outcome was self-reported post-transition suicide attempts (SA). The models were validated in the 30% test sample.

Results

Twelve-month post-transition SA prevalence was 1.0% (s.e. = 0.1). The best constrained model, with only 17 predictors, had a test sample ROC-AUC of 0.85 (s.e. = 0.03). The 10–30% of respondents with the highest predicted risk included 44.9–92.5% of 12-month SAs.

Conclusions

An accurate SA risk calculator based on a short self-report survey can target transitioning soldiers shortly before leaving service for intervention to prevent post-transition SA.

Keywords

Machine learningsuicide attemptsuicide preventionveterans
Type
Original Article
Information
Psychological Medicine , Volume 53 , Issue 15 , November 2023 , pp. 7096 - 7105
Check for updates
Copyright
Copyright © The Author(s), 2023. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

116th Congress (2019–2020). (2020). Commander John Scott Hannon Veterans Mental Health Care Improvement Act of 2019. Retrieved from https://www.congress.gov/bill/116th-congress/senate-bill/785.Google Scholar
Beymer, M. R., Reagan, J. J., Rabbitt, M. P., Webster, A. E., & Watkins, E. Y. (2021). Association between food insecurity, mental health, and intentions to leave the U.S. Army in a cross-sectional sample of U.S. Soldiers. Journal of Nutrition, 151(7), 2051–2058. doi: 10.1093/jn/nxab089CrossRefGoogle Scholar
Bullman, T., Hoffmire, C., Schneiderman, A., & Bossarte, R. (2015). Time dependent gender differences in suicide risk among operation enduring freedom and operation Iraqi freedom veterans. Annals of Epidemiology, 25(12), 964–965. doi: 10.1016/j.annepidem.2015.09.008CrossRefGoogle ScholarPubMed
Chen, R. J., Chen, T. Y., Lipkova, J., Wang, J. J., Williamson, D. F. K., Lu, M. Y., … Mahmood, F. (2022). Algorithm Fairness in AI for Medicine and Healthcare. Retrieved from https://arxiv.org/abs/2110.00603.Google Scholar
Chu, C., Zuromski, K. L., Bernecker, S. L., Gutierrez, P. M., Joiner, T. E., Liu, H., … Nock, M. K. (2020). A test of the interpersonal theory of suicide in a large, representative, retrospective and prospective study: Results from the Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS). Behaviour Research and Therapy, 132, 103688. doi: 10.1016/j.brat.2020.103688CrossRefGoogle Scholar
Executive Office of the President. (2018). Supporting our veterans during their transition from uniformed service to civilian life. Retrieved from https://www.federalregister.gov/documents/2018/01/12/2018-00630/supporting-our-veterans-during-their-transition-from-uniformed-service-to-civilian-life.Google Scholar
Franklin, J. C., Ribeiro, J. D., Fox, K. R., Bentley, K. H., Kleiman, E. M., Huang, X., … Nock, M. K. (2017). Risk factors for suicidal thoughts and behaviors: A meta-analysis of 50 years of research. Psychological Bulletin, 143(2), 187–232. doi: 10.1037/bul0000084CrossRefGoogle ScholarPubMed
Hastie, T., Tibshirani, R., & Wainwright, M. (2016). Statistical learning with sparsity: The lasso and generalizations. New York, NY: Chapman and Hall/CRC.Google Scholar
Heeringa, S. G., Gebler, N., Colpe, L. J., Fullerton, C. S., Hwang, I., Kessler, R. C., … Ursano, R. J. (2013). Field procedures in the Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS). International Journal of Methods in Psychiatric Research, 22(4), 276–287. doi: 10.1002/mpr.1400CrossRefGoogle ScholarPubMed
Holliday, R., Borges, L. M., Stearns-Yoder, K. A., Hoffberg, A. S., Brenner, L. A., & Monteith, L. L. (2020). Posttraumatic stress disorder, suicidal ideation, and suicidal self-directed violence among U.S. Military personnel and veterans: A systematic review of the literature from 2010 to 2018. Frontiers in Psychology, 11, 1998. doi: 10.3389/fpsyg.2020.01998CrossRefGoogle ScholarPubMed
Hubbard, A. E., Kennedy, C. J., & van der Laan, M. J. (2018). Data-Adaptive target parameters. In van der Laan, Mark, & Rose, Sherri (Eds.), Targeted learning in data science: Causal inference for Complex longitudinal studies (pp. 125–142). Cham, Switzerland: Springer International Publishing.CrossRefGoogle Scholar
Kennedy, C. J. (2017). Guide to SuperLearner. Retrieved from https://cran.r-project.org/web/packages/SuperLearner/vignettes/Guide-to-SuperLearner.html.Google Scholar
Kessler, R. C., Bernecker, S. L., Bossarte, R. M., Luedtke, A. R., McCarthy, J. F., Nock, M. K., … Zaslavsky, A. M. (2019). The role of big data analytics in predicting suicide. In Passos, M. B. & Kapczinski, I. F. (Ed.), Personalized psychiatry (pp. 77–98). Cham, Switzerland: Springer International Publishing.CrossRefGoogle Scholar
Kessler, R. C., Colpe, L. J., Fullerton, C. S., Gebler, N., Naifeh, J. A., Nock, M. K., … Heeringa, S. G. (2013a). Design of the Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS). International Journal of Methods in Psychiatric Research, 22(4), 267–275. doi: 10.1002/mpr.1401CrossRefGoogle ScholarPubMed
Kessler, R. C., Furukawa, T. A., Kato, T., Luedtke, A., Petukhova, M., Sadikova, E., … Sampson, N. A. (2021). An individualized treatment rule to optimize probability of remission by continuation, switching, or combining antidepressant medications after failing a first-line antidepressant in a two-stage randomized trial. Psychological Medicine, 52(15), 3371–3380. doi: 10.1017/S0033291721000027CrossRefGoogle Scholar
Kessler, R. C., Heeringa, S. G., Colpe, L. J., Fullerton, C. S., Gebler, N., Hwang, I., … Ursano, R. J. (2013b). Response bias, weighting adjustments, and design effects in the Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS). International Journal of Methods in Psychiatric Research, 22(4), 288–302. doi: 10.1002/mpr.1399CrossRefGoogle ScholarPubMed
Klonsky, E. D., May, A. M., & Saffer, B. Y. (2016). Suicide, suicide attempts, and suicidal ideation. Annual Review of Clinical Psychology, 12, 307–330. doi: 10.1146/annurev-clinpsy-021815-093204CrossRefGoogle ScholarPubMed
Kraemer, H. C., Kazdin, A. E., Offord, D. R., Kessler, R. C., Jensen, P. S., & Kupfer, D. J. (1997). Coming to terms with the terms of risk. Archives of General Psychiatry, 54(4), 337–343. doi: 10.1001/archpsyc.1997.01830160065009CrossRefGoogle ScholarPubMed
Lee, D. J., Kearns, J. C., Wisco, B. E., Green, J. D., Gradus, J. L., Sloan, D. M., … Marx, B. P. (2018). A longitudinal study of risk factors for suicide attempts among Operation Enduring Freedom and Operation Iraqi Freedom veterans. Depression and Anxiety, 35(7), 609–618. doi: 10.1002/da.22736CrossRefGoogle ScholarPubMed
Lundberg, S., & Lee, S.-I. (2017). A unified approach to interpreting model predictions. Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS'17), Red Hook, NY. Retrieved from https://papers.nips.cc/paper/2017/hash/8a20a8621978632d76c43dfd28b67767-Abstract.html.Google Scholar
Manuel, J. I., Nizza, M., Herman, D. B., Conover, S., Esquivel, L., Yuan, Y., … Susser, E. (2022). Supporting vulnerable people during challenging transitions: A systematic review of critical time intervention. Administration and Policy in Mental Health and Mental Health Services Research, 50, 100–113. doi: 10.1007/s10488-022-01224-zCrossRefGoogle ScholarPubMed
Millner, A. J., Lee, M. D., & Nock, M. K. (2015). Single-item measurement of suicidal behaviors: Validity and consequences of misclassification. PLOS One, 10(10), e0141606. doi: 10.1371/journal.pone.0141606CrossRefGoogle ScholarPubMed
Moselli, M., Casini, M. P., Frattini, C., & Williams, R. (2023). Suicidality and personality pathology in adolescence: A systematic review. Child Psychiatry and Human Development, 54, 290–311.CrossRefGoogle ScholarPubMed
Nock, M. K., Deming, C. A., Fullerton, C. S., Gilman, S. E., Goldenberg, M., Kessler, R. C., … Ursano, R. J. (2013). Suicide among soldiers: A review of psychosocial risk and protective factors. Psychiatry: Interpersonal and Biological Processes, 76(2), 97–125. doi: 10.1521/psyc.2013.76.2.97CrossRefGoogle ScholarPubMed
Nock, M. K., Millner, A. J., Ross, E. L., Kennedy, C. J., Al-Suwaidi, M., Barak-Corren, Y., … Kessler, R. C. (2022). Prediction of suicide attempts using clinician assessment, patient self-report, and electronic health records. JAMA Network Open, 5(1), e2144373. doi: 10.1001/jamanetworkopen.2021.44373CrossRefGoogle ScholarPubMed
Plöderl, M., Wagenmakers, E. J., Tremblay, P., Ramsay, R., Kralovec, K., Fartacek, C., … Fartacek, R. (2013). Suicide risk and sexual orientation: A critical review. Archives of Sexual Behavior, 42(5), 715–727. doi: 10.1007/s10508-012-0056-yCrossRefGoogle ScholarPubMed
Polley, E. C., LeDell, E., Kennedy, C., Lendle, S., & van der Laan, M. J. (2018). SuperLearner: Super Learner Prediction [R package] (Version 2.0–24).Google Scholar
Posner, K., Brown, G. K., Stanley, B., Brent, D. A., Yershova, K. V., Oquendo, M. A., … Mann, J. J. (2011). The Columbia-suicide severity rating scale: Initial validity and internal consistency findings from three multisite studies with adolescents and adults. The American Journal of Psychiatry, 168(12), 1266–1277. doi: 10.1176/appi.ajp.2011.10111704CrossRefGoogle ScholarPubMed
R Core Team. (2021). R: A language and environment for statistical computing. Retrieved from https://www.r-project.org/.Google Scholar
Ravindran, C., Morley, S. W., Stephens, B. M., Stanley, I. H., & Reger, M. A. (2020). Association of suicide risk with transition to civilian life among U.S. Military service members. JAMA Network Open, 3(9), e2016261. doi: 10.1001/jamanetworkopen.2020.16261CrossRefGoogle ScholarPubMed
Reger, M. A., Smolenski, D. J., Skopp, N. A., Metzger-Abamukang, M. J., Kang, H. K., Bullman, T. A., … Gahm, G. A. (2015). Risk of suicide among U.S. Military service members following operation enduring freedom or operation Iraqi freedom deployment and separation from the U.S. Military. JAMA Psychiatry, 72(6), 561–569. doi: 10.1001/jamapsychiatry.2014.3195CrossRefGoogle ScholarPubMed
SAS Institute (2013). SAS ®software 9.4 (version 9.4). Cary, NC: SAS Institute Inc.Google Scholar
Shen, Y. C., Cunha, J. M., & Williams, T. V. (2016). Time-varying associations of suicide with deployments, mental health conditions, and stressful life events among current and former U.S. military personnel: A retrospective multivariate analysis. The Lancet. Psychiatry, 3(11), 1039–1048. doi: 10.1016/s2215-0366(16)30304-2CrossRefGoogle ScholarPubMed
Stanley, I. H., Chu, C., Gildea, S. M., Hwang, I. H., King, A. J., Kennedy, C. J., … Kessler, R. C. (2022). Predicting suicide attempts among U.S. Army soldiers after leaving active duty using information available before leaving active duty: Results from the Study to Assess Risk and Resilience in Servicemembers-Longitudinal Study (STARRS-LS). Molecular Psychiatry, 27(3), 1631–1639. doi: 10.1038/s41380-021-01423-4CrossRefGoogle Scholar
Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society. Series B (Methodological), 58(1), 267–288. Retrieved from http://www.jstor.org/stable/2346178.CrossRefGoogle Scholar
Ursano, R. J., Colpe, L. J., Heeringa, S. G., Kessler, R. C., Schoenbaum, M., & Stein, M. B. (2014). The Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS). Psychiatry: Interpersonal and Biological Processes, 77(2), 107–119. doi: 10.1521/psyc.2014.77.2.107CrossRefGoogle ScholarPubMed
U.S. Department of Veterans Affairs. (2014). Analysis of VA Health Care utilization among Operation Enduring Freedom (OEF), Operation Iraqi Freedom (OIF), and Operation New Dawn (OND) veterans. Retrieved from https://www.publichealth.va.gov/docs/epidemiology/healthcare-utilization-report-fy2013-qtr4.pdf.Google Scholar
U.S. Department of Veterans Affairs. (2020). VA Solid Start. Retrieved from https://www.benefits.va.gov/transition/solid-start.asp.Google Scholar
U.S. Department of Veterans Affairs. (2022). National Veteran Suicide Prevention Annual Report. Retrieved from https://www.mentalhealth.va.gov/docs/data-sheets/2022/2022-National-Veteran-Suicide-Prevention-Annual-Report-FINAL-508.pdf.Google Scholar
U.S. Secretary of Department of Defense, U.S. Secretary of Department of Veterans Affairs, & U.S. Secretary of Department of Homeland Security. (2018). Joint action plan for supporting veterans during their transition from uniformed service to civilian life. Retrieved from https://www.va.gov/opa/docs/joint-action-plan-05-03-18.pdf.Google Scholar
van Smeden, M., de Groot, J. A. H., Moons, K. G. M., Collins, G. S., Altman, D. G., Eijkemans, M. J. C., & Reitsma, J. B. (2016). No rationale for 1 variable per 10 events criterion for binary logistic regression analysis. BMC Medical Research Methodology, 16(1), 163. doi: 10.1186/s12874-016-0267-3CrossRefGoogle ScholarPubMed
Wilson, J., Jones, M., Fear, N. T., Hull, L., Hotopf, M., Wessely, S., … Rona, R. J. (2009). Is previous psychological health associated with the likelihood of Iraq War deployment? An investigation of the “Healthy Warrior Effect”. American Journal of Epidemiology, 169(11), 1362–1369. doi: 10.1093/aje/kwp044CrossRefGoogle ScholarPubMed
Supplementary material: File

Kearns et al. supplementary material
Download undefined(File)
File 153.3 KB