Hostname: page-component-7bb8b95d7b-nptnm Total loading time: 0 Render date: 2024-09-19T07:41:15.990Z Has data issue: true hasContentIssue false
  • English
  • Français

Characterizing Veteran suicide decedents that were not classified as high-suicide-risk

Published online by Cambridge University Press:  16 September 2024

Maxwell Levis Open the ORCID record for Maxwell Levis [Opens in a new window] ,
Monica Dimambro ,
Joshua Levy ,
Vincent Dufort ,
Abby Fraade ,
Max Winer  and
Brian Shiner
Maxwell Levis*
Affiliation:
White River Junction VA Medical Center, White River Junction, VT, USA Department of Psychiatry, Geisel School of Medicine at Dartmouth, Hanover, NH, USA
Monica Dimambro
Affiliation:
White River Junction VA Medical Center, White River Junction, VT, USA
Joshua Levy
Affiliation:
Pathology and Computational Biomedicine, Cedars Sinai Medical Center, Los Angeles, CA, USA
Vincent Dufort
Affiliation:
White River Junction VA Medical Center, White River Junction, VT, USA
Abby Fraade
Affiliation:
Long Island University, Brooklyn, NY, USA
Max Winer
Affiliation:
White River Junction VA Medical Center, White River Junction, VT, USA
Brian Shiner
Affiliation:
White River Junction VA Medical Center, White River Junction, VT, USA Department of Psychiatry, Geisel School of Medicine at Dartmouth, Hanover, NH, USA National Center for PTSD Executive Division, White River Junction, VTS, USA
*
Corresponding author: Maxwell Levis; Email: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

Background

Although the Department of Veterans Affairs (VA) has made important suicide prevention advances, efforts primarily target high-risk patients with documented suicide risk, such as suicidal ideation, prior suicide attempts, and recent psychiatric hospitalization. Approximately 90% of VA patients that go on to die by suicide do not meet these high-risk criteria and therefore do not receive targeted suicide prevention services. In this study, we used national VA data to focus on patients that were not classified as high-risk, but died by suicide.

Methods

Our sample included all VA patients who died by suicide in 2017 or 2018. We determined whether patients were classified as high-risk using the VA's machine learning risk prediction algorithm. After excluding these patients, we used principal component analysis to identify moderate-risk and low-risk patients and investigated demographics, service-usage, diagnoses, and social determinants of health differences across high-, moderate-, and low-risk subgroups.

Results

High-risk (n = 452) patients tended to be younger, White, unmarried, homeless, and have more mental health diagnoses compared to moderate- (n = 2149) and low-risk (n = 2209) patients. Moderate- and low-risk patients tended to be older, married, Black, and Native American or Pacific Islander, and have more physical health diagnoses compared to high-risk patients. Low-risk patients had more missing data than higher-risk patients.

Conclusions

Study expands epidemiological understanding about non-high-risk suicide decedents, historically understudied and underserved populations. Findings raise concerns about reliance on machine learning risk prediction models that may be biased by relative underrepresentation of racial/ethnic minorities within health system.

Keywords

machine learning biasmachine learningsuicide preventionveterans and military
Type
Original Article
Information
Psychological Medicine , First View , pp. 1 - 10
Check for updates
Creative Commons
This is a work of the US Government and is not subject to copyright protection within the United States. Published by Cambridge University Press
Copyright
Copyright © US Department of Veterans Affairs, 2024

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Afrose, S., Song, W., Nemeroff, C. B., Lu, C., & Yao, D. (2022). Subpopulation-specific machine learning prognosis for underrepresented patients with double prioritized bias correction. Communications Medicine, 2(1), 111. https://doi.org/10.1038/s43856-022-00165-wGoogle Scholar
Ahmedani, B. K., & Vannoy, S. (2014). National pathways for suicide prevention and health services research. American Journal of Preventive Medicine, 47(3), S222–S228. https://doi.org/10.1016/j.amepre.2014.05.038Google Scholar
Bailey, R., Mokonogho, J., & Kumar, A. (2019). Racial and ethnic differences in depression: Current perspectives. Neuropsychiatric Disease and Treatment, 15, 603–609. https://doi.org/10.2147/NDT.S128584Google Scholar
Botero, G., Rivera, N. I., Calloway, S. C., Ortiz, P. L., Edwards, E., Chae, J., & Geraci, J. C. (2020). A lifeline in the dark: Breaking through the stigma of Veteran mental health and treating America's combat Veterans. Journal of Clinical Psychology, 76(5), 831–840. https://doi.org/10.1002/jclp.22918Google Scholar
Cannizzaro, K. (2017). REACH VET and the possible impact on integrated healthcare. VA Health Care. Rocky Mountain Regional VAMC, Aurora, CO. https://avapl.org/conference/pubs/2018%20Conference%20Presentations/Cannizzaro%20-%20slides%20-%20Open%20Plenary.pdfGoogle Scholar
Ding, C., & He, X. (2004). K-means clustering via principal component analysis. Twenty-first international conference on machine learning – ICML ’04, 29. https://doi.org/10.1145/1015330.1015408Google Scholar
Elnitsky, C. A., Andresen, E. M., Clark, M. E., McGarity, S., Hall, C. G., & Kerns, R. D. (2013). Access to the US Department of Veterans Affairs health system: Self-reported barriers to care among returnees of operations enduring freedom and Iraqi freedom. BMC Health Services Research, 13(1), 498. https://doi.org/10.1186/1472-6963-13-498Google Scholar
Gianfrancesco, M. A., Tamang, S., Yazdany, J., & Schmajuk, G. (2018). Potential biases in machine learning algorithms using electronic health record data. JAMA Internal Medicine, 178(11), 1544. https://doi.org/10.1001/jamainternmed.2018.3763Google Scholar
Hein, T. C., Peltzman, T., Hallows, J., Theriot, N., & McCarthy, J. F. (2021). Suicide mortality among Veterans health administration care recipients with suicide risk record flags. Psychiatric Services, 73(3), 259–264. https://doi.org/10.1176/appi.ps.202000771Google Scholar
Huang, J., Galal, G., Etemadi, M., & Vaidyanathan, M. (2022). Evaluation and mitigation of racial bias in clinical machine learning models: Scoping review. JMIR Medical Informatics, 10(5), e36388. https://doi.org/10.2196/36388Google Scholar
Jobes, D. A., Haddock, L. A., & Olivares, M. R. (2019). Military and Veteran suicide prevention. In Ritchie, E. C., & Llorente, M. D. (Eds.), Veteran psychiatry in the US (pp. 51–71). New York, NY: Springer International Publishing. https://doi.org/10.1007/978-3-030-05384-0_5Google Scholar
Jolliffe, I. T. (2002). Principal component analysis. New York, NY: Springer-Verlag. https://doi.org/10.1007/b98835Google Scholar
Kessler, R. C., Bossarte, R. M., Luedtke, A., Zaslavsky, A. M., & Zubizarreta, J. R. (2020). Suicide prediction models: A critical review of recent research with recommendations for the way forward. Molecular Psychiatry, 25(1), 168–179. https://doi.org/10.1038/s41380-019-0531-0Google Scholar
Kessler, R. C., Hwang, I., Hoffmire, C. A., McCarthy, J. F., Petukhova, M. V., Rosellini, A. J., … Bossarte, R. M. (2017). Developing a practical suicide risk prediction model for targeting high-risk patients in the Veterans health administration. International Journal of Methods in Psychiatric Research, 26(3), e1575. https://doi.org/10.1002/mpr.1575Google Scholar
Klimes-Dougan, B., Klingbeil, D. A., & Meller, S. J. (2013). The impact of universal suicide-prevention programs on the help-seeking attitudes and behaviors of youths. Crisis, 34(2), 82–97. https://doi.org/10.1027/0227-5910/a000178Google Scholar
Lavingia, R., Jones, K., & Asghar-Ali, A. A. (2020). A systematic review of barriers faced by older adults in seeking and accessing mental health care. Journal of Psychiatric Practice, 26(5), 367–382. https://doi.org/10.1097/PRA.0000000000000491Google Scholar
Levis, M., Levy, J., Dent, K. R., Dufort, V., Gobbel, G. T., Watts, B. V., & Shiner, B. (2023a). Leveraging natural language processing to improve electronic health record suicide risk prediction for Veterans health administration users. The Journal of Clinical Psychiatry, 84(4), 22m14568. https://doi.org/10.4088/JCP.22m14568Google Scholar
Levis, M., Levy, J., Dufort, V., Russ, C. J., & Shiner, B. (2023b). Dynamic suicide topic modelling: Deriving population-specific, psychosocial and time-sensitive suicide risk variables from electronic health record psychotherapy notes. Clinical Psychology & Psychotherapy, 30(4), 795810. https://doi.org/10.1002/cpp.2842Google Scholar
Mann, J. J., Michel, C. A., & Auerbach, R. P. (2021). Improving suicide prevention through evidence-based strategies: A systematic review. American Journal of Psychiatry, 178(7), 611–624. https://doi.org/10.1176/appi.ajp.2020.20060864Google Scholar
Matarazzo, B. B., Brenner, L. A., & Reger, M. A. (2019). Positive predictive values and potential success of suicide prediction models. JAMA Psychiatry, 76(8), 869. https://doi.org/10.1001/jamapsychiatry.2019.1519Google Scholar
Matarazzo, B. B., Eagan, A., Landes, S. J., Mina, L. K., Clark, K., Gerard, G. R., … Reger, M. A. (2023). The Veterans health administration REACH VET program: Suicide predictive modeling in practice. Psychiatric Services, 74(2), 206–209. https://doi.org/10.1176/appi.ps.202100629Google Scholar
Mattocks, K. M., Cunningham, K., Elwy, A. R., Finley, E. P., Greenstone, C., Mengeling, M. A., Pizer, S. D., … Bastian, L. A. (2019). Recommendations for the evaluation of cross-system care coordination from the VA state-of-the-art working group on VA/non-VA care. Journal of General Internal Medicine, 34(S1), 18–23. https://doi.org/10.1007/s11606-019-04972-1Google Scholar
McCarthy, J. F., Bossarte, R. M., Katz, I. R., Thompson, C., Kemp, J., Hannemann, C. M., … Schoenbaum, M. (2015). Predictive modeling and concentration of the risk of suicide: Implications for preventive interventions in the US Department of Veterans Affairs. American Journal of Public Health, 105(9), 1935–1942. https://doi.org/10.2105/AJPH.2015.302737Google Scholar
McCarthy, J. F., Cooper, S. A., Dent, K. R., Eagan, A. E., Matarazzo, B. B., Hannemann, C. M., … Katz, I. R. (2021). Evaluation of the recovery engagement and coordination for health – Veterans enhanced treatment suicide risk modeling clinical program in the Veterans health administration. JAMA Network Open, 4(10), e2129900. https://doi.org/10.1001/jamanetworkopen.2021.29900Google Scholar
Meffert, B. N., Morabito, D. M., Sawicki, D. A., Hausman, C., Southwick, S. M., Pietrzak, R. H., & Heinz, A. J. (2019). US Veterans who do and do not utilize Veterans Affairs health care services: Demographic, military, medical, and psychosocial characteristics. The Primary Care Companion for CNS Disorders, 21(1), 18m02350. https://doi.org/10.4088/PCC.18m02350Google Scholar
Miller-Matero, L. R., Yeh, H.-H., Maffett, A., Mooney, J. T., Sala-Hamrick, K., Frank, C. B., … Ahmedani, B. K. (2023). Racial-ethnic differences in receipt of past-year health care services among suicide decedents: A case-control study. Psychiatric Services, 75(2), 124–130. https://doi.org/10.1176/appi.ps.20220578Google Scholar
Nock, M. K., Ramirez, F., & Rankin, O. (2019). Advancing our understanding of the who, when, and why of suicide risk. JAMA Psychiatry, 76(1), 11. https://doi.org/10.1001/jamapsychiatry.2018.3164Google Scholar
Nong, P., Williamson, A., Anthony, D., Platt, J., & Kardia, S. (2022). Discrimination, trust, and withholding information from providers: Implications for missing data and inequity. SSM – Population Health, 18, 101092. https://doi.org/10.1016/j.ssmph.2022.101092Google Scholar
OMHSP, . (2021). National Veteran Suicide Prevention annual report. VA Office of Mental Health and Suicide Prevention, Washington DC. https://www.mentalhealth.va.gov/docs/data-sheets/2021/2021-National-Veteran-Suicide-Prevention-Annual-Report-FINAL-9-8-21.pdfGoogle Scholar
Peltzman, T., Rice, K., Jones, K. T., Washington, D. L., & Shiner, B. (2022). Optimizing data on race and ethnicity for Veterans Affairs patients. Military Medicine, 187(7–8), e955–e962. https://doi.org/10.1093/milmed/usac066Google Scholar
Serneels, S., & Verdonck, T. (2008). Principal component analysis for data containing outliers and missing elements. Computational Statistics & Data Analysis, 52(3), 1712–1727. https://doi.org/10.1016/j.csda.2007.05.024Google Scholar
Shiner, B., Levis, M., Dufort, V. M., Patterson, O. V., Watts, B. V., DuVall, S. L., … Maguen, S. (2021). Improvements to PTSD quality metrics with natural language processing. Journal of Evaluation in Clinical Practice, 28(4), 520–530. https://doi.org/10.1111/jep.13587Google Scholar
Stanley, B., Brown, G., Brent, D. A., Wells, K., Poling, K., Curry, J., … Hughes, J. (2009). Cognitive-behavioral therapy for suicide prevention (CBT-SP): Treatment model, feasibility, and acceptability. Journal of the American Academy of Child & Adolescent Psychiatry, 48(10), 1005–1013. https://doi.org/10.1097/CHI.0b013e3181b5dbfeGoogle Scholar
Steinley, D. (2004). Properties of the Hubert-Arable adjusted rand index. Psychological Methods, 9(3), 386–396. https://doi.org/10.1037/1082-989X.9.3.386Google Scholar
Sullivan, J. L., Simons, K. V., Mills, W. L., Hilgeman, M. M., Freytes, I. M., Morin, R. T., … Byers, A. L. (2023). A paucity of data on Veterans 65 and older and risk of suicide: A systematic review. The American Journal of Geriatric Psychiatry, 31(7), 525–539. https://doi.org/10.1016/j.jagp.2022.11.005Google Scholar
VA/DoD. (2017). Joint Department of Veterans Affairs (VA) and Department of Defense (DoD) Mortality Data Repository – National Death Index (NDI). Center of Excellence for Suicide Prevention, Washington DC. https://www.mirecc.va.gov/suicideprevention/documents/VA_DoD-MDR_Flyer.pdfGoogle Scholar
VA/DoD. (2019). VA/DoD clinical practice guideline for the assessment and management of patients at risk for suicide (2.0). The Assessment and Management of Suicide Risk Work Group, Washington DC. https://www.healthquality.va.gov/guidelines/mh/srb/Google Scholar
Wasserman, D., Tadić, I., & Bec, C. (2023). Vision zero in suicide prevention and suicide preventive methods. In Björnberg, K. E., Hansson, S. O., Belin, M.-Å., & Tingvall, C. (Eds.), The vision zero handbook (pp. 1117–1142). New York, NY: Springer International Publishing. https://doi.org/10.1007/978-3-030-76505-7_43Google Scholar