Hostname: page-component-586b7cd67f-t7czq Total loading time: 0 Render date: 2024-11-24T19:24:39.883Z Has data issue: false hasContentIssue false

298 Improving AI Assessment of Cutaneous Chronic Graft-Versus-Host Disease using Unlabeled Patient Photographs

Published online by Cambridge University Press:  03 April 2024

Andrew McNeil
Affiliation:
Vanderbilt University
Kesley Parks
Affiliation:
Vanderbilt University Medical Center
Edward Cowen
Affiliation:
National Institute of Arthritis and Musculoskeletal and Skin Diseases
Julia Lehman
Affiliation:
Mayo Clinic
Dominique Pichard
Affiliation:
National Center for Advancing Translational Sciences (NCATS)
Michi Shinohara
Affiliation:
University of Washington
Mary Flowers
Affiliation:
Fred Hutchinson Cancer Center
Benoit Dawant
Affiliation:
Vanderbilt University
Eric Tkaczyk
Affiliation:
Vanderbilt University Medical Center
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

OBJECTIVES/GOALS: Measuring the area of skin involvement in chronic graft-versus-host disease (cGVHD) relies on costly, time-consuming manual assessment, with high disagreement among experts (>20%). Our published AI method, trained on labeled 3D photos, showed promise for delineating affected areas. We aim to improve its performance using unlabeled 2D photos. METHODS/STUDY POPULATION: Our published AI model (baseline) was trained on 360 labeled photos of 36 cGVHD patients,from a 3D camera with calibrated distance and lighting.Our gold standard labels were contours around affected skin, marked by a trained expert. A second unlabeledcohort of 974 standard 2D photos of 8 cGVHD patients was used to improve the baseline model. First the baseline model predicted affected areas on the unlabeled photos. Photos with good predictions were added to the training set with their AI-predicted labels. The model was then re-trained with the expanded labeled set. Models were successively trained with more AI labels until performance stopped improving. AI performance was assessed on a test set of 20 photos from 20 patients unseen during training, labeled by 4 experts to improve accuracy. RESULTS/ANTICIPATED RESULTS: Model performance was calculated by comparing against the gold standard labels on the test set. To quantify the spatial overlap of labeled areas the Dice coefficient was used (0 is no overlap, 1 is complete agreement), where higher values are better. To estimate clinical error we used surface area error (Error), where lower values are better. On the test set, the baseline model had a median Dice of 0.57 [interquartile range: 0.39 – 0.82] and Error of 57.6% [20.2 – 103.3%]. Re-training with additional AI-predicted labels from 8 new patients, the model yielded a median Dice of 0.60 [0.35 – 0.80] and Error of 50% [12.5 – 103.8%]. This approach is being expanded to a further 300 unlabeled patients, where we anticipate significant improvements to AI performance and consistency. DISCUSSION/SIGNIFICANCE: Evaluating AI models in standard photos could provide a consistent method of assessing and tracking cutaneous cGVHD and relieve the burden of costly expert assessment. A reliable automated AI tool would provide a meaningful improvement to the current standard of manual assessment and could be easily applied to large patient cohorts.

Type
Informatics and Data Science
Creative Commons
Creative Common License - CCCreative Common License - BYCreative Common License - NCCreative Common License - ND
This is an Open Access article, distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives licence (https://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is unaltered and is properly cited. The written permission of Cambridge University Press must be obtained for commercial re-use or in order to create a derivative work.
Copyright
© The Author(s), 2024. The Association for Clinical and Translational Science